As if walking a robot dog wasn’t strange enough, Boston Dynamics’ Spot can now take visitors for a walk, as it takes on the role of an English tour guide.
In a new video, the engineering firm showed off Spot’s new ability to answer questions and crack jokes using a range of accents, as well as several distinct personalities.
The robot, decorated with tiny hats and googly eyes, leads guests to different locations and describes what it is seeing.
Opening and closing its grabber to mimic a mouth and turning to ‘look’ at people, Spot’s performance is impressively close to that of a real guide.
Powered by ChatGPT, Spot’s creators say they have been surprised by some of the unexpected responses the robot dog has come up with.
Spot the robot dog now has the ability to play tour guide using ChatGPT to answer questions and describe the world around it
What is ChatGPT?
ChatGPT is an AI chatbot designed by Open AI in November 2022.
Upon release it became the quickest app in history to reach 100 million users.
It uses a specially trained Large Language Model to predict what should follow in strings of text.
This allows the AI to respond to prompts in natural language and help with a wide range of tasks.
ChatGPT has been used for everything from coding to drafting legal statements with varying degrees of success
The footage shows the $75,000 (£61,857) robot adopt the personality of a ‘fancy butler’, saying: ‘My linguistics have been meticulously crafted to provide an authentic British experience.’
Asked by principle software engineer Matt Klingensmith if it enjoys its job, the robot responds: ‘Ah Mr Matt, my employment as a tour guide provides great satisfaction.
‘I find the dissemination of knowledge rather rewarding, don’t you agree?’
However, this is far from the only character played by the robot dog, as it takes on the voices and characteristics of a ‘precious metal cowgirl’, ‘excited tour guide’, or a ‘nature documentary’.
The robot also displays an impressive capability to recognize and respond to objects in the world.
While taking on the role of a ‘1920s archaeologist’, Spot refers to the camera crew, calling them ‘a fellow explorer with a camera’.
One of the most surprising personalities is ‘Josh’, a sarcastic and moody robotic guide, which Mr Klingensmith says is ‘an experience I’ve never had with a robot before in my entire life’.
Asked for a haiku about the room, the Josh personality replies: ‘Generator hums low in a room devoid of joy, much like my soul, Matt.’
However, all these diverse functions and personalities are the product of simple adjustments to the same code.
Using different prompts, the large language model behind Spot is able to create an entire spectrum of personalities from ‘Fancy Butler’ to ‘Josh’
Boston Dynamics provide the AI with this map of office, including a few labeled locations and some short descriptions
The team provided ChatGPT with a carefully designed prompt that converts visual and auditory information into speech and commands for the robot.
Prompts include information such as ‘use the tour guide API to guide guests through a building using a robot. Tell the guests about what you see, and make up interesting stories about it.’
They add context such as: ‘Personality: “You are a snarky, sarcastic robot who is unhelpful”‘.
The model is then provided with a map of the building, featuring some labelled rooms and brief descriptions.
However, the AI is only given just enough information to get started.
Its creators say ‘the LLM can be thought of as an improv actor—we provide a broad strokes script and the LLM fills in the blanks on the fly.’
To convert visual information into text that ChatGPT can use, Spot uses a program called a Visual Question Answering model which captions images from the robot’s cameras.
With all this information, ChatGPT 4 can command the robot to move around its environment, respond to questions, and comment on its surroundings.
Some of the behaviour Spot has been able to produce with ChatGPT at its core has surprised even its creators.
‘Josh’ the sarcastic tour guide was one of the most surprising personalities and a very different experience to most previous robot-human interactions
This diagram shows how information in the real world is converted into prompts that the AI can use to control the robot’s speech and movement
In a blog post, Mr Klingensmith and his team point out that large language models (LLMs) like ChatGPT tend to produce ’emergent behaviour’ that is not predicted by the code.
In one example the team asked Spot who its parents were, and the robot led them to the display of older Spot models, saying that these were its ‘elders’.
Likewise, when asked ‘who is Marc Raibert?’ the robot responded: ‘I don’t know. Let’s go to the IT help desk and ask.’
The concepts of parents or a command to ask for help had not been programmed in, although the team insist this is not evidence that the robot is thinking.
‘To be clear, these anecdotes don’t suggest the LLM is conscious or even intelligent in a human sense—they just show the power of statistical association,’ the team writes.
‘But the smoke and mirrors the LLM puts up to seem intelligent can be quite convincing,’ they added.
Boston Dynamics added (1) a navigation system, (2) a microphone, (3) a speaker, and (4) a gripper arm and camera
This isn’t the first role that has been suggested for Spot by Boston Dynamics, as the company has outfitted the robot for a number of different purposes.
Spot engineer Zack Jackowski previously said that the next big application of the technology would be creating robotic guards that can patrol industrial plants and factories.
Mr Jackowski suggests that Spot could autonomously walk around sites collecting data with its sensors to spot issues like open doors or fire hazards.
Spot has already been deployed to inspect nuclear power plants, oil rigs, and construction sites, and even keep an eye on the ruins of Pompeii.
Boston Dynamics’ other robots have also set the bar for bipedal robotic mobility as videos show Atlas effortlessly jumping and performing parkour.
Recently, a video showed how Atlas was able to help on construction sites as it carries bags of tools and lumber up and down scaffolding.
Boston Dynamics first showed off Spot, the most advanced robot dog ever created, in a video posted in November 2017.
The firm, best known for Atlas, its 5 foot 9 (1.7 metre) humanoid robot, has revealed a new ‘lightweight’ version of its robot Spot.
The robotic canine was shown trotting around a yard, with the promise that more information from the notoriously secretive firm is ‘coming soon’.
‘Spot is a small four-legged robot that comfortably fits in an office or home’ the firm says on its website.
It weighs 25 kg (55 lb), or 30 kg (66 lb) when you include the robotic arm.
Spot is all-electric and can go for about 90 minutes on a charge, depending on what it is doing, the firm says, boasting ‘Spot is the quietest robot we have built.’
Spot was first unveiled in 2016, and a previous version of the mini version of spot with a strange extendable neck has been shown off helping around the house.
In the firm’s previous video, the robot is shown walking out of the firm’s HQ and into what appears to be a home.
There, it helps load a dishwasher and carries a can to the trash.
It also at one point encounters a dropped banana skin and falls dramatically – but uses its extendable neck to push itself back up.
‘Spot is one of the quietest robots we have ever built, the firm says, due to its electric motors.
‘It has a variety of sensors, including depth cameras, a solid state gyro (IMU) and proprioception sensors in the limbs.
‘These sensors help with navigation and mobile manipulation.
‘Spot performs some tasks autonomously, but often uses a human for high-level guidance.’