Cadillac – Reimagining Cadillac’s User Experience
Rightpoint Helps Boston Dynamics Create Bespoke Experiences to Ready Spot for Market
Boston Dynamics
Boston Dynamics is an engineering and robotics design company founded in 1992 as a spin-off from work done at the Massachusetts Institute of Technology and Carnegie Mellon. The company’s mission is to imagine and create exceptional robots that enrich people's lives.
As a proponent of dynamic robotics, Boston Dynamics has created and published excellent content on the potential of robotics and has caught the imagination of the public. Spot was Boston Dynamics’ first commercial product, and is arguably the most well-known and beloved robot.
Starting with a Strategy
Spot is truly a groundbreaking robot with a wide array of industrial use cases, from work safety, security, operational reliability and downtime reduction, to digital data collection and analysis and site awareness. The engineers designed Spot to replace humans in dangerous situations, such as monitoring nuclear power plant facilities and conducting remote inspections.
Spot is more than a robot, however. It is a platform on which its users can mount a wide range of “payloads” to assist in data gathering activities such as thermal imaging, LiDAR, 3D laser scanning technology for digital twin creation, noise anomaly detection, and gas leak detection sensors. Spot not only gathers data, but also performs tasks, such as gauge reading, remote inspections, and autonomous security detection and prevention. Spot was designed with an articulating arm that can perform a range of action-oriented tasks, such as flipping switches, turning valves, and opening doors just to name a few, to meet the user’s custom needs.
The Challenge: A User Experience that Didn’t Live Up to Expectations
As Boston Dynamics got closer to commercial release with Spot, they realized that the whole out-of-box experience from setup-to-control needed work. This is understandable, as Boston Dynamics’ expertise is in robotics, not user experience or mobile app development.
Initial versions of Spot required a laptop that was hooked up to a gaming controller. To control the robot in the real world, the user needed to strap the laptop into a sling and walk around next to the robot with it. This was hardly the futuristic experience the company envisioned. The company wanted to replace the laptop with a more manageable tablet experience, but didn’t have expertise developing mobile applications.
Other challenges around creating a great user experience included issues with app-to-robot communication latency, video feeds from the on-board cameras that weren’t optimized for teleoperation, and myriad other issues to work through. For instance, latency between a command entered by the operator and the robot’s execution of it led to some problems, which in turn made it difficult for the robot to execute some tasks correctly. These kinks are to be expected, as any head of R&D, Chief Digital Officers or Chief Product Officers working to enable IoT devices knows well. In working on any connected device, the digital experience is often designed and developed as the device itself is under development.
Boston Dynamics also wanted Spot to work in facilities that are remote or unsafe for humans. To enable teleoperation, the controllers would need a clear video stream so that they could steer the robot through a facility remotely. The first cameras on Spot weren’t intended to produce a streamed video feed for a controller, but the team quickly realized it was essential, and could support a new range of use cases for Spot.
“The cameras on board were really never intended to produce human-readable video, they were really intended to help the onboard processors figure out what steps to take, how to balance, and what the environment looks like. The Rightpoint team helped correct this through advanced on-device image processing.” said Andy Watson, Digital Product - IoT Lead at Rightpoint.
As Boston Dynamics got closer to market release, they realized they needed an expert in user experience design & development to help them conceive, create and deploy. That’s when they contacted Rightpoint.
Work
Rightpoint is known for unparalleled expertise in thinking about and developing innovative connected-user experiences, as well as for building mobile and custom applications. Boston Dynamics was keen to tap into Rightpoint’s expertise in human machine interfaces. Additionally, Rightpoint’s global presence enabled it to provide a local team to work collaboratively onsite with Boston Dynamics.
“Boston Dynamics came to us knowing there was a real gap in Spot’s control experience,” Ben Johnson, SVP and Global Leader of Rightpoint’s Digital Product team explained. “At the time, only a few engineers within their company had the skills to operate Spot. We needed to help them create a control experience that would work for anyone.”
Spot was a wholly unique robot in a new product category, which meant that Rightpoint needed to pull together a wide array of resources to work on the project, including experts in robotics, mobile apps, visual and motion design.
The Rightpoint team began with a discovery process. We asked: what are all the things that must seamlessly happen so that people who aren’t experts in robotics can use Spot out of the box and customize it to meet their use cases?
The team developed a number of iterations, drawing from shared inspirations both the Rightpoint and Boston Dynamics team members had a love of Star Wars and video games (many of the Boston Dynamics Engineers said they were inspired by Star Wars when they chose robotics as a career).
Rightpoint started to explore exceptional examples of joystick controls on mobile games and infused the Spot experience with the best of the best. “A lot of people play those games, which means the Spot interface would be intuitive for them,” Dave Counts, Rightpoint’s Group Design Director, explained. “Star Wars also influenced the way we thought about motion for instructional purposes.”
Rightpoint also proposed various intuitive ways for controlling Spot’s robotic arm and switching it through the different control modes. The Rightpoint team created animations and graphical treatments of how Spot (and the robotic arm) could look on the tablet as it moves through a facility.
Rightpoint Gets to Work
Over the next six months, Rightpoint team members were regular guests at Boston Dynamics offices, experimenting with early prototypes of Spot to refine the control system, refine the app, and work through issues together.
The first order of business was to design, develop and launch an Android app to control Spot. The app featured a slick, Star Wars-like interface with added joystick controls on both sides. Because the Rightpoint team used the Spot prototypes regularly, they were able to identify issue areas with the overall experience. For instance, Rightpoint saw a need to incorporate a video feed into the app controller to enhance and, ultimately, extend the user experience for teleoperations. To address some of the challenges with the on-board stereo imagery captured by the robot, Rightpoint brought in an engineer with extensive computer vision experience to stitch the two video feeds together into a single seamless video feed. This extended the capability of the on-board video the robot used for decision making to produce a video feed the end user can use in the controller app to plan the robot’s next move.
The Android controller app comes preinstalled on a tablet that is now included with every robot sold. The device also includes extensive instructions for setting the robot up along with providing manual control capabilities.
Outcomes
Rightpoint delivered a sophisticated yet fun and fully functioning interface in just six months. The interface met Boston Dynamics' goal of making Spot usable to a wide variety of people who might not necessarily be trained in robotics. Put another way, Rightpoint helped transform Spot from a device that only engineers could use to one that had commercial viability. Rightpoint was also able to accelerate the company's time to market by providing them with the expertise needed to quickly build an intuitive controller app experience and achieve the goal to sell their first 100 Spot robots. The Android tablet that is included with Spot is a key part of the experience for every customer.
“Overall, we created a great canvas for Boston Dynamics’ to expand upon,” said Ben Johnson. “We’re proud to be a part of their story.”
Due to the ease of use, control, configuration, capabilities, and publicity received, Spot quickly became the most renowned and beloved robot in the market.
Our Contributions
Capabilities
Digital Product
Our Contributions
Technology
IoT
Digital Twin
Robotics
