As some societies grapple with labor shortages exacerbated by declining and aging populations, Mitsubishi’s AVATAR aspires to be a game-changing solution. Through its remote operation capabilities, AVATAR provides a platform (software and hardware) designed to make it easier for anyone to perform tasks from any location, thereby democratizing labor and potentially improving safety in hazardous work environments.
Mitsubishi thinks the AVATAR operation interface is simple and intuitive. I didn’t get to operate it, but it did look reasonably simple to get the robots to do its various tasks.
Drawing on human cognitive characteristics, the interface employs Visual Haptics technology to present force levels at remote locations as Augmented Reality (AR) images. Powered by Mitsubishi Electric’s proprietary AI technology Maisart, the system is said to learn and replicate movements automatically. This simplifies tasks like moving objects from one point to another and reduces the operator’s work.
At CEATEC 2023, Mitsubishi demonstrated the entire stack of hardware and software using a simple robot that could move around and move objects using two arms. Obviously, this demo doesn’t achieve the full AVATAR vision, but the company wanted to show tangible progress.
To make teleoperation practical and user-friendly, AVATAR relies on two core strategies:
First, simple and intuitive vision-based operation: Mobile smart devices serve as the operating interface, transmitting motion and sensory information between the operator and AVATAR. Four specific technologies have been developed to facilitate this:
- Visual Haptics: Uses AR to present force haptic information at the point of contact.
- Human Eye Display: Adjusts the resolution of the central and peripheral fields of view according to the task.
- One-Click Operation: Allows users to control the system through simple taps on a touch panel.
- Eye Contact: Creates a more natural interaction by linking a smartphone camera to an extended monitor.
- Augmented Avatar Technology: This flexible system separates the “mind” and the “body” of the avatar, enabling task-specific designs and implementations. Basically, it separates intent from action.
Secondly, task operations can be broken down into sub-tasks, some of which can be automated to reduce the operator’s workload further. This step-by-step automation allows one operator to manage multiple machines simultaneously. You can think of it as macro-level operating instead of micro-managing every movement from the avatar.
If you want to learn more, I find this white paper (PDF) by Mitsubishi’s award-winning team, “Last Mile,” fascinating.
Social Implications
Beyond addressing labor shortages and safety concerns, AVATAR technology aims to solve the issue of uneven population distribution. It gives individuals the freedom to live and work wherever they prefer, thus creating a more balanced distribution of the working population. By the way, the movie Surrogates was based on a similar idea.
This vision of the future will require time and more research to take shape as envisioned, however, it will probably lead to significant learning and lessons for future designs.
Filed in . Read more about CEATEC, Japan and Mitsubishi Electric.