Reverse Bridge Using HTC Vive to Control Real Robot Arms (Drawings)

Virtual Reality Studio 219 Design hopes to use the newly discovered capabilities in VR to build bridges between the virtual world and the real world. After considering existing technologies and the needs of their users, they decided to develop a new tool to manipulate real-world devices through a virtual user interface. They chose a robotic arm that can be intuitively manipulated through a virtual application as a proof of concept for the target.

To create this system quickly and efficiently, 219 Design uses rapid prototyping tools and iteratively developed, flexible, cross-domain design flows. Mathematical modeling, MATLAB simulation, computer-aided design, open source electronic hardware, 3D printing, and rapid production methods are all key to the success of this project.

Develop virtual controls

Now, 219 Design already has a virtual application that can precisely and intuitively control the robot arm. This application can not only control in real time, but also easily record the motion path of a robot arm. This allows the user to "teach" the robot arm for specific actions without any advanced programming knowledge. Recording a new action is as simple as pressing the record button, enabling the robot's end effector in the VR and dragging it into the 3D scene.

Potential applications

In this proof of concept, the 219 Design uses a smaller robot arm, but this method can also be used in powerful industrial machines. Imagine that the entire plant can be designed and programmed in VR, including all mechanical actions. There is no longer any need for specialists in the field of industrial robots, nor is it necessary to spend a great deal of time and experience coding the low-level robotic arms. With this architecture, anyone can easily and quickly program or reprogram the behavior of the machine. Some companies have developed robotic arms that can be easily programmed, but few companies currently use VR's intuitiveness and immersion.

This virtual control system allows users to interact with remote machine systems and even interact with dangerous machine systems. For example, a surgeon in California can intuitively perform surgery on a patient in Tokyo, Japan, and it can be easily implemented without the need for high-intensity training on mechanical controls.

Learning in development

Although this project is only a proof of concept, in the process of making a VR robotic arm system, 219 Design has accumulated experience and knowledge and knows how to connect the virtual world and the real world. 219 Design said to Xiao Bian that the experience accumulated in designing this innovative tool is helpful to users, and hopes that this project can inspire other engineers to develop a control system that integrates VR's intuitiveness.