Research and implementation of an electromechanical operation simulation system based on virtual reality

HONG KONG, July 14 10, 2022 (GLOBE NEWSWIRE) — The WIMI Hologram Academy, in partnership with the Holographic Science Innovation Center, has written a new technical paper describing its research and implementation of a virtual reality-based electromechanical operating simulation system. This article follows below:

Electromechanical simulation technology based on virtual reality, through the establishment of a three-dimensional simulation structure of a virtual reality scene and connected to virtual reality equipment such as posture capture systems and screens head-mounted, can observe the operating conditions of equipment from different points of view, relative position relationships, simulated tracking monitoring and live roaming, and a variety of interactive operations. It realizes multi-person collaborative operation of electromechanical systems and greatly facilitates operators familiar with actual electromechanical equipment. Scientists from the WIMI Hologram Cloud Inc.’s (NASDAQ: WIMI) WIMI Hologram Academy studied the research and implementation of electromechanical operations simulation systems based on virtual reality.

Virtual reality-based electromechanical system simulation mainly uses virtual reality technology, smart wearable technology and distributed computer simulation technology to achieve comprehensive interactive simulation analysis and immersive simulation operation simulation in the world real of the system. The system builds real-size virtual models of real scenes. At the design and development stage, the system can be used to check whether the structure, layout and control process of the equipment is reasonable, and then provide a design basis for the adaptation study of the equipment. ‘equipment. The system supports multi-person collaborative operation, which can provide strong technical support for operators to thoroughly familiarize with the function, structure and composition of the actual equipment, and quickly master the methods. of equipment operation, maintenance, repair and emergency safety, and is important equipment as auxiliary training and simulation operation training after equipment manufacturing is completed.

1.Hcomputer system Ddesign scheme

Motion capture devices, data handles and data gloves are included in posture capture control equipment.

2. Software ssolutions

2.1 Overall structure of the software

The main functions of the software are: driving 3D models, simulating real working conditions, switching scene views, remote communication, capturing equipment data and driving character models, etc. The software includes operation simulation software, human-computer interaction interface software, model calculation and interface control software, and control simulation software. The HCI interface is mainly responsible for acquiring the character movement, hand joint and posture information. It also drives the character model and renders the scene to the corresponding display device in real time. The operation simulation software is responsible for managing the simulation, driving the 3D model and simulating the real working conditions. The model calculation and interface control software is responsible for analyzing, processing and transmitting the data to the corresponding receiver. The control simulation software is able to receive commands entered by the operator on the control panel and control the entire flow of the 3D scene and model accordingly, while correctly displaying status parameters in the system.

2.2 Operation simulation software

2.2.1 The overall structure of the operation simulation software

The VR-based electromechanical system simulation is built on the Unity3D platform, complemented by scripting programs written in C#. The system enables model driving, physics effects calculation, animation rendering, human-computer interface control, communication and motion response. The interface control software receives input commands and motion control parameters in real time and sends them to the digital simulation calculation program. The digital simulation calculation program processes input commands and motion control parameters, calls its corresponding 3D model and digital model, and realizes the simulation physical effects such as model collision, articulation and force with the help of the physics engine, calculates model parameters such as speed and direction in real time, and sends these physical state parameters to the virtual reality simulation subsystem. The virtual reality simulation subsystem renders and displays the 3D model according to the corresponding state parameters of each model, and sends the model feedback parameters to the digital simulation calculation program, which sends the state information (position, speed, etc.) to the interface control server software communication after processing by the digital simulation calculation program.

2.2.2 3D and digital models

Virtual reality-based electromechanical system simulation contains 3D models and digital models of each subsystem. The 3D model is built according to the 1:1 scale of the physical model, and the appearance is consistent with the physical model. The digital model is based on the 3D model and adds the corresponding physical characteristics, such as rigid body, joint, collision body and material. The digital simulation calculation program calls different 3D models and corresponding digital models according to different control instructions, and initializes the status values ​​and flag bits of the digital models in the initial step. After the system starts, the control flag bits are scanned frame by frame according to the frame rate interval, and the operating conditions of the equipment are updated in the simulation subsystem of virtual reality, and the physical calculations are performed according to the fixed physical time interval, and the physical state of the equipment is refreshed.

2.3 Human-computer interaction interface

A virtual reality scene with a 3D simulated structure is created and connected to human-computer interaction devices such as posture capture systems, data gloves, head-mounted displays, and data handles. Engineers can immerse themselves in the virtual reality scene, realize interactive device operation, and immersively interact with the virtual reality scene of the 3D simulated structure. The posture capture system is mainly used to capture the movement process of the human torso, hands and feet and other major joints. Based on the posture capture system, the walking and movement of the person in the simulated equipment scene is realized. The character’s motion and posture data is collected through the appropriate capture software and communicated with the professional software through the Transmission Control Protocol / Internetwork Protocol. The character’s motion and posture data is corrected in the professional software to derive joint position and rotation information, which drives the human model.

The data glove can accurately capture the movement process of human hand and finger joints, enabling more delicate operations in simulated device scenarios. The flexion angle of the fingers is obtained using sensors on the glove, and a communication program is written in C# to communicate with specialized local software. The data correction is then completed in the software and the movement of the character’s hand is controlled in real time. The data handle enables control of the movement of the virtual device and the virtual human body, mainly for checking operation, spatial and layout analysis, etc. The data handle has a special interface in the professional software, and the control program can be written to set the function of the handle buttons to achieve a wide range of character movement and viewpoint switching. The head-mounted display is connected to the software through an official development kit, and two cameras are set in the head-mounted display to simulate the binocular vision of the human eye, and the corresponding parameters are set in the appropriate software to complete the immersive human – computer interaction.

Using posture capture devices, gloves, and head-mounted displays, operators can use virtual characters to perform interactions in virtual scenes. The implementation of interaction technology is mainly achieved by collision detection, trigger collision detection and logic processing algorithms. During the actual operation of human-computer interaction, the phenomenon of characters intersecting with virtual objects in virtual scenes may occur due to the conflict between the physics engine and the external character capture. Considering the reasonableness and reliability of the actual process, the following processing will be carried out. First, when the character in the virtual scene is about to pass through the object, an illegal operation prompt appears on the screen, and the prompting process does not conform to reality. Second, when the character has passed through the object in the virtual scene, the character capture system will stop working and the character model will be stationary. Finally, when the character in the virtual scene is about to leave the object, the character capture can be resumed and the character model can be trained when the distance is judged to meet the conditions.

3. Conclusion

The design scheme of this system realizes interactive simulation analysis and real-world immersive simulation operation simulation of the whole process of the electromechanical system. At the stage of product design and development, the system can be used to check whether the equipment structure, layout and control process are reasonable, and provide a design basis for the study of adaptability equipment. At the same time, help operators to familiarize themselves with the functions of the equipment, the structure and the composition of the equipment. It provides important technical support for operators to quickly master the operation, maintenance, repair and emergency safety management methods of equipment, and can also be used to assist the training and operation training of simulation once the manufacture of the equipment is complete.

Founded in August 2020, the WIMI Hologram Academy is dedicated to exploring AI holographic vision and researching basic science and innovative technologies, guided by human vision. Holographic Science Innovation Center, in partnership with WIMI Hologram Academy, is committed to exploring the unknown technology of AI holographic vision, attracting, gathering and integrating relevant global resources and superior forces, promoting comprehensive innovation with science and technology innovation as the core, and conduct research in basic science and innovative technology.

Holographic Science Innovation Center
Email: [email protected] com

Comments are closed.