A digital twin is a method where complex systems and processes are developed or monitored using a digital equivalent of the physical system. A True Digital Twin^SM replicates all aspects of the physical twin, and including all mechanical, robotic, sensor, vision, AI, and computer systems, while simultaneously incorporating real human action and interaction. Humans experience the system in virtual reality (VR), with concurrent motion capture driving their avatars in real time. The avatars interact with virtual objects and their physical “stand ins”, which are also motion-tracked. All aspects of the system, including hardware, software, AI training data/models, and human interaction, are developed, tested and optimized virtually.

The result is faster design discovery and iteration at a much lower cost. Once tested and optimized, the virtual system is deployed physically. Team collaboration is also improved, as key stakeholders in training, operations, safety, and other groups can join the virtual development environment as needed, even if located thousands of miles apart. Finally, the use of a true digital twin removes the requirements for in-person human interaction, which allows for remote system development.

Digital twins not only accurately represent the system's layout and motion, but also the software control systems. Specialized components such as industrial robots typically come with control and training software, and that software is utilized to integrate the robot into the overall real-time simulation. Virtual robots and other operations are tested within the simulation. This approach streamlines custom control software development as the code is designed, tested and debugged within the virtual environment before being deployed on the physical counterpart.

Kinetic Vision's digital twin development services not only reduce time and cost, but also improve communication and collaboration between other functional product teams. The virtual environment is concurrently deployed to all groups, which enables training, safety and maintenance personnel to easily visualize and understand all aspects of the system, even if located thousands of miles apart. System monitoring sensors and devices are linked together both in physical and virtual space, allowing for remote troubleshooting and process monitoring.

Learn more about digital twins at Wikipedia.

Automated Product Inspection

This example is an automated system to receive just-in-time shipments of automotive oil filters, perform a quality inspection using industrial computed tomography, and route the acceptable products to the manufacturing line. Shown is the 3D virtual environment showing the incoming product shipments, CT scanner, and conveying of qualified products to production. The entire system’s design, optimization, and testing was done using this environment. Commercial simulation software drove the robot in the twin.

This is a side-by-side comparison of the twins during qualification testing. The physical robot was programmed in the digital twin to pick the incoming oil filters, place them in the scanner, perform the scan, and remove the filters. Those that pass inspection are routed to the manufacturing line, those that don’t are collected and returned to the vendor. All system development was done virtually in the digital twin before any physical testing.

Hospital Surgical Suite

In this example a new surgical suite is being developed, including room configuration and equipment layout, and new device concepts and new procedures are being evaluated. The surgeon avatars are driven in real time using concurrent motion capture, allowing true subject matter experts to be present in the digital twin.

The subject matter experts, in this case surgeons evaluating a new procedure, experience the twin through virtual reality. They feel just like they’re in the actual space. Their motion, including hands and figures, is tracked in real time, as is the position of all objects in the scene: the patient, equipment, tables and medical devices.

With Kinetic Vision's digital twin development services, new devices can be evaluated virtually; they just need to feel and act like the proposed device, but complex functionality is done digitally. A 3D printed model with a spring-loaded handle can digitally replicate a prototype that would take many months and $50,000 or more to create. Device design iterations take hours or days instead of weeks or months, and the subject matter experts can be located anywhere.

A composite of the surgeon headset views, motion capture process, and digital twin third-person camera is shown below. Room configuration, equipment layout, procedures and devices are tested virtually, allowing for iteration and innovation to occur at a much faster pace. Although not shown here, it is even possible to simulate actual human tissue and procedures for training and to evaluate medical device concepts.

Retail AI System

In this example a retail AI system is being developed with a goal of using in-store cameras to monitor and detect shopper product selections. Project goals include the creation of synthetic training data, training deep learning models, and testing model performance using real shopper behavior. A true digital twin was utilized to train and test a deep learning model. Initial testing within the digital twin uncovered a deficiency in the system, as object detection failed once items were placed in the shopping cart. This was quickly remedied by adding synthetic images of products in the cart to the training data, and continuing the model training. The updated model met the required performance metrics, and was subsequently verified with physical testing.

Using True Digital Twins to Develop, Test and Optimize Complex Systems