Industrial

The challenge: The blind metaverse

A robot in simulation performs perfectly, but only in theory. As soon as it encounters the real world, algorithms often fail due to light reflections, shadows, or minimal deviations. This is the Sim2Real Gap, the biggest hurdle for real physical AI. 

Manual data collection is considered the "gold standard," but it is prone to human error and simply not scalable for SMEs. When geometries or materials change, constantly maintaining data records manually becomes uneconomical. The realistic solution therefore lies in synthetic data: our research proves that even purely synthetically trained models perform well and the targeted addition of just a few real images ("hybrid approach") even deliver state-of-the-art results.

Stages of perception

Our research in the real-world laboratory has shown that robust perception is an evolutionary process. We have validated our systems step by step:

1. Localization (bounding boxes): The first step. The AI learns where an object is located in space. This rough framing is often sufficient for pure counting tasks or inventories, but does not yet provide any information about the actual shape or orientation of the component for a gripping operation.
2. Precision (semantic segmentation): The system learns to precisely separate object boundaries from the surrounding environment—essential for safe interaction. This pixel-precise masking enables the robot to recognize exact contours, preventing accidental damage to neighboring objects or container walls during gripping.
3. Understanding (6D pose estimation): The key to physical AI. AI understands the exact position and rotation of an object in 3D space in order to manipulate it physically. Only by combining three position and three rotation coordinates does the robot arm know exactly from which angle it must approach a randomly positioned workpiece in order to assemble it safely.

The strategy: generalization instead of copying

The goal is not to recreate reality in the metaverse with pixel-perfect accuracy—that would be inefficient. Instead, we are pursuing a domain randomization approach. We train the AI with thousands of variations of the environment. This teaches the system not to be distracted by changing light or backgrounds, but to focus on the physical geometry of the component.

Application: The self-updating twin

The goal is a seamless cycle: a sensor scans the hall, the AI detects changes (e.g., a machine that has been moved) and automatically updates the digital twin. The metaverse remains synchronized with reality.