# Control of an excavator robot with VR glasses through 5G network - Savonia AMK

> The remote control of heavy and risky construction machines such as excavators has always been a topic of interest. However, the lack of high-quality video streaming and low-latency control has posed a major obstacle to remote control.

## Overview
The remote control of heavy and risky construction machines has historically been hindered by insufficient video streaming quality and high latency. The emergence of 5G technology provides a solution by offering high bandwidth and low latency. This study details a pilot application where an excavator robot was remotely controlled using VR goggles and control equipment over the Savonia 5G test network, providing the operator with a safe and comfortable environment.

## Implementation Process
The pilot began with the preparation of the excavator robot and the development of a virtual environment in the Unity platform. Key steps included:
*   Configuring VR controllers to send task commands.
*   Establishing a communication protocol between Unity and the excavator.
*   Setting up camera connections and video streaming protocols.
*   Connecting both the excavator and the Unity workstation to the 5G network.

## Preparation of the Excavator Robot
The excavator robot is a Raspberry Pi-based device. Its movement is managed by Python-based control software that operates a servo motor-controlled hydraulic system and a tank track system. The software listens for client commands, which are processed and sent to the hardware via an ADC converter.

## Unity Platform and VR Integration
Unity is used to render a 180-degree video stream from the excavator to the operator's VR goggles. 
*   **Video Streaming:** An MJPEG streamer transmits real-time video from the robot to the Unity platform via a dedicated IP address and socket communication.
*   **Control Logic:** VR controllers are mapped to specific tasks. Commands are sent from the Unity platform back to the excavator robot using the same socket communication protocol.
*   **Visuals:** A 180-degree USB camera provides a broad field of view, with video parameters like frame rate and resolution configured within the MJPEG streamer.

## 5G Network Infrastructure
The Savonia 5G test network was utilized to ensure the necessary bandwidth and low latency for real-time operation. Both the VR workstation and the excavator robot were equipped with 5G mmWave modems, connecting to radio base stations via a private network using dedicated SIM cards and IP addresses. This setup ensured a secure, high-speed, and low-latency connection.

## Results and Conclusion
Testing on the Savonia 5G network demonstrated that the system could handle delicate tasks with high precision. The operator reported ease of use, and the video stream proved more stable and reliable than comparable Wi-Fi connections. This study confirms that 5G technology enables reliable remote operation of heavy machinery, with future work planned to expand these capabilities to other types of devices.

## Video
[Watch the demonstration on YouTube](https://www.youtube.com/watch?v=im2QZc-_jdU)

## Project Information
*   **Project:** 5G:llä vauhtia digitalisaatioon (REACT-EU EAKR / Pohjois-Savon liitto, A77467)

## Authors
*   **Osman Torunoglu**, R&D Specialist, DigiCenter, Savonia University of Applied Sciences, osman.torunoglu@savonia.fi
*   **Timo Lassila**, Test Engineer, DigiCenter, Savonia University of Applied Sciences, timo.lassila@savonia.fi
*   **Aki Happonen**, Project Manager, DigiCenter, Savonia University of Applied Sciences, aki.happonen@savonia.fi

## Unity3D VR Environment Developers
*   **Mahdi Belila**, Savonia University of Applied Sciences, mahdi.belila@savonia.fi
*   **Premton Canamusa**, Savonia University of Applied Sciences, premton.canamusa@savonia.fi

## Topic Categories
Tekniikan ala, TKI, 5G (Technology, R&D, 5G)