# Imitating hydraulics in Simulation: Can neural networks model physics?

> A small MLP learns an excavator’s hydraulic dynamics from drive recordings, replacing hand-built physics in simulation.

## The Challenge of Hydraulic Simulation
Simulating an electric robot arm is comparatively easy. Motor torque, gear ratio, inertia, done. Hydraulics are a different animal. They’re highly nonlinear, with pressure dynamics, valve dead bands, flow saturation, oil compressibility and lag everywhere. It can be modelled with stout math, and people do it well, but it takes a lot of work and a lot of domain knowledge. You also end up with a pile of parameters nobody has ever measured for your specific machine.

Then there’s the linkage problem. In a hydraulically actuated boom, as in an excavator, the cylinders produce linear motion. That linear motion gets converted into joint rotation through a mechanical linkage, and the conversion is not linear. The same cylinder speed produces a different joint velocity depending on where the joint currently sits.

## The Neural Network Approach
A neural network sidesteps the whole taxonomy. It doesn’t care whether a given effect came from the valve, the cylinder, the linkage geometry or the pump. It learns the whole system as one input-to-output relationship.

*   **Architecture:** A multilayer perceptron (MLP) with three hidden layers of 128 units each, using ReLU activations.
*   **Input:** 138 dimensions (3 joint positions, 0.1 s of velocity history, and 0.99 s of command history).
*   **Output:** The change in joint velocity over the next step.
*   **Training:** Trained at 100 Hz using 90 minutes of data collected from an excavator moving semi-randomly.

## Simulation Integration
The model’s velocity output is integrated into position before feeding it to the simulation. Two versions were developed for the controller:

1.  **Direct:** Integrates the learned velocity into joint state and writes it straight to the simulator. The model *is* the dynamics. This is the most faithful reproduction of the learned hydraulics but does not react to contact.
2.  **Target:** Integrates the velocity into a position target and lets the simulation’s internal joint controller chase it. Contact and load response are somewhat preserved, at the cost of introducing a second dynamic system with its own lag.

## Project Applications and Limitations
For the AI-MaSI project, this approach allows for consistent controller gains between simulation and the real machine. It enables policies to learn actual system dynamics during Reinforcement Learning (RL) training rather than idealized versions.

**Caveats:**
*   **Drift:** The model has not learned contact or system load, leading to position drift over time. This is acceptable for short simulation runs (1–5 seconds).
*   **Sensor Dependency:** The model learns the sensors, not the robot. IMU noise or poor data fusion can compromise the model.
*   **Blind Spots:** Like any learned model, it may produce unpredictable results if fed inputs outside the training distribution.

## References
*   Egli, P. and Hutter, M. (2020) ‘Towards RL-Based Hydraulic Excavator Automation’, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2692–2697.
*   Miettinen, E. (2026) ‘Isaac Hydraulic Actuator’, GitHub repository. Available at: https://github.com/eetmie/Isaac-hydraulic-actuator

*This work was carried out as a part of AI-MaSi project, Pohjois-Savon Liitto, EAKR.*