Simulating an electric robot arm is comparatively easy. Motor torque, gear ratio, inertia, done. Hydraulics are a different animal. They’re highly nonlinear, with pressure dynamics, valve dead bands, flow saturation, oil compressibility and lag everywhere. It can be modelled with stout math, and people do it well, but it takes a lot of work and a lot of domain knowledge. Sadly, I’m not one of those people. You also end up with a pile of parameters nobody has ever measured for your specific machine, which is a problem even for the people who are.

Then there’s the linkage problem. In a hydraulically actuated boom, as in an excavator, the cylinders produce linear motion. That linear motion gets converted into joint rotation through a mechanical linkage, and the conversion is not linear. The same cylinder speed produces a different joint velocity depending on where the joint currently sits. So you either simulate the linkage geometry properly, or you compensate by modifying the joint velocity gain based on the joint’s position relative to the previous link. Top all that off with non-standard hydraulic valves, as in our case, and head scratching is guaranteed.

A neural network sidesteps the whole taxonomy. It doesn’t care whether a given effect came from the valve, the cylinder, the linkage geometry or the pump. It learns the whole system as one input-to-output relationship, which is all the log data really contains anyway.

The Model

The architecture is an MLP, a multilayer perceptron. Three hidden layers of 128 units each, with ReLU activations. Cheap and enough to bend a linear map into a nonlinear one. That’s a small network by any modern standard, which is the point. It is meant to be run fast.

The input dimension is 138: 3 joint positions, 0.1 s of velocity history and 0.99 s of command history. The output is the change in joint velocity over the next step. One network handles all three joints together so it can learn the coupling between them. On a real machine the axes share a pump, and moving two or more at once changes them all.

I trained the model at 100 Hz because that’s what our little excavator is running at. Data was collected for 90 minutes. For most of that time the control was automated, and the excavator was just semi-randomly moving around.

The model’s velocity output is integrated into position before feeding it to the sim. I felt it makes more sense to do it this way as an excavator boom doesn’t suddenly drop when the command is zeroed, position control keeps it where it stops.

Ten seconds of never-seen recorded valve commands replayed through the sim (NVIDIA Isaac Lab), against the real machine. Green is the given command, blue is the measurement from the real machine, red and yellow are the model’s predicted output through the sim, one for each of the two integration routes mentioned below.

Two ways are better than one

I wasn’t sure how I’d build the controller, so I split it into two versions.

Direct integrates the learned velocity into joint state and writes it straight to the simulator. The model *is* the dynamics, the simulation engine doesn’t get a say. This is the most faithful reproduction of the learned hydraulics, and the one to use when you want the model to be ground truth. The trade-off is that those joints won’t react to contact at all.

Target integrates the same velocity into a position target and lets the simulation’s internal joint controller chase it. Contact and load response are somewhat preserved, at the cost of introducing a second dynamic system the real machine does not have, with its own lag. On paper this should behave better in contact rich scenes. In practice it still needs proper testing.

What this buys us?

For our AI-MaSI project work the approach looks very promising. For example, the model allows us to use the same controller gains in simulation and on the real machine. We can drop the model into RL training environments so that policies learn the actual system dynamics instead of an idealized version of them, which has been an issue in basically all our RL tasks. We can even flip it around entirely: train a model that corrects these nonlinearities, and deploy that on the real excavator.

Have we struck gold?

Well, yes and no. The system is comparatively fast to set up, and the initial results look great. But there are real caveats.

The model has learned neither contact nor system load. There’s no ground truth anywhere in the loop, so position drifts. The drift is fine for shorter runs of roughly one to five seconds, which sounds damning but is actually plenty for simulation use, as we can reset and teleport the robot how we like and just take the nice parts.

More subtly: the model learns the sensors, not the robot. IMU noise or a bad fusion pipeline can make or break it, and no amount of training fixes a dataset that was lying to you.

And like any learned model, it has blind spots. Feed it a region of the input space the data never covered and it will happily produce something weird with full confidence.

The model can be a magical tool for some things, while not suiting others at all. Knowing which is which is the skill.

---

Author

Eetu Miettinen, Research Engineer at AI-MaSi, Savonia University of Applied Sciences.

This article was prepared with the help of AI.

---

References

Egli, P. and Hutter, M. (2020) ‘Towards RL-Based Hydraulic Excavator Automation’, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 2692–2697. (research-collection.ethz.ch)

Miettinen, E. (2026) ‘Isaac Hydraulic Actuator’, GitHub repository. Available at: https://github.com/eetmie/Isaac-hydraulic-actuator (Accessed: 18 August 2026). (GitHub)

---

This work was carried out as a part of AI-MaSi project, Pohjois-Savon Liitto, EAKR