# Savonia Article Pro: It can do it better! Pushing VLAs all out on Jetson Orin Nano 8 GB

##### Savonia Article Pro is a collection of multidisciplinary Savonia expertise on various topics.

20.10.2026 [Articles Pro](https://www.savonia.fi/en/category/articles-pro)[Hankkeet](https://www.savonia.fi/en/category/hankkeet)

This work is licensed under [CC BY-SA 4.0![Creative Commons logo](https://mirrors.creativecommons.org/presskit/icons/cc.svg?ref=chooser-v1)![Creative Commons Attribution logo](https://mirrors.creativecommons.org/presskit/icons/by.svg?ref=chooser-v1)![Creative Commons Share Alike logo](https://mirrors.creativecommons.org/presskit/icons/sa.svg?ref=chooser-v1)](http://creativecommons.org/licenses/by-sa/4.0/?ref=chooser-v1)

This is a short technical follow-up to my previous article, [[It can do it! Making VLA models fit an 8 GB Jetson Orin Nano](https://www.savonia.fi/en/articles-pro/it-can-do-it-making-vla-models-fit-an-8-gb-jetson-orin-nano/),], where I managed to run sub-1 B models with TensorRT by splitting them into ONNX graphs and building them on the Jetson one graph at a time. This article continues that work with much larger models: NVIDIA’s GR00T [N1.6, [https://huggingface.co/nvidia/GR00T-N1.6-3B](https://huggingface.co/nvidia/GR00T-N1.6-3B)] and [N1.7, [https://huggingface.co/nvidia/GR00T-N1.7-3B](https://huggingface.co/nvidia/GR00T-N1.7-3B)] at around 3 B, and Physical Intelligence’s [π0.5, [https://arxiv.org/abs/2504.16054](https://arxiv.org/abs/2504.16054)] at 3.3 B parameters.

Since the ONNX splits let me fit and test quite large models, I decided to try fitting the largest ones I could. π0.5 was my end goal, as that seems to be _the model_ to use when building custom, open-world robots!

While doing the research, I found very little information about running models over 1 B on the Orin Nano, and π0.5 specifically was said NOT to fit (without quants). Well, it does now, though with very little RAM to spare.

## How 3 B models fit without quantization

I didn’t want to hack my way there with crazy-low quants, so I ran all the models in mostly FP16 (some smaller parts stay FP32 for accuracy), which is at most a very, very light quantization. The model checkpoints are published as FP32 or BF16 weights, and the FP16 versions reproduce their actions almost exactly: in my tests the cosine similarity was at least 0.99999, and the difference stayed within 0.13 % of the action range (0.63 % for π0.5 base).

Going lower would also have been a lot of extra work. The Orin Nano has no hardware FP8 or FP4 support, so it would mean INT8 calibration or custom INT4 kernels for every model, plus an accuracy hit to measure on the robot.

So the room had to come from cutting waste:

\1. One template per layer type. π0.5’s layers repeat, so its bundle ships 7 layer templates plus the weights of all 67 layers. The Jetson builds each template once and fills in the weights at load, so building never needs more than one layer’s worth of memory.

\2. Big tables off the GPU. The token-embedding tables of EVO-1 and π0.5 stay on the CPU; only the rows the prompt needs are fetched.

\3. Skip work nobody uses. π0.5’s last language layer only produces what the action expert reads (saves 213 MB). GR00T only computes attention for the rows each block can actually see (saves 99 MB on N1.6, 157 MB on N1.7).

\4. Share buffers. Chained engines take turns on two shared buffers instead of each having its own (saves 22–96 MB).

\5. Export on a PC. The full FP32 model is only ever loaded on the PC that does the export, never on the Jetson.

## Results

Below are the median timings from 300 s runs, split by stage, with each model’s median latency and the board’s RAM in use at the end of its bar.

Setup: JetPack 7.2.1, TensorRT 10.16, MAXN_SUPER with pinned clocks, desktop running (headless recommended!), synthetic camera images and each model’s default number of views and denoising steps. RAM is whole-board usage during inference.

![Pylväsdiagrammi, jossa verrataan eri VLA-mallien viivettä ja RAM-muistin käyttöä Jetson Orin Nano -laitteella, jaoteltuna käsittelyvaiheittain: esikäsittely, kuvankäsittely, kielenkäsittely ja kohinanpoisto.](https://www.savonia.fi/app/uploads/2026/10/plot-1200x531.png)

Per-stage latency and RAM use of each model on the Jetson Orin Nano 8 GB. Image: Claude Opus 5.5.

π0.5 spends most of its time in the language model. The two π0.5 versions are the same 3.3 B model with different inputs: [openpi’s, [https://github.com/Physical-Intelligence/openpi](https://github.com/Physical-Intelligence/openpi)] LIBERO fine-tune uses two cameras and a 521-token prefix, while LeRobot’s base checkpoint uses three cameras and also writes the robot state into its prompt, for up to 912 tokens. X-VLA spends its time in denoising and EVO-1 in its vision encoder.

## Under the hood: Triton kernels inside TensorRT

[TensorRT, [https://developer.nvidia.com/tensorrt](https://developer.nvidia.com/tensorrt)] does most of the work. Custom kernels made with [Triton, [https://github.com/triton-lang/triton](https://github.com/triton-lang/triton)] go only where a profile shows TensorRT doing something wasteful. TensorRT 10’s Python plugin API lets a Triton kernel be compiled ahead of time to PTX (NVIDIA’s low-level GPU assembly) for the Orin’s SM87 GPU and stored inside the engine. At run time there is no Python, Triton or PyTorch in the loop: TensorRT launches the kernel like its own. The engines of a policy are chained on the GPU, and the whole sequence is captured once and replayed per inference as a CUDA graph (a recorded sequence of GPU work that runs with a single launch).

Each change was handled the same way: trace the board with NVIDIA’s [Nsight Systems, [https://developer.nvidia.com/nsight-systems](https://developer.nvidia.com/nsight-systems)] and read the engine’s per-layer tactics before writing a kernel. The biggest wins:

The biggest speed-ups from custom kernels and export fixes. Image: Claude Opus 5.5.

Some things didn’t pay off. Custom matrix-multiply kernels were pointless, since TensorRT already uses 86–91 GB/s of the board’s ~96 GB/s memory bandwidth. Triton didn’t beat TensorRT in π0.5’s language model either. Memory has a price too. On 8 GB every speed-up has to pay for the RAM it costs, so a few working tricks were dropped. All in all, the Orin Nano can do a lot more than its 8 GB suggests. Models up to 3.3 B run at full FP16 accuracy, and [the repo, [https://github.com/eetmie/jetson-orin-nano-vla](https://github.com/eetmie/jetson-orin-nano-vla)] makes a great base to keep building on. Your own fine-tunes can go through the same provided export and run scripts.

Also, here’s an Orin Nano controlling a real excavator (X-VLA for now, hopefully π0.5 soon!):

(VIDEO: kaivuri.mp4)

Excavator kicking cones autonomously. Video: Joni Hänninen.

This work was carried out as part of the T.A.I.T.O project, supported by Pohjois-Savon liitto and the European Regional Development Fund (EAKR).

References

Black, K. et al. 2025. [π0.5: a Vision-Language-Action Model with Open-World Generalization, [https://arxiv.org/abs/2504.16054](https://arxiv.org/abs/2504.16054)]. arXiv:2504.16054.

Miettinen, E. 2026a. [It can do it! Making VLA models fit an 8 GB Jetson Orin Nano, [https://www.savonia.fi/en/articles-pro/it-can-do-it-making-vla-models-fit-an-8-gb-jetson-orin-nano/](https://www.savonia.fi/en/articles-pro/it-can-do-it-making-vla-models-fit-an-8-gb-jetson-orin-nano/)]. Savonia Article Pro. Accessed 14 October 2026.

Miettinen, E. 2026b. [jetson-orin-nano-vla, [https://github.com/eetmie/jetson-orin-nano-vla](https://github.com/eetmie/jetson-orin-nano-vla)]. Source code and retained benchmark results. Revision 6e2d857. Accessed 14 October 2026.

NVIDIA et al. 2025. [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots, [https://arxiv.org/abs/2503.14734](https://arxiv.org/abs/2503.14734)]. arXiv:2503.14734.

NVIDIA. n.d.-c. [Nsight Systems, [https://developer.nvidia.com/nsight-systems](https://developer.nvidia.com/nsight-systems)]. Accessed 14 October 2026.

NVIDIA. n.d.-d. [TensorRT, [https://developer.nvidia.com/tensorrt](https://developer.nvidia.com/tensorrt)]. Accessed 14 October 2026.

Triton. n.d. [Triton language and compiler, [https://github.com/triton-lang/triton](https://github.com/triton-lang/triton)]. Source code. Accessed 14 October 2026.

#### Authors

Eetu Miettinen, Research Engineer at T.A.I.T.O / AI-MaSi, Savonia University of Applied Sciences.

This article was prepared with the help of AI.