Hacker News new | past | comments | ask | show | jobs | submit
Aren't there other ONNX runtimes out there? A quick google search shows a few written in Rust: Tract, RTen, WONNX, Burn&Candle. Also, IIRC there's a Julia-based one as well.
loading story #50543994
https://github.com/tracel-ai/burn-onnx

I maintain this project at the moment.

It convert ONNX models into native, backend-agnostic Burn code for inference and fine-tuning.

This looks awesome, i may be using this in prod very soon. Ive had some ONNX issues this looks to solve
Have used it with wasm and it's lovely, but I was kindof disappointed to learn internally it's basically another serialisation format. I imagined gigabytes of generated Rust code hehe
The graph is converted into Rust code that calls Burn APIs but the weights are converted into Burn's format, called burnpack (similar to safetensors but instead of JSON we use CBOR for weights meta data).
Yet another alternative: convert your `.onnx` file to GoMLX [0] using onnx-gomlx [1].

For CPUs using SIMD it's significantly faster than ONNX. For NVidia GPUs, depending on the model, you can use an XLA backend. On Apple you can use an MLX backend (but it supports Neon/SME as well).

- [0] : https://github.com/gomlx/gomlx - [1] : https://github.com/gomlx/onnx-gomlx