NVIDIA wants to make the trip from model checkpoint to production inference a two-command affair. TensorRT Model Connect, now in public preview under an Apache-2.0 license, takes a supported Hugging Face or local checkpoint straight to TensorRT inference with no ONNX export in between.
The project’s core artifact is a versioned bundle that runs through native C++ task APIs, so inference can live inside embedded systems or robotics stacks without PyTorch in the runtime. Rather than one generic converter, it ships family-owned reference implementations spanning text generation, speech, image generation, segmentation, embeddings, and more, with task APIs such as generate, transcribe, and embed.
Wheels today target Linux aarch64 with Python 3.10 or 3.12 and TensorRT 11.1; x86-64 users build from source via Docker. NVIDIA’s GB300 snapshot covers 105 profiles across 76 model families, 102 of them beating their declared reference by more than 5 percent.
The project also carries a curious footnote: NVIDIA says the entire codebase, from model implementations to tests and docs, was built by OpenAI Codex agents under human direction.
