OctoAI was a hosted service for running generative AI models, made by the company OctoAI. It served inference, which means running trained models and returning their output, for models such as Llama 3.1, Mistral NeMo and the image model FLUX.1. It was offered as SaaS (software as a service) or, through its product OctoStack, inside a customer's own environment.
Developers called the models through an API. The service let them mix models, fine-tuned versions and LoRAs (small add-on weight files that adapt a model to a task) at the serving layer. It also offered retrieval-augmented generation with embeddings, function calling for AI agents, and a JSON mode for structured output. OctoAI said its platform was built on systems and compiler technologies that it had pioneered, such as TVM and MLC.
The service is closed. The newest archived copy of its own home page, from 12 December 2024 on the Internet Archive, carries the heading "OctoAI is now NVIDIA". In October 2026 the address octo.ai redirects to nvidia.com.