Foundry Local

What is Foundry Local?

Foundry Local is a powerful tool that enables developers to run AI models directly on user devices, ensuring complete data privacy and eliminating the need for cloud subscriptions. With native SDKs for Python, JavaScript, C#, and Rust, you can easily download, cache, load, and call optimized local models in-process. The SDK automatically selects the best execution provider—NPU, GPU, or CPU—so your app can leverage available hardware without custom plumbing. This means faster inference, lower latency, and offline capability, as all runtime and cached models stay local. Use cases include building AI-powered applications that require real-time responses, such as virtual assistants, content generation, or audio processing, all while keeping sensitive data on-device. For development, you can start with CLI or REST tools, but for production, the in-process SDK clients offer seamless integration. Optionally, you can start an OpenAI-compatible server to work with frameworks like LangChain. Key benefits include enhanced user privacy, reduced cloud costs, and the ability to run AI features even without internet connectivity. Technical details: initialize a manager, choose a model alias, download and cache it, then call chat or audio clients. The SDK handles execution provider registration, making it easy to target NPU, GPU, or CPU. Foundry Local is ideal for developers who want to build secure, offline-capable AI applications with minimal overhead. Whether you're prototyping in Python or shipping in C#, Foundry Local provides a streamlined workflow for on-device AI inference.

Key Features

on device AI inference complete data privacy native SDKs for Python/JS/C#/Rust automatic execution provider selection offline capability reduced cloud costs real time responses OpenAI compatible server

Who is it for?

mobile app developers, edge computing engineers, privacy, focused developers, AI application builders, IoT developers, enterprise IT teams