Set up Ollama in the AppHost

⭐ Community Toolkit

This article is the reference for the Aspire Ollama hosting integration from the Aspire Community Toolkit. It enumerates the AppHost APIs — with examples for both AppHost.cs and apphost.mts — that you use to model an Ollama server and its model resources in your AppHost project.

If you’re new to the Ollama integration, start with the Get started with Ollama integrations guide. For how consuming apps read the connection information this page exposes, see Connect to Ollama.

Installation

To start building an Aspire app that uses Ollama, install the 📦 CommunityToolkit.Aspire.Hosting.Ollama NuGet package:

C#

Terminal

aspire add ollama --source CommunityToolkit

Learn more about aspire add in the command reference.

Or, choose a manual installation approach:

AppHost.cs

#:package CommunityToolkit.Aspire.Hosting.Ollama@*

AppHost.csproj

<PackageReference Include="CommunityToolkit.Aspire.Hosting.Ollama" Version="*" />

TypeScript

Terminal

aspire add ollama --source CommunityToolkit

Learn more about aspire add in the command reference.

This updates your aspire.config.json with the Ollama hosting integration package:

aspire.config.json

{
  "packages": {
    "CommunityToolkit.Aspire.Hosting.Ollama": "*"
  }
}

Note

The TypeScript AppHost bindings for Ollama are provided by the Community Toolkit package and are not part of the core Aspire TypeScript SDK. The .aspire/modules/aspire.mjs file in your project will include addOllama and related APIs after installation.

Add Ollama resource

Once you’ve installed the hosting integration in your AppHost project, you can add an Ollama server resource and then add model resources as shown in the following examples:

C#

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama");
var llama3 = ollama.AddModel("llama3");

var exampleProject = builder.AddProject<Projects.ExampleProject>("apiservice")
    .WithReference(llama3);

// After adding all resources, run the app...

TypeScript

apphost.mts

import { createBuilder } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");
const llama3 = await ollama.addModel("llama3");

await builder.addNodeApp("api", "./api", "index.js")
    .withReference(llama3);

// After adding all resources, run the app...

1. When Aspire adds a container image to the AppHost, as shown in the preceding example with the docker.io/ollama/ollama image, it creates a new Ollama instance on your local machine.

2. The resource name passed to AddOllama is used as the connection string name when referenced in a dependency. Model sub-resources are named by sanitizing the model name (for example, "llama3" produces the resource name "ollama-llama3").

3. The AppHost reference call configures a connection in the consuming project named after the referenced model resource.

Note

When you reference an Ollama model resource from the AppHost, Aspire makes several properties available to the consuming project, such as the HTTP endpoint, host, and port. For a complete list of these properties and per-language connection examples, see Connect to Ollama.

Add Ollama model resource

Models are sub-resources of an Ollama server resource. You add them with the AddModel (or addModel) method. Each model is downloaded when the Ollama container first starts.

C#

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama");

// Add by model name — resource name is generated automatically
var phi35 = ollama.AddModel("phi3.5");

// Add with an explicit resource name
var llama3 = ollama.AddModel("ollama-llama3", "llama3");

var exampleProject = builder.AddProject<Projects.ExampleProject>("apiservice")
    .WithReference(phi35)
    .WithReference(llama3);

// After adding all resources, run the app...

TypeScript

apphost.mts

import { createBuilder } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");

// Add by model name — resource name is generated automatically
const phi35 = await ollama.addModel("phi3.5");

// Add with an explicit resource name
const llama3 = await ollama.addNamedModel("ollama-llama3", "llama3");

await builder.addNodeApp("api", "./api", "index.js")
    .withReference(phi35)
    .withReference(llama3);

// After adding all resources, run the app...

Download the LLM

When the Ollama container for this integration first spins up, it downloads the configured LLMs. The progress of this download displays in the State column for this integration on the Aspire dashboard.

Caution

Keep the Aspire orchestration app open until the download is complete, otherwise the download will be cancelled.

Add Ollama resource with data volume

Add a data volume to the Ollama resource to persist downloaded models across container restarts:

C#

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama")
    .WithDataVolume();

var llama3 = ollama.AddModel("llama3");

var exampleProject = builder.AddProject<Projects.ExampleProject>()
    .WithReference(llama3);

// After adding all resources, run the app...

TypeScript

apphost.mts

import { createBuilder } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");
await ollama.withDataVolume();

const llama3 = await ollama.addModel("llama3");

await builder.addNodeApp("api", "./api", "index.js")
    .withReference(llama3);

// After adding all resources, run the app...

The data volume is mounted at /root/.ollama in the Ollama container. When a name parameter isn’t provided, the volume name is generated automatically. For more information on data volumes and details on why they’re preferred over bind mounts, see Docker docs: Volumes.

Use GPUs when available

By default, the Ollama container runs on CPU. To enable GPU acceleration, use the WithGPUSupport (or withGPUSupport) extension method:

C#

Nvidia:

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama")
    .WithGPUSupport();

ollama.AddModel("llama3");

// After adding all resources, run the app...

AMD:

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama")
    .WithGPUSupport(OllamaGpuVendor.AMD);

ollama.AddModel("llama3");

// After adding all resources, run the app...

TypeScript

Nvidia:

apphost.mts

import { createBuilder } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");
await ollama.withGPUSupport();

await ollama.addModel("llama3");

// After adding all resources, run the app...

AMD:

apphost.mts

import { createBuilder, OllamaGpuVendor } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");
await ollama.withGPUSupport({ vendor: OllamaGpuVendor.AMD });

await ollama.addModel("llama3");

// After adding all resources, run the app...

Note

In TypeScript, select the GPU vendor with the OllamaGpuVendor enum imported from the generated .aspire/modules/aspire.mjs — for example, OllamaGpuVendor.AMD or OllamaGpuVendor.Nvidia.

For more information, see GPU support in Docker Desktop and GPU support in Podman.

Add a model from Hugging Face

To use a model in GGUF format from Hugging Face, use the AddHuggingFaceModel (or addHuggingFaceModel) extension method:

C#

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama");

var llama = ollama.AddHuggingFaceModel(
    "llama",
    "bartowski/Llama-3.2-1B-Instruct-GGUF:IQ4_XS");

var exampleProject = builder.AddProject<Projects.ExampleProject>()
    .WithReference(llama);

// After adding all resources, run the app...

TypeScript

apphost.mts

import { createBuilder } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");

const llama = await ollama.addHuggingFaceModel(
    "llama",
    "bartowski/Llama-3.2-1B-Instruct-GGUF:IQ4_XS");

await builder.addNodeApp("api", "./api", "index.js")
    .withReference(llama);

// After adding all resources, run the app...

Only models in GGUF format are supported. The integration automatically prefixes hf.co/ to the model name if it doesn’t already start with hf.co/ or huggingface.co/, so Ollama can resolve it from Hugging Face.

Add Ollama resource with parameters

To use a fixed port for the Ollama container, pass it as a parameter:

C#

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama", port: 11434);

ollama.AddModel("llama3");

// After adding all resources, run the app...

TypeScript

apphost.mts

import { createBuilder } from './.aspire/modules/aspire.mjs';

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama", { port: 11434 });

await ollama.addModel("llama3");

// After adding all resources, run the app...

Open WebUI support

The Ollama integration also provides support for running Open WebUI alongside the Ollama container:

C#

AppHost.cs

var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama("ollama")
    .WithOpenWebUI();

ollama.AddModel("llama3");

// After adding all resources, run the app...

TypeScript

apphost.mts

import { createBuilder } from "./.aspire/modules/aspire.mjs";

const builder = await createBuilder();

const ollama = await builder.addOllama("ollama");
await ollama.withOpenWebUI();

await ollama.addModel("llama3");

// After adding all resources, run the app...

Connection properties

For the full reference of Ollama connection properties — and how consuming apps in C#, TypeScript, Python, and Go read them — see Connect to Ollama.

Hosting integration health checks

The Ollama hosting integration automatically adds health checks for both the Ollama server and each model resource:

• Server health check. Verifies that the Ollama server is running and that a connection can be established to it.
• Model health check. Verifies that a model has been downloaded and is available. The model resource is marked as unhealthy until the download completes.