AI Assistant and MCP Agent Setup

Trellilume can work with AI in two different ways:

Feature	What it does	What you configure
Built-in Assistant	Uses a model to understand a request and prepare a safe plan inside Trellilume.	An AI provider such as Ollama, LM Studio, or OpenAI.
MCP Agent Access	Lets a separate, MCP-compatible agent search your library or request allowed actions.	The Trellilume MCP server, a scoped Agent Grant, and an external MCP client.

These features are independent. You can enable either one or both. Connecting an AI provider does not give that provider MCP access, and creating an MCP grant does not select an AI model.

Only a server administrator can change these settings. Start with read-only access, keep privacy controls at their defaults until you understand them, and review every proposed action.

Set up the built-in Assistant in the app

1. Sign in to Trellilume with a server administrator account.
2. Open Settings > Server > AI Providers.
3. Turn on Enable AI provider.
4. Choose a provider type and enter the provider details described below.
5. Leave the privacy controls off unless you intentionally want to share those data types with the provider.
6. Select Save.
7. Select Test Provider. A passing check confirms that the Trellilume backend can reach the model and receive a response.

The API key is saved on your Trellilume server, is never displayed again in the app, and is not stored on the phone. A successful provider check verifies the connection; it does not guarantee that every model will produce a valid plan.

Important: Which machine does the endpoint describe?

The Trellilume backend contacts the AI provider. Your phone does not. This means localhost always refers to the machine or container running the backend.

• Direct backend install, provider on the same computer: use localhost.
• Trellilume in Docker Desktop, provider on the host: use host.docker.internal.
• Provider on another computer: use that computer's private IP address and allow the provider to listen on the local network.

Do not expose an unauthenticated local model server directly to the internet.

Option A: Ollama

Ollama runs models on your own computer. This avoids cloud API charges, but the computer running the model needs enough memory for the model you choose.

1. Install Ollama on the same computer as the Trellilume backend, or on another trusted computer on your local network.

2. Download a model. For example:

   ollama pull qwen3:14b

3. Confirm the model is installed:

   ollama list

4. In Settings > Server > AI Providers, enter:

   Setting	Value
   Provider type	Ollama
   Model	qwen3:14b, or the exact name shown by ollama list
   Endpoint, direct install	http://localhost:11434/v1
   Endpoint, Docker Desktop	http://host.docker.internal:11434/v1
   API key	Leave blank

5. Save and test the provider.

If Ollama is on another computer, its default loopback-only listener will not accept the connection. Configure OLLAMA_HOST for local-network access, restart Ollama, and use an endpoint such as http://192.168.1.50:11434/v1. Restrict the port to your trusted network. See Ollama's server configuration and network guidance and OpenAI-compatible API documentation.

Option B: LM Studio

LM Studio provides a desktop interface for downloading and running local models.

1. Install LM Studio and download a model that supports structured JSON output.

2. Open LM Studio's Developer page, load the model if needed, and start the local server. The default port is 1234.

3. Copy the exact model identifier shown by LM Studio.

4. In Settings > Server > AI Providers, enter:

   Setting	Value
   Provider type	LM Studio
   Model	The exact LM Studio model identifier
   Endpoint, direct install	http://localhost:1234/v1
   Endpoint, Docker Desktop	http://host.docker.internal:1234/v1
   API key	Leave blank unless LM Studio authentication is enabled

5. Save and test the provider.

If LM Studio is on another computer, turn on Serve on Local Network, enable authentication, and use the address LM Studio displays. See the official local server quickstart, network guidance, and OpenAI-compatible endpoint guide.

Option C: OpenAI

OpenAI runs the model in the cloud. Text and any context allowed by your Trellilume privacy settings are sent to OpenAI for processing.

1. Create an API project, configure API billing, and create a secret key on the OpenAI API key page. API billing is separate from a ChatGPT subscription.

2. Copy the key when it is created. OpenAI does not show the complete secret again later.

3. In Settings > Server > AI Providers, enter:

   Setting	Value
   Provider type	Custom OpenAI-compatible
   Model	gpt-4.1-mini, or another compatible model enabled for your project
   Endpoint	https://api.openai.com/v1
   New API key	Your OpenAI project API key

4. Save and test the provider.

Keep the key private and never put it in a screenshot, source file, browser script, or mobile app. See OpenAI's API quickstart, API key guidance, and API billing explanation.

Review the Assistant privacy controls

Open Settings > Server > AI Providers > Privacy controls before using a cloud provider. Trellilume blocks these data types by default:

• media bytes;
• location metadata;
• raw EXIF metadata;
• storage paths;
• filenames; and
• sensitive metadata.

Turn on only the data types that are necessary for your use case. Local Ollama or LM Studio reduces cloud disclosure, but the selected endpoint still receives the context that Trellilume allows.

Set up MCP Agent Access

MCP, or Model Context Protocol, is a standard that lets an external AI client discover and call Trellilume tools. Trellilume uses Streamable HTTP and scoped Agent Grant secrets.

1. Enable MCP on the server

MCP is disabled by default. In the Trellilume server .env file, add:

NESTPICS_MCP_ENABLED=1
NESTPICS_MCP_ROUTE_PREFIX=/mcp

Restart the backend after changing .env:

docker compose up -d

Keep the default HTTPS requirement. Plain HTTP is allowed by default only when the MCP client reaches the backend as localhost. For a browser-based MCP client, list its exact origin with NESTPICS_MCP_ALLOWED_ORIGINS; do not use a wildcard.

2. Create a scoped Agent Grant in the app

1. Sign in as a server administrator.
2. Open Settings > Server > MCP Agent Access.
3. Confirm the MCP Server card says Enabled and review any warnings.
4. Select New Grant.
5. Enter a descriptive name, such as LM Studio on family desktop.
6. Choose a short expiry. The default is 90 days; 7 or 30 days is safer for a test connection.
7. Select only the scopes the agent needs. Start with library:read. Add assets:read, albums:read, or tags:read only when needed.
8. Select Create, then copy the full secret immediately. Trellilume shows it only once and stores only a hash on the server.

Treat the secret like a password. If it is lost, create a new grant. If it is exposed, revoke the grant in MCP Agent Access.

3. Add Trellilume to an MCP client

Use your normal Trellilume server URL plus /mcp and send the grant secret as a Bearer token. Generic MCP client settings look like this:

{
  "url": "https://photos.example.com/mcp",
  "headers": {
    "Authorization": "Bearer PASTE_THE_AGENT_GRANT_SECRET_HERE"
  }
}

The exact field names vary by MCP client. The transport must be Streamable HTTP. A compatible client automatically sends the MCP protocol and Accept headers.

4. Use LM Studio as the external MCP client

LM Studio can be both an AI provider for Trellilume's built-in Assistant and a separate MCP client. These are two different connections.

1. In LM Studio, open the Program tab in the right sidebar.

2. Select Install > Edit mcp.json.

3. Add this entry, replacing the URL and secret:

   {
     "mcpServers": {
       "trellilume": {
         "url": "https://photos.example.com/mcp",
         "headers": {
           "Authorization": "Bearer PASTE_THE_AGENT_GRANT_SECRET_HERE"
         }
       }
     }
   }

4. Save the file, load a model that supports tool use, and enable the Trellilume MCP server for the chat.

5. Start with a read-only request such as, "Summarize the albums I can access."

6. Return to Settings > Server > MCP Agent Access in Trellilume and confirm that the grant's Last used time changed.

LM Studio stores the secret in mcp.json, so protect that file like a password file. See LM Studio's official MCP setup guide.

Understand MCP scopes and action safety

• Read scopes allow access only to the visible summaries described by that scope.
• actions:propose:* scopes let the agent prepare an action for review inside Trellilume. The agent cannot confirm its own proposal.
• actions:execute:* scopes request direct execution and are not recommended for a first setup. A matching scope alone is not enough: server policy, user permission, pause controls, resource checks, and any required short-lived authorization must also allow the exact action.

Prefer read-only or proposal scopes. Review Settings > Server > Agent action policy before considering any direct execution scope.

Network and security rules

1. Keep MCP on localhost, a trusted private network, or a private VPN when possible.
2. Use HTTPS for any connection that is not localhost.
3. Do not broadly expose the MCP endpoint to the public internet. Trellilume's broad remote OAuth deployment path still requires operator configuration and target-specific security testing.
4. Use one grant per client so it can be revoked without affecting other agents.
5. Use the shortest practical expiry and the fewest scopes.
6. Revoke grants you no longer use and review their Last used times.

Troubleshooting

The provider check says connection refused

• Confirm Ollama or LM Studio is running and a model is installed.
• If Trellilume runs in Docker Desktop, replace localhost with host.docker.internal in the provider endpoint.
• If the provider is on another computer, enable local-network serving and check that its firewall allows the provider port only from trusted devices.
• Confirm the endpoint ends in /v1.

The provider check says the model was not found

Copy the exact identifier from ollama list, LM Studio, or your OpenAI project. Model names are case-sensitive and may include a size or version suffix.

MCP Agent Access says Disabled

Set NESTPICS_MCP_ENABLED=1 in the environment used by the backend and restart the backend. Editing a different .env file does not change a running container.

The MCP client gets 401 or an authentication error

Create a new grant if the secret was not copied, and send it as Authorization: Bearer <secret>. Confirm the grant is active and not expired or revoked.

The MCP client gets 403

Confirm the URL uses HTTPS unless the client connects through localhost. For a client that sends an Origin header, add that exact origin to NESTPICS_MCP_ALLOWED_ORIGINS, restart the backend, and avoid *.

The agent cannot see a tool or item

Tool discovery is filtered by the grant's scopes and current Trellilume policy. Create a new narrowly scoped grant if additional access is necessary. Locked or otherwise restricted content stays subject to its normal access rules.