Chat

Conversational AI for your knowledge base.

How It Works

Agency's chat feature lets you have a conversation with your documents:

Ask a question in natural language
Agency searches your knowledge base (semantic + keyword + filters)
Local AI generates an answer from the retrieved chunks
Sources are cited for verification with deep links to the excerpt

Chat Capabilities

Question answering: "What was the revenue in Q3?"
Summarization: "Summarize the key points of this report"
Comparison: "How do these two contracts differ?"
Analysis: "What trends do you see in these reports?"
Context switching: Follow-ups reuse the current conversation context.

Privacy

All chat processing happens locally:

Your questions never leave your device
AI models run entirely on your machine (downloaded and managed by the app)
No cloud API calls for inference
Complete conversation privacy

Tips & Controls

Choose model size to balance speed vs depth; switch in settings without re-ingesting data.
Citations link back to exact chunks; click to open the excerpt.
Long documents are chunked; keep questions specific to tighten grounding.
If cloud history sync is enabled, per-query encryption keys keep conversations private to you.

Models Included by Default

The app uses local Ollama at http://localhost:11434.
New teams default to the Balanced preset: LLM qwen3:8b with embedding mxbai-embed-large.
Other presets you can pick in Settings → Models:
Lightweight: llama3.2:3b + nomic-embed-text
Balanced (default): qwen3:8b + mxbai-embed-large
Quality: mistral:latest + BAAI/bge-m3
Reasoning: gemma3:12b + embeddinggemma:latest
Multilingual: qwen3:14b + BAAI/bge-m3
Additional LLM options available in the selector: phi4:mini, llama3.2:3b, llama3.1:8b, qwen3:8b, qwen3:14b, mistral:latest, gemma3:4b, gemma3:12b.
Embedding options: all-minilm, nomic-embed-text, mxbai-embed-large, BAAI/bge-m3, embeddinggemma:latest, plus legacy all-MiniLM-L6-v2 and all-mpnet-base-v2.
Changing the embedding model requires a full re-index of documents; LLM changes do not.

Bringing Your Own Model (Ollama-compatible)

Train or fine-tune: Use a base model (e.g., Llama 3 or Mistral). If you fine-tune with LoRA using tools like *unsloth*, merge to a GGUF or other Ollama-compatible artifact.
Create an Ollama model: Write a Modelfile that points to your base or merged weights, set context length and quantization, then run ollama create my-custom -f Modelfile.
Verify locally: ollama run my-custom "Hello" to ensure it responds and fits your hardware.
Add to Agency: In Settings → Models → Add Custom Model (Ollama), enter the tag (e.g., my-custom or an Ollama library tag) and click Pull Model. It will appear under LLM choices when installed.
Set for the team: Select the custom model as the LLM for the team. Embedding models must come from the supported list to keep dimensions consistent; switching embeddings triggers re-indexing.

Notes:

Ensure your GPU/CPU and disk space match the model’s size; downloads can be large.
Keep context length in mind—very long contexts need models that support it (e.g., 32K/128K).
If a pull fails, retry from Settings; pending downloads and status are shown in the Models panel.

Tips & Controls

Choose model size to balance speed vs depth; switch in settings without re-ingesting data.

Citations link back to exact chunks; click to open the excerpt.

Long documents are chunked; keep questions specific to tighten grounding.

If cloud history sync is enabled, per-query encryption keys keep conversations private to you.

Models Included by Default

The app uses local Ollama at http://localhost:11434.

New teams default to the Balanced preset: LLM qwen3:8b with embedding mxbai-embed-large.

Other presets you can pick in Settings → Models:

Lightweight: llama3.2:3b + nomic-embed-text

Balanced (default): qwen3:8b + mxbai-embed-large

Quality: mistral:latest + BAAI/bge-m3

Reasoning: gemma3:12b + embeddinggemma:latest

Multilingual: qwen3:14b + BAAI/bge-m3

Additional LLM options available in the selector: phi4:mini, llama3.2:3b, llama3.1:8b, qwen3:8b, qwen3:14b, mistral:latest, gemma3:4b, gemma3:12b.

Embedding options: all-minilm, nomic-embed-text, mxbai-embed-large, BAAI/bge-m3, embeddinggemma:latest, plus legacy all-MiniLM-L6-v2 and all-mpnet-base-v2.

Changing the embedding model requires a full re-index of documents; LLM changes do not.

Bringing Your Own Model (Ollama-compatible)

Train or fine-tune: Use a base model (e.g., Llama 3 or Mistral). If you fine-tune with LoRA using tools like *unsloth*, merge to a GGUF or other Ollama-compatible artifact.

Create an Ollama model: Write a Modelfile that points to your base or merged weights, set context length and quantization, then run ollama create my-custom -f Modelfile.

Verify locally: ollama run my-custom "Hello" to ensure it responds and fits your hardware.

Add to Agency: In Settings → Models → Add Custom Model (Ollama), enter the tag (e.g., my-custom or an Ollama library tag) and click Pull Model. It will appear under LLM choices when installed.

Set for the team: Select the custom model as the LLM for the team. Embedding models must come from the supported list to keep dimensions consistent; switching embeddings triggers re-indexing.

Notes:

Ensure your GPU/CPU and disk space match the model’s size; downloads can be large.

Keep context length in mind—very long contexts need models that support it (e.g., 32K/128K).

If a pull fails, retry from Settings; pending downloads and status are shown in the Models panel.