llama-nest

local-first memory and context infrastructure for ollama

goollamajsonllocal-firstearly experimental

Context should outlive models.

What llama-nest Is

llama-nest is an experimental runtime wrapper around Ollama that adds inspectable local memory, conversational context persistence, model-to-model context transfer, token usage tracking, local search, catch-up briefs, and runtime monitoring foundations.

It sits between your applications and Ollama, capturing and structuring conversational context locally while remaining fully model-agnostic.

What Problem Does This Solve

Ollama makes running local models easy. But local AI workflows still have major gaps:

- conversations become fragmented
- switching models loses useful context
- there is no inspectable memory layer
- usage and performance are difficult to observe
- context continuity is tied to a single model session
- local AI tooling lacks portable conversational infrastructure

How It Works

Instead of applications talking directly to Ollama, they talk to llama-nest, which proxies requests:

            ┌────────────────────────────┐
            │         Your App           │
            │ CLI / scripts / agents/UI │
            └─────────────┬──────────────┘
                          │
                          │ requests
                          ▼
            ┌────────────────────────────┐
            │         llama-nest         │
            │                            │
            │  local proxy + memory      │
            │                            │
            │  localhost:11435           │
            └─────────────┬──────────────┘
                          │
                          │ proxied requests
                          ▼
            ┌────────────────────────────┐
            │           Ollama           │
            │      localhost:11434      │
            └────────────────────────────┘

Because llama-nest sits in the middle, it can capture conversations, persist local memory, track token usage, monitor model performance, search prior context, transfer context between models, and export/import conversational state.

Features

Interactive local chat

Instead of ollama run llama3.2, use:

llama-nest run llama3.2

Routes chat through the proxy so context, usage, and transfers are captured automatically.

Context transfer between models

Move recent conversational context between local models:

llama-nest transfer qwen2.5:0.5b

Checks if the target model exists locally, pulls it automatically if missing, builds a local context pack, transfers recent conversational context, and asks the target model to acknowledge the session.

Catch-up briefs

Generate a brief of recent conversational context:

llama-nest catch-up

Useful for re-entering long sessions, restoring conversational continuity, or summarizing recent local memory.

Token usage tracking

Tracks prompt tokens, completion tokens, total tokens, and usage by model. Future monitoring support will include latency, tokens/sec, request throughput, and model performance comparisons.

Export local context

Export captured conversational state into a portable .nest bundle:

llama-nest export

The long-term goal is portable conversational infrastructure across machines and models.

Quick Start

Prerequisite: Ollama installed and running locally on port 11434.

# start ollama

ollama serve

# clone and build

git clone https://github.com/llama-nest/llama-nest.git
cd llama-nest
make build

# initialize and start

./bin/llama-nest init
./bin/llama-nest start

API server

http://localhost:8787

Ollama proxy

http://localhost:11435

Commands

llama-nest init             # initialize local config and storage
llama-nest start            # start local proxy + API
llama-nest stop             # stop running llama-nest services

llama-nest run MODEL        # interactive chat through llama-nest
llama-nest transfer MODEL   # transfer recent context to another model

llama-nest status           # show local status
llama-nest usage            # show token usage
llama-nest search QUERY     # search local context
llama-nest catch-up         # generate memory brief

llama-nest export           # export local context bundle
llama-nest wipe --yes       # delete captured local memory

llama-nest doctor           # validate local setup

UI

The UI is a lightweight Vite React application that supports session inspection, message browsing, local search, transfer history, token usage tracking, and catch-up briefs.

cd ui
npm install
npm run dev
# Open http://localhost:5173

Data Storage

Current local storage uses JSONL-backed local persistence inside:

~/.llama-nest/

Files include: sessions.jsonl, messages.jsonl, transfers.jsonl, usage.jsonl

Design Principles

*local-first by default
*no telemetry
*inspectable before autonomous
*model-agnostic conversational context
*raw context before derived memory
*portability over lock-in
*user-controlled memory lifecycle

Roadmap

[ ]latency + throughput monitoring
[ ]Grafana-style monitoring dashboard
[ ]encrypted local event store
[ ]structured memory extraction
[ ]semantic memory graph
[ ]sqlite-vec / vector backend
[ ]model routing engine
[ ]import .nest bundles
[ ]shared memory across devices
[ ]MCP server
[ ]Tauri desktop app
[ ]Homebrew tap
[ ]Docker image

Status

Early experimental. This is being built openly as I learn. Expect breaking changes, incomplete features, and occasional moments of "why did I think that would work?"

If you're interested in local AI memory systems or just want to see how this evolves, follow along on GitHub.

License

Apache-2.0