Gateway (drop-in)
Do not replace your model provider. Put NeuraFrame™ in front of it. Point your app at NeuraFrame™ instead of your model, change nothing else, and get the savings.
How it works
NeuraFrame™ runs a reuse-aware reverse proxy in front of your model. Your app sends its normal request to NeuraFrame™. On a repeat, NeuraFrame™ answers from memory; on a miss, it forwards the request to your real model and returns that response unchanged, then remembers it.
client app -> NeuraFrame gateway -> your model (upstream)
hit : served from memory, no upstream call
miss : forwarded, model response returned, remembered
The only change on your side is one base URL.
Works with any model API
The gateway has two layers of compatibility:
- Universal exact layer. It keys on the normalized request, so an identical repeat replays the stored response byte for byte. This needs no knowledge of your API schema, so it works with any HTTP JSON model endpoint (OpenAI style, other local servers, custom APIs). It delivers the exact-recurrence savings.
- Meaning layer. For known profiles (OpenAI and OpenAI-compatible chat and completions, Anthropic Messages, Grok and xAI, and the native llama.cpp endpoints) it also reads the prompt and answer so it can apply semantic reuse and routing, the deeper NeuraFrame™ benefits.
The meaning layer needs an embedding endpoint on your model server (embedding_url in the config, default http://127.0.0.1:8080/embedding). The gateway checks it at startup: if it is missing, unreachable, or misconfigured, a warning is printed in the service log and the live state is reported in the semantic block at GET /_nf/savings, so paraphrase reuse can never be silently off. Exact reuse works regardless.
Freshness: reuse in time
Some answers are stable and some go out of date. "What is a widget" answers the same next week; "what is the weather in Arizona" does not. A cache keyed only on the input would replay an hour-old weather answer as if it were current. NeuraFrame™ gives every stored answer a freshness window: a stable answer is reused for as long as it stays true, and a time-sensitive one is re-fetched once its window passes. It is the same safety rule as vision mode (never reuse over a material change), applied to time instead of pixels.
It is on by default and safe by default, so it works with no configuration: stable answers never expire (reuse is exactly as before), and answers that look time-sensitive get a short window. The classifier is conservative, so turning it on can only ever cost model calls, never serve a stale answer as fresh. NeuraFrame™ decides the window from a few signals, in order: an explicit per-path rule you set, then the question's temporal intent (words like weather, price, now, latest), then the stable default. Reading the answer's shape as well (clock times, currency figures, temperatures) is available behind the detect_answer_signals knob, off by default because it over-triggers on code and arithmetic. And a mis-flagged class corrects itself: every expiry re-fetch doubles as a free verification, and a class verified unchanged twice earns its never-expires window back, so stable content is never taxed for long.
Each stored answer also carries provenance and trust: a human correction outranks a model answer, and a low-trust answer is re-checked sooner. If NeuraFrame™ ever re-verifies a reuse and the model comes back with a different answer, that contradiction is surfaced and the answer's window is tightened, never silently left stale. You can inspect all of it live: the gateway answers GET /_nf/freshness with the current windows, drift checks, and any contradictions it has surfaced.
| Control | CLI | Default |
|---|---|---|
| On / off | neuraframe freshness on / off | on |
| Off restores timeless reuse (a stored answer never expires). | ||
| Stable window | neuraframe freshness default_ttl <secs> | 0 |
| Seconds a stable answer stays reusable; 0 means never expires. | ||
| Volatile window | neuraframe freshness volatile_ttl <secs> | 60 |
| Seconds a time-sensitive answer stays reusable before a re-fetch. | ||
| Auto-detect | neuraframe freshness detect on / off | on |
| Detect volatility from the question's temporal intent. | ||
| Answer-side signals | neuraframe freshness detect_answer_signals on / off | off |
| Also read the answer's shape (clock times, currency, temperatures). Off by default: it over-triggers on code and arithmetic. | ||
| Restore stability | neuraframe freshness stable_after_unchanged <n> | 2 |
| A heuristically-flagged class earns back a never-expires window after this many verified-unchanged re-fetches. Operator settings (fresh paths, per-path windows) are never overridden. | ||
| Always fresh / stable paths | neuraframe freshness fresh_paths <p1,p2> / stable_paths | none |
| Force a path to be volatile (short window) or stable (never expires). | ||
| Per-path window | neuraframe freshness path_ttl <path> <secs> | none |
An explicit window for one path, e.g. /quote 30. | ||
| Extra volatile words | neuraframe freshness extra_volatile_terms <w1,w2> | none |
| Add your own terms to the built-in volatility vocabulary. | ||
| Drift sampling | neuraframe freshness drift_sample_rate <0-1> | 0 |
| Chance of re-verifying a reuse against the model to catch a class that is volatile after all (costs a call when it fires). | ||
| Trust floor | neuraframe freshness min_trust_reuse <0-1> | 0 |
| Do not reuse an answer whose trust is below this. | ||
| Self-learning | neuraframe freshness adaptive on / off | on |
| Learn how fast each class of query actually changes and tighten its window to match. Only ever shortens a window, never lengthens one, so it can only make answers fresher. | ||
| Forget learned | neuraframe freshness forget | - |
| Clear everything the layer has self-learned (reversibility control). | ||
Run neuraframe freshness with no arguments to list every knob, its default, and what it does. The settings live in the config's freshness block; the vision layer has its own change-awareness and is unaffected by these.
sudo systemctl restart neuraframe-studio after a change. The live counters at /_nf describe the running process, so they reflect a change only after that restart.Pins: teach the gateway a fixed answer
Sometimes you want the gateway to always give a particular answer for a particular kind of request: a corrected reply for something the model gets wrong, a canonical policy response, a fixed answer for a health check. A pin does exactly that, and only that. It returns exactly what you type, for requests that match a rule you write. It is deliberately not learning and does not generalise (this is a reuse layer, not a mind): every pin is listed and can be revoked, and pins are served before the reuse layers, so a pin always wins.
sudo neuraframe pin add "refund policy" '{"answer":"Refunds within 30 days."}' # fires when a request contains "refund policy"
sudo neuraframe pin add "/v1/health" "ok" --exact --path /v1/health # exact match, scoped to one path
neuraframe pin list # see every pin
sudo neuraframe pin revoke <id> # remove one
What reuse is saving
The gateway keeps a running readout of what reuse is buying you: how many model calls it avoided, the reuse rate, an estimate of the tokens saved, and a breakdown by where the saving came from (exact repeats, paraphrases, vision, and pins). The call counts are exact; the token figure is an estimate at roughly four bytes per token.
neuraframe savings # calls avoided, reuse rate, tokens saved, by source curl http://127.0.0.1:8081/_nf/savings # the same, live from the gateway curl http://127.0.0.1:8081/_nf/freshness # windows, drift checks, contradictions, learned classes
Point your app at it
Two steps: set the gateway's upstream to your provider, then change your app's base URL to the gateway (default http://127.0.0.1:8081). Set gateway.upstream_url to your provider and gateway.profile to match it. The auto profile detects OpenAI-style and Anthropic-style requests by path, so one gateway can front both.
sudo nano /etc/neuraframe-studio/config.json # edit the "gateway" block
sudo systemctl restart neuraframe-studio
| Provider | gateway.upstream_url | gateway.profile | point your app at |
|---|---|---|---|
| OpenAI | https://api.openai.com | openai or auto | http://127.0.0.1:8081/v1 |
| Anthropic | https://api.anthropic.com | anthropic or auto | http://127.0.0.1:8081 |
| Grok / xAI | https://api.x.ai | grok or auto | http://127.0.0.1:8081/v1 |
| Compatible / local | your endpoint base URL | auto | http://127.0.0.1:8081 |
Your provider API key travels on the request from your app exactly as it does today; the gateway passes it straight through. OpenAI and xAI use Authorization: Bearer, Anthropic uses x-api-key and anthropic-version, and any custom header a compatible endpoint needs is forwarded too.
# example: Anthropic through the gateway, the same request you send today
curl http://127.0.0.1:8081/v1/messages \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{"model":"claude-sonnet-5","max_tokens":64,"messages":[{"role":"user","content":"say hi"}]}'
Streaming and misses
A miss is exactly like talking to your model directly: the request is forwarded unchanged and the response is returned unchanged (byte identical), including streamed responses. Streaming repeats use the exact layer; semantic reuse applies to non streaming requests.
If your provider is unreachable
NeuraFrame™ only ever returns a verified answer from memory or your provider's real response. If a request is not in memory and the gateway cannot reach your provider (down, refused, or timed out), it returns a clear error saying so rather than inventing an answer. Anything it has already learned still serves from memory, so a growing NeuraFrame™ keeps working even when calls to the provider become rare.
Licensing and pass-through
When unlicensed or expired, the gateway forwards everything and serves nothing from memory, so it is fully transparent. Your model keeps working; you simply lose the savings until you renew.
Teaching is optional
A plain model API has no concept of a correction, so teaching happens through the neuraframe CLI or the native API, not the proxy path. You get reuse and energy savings with zero code change; teaching is an opt-in extra when you want future answers to change.