Make fast, calibrated, explainable yes/no decisions.

Tiny Decision runs small System One models locally and turns messy input into a calibrated yes/no, a confidence score, and a full audit log — no cloud round-trip, no leaked data.

eligible → yes
confidence
model: microsoft/phi-2 · calibrated: true · latency: 42ms

Most classification calls don't need a model to reason step by step — they need one confident yes or no, fast. Tiny Decision wraps small 1B–7B models like Phi-2, Gemma-2-2B, and Qwen-1.5-1.8B behind a single HTTP contract called SOMI, so every model your app talks to looks identical, answers in milliseconds, and comes with a calibration score you can actually trust.

Fully local

No data leaves your machine, unless you deliberately opt into the OpenRouter adapter for cloud fallback.

Sub-100ms decisions

Runs on Apple Silicon and modern CPUs via llama.cpp GGUF — fast enough to sit in the request path.

Calibrated confidence

Platt scaling, isotonic regression, and temperature scaling turn raw logits into probabilities you can act on.

Auto-router

Automatically picks the smallest model that still clears your accuracy and latency budget.

Plugin system

WASM-sandboxed plugins for model adapters, alerting, and pipeline nodes, run through wasmtime.

MCP compatible

Expose any decision template as a tool for Claude Desktop, Cursor, or any MCP-aware host.

Where a fast yes/no earns its keep

Should this ticket be escalated to a human? Route the routine 90% straight to an automated reply, and flag the rest before they go stale.

Is this return request eligible for an automatic refund? Approve clean cases in milliseconds and send edge cases to a reviewer with the reasoning attached.

Is this message spam or phishing? Screen inbound email and content at submission time, locally, before it ever reaches a person.

Does this applicant meet minimum screening criteria? Apply a consistent, auditable bar at the top of the funnel instead of ad hoc judgment calls.

Does this transaction look anomalous? Flag likely fraud in real time with a confidence score that tells you how sure the model actually is.

Architecture

A React desktop shell talks to a Rust core over Tauri's IPC bridge. The core does all the real work — inference, calibration, routing, plugins — and can reach out to a few optional external services.

Desktop App

Tauri 2 / React
  • React UIModel browser, template editor, run history, calibration dashboard
  • Tauri CommandsIPC bridge into the Rust core (commands.rs)

Core Engine

Rust
  • Axum HTTP ServerSOMI API on :11535
  • SQLiteDecisions, runs, calibration params
  • Inference Enginellama.cpp via llama-rs
  • Calibration PipelinePlatt / isotonic / temperature
  • Auto-RouterSmallest model that clears the bar
  • Plugin HostWASM sandbox via wasmtime

External / Optional

Only if you opt in
  • HuggingFace HubGGUF model downloads
  • OpenRouter APICloud fallback via SOMI adapter
  • MCP ClientClaude Desktop, Cursor, etc.
  • Slack / WebhookAlert plugins

Quick start

ToolVersionNotes
Rust1.77+rustup update
Node.js20+For the Tauri frontend
pnpm8+npm i -g pnpm
Tauri CLI2.xcargo install tauri-cli
1

Clone the repo

git clone https://github.com/aryanthegamedev3465-collab/tiny-decision.git
cd tiny-decision
2

Install frontend dependencies

cd app
pnpm install
3

Download a model

cargo run -p td-cli -- models download microsoft/phi-2 --quant Q4_K_M
# or use the in-app Model Browser
4

Run in development mode

cargo tauri dev
5

Make your first decision

curl -X POST http://localhost:11535/somi/v1/decide \
  -H "Content-Type: application/json" \
  -d '{
    "model_id": "microsoft/phi-2",
    "template_id": "refund-eligibility",
    "state": { "order_value": 45.99, "customer_tier": "gold" },
    "questions": [
      { "id": "eligible", "type": "choice", "text": "Is this eligible for an automatic refund?", "choices": ["yes","no"] }
    ]
  }'

SOMI — the System One Model Interface

SOMI is the HTTP contract every model and adapter satisfies, so your application never has to know which model answered.

POST /somi/v1/decide
{
  "model_id": "microsoft/phi-2",
  "template_id": "refund-eligibility",
  "state": { "order_value": 45.99 },
  "questions": [{
    "id": "eligible",
    "type": "choice",
    "choices": ["yes","no"]
  }],
  "options": { "calibrate": true }
}
Response
{
  "run_id": "run_01J9XK...",
  "answers": [{
    "question_id": "eligible",
    "value": "yes",
    "confidence": 0.87,
    "calibrated": true,
    "latency_ms": 42
  }],
  "somi_version": "v1"
}
TypeDescriptionExample
choicePick one from N choiceschoices: ["yes","no","maybe"]
noulNull or value — present / absentlabel: "escalate"
scoreContinuous 0–1 scorelabel: "risk_score"
Native — direct logit probabilities, no prompting needed Adapted — wrapped by a SOMI adapter plugin

CLI usage

USAGE:
    td <COMMAND>

COMMANDS:
    eval        Evaluate a template against an eval set (CI-friendly)
    run         Run a pipeline or single decision
    models      Model management (list, download, info)
    server      Start or stop the local SOMI server
    plugin      Plugin management (install, list, remove)
CI gate example — exits with code 1 if accuracy drops below 0.92 or the expected calibration error rises above 0.05. td eval --template refund-eligibility --eval-set ./evals/refund-v2.jsonl --fail-under-accuracy 0.92 --fail-over-ece 0.05

Plugins & MCP

Plugins are WASM modules sandboxed by wasmtime, declared through a small manifest. Any decision template can also be published as an MCP tool for Claude Desktop, Cursor, or another MCP-aware host.

plugin.json
{
  "name": "my-alert-plugin",
  "type": "pipeline_node",
  "interface": "process(node_input) -> node_output"
}
Claude Desktop config
{
  "mcpServers": {
    "tiny-decision": {
      "command": "td",
      "args": ["server", "start", "--mcp"]
    }
  }
}