Make fast, calibrated, explainable yes/no decisions.
Tiny Decision runs small System One models locally and turns messy input into a calibrated yes/no, a confidence score, and a full audit log — no cloud round-trip, no leaked data.
Most classification calls don't need a model to reason step by step — they need one confident yes or no, fast. Tiny Decision wraps small 1B–7B models like Phi-2, Gemma-2-2B, and Qwen-1.5-1.8B behind a single HTTP contract called SOMI, so every model your app talks to looks identical, answers in milliseconds, and comes with a calibration score you can actually trust.
Fully local
No data leaves your machine, unless you deliberately opt into the OpenRouter adapter for cloud fallback.
Sub-100ms decisions
Runs on Apple Silicon and modern CPUs via llama.cpp GGUF — fast enough to sit in the request path.
Calibrated confidence
Platt scaling, isotonic regression, and temperature scaling turn raw logits into probabilities you can act on.
Auto-router
Automatically picks the smallest model that still clears your accuracy and latency budget.
Plugin system
WASM-sandboxed plugins for model adapters, alerting, and pipeline nodes, run through wasmtime.
MCP compatible
Expose any decision template as a tool for Claude Desktop, Cursor, or any MCP-aware host.
Where a fast yes/no earns its keep
Should this ticket be escalated to a human? Route the routine 90% straight to an automated reply, and flag the rest before they go stale.
Is this return request eligible for an automatic refund? Approve clean cases in milliseconds and send edge cases to a reviewer with the reasoning attached.
Is this message spam or phishing? Screen inbound email and content at submission time, locally, before it ever reaches a person.
Does this applicant meet minimum screening criteria? Apply a consistent, auditable bar at the top of the funnel instead of ad hoc judgment calls.
Does this transaction look anomalous? Flag likely fraud in real time with a confidence score that tells you how sure the model actually is.
Architecture
A React desktop shell talks to a Rust core over Tauri's IPC bridge. The core does all the real work — inference, calibration, routing, plugins — and can reach out to a few optional external services.
Desktop App
Tauri 2 / React- React UIModel browser, template editor, run history, calibration dashboard
- Tauri CommandsIPC bridge into the Rust core (commands.rs)
Core Engine
Rust- Axum HTTP ServerSOMI API on :11535
- SQLiteDecisions, runs, calibration params
- Inference Enginellama.cpp via llama-rs
- Calibration PipelinePlatt / isotonic / temperature
- Auto-RouterSmallest model that clears the bar
- Plugin HostWASM sandbox via wasmtime
External / Optional
Only if you opt in- HuggingFace HubGGUF model downloads
- OpenRouter APICloud fallback via SOMI adapter
- MCP ClientClaude Desktop, Cursor, etc.
- Slack / WebhookAlert plugins
Quick start
| Tool | Version | Notes |
|---|---|---|
| Rust | 1.77+ | rustup update |
| Node.js | 20+ | For the Tauri frontend |
| pnpm | 8+ | npm i -g pnpm |
| Tauri CLI | 2.x | cargo install tauri-cli |
Clone the repo
git clone https://github.com/aryanthegamedev3465-collab/tiny-decision.git cd tiny-decision
Install frontend dependencies
cd app pnpm install
Download a model
cargo run -p td-cli -- models download microsoft/phi-2 --quant Q4_K_M
# or use the in-app Model Browser
Run in development mode
cargo tauri dev
Make your first decision
curl -X POST http://localhost:11535/somi/v1/decide \
-H "Content-Type: application/json" \
-d '{
"model_id": "microsoft/phi-2",
"template_id": "refund-eligibility",
"state": { "order_value": 45.99, "customer_tier": "gold" },
"questions": [
{ "id": "eligible", "type": "choice", "text": "Is this eligible for an automatic refund?", "choices": ["yes","no"] }
]
}'
SOMI — the System One Model Interface
SOMI is the HTTP contract every model and adapter satisfies, so your application never has to know which model answered.
{
"model_id": "microsoft/phi-2",
"template_id": "refund-eligibility",
"state": { "order_value": 45.99 },
"questions": [{
"id": "eligible",
"type": "choice",
"choices": ["yes","no"]
}],
"options": { "calibrate": true }
}
{
"run_id": "run_01J9XK...",
"answers": [{
"question_id": "eligible",
"value": "yes",
"confidence": 0.87,
"calibrated": true,
"latency_ms": 42
}],
"somi_version": "v1"
}
| Type | Description | Example |
|---|---|---|
| choice | Pick one from N choices | choices: ["yes","no","maybe"] |
| noul | Null or value — present / absent | label: "escalate" |
| score | Continuous 0–1 score | label: "risk_score" |
CLI usage
USAGE:
td <COMMAND>
COMMANDS:
eval Evaluate a template against an eval set (CI-friendly)
run Run a pipeline or single decision
models Model management (list, download, info)
server Start or stop the local SOMI server
plugin Plugin management (install, list, remove)
CI gate example — exits with code 1 if accuracy drops below 0.92 or the expected calibration error rises above 0.05. td eval --template refund-eligibility --eval-set ./evals/refund-v2.jsonl --fail-under-accuracy 0.92 --fail-over-ece 0.05
Plugins & MCP
Plugins are WASM modules sandboxed by wasmtime, declared through a small manifest. Any decision template can also be published as an MCP tool for Claude Desktop, Cursor, or another MCP-aware host.
{
"name": "my-alert-plugin",
"type": "pipeline_node",
"interface": "process(node_input) -> node_output"
}
{
"mcpServers": {
"tiny-decision": {
"command": "td",
"args": ["server", "start", "--mcp"]
}
}
}