RunWeights.com — Sovereign Local & Hybrid Inference
Run your weights where you choose: local GPU, hybrid spill to CPU/RAM, or routed across a fleet. This guide covers the routing decision and ships a live calculator that estimates whether a model at a given quantization actually fits your VRAM.
VRAM / Quantization Fit Calculator
Enter model parameters and a quantization level. The calculator estimates weight + KV-cache footprint against your VRAM budget and recommends a quant — sovereign local-first inference routing math, client-side.
Set parameters and press Calculate Fit.
Routing tiers
- Local-first — model + context fit in VRAM at the chosen quant; zero egress, full sovereignty.
- Hybrid — weights resident, KV cache or overflow layers on CPU/RAM; slower but capable on consumer GPUs.
- Fleet-routed — request routed to whichever node has headroom; the calculator tells you the per-node fit.
- Quant ladder — Q8_0 for fidelity, Q4_K_M the quality/size sweet spot, Q2_K last resort before hybrid.
Reading the calculator
Weight footprint scales linearly with bits-per-weight; KV cache scales with layers × context × bytes-per-element. The calculator uses a layer-count heuristic from parameter count — treat results as planning estimates, then verify against your runtime's real memory report before committing a deployment.
Cluster
Sibling properties in the MCP/A2A protocol bridge and guardrail runtimes.
MCP Context ProMCP↔A2A HubA2A→MCP CompilerOverwatcherTypedBotsStrictBotsRunWeights · ForwardRouterReverseHarnessLinkProducers