One DGX Spark · four coding agents

Inference you can sleep on.

A 125-billion-parameter coding model, served to four agents at once, all night. Built from scratch in nine days.

See the milestones

23×

faster on a 131k-token prompt than the first build

2.5 s

median start of an agent turn at 100k+ context, three others busy

errors in 7,821 replayed agent requests

5.1M

grammar masks checked against xgrammar, 0 wrong

Why hypnos exists

Fast engines are built for every model. My pipeline needs one.

Each fast engine-and-model pairing is tuned on its own. Every pairing is one more configuration to keep working.

TensorFold with Qwen3.8-Flash-Next on one DGX Spark: the one feature my pipeline needed.

Engine developers chase the newest models, so a compatibility layer for one pipeline waits. I don't need a smarter model. I need Qwen3.8-Flash-Next, accurate and stable, every night. So I built hypnos, named for the god of sleep, with one goal: inference I can sleep on.

The spark

Single focus, from 0xBakeer's TandemLLM: one inference engine, Qwen3.8-27B, one DGX Spark. That's it. One pairing is one thing to control.

≈

Speed around the industry average is enough.

∞

Running sleepless while I sleep is the point.

Built for my pipeline. Different people need different approaches.

With respect to the people building inference engines, especially Mia, Ash and 0xBakeer, who welcomed me into local AI. I use their engines every day. My production pipeline runs on hypnos.

What the pipeline needs

Agents are long, many and unattended.

150k

tokens in a typical agent turn. Only the last thousand are new.

4 → 1

agents share one GPU. A long prompt must not freeze the others.

03:00

no one is watching. It must parse, validate and recover on its own.

How hypnos works

Built for one model on one machine, end to end.

95–97 %

of agent prompt tokens from cache

+42 %

one stream with drafts, replies identical

0 / 708,899

invalid strict-JSON documents

0.28 s

to encode a 1280 × 800 screenshot

1.57×

streams' worst gap beside a 156k prompt (limit 2×)

≈ 85 s

from a hung GPU to serving again

Milestones

Ten days, six releases.

Then and now

Measured on the box, every day.

Reliability

Correct, and still running in the morning.

Ten soaks of replayed agent traffic, requests per soak0 errors

A hung GPU, recovered with no one there

99.2 %

token agreement with the vLLM reference; misses only at near-ties

1,977 / 1,977

tool calls that parsed, over 14 hours of agents

3,601 / 3,601

disagreements with xgrammar where hypnos was right

Where it stands

Against vLLM on the same box.

Prompt processing, 131k tokens, cold

vLLM default2,218 tok/s

vLLM, same precision1,584 tok/s

hypnos1,157 tok/s

vLLM referencehypnos

Reply speed, one stream

vLLM, 4 drafts48.3 tok/s

hypnos, 2 drafts31.9 tok/s

hypnos, no drafts23.6 tok/s

vLLM referencehypnos

now

72-hour soak

Running since Oct 27: the last gate before v1.0.

next

A faster decode step

Fuse ≈ 1,800 small kernels a token into fewer.

then

v1.0

Targets: 28 tok/s a stream, 40 with drafts.

hypnos

Run your agents overnight. Sleep.

hypnos serve --listen 127.0.0.1:8091

An OpenAI-compatible chat-completions API. Point OpenCode, or any agent that speaks it, at the box.