GLM-5.3-Flash is a 320B mixture-of-experts model. It has no business running next to a router in a spare room. It does anyway — split across two NVIDIA DGX Sparks wired together with a 200-gigabit cable, quantized to four bits, and taught to draft its own tokens. Every number on this page was measured on that machine on Sep 5, 2026, the day before it was stood down.
A single DGX Spark has 128 GB of unified memory. The quantized model needs 164 GB. So the model gets cut in half — every attention head and every expert layer is split across both machines, and the two halves talk to each other on every single token.
164 GB of weights will not fit in 128 GB. Tensor parallelism splits each layer's matrices column-wise, so each box stores and computes exactly half.
Both halves must be recombined at every layer. Over ordinary ethernet that sync dominates. The 200 Gb ConnectX link makes it small enough to ignore.
A 756,736-token KV cache and a 262,144-token context window — long enough to hold an entire codebase in one conversation.
Quantization is where most home-lab attempts quietly lose the model's intelligence. The honest way to choose is KL divergence against the full-precision model: how far the compressed model's probability distribution drifts from the original. Lower is better. Click a rung.
Loading 164 GB of weights off disk and warming CUDA graphs is not instant. Plan for it, and never let a watchdog restart the engine casually.
A voice assistant wants an answer in under a second. An autonomous coding agent wants deep reasoning and does not care if it waits a minute. Running two models would need memory nobody has. Instead one engine serves two ports with different priorities and different reasoning budgets — and the scheduler always lets the human jump the queue.
--scheduling-policy priority.
GLM-5.3 defaults reasoning_effort to
max. Not medium. Max. A one-line question took 33 seconds and often returned an
empty message, because the model spent its whole token budget thinking and had nothing left to say. Setting
it explicitly per lane made the assistant 25× faster with no loss of quality
on short questions.
# every request, on both lanes — never rely on the default { "model": "brandonmusic/GLM-5.3-Flash-tr3-4bpw", "messages": [...], "chat_template_kwargs": { "reasoning_effort": "low", // "high" on the coding lane "thinking": false // true on the coding lane } }
Two tricks do almost all the work. The model drafts its own next tokens and checks them in a batch, and it refuses to re-read text it has already seen.
A small draft model proposes seven tokens ahead. The big model verifies all seven in one pass and keeps the ones it agrees with. Generating tokens one at a time is bandwidth-bound; verifying seven at once is not. Below is the real per-position acceptance rate from a live request — position 1 is almost always right, and acceptance decays with distance.
An agent sends the same 18,000-token codebase on every turn. Without caching, the model re-reads all of it every time. With caching, the second turn skips straight to the new part. This is the single largest win available to anyone running an agent loop.
GLM-5.3's hybrid attention caches in 3584-token pages. If
max-num-batched-tokens is not a multiple of that page size, the cache silently never hits and
you will never see an error — just a mysteriously slow agent.
Single-stream numbers on an otherwise idle cluster, agent lane, reasoning effort low. Generation rate varies with draft acceptance — a request the drafter predicts well runs considerably faster than one it doesn't.
One Spark sat at 93.7 °C while barely generating anything. The GPU was not the problem. vLLM's worker process was spinning in a one-second busy-wait loop, burning 20 CPU cores doing nothing while it waited for work.
The worker polls for new work. One second of busy-waiting per poll pins every core at maximum clock.
The performance governor holds all 20 cores at peak frequency forever. Schedutil lets them idle down between tokens.
The last few hundred megahertz cost disproportionate heat and buy no tokens per second.
The write-ups make this look linear. It was not.
The reference NVFP4 recipe simply would not fit. Four separate configurations died during weight load before the cause was clear.
An in-place upgrade allocated a one-million-token KV cache, then never brought the health endpoint up. Rolled back.
33 seconds for a one-line question, sometimes answering with nothing at all. The reasoning-effort default was eating the entire token budget.
Prefix caching was enabled and reported no errors, and still every request paid full price. The batch size was not page-aligned.
Priority scheduling separated the human from the robots. The spinwait fix took 13 °C off the hot Spark for free. This is the setup running now.
reasoning_effort explicitly on every single request.Configuration files, benchmark logs and the longer write-ups. Nothing here is theoretical — it all came off this machine.
The cluster was stood down on Sep 5 so one Spark could train. Every file needed to restore it in ~8 minutes.
Bring-up saga, the quant ladder, two-lane design and thermal engineering, at length.
Splitting one EXL3 engine into a priority agent lane and a background coding lane.
The million-token-context experiment that taught us how two Sparks behave under load.
The full performance picture across models, quantizations and speculative-decoding settings.
Benchmarks, soak tests and the corrections we had to publish when we got something wrong.
Scripts, environment files, raw benchmark output and every page on this site.
Before GLM-5.3 there was a Nemotron orchestrator and a Qwen coder sharing a single GB10. Kept for the numbers.
Six lines that took a week to arrive at.
# GLM-5.3-Flash-EXL3-2x-DGX-Sparks/.env MODEL=brandonmusic/GLM-5.3-Flash-tr3-4bpw # EXL3 trellis, KLD 0.0246 MAX_MODEL_LEN=262144 MAX_NUM_BATCHED_TOKENS=3584 # page-aligned, or no cache hits GLM53_SPINWAIT_MS=16 # was 1000. 13 degrees C. EXTRA_ARGS="--scheduling-policy priority" # quotes are load-bearing HEAD_IP=10.100.128.1 WORKER_IP=10.100.128.2 # the 200G fabric