How to run OxCoder-9B Q5_K_M on 2× RTX 5060 Ti 16 GB

Retrieval nearly carries the slot — 28/30 needle hits through 90 % of the served 262,144-token window, only the literal edge wobbles (1/3). The code deliverable is the opposite: one run thought for 30 minutes and emitted nothing, one produced four fences where the contract says two, one produced zero fences at all — and none of the four extractable units compiles (median quality 1/5). It reads long documents well and cannot write Elixir you can merge. Route no implementation work through it.

Evaluated by Qwen3.8-27B/low (UD-Q4_K_XL 3.1) · Written by Qwen3.8-27B/low (UD-Q4_K_XL 3.1)

Results

LoadWallPPTGPredFinishQuality
6.7 s 159.1 s 3389 106.3 14984 stop 1/5

Medians of three runs (wall 159.1 / 1800.1 / 123.2 s); per-run quality 1 / 1 / 1. PP is run 1’s cold prefill — run 3 prefix-cached the identical prompt (1,050 prompt tokens, 60.2 tok/s) and is excluded. Run 2 spiraled ~151 k reasoning tokens into the 1,800 s client cap with zero content; slot postmortem 152,208 tokens occupied. Thinking tax 58 % of emitted characters on both completing runs; dup-8 56.7 / 58.6 %, inflated by redelivered fences.

Needle retrieval

Window
262144
Depth
26,214 → 262,080 (every 10 % of the slot)
Result
hit
pp/tg
28/30 — 3/3 through 90 %, coin flip at the edge (1/3); pp 4,229→2,364 · tg 99.2→53.0 across the ladder
Anchor
tesseract-1c7d3f-vault

Quality of result

Adversarial review before code 2/5 Thirteen concrete gap → resolution lines, mostly right arities and semantics — then contradicted wholesale by the code; one run emitted no review because it emitted nothing.
Public API present (put/get/delete/stats) 1/5 run 1: twelve @specs for put/3,4, get/2, delete/2, stats/1, zero defs — only start_link exists; run 3: get/1, delete/1, stats/0, no put/3 — wrong arity everywhere.
TTL + periodic eviction + named instances + hit/miss stats 1/5 Eviction is unreachable code in pair A and uncompilable code in pair B; {:ok, state, timeout: 5_000} with no handle_info(:timeout) clause crashes every instance after 5 s idle; run 3 mixes :timer.now() triples with integer TTLs (3,600 s vs 60,000 ms) and stats() hard-codes __MODULE__, so named instances are unaddressable by construction.
Idiomatic OTP / would it compile? 1/5 0/4 units compile on Elixir 1.20.4 / OTP 29.
Tests shipped, not promised 1/5 0/4 pass; two are SyntaxErrors, one fails compile on unbound vars, the survivor calls the message shapes wrong.
Finish reason + thinking tax 1/5 Two stops in ~2 min and one 1,800 s silent spiral; the revision loop moves away from correctness.

Scale: 5 strong · 4 capable · 3 mixed · 2 weak · 1 broken.

Prompt

This run used v5 (The benchmarking prompt) — It answers a single Elixir-specific question, so YMMV.

Elixir code smells 8

  • kv_1a.ex:6-15 — run 1 pair A — the review promises def get(key, _opts); the module never defines it. Twelve orphan specs for an API with no implementations.
  • kv_1a.ex:18-21 — run 1 pair A — named instances done right ({__MODULE__, name} tuples) and then pointless: the accessors that would use them do not exist.
  • kv_1a.ex:23-31 — run 1 pair A — init/1 returns {:ok, state, timeout: 5_000} with no :timeout handler: a self-inflicted crash timer.
  • kv_1b.ex:19 — run 1 pair B — the self-revision deletes all twelve typespecs and introduces an undefined variable loop inside its own Task.
  • kv_3a.ex:27,34 — run 3 pair A — shadowed def start_link([]) clause after the guarded start_link(opts); duplicate put/4 heads.
  • kv_3a.ex:61,74 — run 3 pair A — send(self(), :evict, …) against a handle_info({:evict, _}, …) pattern: the timer fires a shape nothing matches; a no-op {:tick, _} handler for a message nobody sends (:92).
  • kv_3a.ex:97,110 — run 3 pair A — :timer.now() + current_ttl: integer arithmetic on a now-triple; @default_ttl 3600 in seconds vs @default_interval 60_000 in ms (:9-13).
  • ekv_run3.md:— — run 3 — the whole deliverable arrives as flat prose with zero fences and a </think> token stranded mid-content: the answer channel leaked its own scratchpad, Analyze the Request: and all.

Defects 7

  • kv_1a.ex — spec for undefined function put/3 — the fixed API has no implementations (compile FAIL — run 1 pair A)
  • kv_1b.ex — undefined variable loop in the Task recursion (compile FAIL — run 1 pair B)
  • kv_3a.ex — spec for undefined function start_link/0; [] clause shadowed at :27 (compile FAIL — run 3 pair A)
  • kv_3b.ex — identical to pair A — the revision changed two comment lines (compile FAIL — run 3 pair B)
  • test_1a.exs — unbound pid in every test body after the first — each block reuses the first block’s binding (compile FAIL — run 1 tests)
  • test_1b.exs — parses and runs, but calls raw GenServer.call(pid, {:put, "key", "v", nil}) — a 4-tuple against the module’s 5-tuple {:put, key, value, ttl, opts} head (never passes — run 1 tests B)
  • test_3a.exs · test_3b.exs — SyntaxError — unexpected expression after keyword list (a [:expires_at… | _] union written inside a keyword list); 3b:20 also binds pid to the whole {:ok, pid} tuple (tests FAIL — run 3)

Host

QuantQ5_K_M
Hardware2x RTX 5060 Ti 16 GB (PCIe 5.0 x8/x8), Ryzen 9700X, 16 GB DDR5-6000
Splittensor-split across the pair, llama-server CUDA (the recipe's `--split-mode tensor`)
Runtimellama-server (build 12268, commit 90e3841d9), CUDA mainline
HarnessGrok Build 1.0.77 (50ca579fa996)
Tested2026-11-12
Weightshuggingface.co/prithivMLmods/OxCoder-9B-GGUF

Reproduce

Two 16 GB cards, --split-mode tensor. The bf16 vision projector loads into host RAM (--no-mmproj-offload) and was unused — every prompt was text. Speculative decoding off by recipe; no -md drafter, no draft acceptance to report. The recipe ships the sampler twice; llama.cpp keeps the last values, so --temp 1.0 --top_p 0.95 --top_k 20 --min-p 0.0 --presence_penalty 0.0 is what the runs decoded at — deliberately: three fresh samples at the served sampler is the test. The client sent no sampling parameters beyond the near-slot max_tokens caps (261,000 for the benchmark runs, 512 for needle answers). --host/--port omitted.
llama-server \
  -m OxCoder-9B.Q5_K_M.gguf \
  --ctx-size 262144 \
  --parallel 1 \
  -b 2048 \
  -ub 512 \
  -t 6 \
  -tb 6 \
  -ngl 999 \
  -ctk q8_0 \
  -ctv q8_0 \
  --kv-offload \
  --cache-ram 0 \
  --no-cache-idle-slots \
  --load-mode mmap \
  --no-context-shift \
  -fa on \
  --jinja \
  --reasoning on \
  --reasoning-preserve \
  --cont-batching \
  --split-mode tensor \
  --temp 1.0 \
  --top_p 0.95 \
  --top_k 20 \
  --min-p 0.0 \
  --presence_penalty 0.0 \
  --image-min-tokens 1024 \
  --mmproj OxCoder-9B.mmproj-bf16.gguf \
  --no-mmproj-offload \
  --chat-template-file froggeric-qwen-chat-template.jinja

Three v5 runs on the 32 GB tensor-split pair, each capped at 261,000 tokens, i.e. the served 262,144 slot minus a margin. The cap tracks the window on purpose: a short mid-answer cap can kill a run the model would have finished, and a run is only comparable to the slot it was given.

The pipeline never delivered. Run 2 is the spiral: ~151 k reasoning tokens against a 1,800 s wall, zero content — the slot postmortem showed 152,208 tokens occupied before the abort freed it. Run 1 emitted module and test, then re-emitted both inside the same reply — four fences where the contract says two — and the revision is worse: all twelve @spec lines deleted, eviction “fixed” as a Task whose recursive lambda references an undefined variable and whose self() is the task, not the server. Run 3 delivered its review and both modules as flat prose with zero fences — two code pairs whose only difference is two comment lines — with a `

` token stranded mid-content, the answer channel leaking its own scratchpad.

The needle is where it earns its keep. One anchor, buried at every tenth of the served slot, three repeats per depth, each request opening with a fresh nonce so every step measures cold prefill. 28 of 30 hit, 3/3 at every depth through 90 % of the window; the 100 % edge is a coin flip — the hit found the marker, the two misses burned the 512-token cap re-analyzing the request (“The user wants to find a specific marker hidden in the text…”) and truncated before transcription. Prefill decays 4,229 → 2,364 tok/s (−44 %) and decode 99.2 → 53.0 tok/s (−47 %) from the 10 % rung to the full slot: smooth, no cliff, no cache hole. The short-probe figure of 165 tok/s at n=105 tokens ages badly: at 26 k real decode is 99 tok/s, at the full slot 53. The ladder is the honest one.

Methodology, disclosed: the served recipe runs --reasoning on with a 512-token answer cap, and 26 of the 28 hits were recovered from the reasoning stream before truncation (28 of the 30 rows finished length). Grounding is N/A: completion-only harness, no tools wired.

The catch

Delivered-pipeline pass rate: 0/3 replies, 0/4 units, 0/4 tests.

  • run 1 pair A (kv_1a.ex) — twelve @specs for put/3,4, get/2, delete/2, stats/1 and zero defs behind them; init/1 returns a 5 s timeout with no handler, so every instance crashes after 5 s idle; the promised periodic eviction is unreachable code.
  • run 1 pair B (kv_1b.ex) — the self-revision that deletes the typespecs: eviction as a Task whose loop is undefined inside its own definition (:19), and whose self() points at the task, not the server. Unreachable code became uncompilable code.
  • run 3 pairs A/B (kv_3a.ex, kv_3b.ex) — :timer.now() + current_ttl integer arithmetic on a now-triple (:97,110), an eviction timer that fires a shape nothing matches (:61,74), stats() hard-coded to __MODULE__; pair B is pair A with two comment lines changed.
  • Tests — test_1a.exs reuses the first block’s pid binding in every later body; test_1b.exs calls a 4-tuple against a 5-tuple head; test_3a.exs and test_3b.exs are SyntaxErrors.

The one honest positive: run 1’s thirteen-line review enumerates every trap with correct arities and semantics — a design doc its own code then ignores. The eval→implement handoff is the broken organ, not the eval.

Representative output

Run 1, pair B — the self-revised eviction, trimmed to the Task:

def init(opts) do
  default_ttl = Keyword.get(opts, :default_ttl, @default_ttl)
  # Start eviction task
  Task.start_link(fn ->
    loop = fn ->
      send(self(), :evict)
      :timer.sleep(60_000)
      loop.()
    end
    loop.()
  end)

loop is referenced inside its own definition before it is bound — a compile error at kv_1b.ex:19. Even if it compiled, self() inside the task is the task’s pid, so :evict would never reach the GenServer.

Route

Route no implementation work through it. The fit is as a long-document reader: the full 262,144-token window fits the pair, 9 of 10 depths retrieve cleanly, and decode still runs 53 tok/s at the slot edge. It reads long documents well and cannot write Elixir you can merge; keep a compile gate between it and any repo.

Have something worth building?

A paragraph about what you are working on is enough to start.

Say hello