How to run Muse-Glimmer NVFP4 on 2× RTX 5060 Ti 16 GB

Warning: this is a temporary briefing, written by an LLM and not yet evaluated or rewritten by a human to get rid of the signature AI-slop writing style. It’s published as a draft despite its shortcomings, so that you can find out how to run it and what to expect in terms of pp and tg on this hardware, as well as what to expect in terms of coding performance on this benchmark, grosso modo.

Three v5 runs on Muse-Glimmer NVFP4 — a 27.8B dense weight with the vision tower baked into the GGUF. It runs at its full 131,072-token native window, layer-split across two RTX 5060 Ti 16 GB cards, with DFlash2 speculative decoding on top. The earlier Oct-28 run scored this model q=0, a spiral of empty content, on the old 65,536 window — the window itself was the cap there. On the real 131k slot the answer is different: three clean stops at ~7-9 k tokens each, ~110 s, and the q=0 spiral was a smaller-window artifact.

The needle ladder is where this model earns its keep. Same anchor planted at every tenth of the window, three repeats per depth, fresh nonce per request so no step inherited a warm cache:

30/30, no cliff. Prefill decays a smooth ~25 % from 2,395 to 1,789; decode holds 63-95 throughout. The full-window steps run to the 512-token answer cap only because the model rambles after it finds the marker — the marker is present every time. At the depth where flint, sliding-window by design, lost retrieval past 60 %, Muse keeps fetching from the last third.

Then the catch: every run shipped its code as prose. The v5 contract asks for two ```elixir fences; runs 1 and 2 emitted bare code (run 1 even had a stray single-backtick where an opening fence should be), and run 3 used untyped ``` without a language tag. An automated extractor reads nothing — it takes a person peeling the fenced blocks off by hand to get a pipeline that compiles. For an agent that means the deliverable only reaches the pipeline when a harness forces the fences.

The record: three clean stops, but the gate is 0/3 as-filed — runs 2 and 3 hit the same classic Elixir trap, def start_link/2 (opts \ []) colliding with start_link/1 at :16/:11. Run 1’s code, once the fences were hand-peeled, compiles and passes its six tests in 4 s including real TTL eviction timing — the only passing pipeline in the measured series. The review pass was the strongest part of the batch: fifteen explicit gap → resolution lines, the most disciplined of the run.

Representative output

Run 1 shipped the largest, best-formed delivery of the batch (11.3 k chars). Trimmed to the public API and the timer-driven eviction handler — the only run whose pipeline compiled and passed its six tests:

defmodule TTLKV.Store do
  def start_link(opts \ []) do
    name = Keyword.get(opts, :name)
    GenServer.start_link(__MODULE__, opts, name: name)
  end

  def start_link(name, opts) when is_atom(name) do
    GenServer.start_link(__MODULE__, opts, name: name)
  end

  def put(name, key, value) do
    GenServer.call(name, {:put, key, value})
  end

  def put(name, key, value, ttl) do
    GenServer.call(name, {:put, key, value, ttl})
  end

  def get(name, key) do
    GenServer.call(name, {:get, key})
  end

  def delete(name, key) do
    GenServer.call(name, {:delete, key})
  end

  def stats(name) do
    GenServer.call(name, :stats)
  end

  def handle_info(:evict, state) do
    now = System.monotonic_time(:millisecond)
    new_entries =
      state.entries
      |> Enum.filter(fn {_k, %{expires_at: exp}} -> now <= exp end)
      |> Map.new()
    Process.send_after(self(), :evict, state.eviction_interval_ms)
    {:noreply, %{state | entries: new_entries}}
  end
end

Route: long-context retrieval and review with a harness that forces fenced output. Not drop-in agentic coding until the format is fixed, or the template is re-packed. The window is the whole 131k — that part is settled; the fence habit is the thing to watch.