Local LLM Leaderboard

Can local models running on a Mac Studio or Mac mini do real HPC and cloud consulting work? Each track below is a reproducible benchmark built from real engagement work, sanitized, and scored by objective checks against a frozen Anthropic Claude Opus baseline.

Background and method: Part I · Part II: AWS IAM policies · Part III: Schrödinger postmortem root cause analysis.

Reading the tables. opus-5-5-high (Claude Opus 5.5) is the frontier baseline, highlighted in each table. studio-* and mini-* are local models served by oMLX on a Mac Studio (M5 Ultra, 256 GB) or Mac mini (M4 Pro, 64 GB); the suffix is the thinking level. One row per complete run; a model can have several runs. Scores are 0–1 with a 90% bootstrap confidence interval over items, except the closed interview track, which reports the judged difference from Opus. Click a model name for its card: full model name, hardware, server settings and its rank on every track.

Snapshot 2026-11-04 (evals commit 0d90ede).

Best local model per use case

Best complete run by a current (not retired) local model on each track, the runner-up from a different model, and the Opus reference.

IAM boundary (tools)

DeepSeek-V4-Flash-0731-oQ4e-mtp-r1029

mean 0.892 · pass 0.886

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Contestant: studio-deepseek-r1029-high · thinking high
Run: iam5t-studio-deepseek-r1029-high-a
Runner-up: studio-qwen-r1024-high (mean 0.891 · pass 0.886)
Opus reference: mean 0.991 · pass 1.000

Audit-fix (agentic)

MiMo-V2.6-Flash-MOPD-oQ4e-mtp

pass 1.000 · mean 1.000

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Contestant: studio-mimo-high · thinking high
Run: afx5-studio-mimo-high-a
Runner-up: studio-deepseek-r1029-high (pass 0.990 · mean 0.998)
Opus reference: pass 1.000 · mean 1.000

Postmortem (assist)

DeepSeek-V4-Flash-0731-oQ4e-mtp

pass 1.000 · mean 1.000

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Contestant: studio-deepseek-high · thinking high
Run: pm-v3a-studio-deepseek-high-a
Runner-up: studio-mimo-high (pass 1.000 · mean 1.000)
Opus reference: pass 1.000 · mean 1.000

Recon checklist (assist)

Qwen3.8-Flash-Next-oQ4e-mtp-r1024

pass 0.750 · mean 0.950

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Contestant: studio-qwen-r1024-high · thinking high
Run: rc-v3a-studio-qwen-r1024-high-a
Runner-up: studio-qwen-r1029-high (pass 0.750 · mean 0.950)
Opus reference: pass 1.000 · mean 1.000

Document reading (vision)

NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit

mean 0.999

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Contestant: studio-nemotron35-vl · thinking off
Run: dr1-studio-nemotron35-vl-a
Runner-up: mini-qwen36-27b-vl (mean 0.998)
Opus reference: mean 1.000

Interview notes (transcripts, closed)

Qwen3.8-Flash-Next-oQ4e-mtp-r1024

Δacc vs Opus -1.430

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Contestant: studio-qwen-r1024-high · thinking high
Run: studio-qwen-r1024-high-a
Runner-up: studio-cards-qwen-r1024-v2 (Δacc vs Opus -4.030)
Opus reference: Δacc vs Opus 0.170

IAM boundary (tools)

Write an AWS IAM permissions boundary or policy from a written spec, with AWS Service Reference and policy-validation tools. Scored by IAM Access Analyzer plus allow/deny probes against the policy simulator; composite 0–1, capped at 0.25 on a critical privilege escalation.

Suite iam-boundary-v5-tools · 7 items · ranked by mean.

#	Model	Pass rate	mean [90% CI]	Samples	Run
1	opus-5-5-high	1.0	0.9907 [0.987, 0.995]	35/0	`iam5t-opus-5-5-high-ref2`
2	opus-5-5-high	1.0	0.9906 [0.987, 0.995]	35/0	`iam5t-opus-5-5-high-ref3`
3	opus-5-5-high	1.0	0.9905 [0.987, 0.995]	35/0	`iam5t-opus-5-5-high-ref1`
4	studio-deepseek-r1029-high	0.886	0.8921 [0.783, 0.976]	35/0	`iam5t-studio-deepseek-r1029-high-a`
5	studio-qwen-r1024-high	0.886	0.8914 [0.727, 0.979]	35/0	`iam5t-studio-qwen-r1024-high-a`
6	studio-mimo-high	0.886	0.8858 [0.706, 0.983]	35/0	`iam5t-studio-mimo-high-c`
7	studio-qwen-r1029-high	0.857	0.8747 [0.669, 0.985]	35/0	`iam5t-studio-qwen-r1029-high-a`
8	studio-mimo-high	0.857	0.8657 [0.646, 0.982]	35/0	`iam5t-studio-mimo-high-b`
9	studio-deepseek-high	0.829	0.8605 [0.7, 0.969]	35/0	`iam5t-studio-deepseek-high-a`
10	studio-swift-high	0.829	0.8443 [0.644, 0.977]	35/0	`iam5t-studio-swift-high-a`
11	studio-qwen-high	0.829	0.8296 [0.621, 0.975]	35/0	`iam5t-studio-qwen-high-a`
12	studio-glm53-q5-high	0.629	0.6954 [0.48, 0.88]	35/0	`iam5t-studio-glm53-q5-high-a`
13	studio-inkling-high	0.229	0.5666 [0.376, 0.751]	35/0	`iam5t-studio-inkling-high-a`

Audit-fix (agentic)

Apply a written audit finding to an infrastructure-as-code repository as a sandboxed, tool-using agent. Scored by the repo's gates (shellcheck, ansible-lint, terraform fmt, a sanitization sweep), change scope, and per-item checks validated against the real fix.

Suite audit-fix-v5 · 32 items · ranked by pass rate.

#	Model	Pass rate	mean score [90% CI]	Samples	Run
1	opus-5-5-high	1.0	1.0 [1.0, 1.0]	96/0	`afx5-opus-5-5-high-ref1`
2	studio-mimo-high	1.0	1.0 [1.0, 1.0]	96/0	`afx5-studio-mimo-high-a`
3	studio-mimo-high	1.0	1.0 [1.0, 1.0]	96/0	`afx5-studio-mimo-high-b`
4	studio-qwen-high	0.99	0.9983 [0.995, 1.0]	96/0	`afx5-studio-qwen-high-a`
5	studio-deepseek-r1029-high	0.99	0.9979 [0.994, 1.0]	96/0	`afx5-studio-deepseek-r1029-high-a`
6	opus-5-5-high	0.989	0.9989 [0.997, 1.0]	375/0	`afx5-opus-5-5-high-pooled`
7	studio-qwen-r1029-high	0.979	1.0 [1.0, 1.0]	96/0	`afx5-studio-qwen-r1029-high-a`
8	studio-deepseek-high	0.979	0.999 [0.997, 1.0]	96/0	`afx5-studio-deepseek-high-a`
9	studio-swift-high	0.979	0.9983 [0.995, 1.0]	96/0	`afx5-studio-swift-high-a`
10	studio-laguna-high	0.969	0.9985 [0.996, 1.0]	96/0	`afx5-studio-laguna-high-a`
11	studio-qwen-r1024-high	0.969	0.9974 [0.992, 1.0]	96/0	`afx5-studio-qwen-r1024-high-a`
12	studio-glm53-q5-high	0.958	0.9931 [0.985, 0.999]	96/0	`afx5-studio-glm53-q5-high-a`
13	studio-nemotron35-high	0.917	0.9834 [0.969, 0.995]	96/0	`afx5-studio-nemotron35-high-a`
14	studio-inkling-high	0.823	0.9731 [0.957, 0.988]	96/0	`afx5-studio-inkling-high-a`

Postmortem (assist)

Diagnose a failed Schrödinger job from its postmortem bundle, with documentation-search tools and at most one gold-free repair turn. Scored on failure category, verbatim evidence on the decisive log lines, required facts, and known-wrong causes.

Suite postmortem-v3-assist · 5 items · ranked by pass rate.

#	Model	Pass rate	mean score [90% CI]	Samples	Run
1	opus-5-5-high	1.0	1.0 [1.0, 1.0]	15/0	`pm-v3a-opus-5-5-high-a`
2	studio-deepseek-high	1.0	1.0 [1.0, 1.0]	15/0	`pm-v3a-studio-deepseek-high-a`
3	studio-mimo-high	1.0	1.0 [1.0, 1.0]	15/0	`pm-v3a-studio-mimo-high-c`
4	studio-qwen-r1029-high	1.0	1.0 [1.0, 1.0]	15/0	`pm-v3a-studio-qwen-r1029-high-a`
5	studio-mimo-high	0.933	0.9926 [0.978, 1.0]	15/0	`pm-v3a-studio-mimo-high-a`
6	studio-mimo-high	0.933	0.9926 [0.978, 1.0]	15/0	`pm-v3a-studio-mimo-high-b`
7	studio-mimo-high	0.933	0.9926 [0.978, 1.0]	15/0	`pm-v3a-studio-mimo-high-d`
8	studio-qwen-high	0.933	0.9926 [0.978, 1.0]	15/0	`pm-v3a-studio-qwen-high-a`
9	studio-deepseek-high	0.867	0.9852 [0.97, 1.0]	15/0	`pm-v3a-studio-deepseek-high-b`
10	studio-deepseek-r1029-high	0.867	0.9852 [0.97, 1.0]	15/0	`pm-v3a-studio-deepseek-r1029-high-a`
11	studio-qwen-r1024-high	0.867	0.9778 [0.948, 1.0]	15/0	`pm-v3a-studio-qwen-r1024-high-a`
12	studio-nemotron35-high	0.333	0.5852 [0.4, 0.763]	15/0	`pm-v3a-studio-nemotron35-high-a`

Recon checklist (assist)

Walk an HPC cluster-assessment findings catalog over a recon evidence bundle: present, absent or undetermined per row, with verbatim citations, and at most one gold-free repair turn.

Suite recon-v3-assist · 4 items · ranked by pass rate.

#	Model	Pass rate	mean score [90% CI]	Samples	Run
1	opus-5-5-high	1.0	1.0 [1.0, 1.0]	12/0	`rc-v3a-opus-5-5-high-a`
2	studio-qwen-r1024-high	0.75	0.95 [0.9, 0.983]	12/0	`rc-v3a-studio-qwen-r1024-high-a`
3	studio-qwen-r1024-high	0.75	0.95 [0.9, 0.983]	12/0	`rc-v3a-studio-qwen-r1024-high-b`
4	studio-qwen-r1029-high	0.75	0.95 [0.9, 0.983]	12/0	`rc-v3a-studio-qwen-r1029-high-b`
5	studio-deepseek-high	0.667	0.9333 [0.883, 0.983]	12/0	`rc-v3a-studio-deepseek-high-b`
6	studio-deepseek-r1029-high	0.583	0.9 [0.833, 0.967]	12/0	`rc-v3a-studio-deepseek-r1029-high-a`
7	studio-qwen-r1029-high	0.583	0.9 [0.833, 0.95]	12/0	`rc-v3a-studio-qwen-r1029-high-a`
8	studio-deepseek-high	0.5	0.9 [0.85, 0.95]	12/0	`rc-v3a-studio-deepseek-high-a`
9	studio-swift-high	0.5	0.9 [0.85, 0.95]	12/0	`rc-v3a-studio-swift-high-a`
10	studio-deepseek-r1029-high	0.5	0.8667 [0.8, 0.933]	12/0	`rc-v3a-studio-deepseek-r1029-high-b`
11	studio-qwen-high	0.5	0.8167 [0.667, 0.933]	12/0	`rc-v3a-studio-qwen-high-a`
12	studio-mimo-high	0.417	0.8167 [0.667, 0.917]	12/0	`rc-v3a-studio-mimo-high-a`
13	studio-nemotron35-high	0.25	0.8 [0.733, 0.867]	12/0	`rc-v3a-studio-nemotron35-high-a`
14	studio-mimo-high	0.25	0.7667 [0.633, 0.867]	12/0	`rc-v3a-studio-mimo-high-b`
15	studio-glm53-q5-high	0.0	0.4167 [0.267, 0.567]	12/0	`rc-v3a-studio-glm53-q5-high-a`

Document reading (vision)

Read values off document images (dashboards, spreadsheet and slide tables, charts, a Word page) into a requested JSON shape. 32 synthetic items with a generated answer key; per-field exact match.

Suite docread-v1 · 32 items · ranked by mean.

#	Model	Mean [90% CI]	Run
1	anthropic/claude-opus-5-5 (cloud, high)	1.000 [1.000, 1.000]	`dr1-opus-5-5-high-a`
2	NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit (studio, off)	0.999 [0.998, 1.000]	`dr1-studio-nemotron35-vl-a`
3	Qwen3.6-27B-oQ4e-mtp (mini, off)	0.998 [0.996, 0.999]	`dr1-mini-qwen36-27b-vl-a`
4	Qwen3.6-35B-A3B-oQ6-fp16-mtp (mini, off)	0.997 [0.994, 0.999]	`dr1-mini-qwen36-35b-vl-a`
5	Qwen3.6-35B-A3B-4bit (mini, off)	0.997 [0.994, 0.999]	`dr1-mini-qwen36-35b-4bit-vl-a`
6	Qwen3.8-Flash-Next-oQ4e-mtp-r1029 (studio, off)	0.996 [0.993, 0.999]	`dr1-studio-qwen-r1029-vl-a`
7	Qwen3.8-Flash-Next-oQ4e-mtp-r1024 (studio, off)	0.995 [0.988, 1.000]	`dr1-studio-qwen-r1024-vl-a`
8	Qwen3.8-27B-oQ4e-fp16-mtp (mini, off)	0.994 [0.988, 0.998]	`dr1-mini-qwen38-27b-vl-a`
9	gemma-4-26B-A4B-it-oQ4e-mtp (mini, off)	0.993 [0.986, 0.998]	`dr1-mini-gemma4-26b-vl-a`
10	Devstral-Small-2-24B-Instruct-2512-4bit (mini, off)	0.963 [0.943, 0.980]	`dr1-mini-devstral-vl-a`
11	MiMo-V2.6-Distill-Qwen-9B-oQ8e-fp16 (mini, off)	0.946 [0.917, 0.970]	`dr1-mini-mimo9b-vl-a`

Interview notes (transcripts, closed)

Write structured notes from stakeholder interview transcripts. Blind pairwise judging against the Opus baseline by a calibrated judge model; Δacc is the judged accuracy difference vs Opus (0 = parity, negative = worse). Closed 2026-09-26.

Suite interviews-v1 · 15 items · ranked by Δacc vs Opus.

#	Model	Δacc [CI]	Run
1	opus-5-5-high	0.17 [-0.47, 0.87]	`opus-5-5-high-rep1`
2	opus-5-5-high	0.17 [-0.47, 0.8]	`opus-5-5-high-rep2`
3	studio-qwen-r1024-high	-1.43 [-2.33, -0.5]	`studio-qwen-r1024-high-a`
4	studio-glm53-high	-2.23 [-2.7, -1.77]	`studio-glm53-high-rc1`
5	studio-cards-qwen-r1024-v2	-4.03 [-4.53, -3.53]	`studio-cards-qwen-r1024-v2-a`
6	mini-cards-qwen-v2	-4.43 [-4.9, -3.93]	`mini-cards-qwen-v2-a`
7	mini-qwen-high	-4.8 [-5.23, -4.37]	`mini-qwen-high-a`
8	mini-cards-qwen	-5.33 [-5.67, -5.0]	`mini-cards-qwen-a`

Models

Every model with a complete run above. Server settings are the host's current oMLX settings for that model (as of the snapshot), not necessarily those of older runs. Retired models no longer take new runs.

Claude Opus 5.5

opus-5-5-high

Runs on: Anthropic API (cloud)
Thinking: high
Family: anthropic

IAM boundary (tools)	#1 of 13	mean 0.991 · pass 1.000
Audit-fix (agentic)	#1 of 14	pass 1.000 · mean 1.000
Postmortem (assist)	#1 of 12	pass 1.000 · mean 1.000
Recon checklist (assist)	#1 of 15	pass 1.000 · mean 1.000
Document reading (vision)	#1 of 11	mean 1.000
Interview notes (transcripts, closed)	#1 of 8	Δacc vs Opus 0.170

Devstral-Small-2-24B-Instruct-2512-4bit

mini-devstral-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: mistral
Context / max out: 393,216 / 16,384 tokens
Decoding: standard

Document reading (vision)	#10 of 11	mean 0.963

gemma-4-26B-A4B-it-oQ4e-mtp

mini-gemma4-26b-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: gemma
Context / max out: 262,144 / 16,384 tokens
Sampling: temp 1 · top_p 0.95 · top_k 64
Decoding: multi-token prediction

Document reading (vision)	#9 of 11	mean 0.993

MiMo-V2.6-Distill-Qwen-9B-oQ8e-fp16

mini-mimo9b-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: mimo
Context / max out: 262,144 / 8,192 tokens
Decoding: standard

Document reading (vision)	#11 of 11	mean 0.946

Qwen3.6-35B-A3B-oQ6-fp16-mtp

mini-qwen-high

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction · TurboQuant KV cache

Interview notes (transcripts, closed)	#7 of 8	Δacc vs Opus -4.800

Qwen3.6-27B-oQ4e-mtp

mini-qwen36-27b-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction · TurboQuant KV cache

Document reading (vision)	#3 of 11	mean 0.998

Qwen3.6-35B-A3B-4bit

mini-qwen36-35b-4bit-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: standard

Document reading (vision)	#5 of 11	mean 0.997

Qwen3.6-35B-A3B-oQ6-fp16-mtp

mini-qwen36-35b-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction · TurboQuant KV cache

Document reading (vision)	#4 of 11	mean 0.997

Qwen3.8-27B-oQ4e-fp16-mtp

mini-qwen38-27b-vl

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: off
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

Document reading (vision)	#8 of 11	mean 0.994

DeepSeek-V4-Flash-0731-oQ4e-mtp

studio-deepseek-high

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: deepseek
Context / max out: 1,048,576 / 65,536 tokens
Sampling: temp 1 · top_p 0.95 · top_k 0
Decoding: multi-token prediction

IAM boundary (tools)	#9 of 13	mean 0.861 · pass 0.829
Audit-fix (agentic)	#8 of 14	pass 0.979 · mean 0.999
Postmortem (assist)	#2 of 12	pass 1.000 · mean 1.000
Recon checklist (assist)	#5 of 15	pass 0.667 · mean 0.933

DeepSeek-V4-Flash-0731-oQ4e-mtp-r1029

studio-deepseek-r1029-high

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: deepseek
Context / max out: 1,048,576 / 65,536 tokens
Sampling: temp 1 · top_p 0.95 · top_k 0
Decoding: multi-token prediction

IAM boundary (tools)	#4 of 13	mean 0.892 · pass 0.886
Audit-fix (agentic)	#5 of 14	pass 0.990 · mean 0.998
Postmortem (assist)	#10 of 12	pass 0.867 · mean 0.985
Recon checklist (assist)	#6 of 15	pass 0.583 · mean 0.900

MiMo-V2.6-Flash-MOPD-oQ4e-mtp

studio-mimo-high

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: mimo
Context / max out: 524,288 / 65,536 tokens
Sampling: temp 1 · top_p 0.95 · top_k 0
Decoding: multi-token prediction

IAM boundary (tools)	#6 of 13	mean 0.886 · pass 0.886
Audit-fix (agentic)	#2 of 14	pass 1.000 · mean 1.000
Postmortem (assist)	#3 of 12	pass 1.000 · mean 1.000
Recon checklist (assist)	#12 of 15	pass 0.417 · mean 0.817

NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit

studio-nemotron35-vl

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: off
Family: nemotron
Context / max out: 131,072 / 16,384 tokens
Sampling: temp 1 · top_p 0.95 · top_k 0
Decoding: standard

Document reading (vision)	#2 of 11	mean 0.999

Qwen3.8-Flash-Next-oQ4e-mtp-r1024

studio-qwen-r1024-high

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

IAM boundary (tools)	#5 of 13	mean 0.891 · pass 0.886
Audit-fix (agentic)	#11 of 14	pass 0.969 · mean 0.997
Postmortem (assist)	#11 of 12	pass 0.867 · mean 0.978
Recon checklist (assist)	#2 of 15	pass 0.750 · mean 0.950
Interview notes (transcripts, closed)	#3 of 8	Δacc vs Opus -1.430

Qwen3.8-Flash-Next-oQ4e-mtp-r1024

studio-qwen-r1024-vl

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: off
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

Document reading (vision)	#7 of 11	mean 0.995

Qwen3.8-Flash-Next-oQ4e-mtp-r1029

studio-qwen-r1029-high

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

IAM boundary (tools)	#7 of 13	mean 0.875 · pass 0.857
Audit-fix (agentic)	#7 of 14	pass 0.979 · mean 1.000
Postmortem (assist)	#4 of 12	pass 1.000 · mean 1.000
Recon checklist (assist)	#4 of 15	pass 0.750 · mean 0.950

Qwen3.8-Flash-Next-oQ4e-mtp-r1029

studio-qwen-r1029-vl

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: off
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

Document reading (vision)	#6 of 11	mean 0.996

Qwen3.6-35B-A3B-oQ6-fp16-mtp

mini-cards-qwen-v2 · multi-step pipeline around a local model

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction · TurboQuant KV cache

Interview notes (transcripts, closed)	#6 of 8	Δacc vs Opus -4.430

Qwen3.8-Flash-Next-oQ4e-mtp-r1024

studio-cards-qwen-r1024-v2 · multi-step pipeline around a local model

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

Interview notes (transcripts, closed)	#5 of 8	Δacc vs Opus -4.030

GLM-5.3-Flash-oQ4e

studio-glm53-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: glm

Interview notes (transcripts, closed)	#4 of 8	Δacc vs Opus -2.230

GLM-5.3-Flash-oQ5e-mtp

studio-glm53-q5-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: glm

IAM boundary (tools)	#12 of 13	mean 0.695 · pass 0.629
Audit-fix (agentic)	#12 of 14	pass 0.958 · mean 0.993
Recon checklist (assist)	#15 of 15	pass 0.000 · mean 0.417

Inkling-Small-mlx-4bit

studio-inkling-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: inkling

IAM boundary (tools)	#13 of 13	mean 0.567 · pass 0.229
Audit-fix (agentic)	#14 of 14	pass 0.823 · mean 0.973

Laguna-S-2.1-oQ6e

studio-laguna-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: laguna

Audit-fix (agentic)	#10 of 14	pass 0.969 · mean 0.999

NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit

studio-nemotron35-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: nemotron
Context / max out: 131,072 / 16,384 tokens
Sampling: temp 1 · top_p 0.95 · top_k 0
Decoding: standard

Audit-fix (agentic)	#13 of 14	pass 0.917 · mean 0.983
Postmortem (assist)	#12 of 12	pass 0.333 · mean 0.585
Recon checklist (assist)	#13 of 15	pass 0.250 · mean 0.800

Qwen3.8-Flash-Next-oQ4e-mtp

studio-qwen-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: qwen

IAM boundary (tools)	#11 of 13	mean 0.830 · pass 0.829
Audit-fix (agentic)	#4 of 14	pass 0.990 · mean 0.998
Postmortem (assist)	#8 of 12	pass 0.933 · mean 0.993
Recon checklist (assist)	#11 of 15	pass 0.500 · mean 0.817

Swift1.5-Qwen3.8-Flash-Next-oQ4e-fp16-mtp

studio-swift-high · retired

Runs on: Mac Studio, Apple M5 Ultra, 256 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction

IAM boundary (tools)	#10 of 13	mean 0.844 · pass 0.829
Audit-fix (agentic)	#9 of 14	pass 0.979 · mean 0.998
Recon checklist (assist)	#9 of 15	pass 0.500 · mean 0.900

Qwen3.6-35B-A3B-oQ6-fp16-mtp

mini-cards-qwen · retired · multi-step pipeline around a local model

Runs on: Mac mini, Apple M4 Pro, 64 GB unified memory
Thinking: high
Family: qwen
Context / max out: 262,144 / 32,768 tokens
Sampling: temp 1 · top_p 0.95 · top_k 20
Decoding: multi-token prediction · TurboQuant KV cache

Interview notes (transcripts, closed)	#8 of 8	Δacc vs Opus -5.330