gfx1151 · Ryzen AI MAX+ 395 · 128 GiB unified memory

The Open Source Strix Halo AI engine

Get started See benchmarks

01 / Principles

Why Gufo

We were tired of owning hardware we couldn't fully use. So we built Gufo: an inference engine tuned end to end for Strix Halo, running only the best model in each class, with no performance left on the table.

Fewer models, done well

The best model for each size, not every model.

Speed never costs quality

Every model has a quality report.

Measure the whole workload

A faster kernel that makes requests slower gets rejected.

Real workloads, reproducible builds

Several users and long conversations are the normal case.

Contributions are welcome. Two community forks are worth a look: a Windows port and Gufo RDMA for two Strix Halos linked together.

02 / Quickstart

Serve in two commands

Download the weights, then start the runtime image with access to the GPU. The server listens on port 8080 and speaks the OpenAI API, so Open WebUI, Codex or any OpenAI SDK can use localhost:8080/v1.

Full quickstart, or build from source
Quickstartsh
# Download a target and its DFlash2 draft
hf download unsloth/Qwen3.8-27B-GGUF Qwen3.8-27B-UD-Q8_K_XL.gguf \
  --local-dir models/Qwen3.8-27B-GGUF

# Serve it with speculative decoding
podman run --rm --userns=keep-id:uid=1000,gid=1000 \
  --device /dev/kfd --device /dev/dri --group-add keep-groups \
  --ulimit memlock=-1 -p 8080:8080 -v ./models:/models:ro \
  ghcr.io/gufo-org/toolboxes/gufo-runtime:latest \
  gufo serve --host 0.0.0.0 --port 8080 llm \
  --model /models/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q8_K_XL.gguf \
  --speculative dflash2 --dflash-model /models/…-DFlash2-Q4_K_M.gguf

03 / Models

Supported models

Each model has its own implementation, kernels and tests under src/models/.

Benchmark tables
Model Task Best result
Qwen3.8 27B Dense text and images 656.33 tok/s pp · up to 70.56 tok/s tg · 123.00 tok/s summed over 8 users
Qwen3.8 Flash-Next MoE text and images 1,700.52 tok/s pp · up to 60.39 tok/s tg · 162.98 tok/s summed over 8 users
DeepSeek V4 Flash MoE text 484.62 tok/s pp · up to 26.62 tok/s tg · 54.74 tok/s summed over 8 users
Qwen3-ASR 1.7B Speech recognition 15.27× realtime
Qwen3-TTS 12Hz 1.7B Speech synthesis and voice cloning Up to 2.54× realtime · 201 ms to first audio
Qwen-Image-2.1 Image generation and editing In progress
MiniMax H3 FL2VA Text to video and audio In progress

Best measured results. Prefill (pp) runs without speculation; decode (tg) uses the model's draft mode, and its peak comes from repetitive output. The 8-user number adds up each request's decode speed.