Name	Last modified	Size	Description
	Parent Directory		-
	AGENTS.md	2026-08-31 17:10	6.5K
	dual_gpu_context_balancing_guide.md	2026-08-25 13:38	13K
	history.md	2026-08-26 14:48	12K
	improvemets_list.md	2026-08-19 19:29	6.7K
	latest_rocm_improvement_c6824a9e.patch	2026-09-16 18:39	126K
	llama_scripts/	2026-09-04 10:59	-
	logs/	2026-08-25 13:40	-
	mtp_compact_rollback.md	2026-08-26 14:48	17K
	mtp_compact_rollback_c6824a9e.patch	2026-09-16 18:39	77K
	mtp_cr_eaman.md	2026-08-26 10:03	16K
	old/	2026-08-31 16:50	-
	patches/	2026-09-16 18:39	-
	research/	2026-09-04 09:48	-
	status	2026-09-16 18:39	23K
	summary_eaman.md	2026-09-16 18:39	98K
	usage_eaman.md	2026-08-26 14:49	770

Eaman ROCm/MTP patch for llama.cpp

What the patches do

• Improve per-GPU context fitting for fixed multi-GPU layer layouts.
• Add control over pipeline-parallel scheduling.
• Reduce MTP scheduler copies and temporary workspace with non-pipeline execution.
• Add a ROCm/HIP quantized-KV VEC path that avoids large F16 conversion buffers.
• Add selectable normal/VEC HIP Flash Attention dispatch.
• Add configurable MTP Compact Rollback.
• Use on-GPU rollback checkpoints with exact per-GPU memory reservation.
• Allocate rollback checkpoints at startup with safe host fallback and multi-slot support.

Improvements now provided by upstream llama.cpp

Recent llama.cpp versions already include joint target/MTP memory fitting, recalculate MTP memory for every candidate context size and pass the final fitted context directly to runtime. These upstream improvements replace parts of the older Eaman fitting and MoE workaround code.

How to build:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout c6824a9e
wget https://store.piffa.net/lm/bug/latest_rocm_improvement_c6824a9e.patch
git apply latest_rocm_improvement_c6824a9e.patch

You can build for both Vulkan and ROCm backends at the same time, the idea is that Vulkan saves some more vRAM while ROCm gives better prefill performance.

HIPCXX="$(hipconfig -l)/clang" \
HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_HIP_ARCHITECTURES="gfx1030;gfx1031" \
    -DGGML_HIP=ON \
    -DGGML_VULKAN=ON \
    -DGGML_CUDA_FA_ALL_QUANTS=ON \
    -DLLAMA_BUILD_SERVER=ON

cmake --build build \
    --config Release \
    -j"$(nproc)"

How to use: options for llama-server:

• --spec-mtp-cr-depth N
  Sets the MTP Compact Rollback depth, recommanded value = 1 with --spec-draft-n-max 7 for coding (5 or less for more prosaic content).
  If omitted, the normal full MTP depth is used.
--spec-draft-adaptive
Dynamically adjusts the MTP draft limit, up to --spec-draft-n-max, based on recent draft acceptance.
Most useful for less predictable text at large context, mixed sessions of code and creative language. Default is off.
• --pipeline-parallel auto|on|off
  Controls pipeline parallelism, use off when prioritizing available context memory.
  The default is auto, llama.cp normal behaviour.
• --hip-fa-force-vec on|off
  Forces the supported quantized-KV HIP Flash Attention VEC path, use on to enable more ctx at the expense of Prompt Processing speed.
  The default is off. This option affects ROCm/HIP only.

     --spec-mtp-cr-depth  1 --spec-draft-adaptive \
     --spec-type draft-mtp --spec-draft-p-min 0.75 --spec-draft-n-max 7 \
     --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 \