Name	Last modified	Size	Description
	Parent Directory		-
	AGENTS.md	2026-08-26 14:48	6.5K
	\	2026-08-26 15:07	3.6K
	dual_gpu_context_balancing_guide.md	2026-08-25 13:38	13K
	history.md	2026-08-26 14:48	12K
	improvemets_list.md	2026-08-19 19:29	6.7K
	latest_rocm_improvement_3737e41.patch	2026-08-26 14:48	99K
	latest_rocm_improvement_no_adaptive_mtp_3737e41.patch	2026-08-25 13:06	79K
	llama_scripts/	2026-08-27 23:52	-
	logs/	2026-08-25 13:40	-
	mtp_compact_rollback.md	2026-08-26 14:48	17K
	mtp_compact_rollback_3737e41.patch	2026-08-26 14:48	80K
	mtp_compact_rollback_no_adaptive_3737e41.patch	2026-08-25 13:06	61K
	mtp_cr_eaman.md	2026-08-26 10:03	16K
	old/	2026-08-25 14:12	-
	patches/	2026-08-26 14:33	-
	summary_eaman.md	2026-08-26 14:48	77K
	usage_eaman.md	2026-08-26 14:49	770

Eaman ROCm/MTP patch for llama.cpp

What the patches do

• Improve per-GPU context fitting for fixed multi-GPU layer layouts.
• Add control over pipeline-parallel scheduling.
• Reduce MTP scheduler copies and temporary workspace with non-pipeline execution.
• Add a ROCm/HIP quantized-KV VEC path that avoids large F16 conversion buffers.
• Add selectable normal/VEC HIP Flash Attention dispatch.
• Add configurable MTP Compact Rollback.
• Use on-GPU rollback checkpoints with exact per-GPU memory reservation.
• Allocate rollback checkpoints at startup with safe host fallback and multi-slot support.

Improvements now provided by upstream llama.cpp

Recent llama.cpp versions already include joint target/MTP memory fitting, recalculate MTP memory for every candidate context size and pass the final fitted context directly to runtime. These upstream improvements replace parts of the older Eaman fitting and MoE workaround code.

How to build:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 3737e41
wget https://store.piffa.net/lm/bug/latest_rocm_improvement_3737e41.patch
git apply latest_rocm_improvement_3737e41.patch

You can build for both Vulkan and ROCm backends at the same time, the idea is that Vulkan saves some more vRAM while ROCm gives better prefill performance.

HIPCXX="$(hipconfig -l)/clang" \
HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build \
    -DCMAKE_BUILD_TYPE=Release \
    -DCMAKE_HIP_ARCHITECTURES="gfx1030;gfx1031" \
    -DGGML_HIP=ON \
    -DGGML_VULKAN=ON \
    -DGGML_CUDA_FA_ALL_QUANTS=ON \
    -DLLAMA_BUILD_SERVER=ON

cmake --build build \
    --config Release \
    -j"$(nproc)"

How to use: options for llama-server:

• --spec-mtp-cr-depth N
  Sets the MTP Compact Rollback depth, recommanded value = 1 with --spec-draft-n-max 7 for coding (5 or less for more prosaic content).
  If omitted, the normal full MTP depth is used.
• --spec-draft-adaptive
  Dynamically adjusts the MTP draft limit, up to --spec-draft-n-max, based on recent draft acceptance.
  Most useful for less predictable text at large context. Default is off.
• --pipeline-parallel auto|on|off
  Controls pipeline parallelism, use off when prioritizing available context memory.
  The default is auto, llama.cp normal behaviour.
• --hip-fa-force-vec on|off
  Forces the supported quantized-KV HIP Flash Attention VEC path, use on to enable more ctx at the expense of Prompt Processing speed.
  The default is off. This option affects ROCm/HIP only.

Simple MTP example

     --spec-mtp-cr-depth  1 --spec-draft-adaptive \
     --spec-type draft-mtp --spec-draft-p-min 0.75 --spec-draft-n-max 7 \
     --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 \
  
  

Example llama-server scripts for 16GB gpu and dual 16+12GB. Made for AMD 6800 plus 6700XT.