# Instructions for the ROCm MTP and VEC investigation

## Purpose

This directory stores the patches, launch notes, validation logs, and session
history for the ROCm MTP fitting and HIP VEC flash-attention work.

## Repository and branch policy

- This notes repository is `/home/eaman/llama/bug`.
- Keep this repository on `master`.
- The source repository is `/home/eaman/llama/llama.cpp`.
- Keep the source repository on `sol` when working on this investigation.
- Do not commit or modify `master` in `llama.cpp` unless the user explicitly asks.
- Do not push or create pull requests.

## Canonical artifacts

- `summary_sol.md` is the main technical record and current status document.
- `status` is the short operational snapshot.
- `sol4.patch` is the canonical complete patch and includes the merged HIP VEC change.
- `rocm_improvement.patch` is a duplicate of `sol4.patch`, not a separate patch.
- `logs/` contains validation logs.
- `old/patches/` contains superseded patches such as `sol2.patch` and `sol3.patch`.
- `llama_scripts/` contains active local launchers.
- Do not recreate the removed `vec_update.md` or `think_pipeline_off_f16.sh` files.

## Current source state

- Source branch: `sol`.
- Source tip includes the MTP fitting and pipeline-control changes plus HIP VEC
  dispatch for supported quantized-KV attention shapes.
- `sol4.patch` is based on `master` commit `7bd8282` and represents source tip
  `e37aae4`.
- The initial Q4/Q4 MTP VEC run reached 147,712 context tokens and completed
  startup plus short generation.
- Long-prompt stability, delayed-OOM, throughput, and draft-acceptance testing
  remain pending.

## Validation workflow

1. Confirm both repository branches and clean worktrees before acting.
2. Apply patches and build in a temporary detached worktree; never build by
   modifying the user's `master` checkout.
3. A CPU/default build checks general compilation.
4. A HIP build is needed to compile `ggml/src/ggml-cuda/fattn.cu` and the
   `fattn-vec` instances.
5. Report separately whether the HIP backend compiled and whether the final
   application link completed.
6. Remove temporary worktrees after validation.
7. Record new measured results in `summary_sol.md` and keep `status` concise.

## Documentation rules

- Use paths from the current layout: logs are under `logs/`, and old patches are
  under `old/patches/`.
- Keep historical results labeled as historical; do not present the initial VEC
  result as a completed stability benchmark.
- Update checksums and commit identifiers when replacing the canonical patch.
- Keep `summary_sol.md` for detailed history and `AGENTS.md` for stable workflow
  rules; do not duplicate large experiment logs here.