IQ4_XS
one shot with patched ROCm: 	58112ctx
	      mainstream:	20480	
	patched vulkan:		78592
	unpatched vulkan:	68352



Think q6
one shot with patched ROCm: 	149248
	      mainstream:	64256  
	patched vulkan:		151296	
	unpatched vulkan:	68864

                 patched    unpatched
Vulkan0          102 MiB      512 MiB
Vulkan1          102 MiB      391 MiB
--------------------------------------
GPU compute      204 MiB      903 MiB



---------------
Think q6 vulkan patched:
I common_memory_breakdown_print: | memory breakdown [MiB]                 | total    free     self   model   context   compute    unaccounted |
I common_memory_breakdown_print: |   - Vulkan0 (RX 6800 (RADV NAVI21))    | 16368 = 15855 + (13118 = 12569 +     446 +     102) +      -12606 |
I common_memory_breakdown_print: |   - Vulkan1 (RX 6700 XT (RADV NAVI22)) | 12272 = 12031 + ( 9465 =  9094 +     267 +     102) +       -9224 |
I common_memory_breakdown_print: |   - Host                               |                   1306 =  1288 +       0 +      18                |
I common_params_fit_impl: context size reduced from 262144 to 151296 using per-device limits


Think q6 vulkan unpatched:

I common_memory_breakdown_print: | memory breakdown [MiB]                 | total    free     self   model   context   compute    unaccounted |
I common_memory_breakdown_print: |   - Vulkan0 (RX 6800 (RADV NAVI21))    | 16368 = 15856 + (13528 = 12569 +     446 +     512) +      -13016 |
I common_memory_breakdown_print: |   - Vulkan1 (RX 6700 XT (RADV NAVI22)) | 12272 = 12031 + ( 9754 =  9094 +     267 +     391) +       -9513 |
I common_memory_breakdown_print: |   - Host                               |                   1316 =  1288 +       0 +      28                |
I common_params_fit_impl: context size reduced from 262144 to 68864 -> need 6330 MiB less memory in total



Test vulkan at max ctx:
------------------------------


performance:
zero
I slot print_timing: id  0 | task 0 | prompt processing, n_tokens =   2048, progress = 0.02, t =   7.94 s / 257.92 tokens per second
-- mid 90k ctx

I slot create_check: id  0 | task 0 | created context checkpoint 1 of 148 (pos_min = 88987, pos_max = 88987, n_tokens = 88988, size = 498.933 MiB)
I slot print_timing: id  0 | task 0 | prompt processing, n_tokens =  89372, progress = 1.00, t = 760.06 s / 117.59 tokens per second

I slot print_timing: id  0 | task 0 | prompt eval time =  761072.39 ms / 89376 tokens (    8.52 ms per token,   117.43 tokens per second)
---- 130k
I slot   operator(): id  0 | task 151 | cached n_tokens = 100726, memory_seq_rm [100726, end)
I slot print_timing: id  0 | task 151 | prompt processing, n_tokens =  12378, progress = 0.79, t = 177.06 s / 69.91 tokens per second
I slot create_check: id  0 | task 151 | created context checkpoint 3 of 148 (pos_min = 128095, pos_max = 128095, n_tokens = 128096, size = 652.444 MiB)
I slot print_timing: id  0 | task 151 | prompt processing, n_tokens =  39108, progress = 1.00, t = 626.75 s / 62.40 tokens per second

------------- Generation at 130K
I slot print_timing: id  0 | task 1341 | n_decoded =    102, tg =  14.90 t/s, tg_3s =  14.90 t/s
--
I slot print_timing: id  0 | task 1341 | prompt eval time =   21651.10 ms /  1007 tokens (   21.50 ms per token,    46.51 tokens per second)
I slot print_timing: id  0 | task 1341 |        eval time =  460890.58 ms /  6944 tokens (   66.37 ms per token,    15.07 tokens per second)
I slot print_timing: id  0 | task 1341 |       total time =  482541.69 ms /  7951 tokens
I slot print_timing: id  0 | task 1341 |    graphs reused =       1525
I slot print_timing: id  0 | task 1341 | draft acceptance = 0.89884 ( 4745 accepted /  5279 generated), mean len =  3.91

-------------------
Summary: 15.06 TG / 62.40 PP at 130k ctx
---------------------------
• Model     ROCm mainline  ROCm patched  Vulkan mainline  Vulkan patched
  IQ4_XS         20,480        58,112          68,352          78,592
  Think Q6       64,256       149,248          68,864         151,296



