RTX 5080 16GB에서 Qwen 3.8 27b를 약 75t/s로 구동하는 방법

다들 안녕,

최근에 여러 LLM 세팅을 이것저것 만져보고 있는데, 다들 Qwen 3.8 27b가 그렇게 좋다고 난리길래 나도 한번 돌려봐야겠다 싶어서 시도해 봤어.

설정 때문에 좀 고생하긴 했는데, 결국 평균 75t/s 정도 뽑아냈고 MTP까지 잘 먹히면 100t/s 넘게도 나오더라.

내가 쓴 건 이거야:
Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller - jrell/Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller · Hugging Face

"Qwen3.8-27B 베이스 모델을 커스텀 하이브리드 양자화한 건데, 16GB VRAM이라는 빡빡한 하드웨어 환경(RTX 4080 / RTX 5080 같은 소비자용 카드)에서도 Multi-Token Prediction(MTP)이랑 긴 컨텍스트를 돌릴 수 있게 설계됐어."

내 llama.cpp 설정은 이래:

$llamaPath = "C:\Tools\llama-cuda2\llama-server.exe"
$modelPath = "D:\models\Qwen3.8-27B-i1-IQ4_XS-GGUF-Smaller.gguf"
$chatTemplatePath = Join-Path (Split-Path $modelPath -Parent) "chat_template.jinja"

$llamaArgs = @(
"-m", $modelPath,
"-a", "qwen3.8-27b",
"-ngl", "99",
"-c", "85000",
"-np", "1",
"-b", "512",
"-ub", "512",
"-fa", "on",
"-t", "8",
"-tb", "8",
"-ctk", "q4_0",
"-ctv", "q4_0",
"--reasoning-preserve",
"--reasoning-effort", "medium",
"--temp", "1",
"--top-p", "0.95",
"--top-k", "20",
"--min-p", "0.0",
"--spec-type", "draft-mtp",
"--spec-draft-n-max", "3",
"--repeat-penalty", "1.0",
"--presence-penalty", "0.0",
"--jinja",
"--chat-template-file", $chatTemplatePath,
"--host", "0.0.0.0",
"--port", "8080"
)

& $llamaPath

최근에 돌려본 결과 예시야:

10.17.039.668 I slot launch_slot_: id 0 | task 10227 | 작업 처리 중, is_child = 0
10.20.936.381 I slot print_timing: id 0 | task 10227 | n_gen = 239, tg = 78.94 t/s, tg_3s = 79.26 t/s
10.23.953.739 I slot print_timing: id 0 | task 10227 | n_gen = 424, tg = 70.12 t/s, tg_3s = 61.31 t/s
10.26.978.406 I slot print_timing: id 0 | task 10227 | n_gen = 642, tg = 70.77 t/s, tg_3s = 72.07 t/s
10.30.010.411 I slot print_timing: id 0 | task 10227 | n_gen = 853, tg = 70.48 t/s, tg_3s = 69.59 t/s
10.33.032.673 I slot print_timing: id 0 | task 10227 | n_gen = 1082, tg = 71.54 t/s, tg_3s = 75.77 t/s
10.36.038.373 I slot print_timing: id 0 | task 10227 | n_gen = 1327, tg = 73.19 t/s, tg_3s = 81.51 t/s
10.39.043.415 I slot print_timing: id 0 | task 10227 | n_gen = 1532, tg = 72.48 t/s, tg_3s = 68.22 t/s
10.42.075.239 I slot print_timing: id 0 | task 10227 | n_gen = 1819, tg = 75.27 t/s, tg_3s = 94.66 t/s
10.45.082.345 I slot print_timing: id 0 | task 10227 | n_gen = 2005, tg = 73.78 t/s, tg_3s = 61.85 t/s 10.48.100.133 I slot print_timing: id 0 | task 10227 | n_gen = 2210, tg = 73.20 t/s, tg_3s = 67.93 t/s 10.51.129.760 I slot print_timing: id 0 | task 10227 | n_gen = 2436, tg = 73.32 t/s, tg_3s = 74.60 t/s 10.54.152.649 I slot print_timing: id 0 | task 10227 | n_gen = 2646, tg = 73.00 t/s, tg_3s = 69.47 t/s 10.57.172.073 I slot print_timing: id 0 | task 10227 | n_gen = 2847, tg = 72.51 t/s, tg_3s = 66.57 t/s 11.00.182.456 I slot print_timing: id 0 | task 10227 | n_gen = 3117, tg = 73.73 t/s, tg_3s = 89.69 t/s 11.03.195.203 I slot print_timing: id 0 | task 10227 | n_gen = 3338, tg = 73.71 t/s, tg_3s = 73.36 t/s 11.06.202.960 I slot print_timing: id 0 | task 10227 | n_gen = 3545, tg = 73.40 t/s, tg_3s = 68.82 t/s 11.09.219.971 I slot print_timing: id 0 | task 10227 | n_gen = 3764, tg = 73.35 t/s, tg_3s = 72.59 t/s 11.12.237.385 I slot print_timing: id 0 | task 10227 | n_gen = 3979, tg = 73.24 t/s, tg_3s = 71.25 t/s 11.15.246.902 I slot print_timing: id 0 | task 10227 | n_gen = 4170, tg = 72.72 t/s, tg_3s = 63.47 t/s 11.18.268.286 I slot print_timing: id 0 | task 10227 | n_gen = 4370, tg = 72.40 t/s, tg_3s = 66.19 t/s 11.21.299.694 I slot print_timing: id 0 | task 10227 | n_gen = 4587, tg = 72.36 t/s, tg_3s = 71.58 t/s 11.24.331.662 I slot print_timing: id 0 | task 10227 | n_gen = 4815, tg = 72.49 t/s, tg_3s = 75.20 t/s 11.27.359.294 I slot print_timing: id 0 | task 10227 | n_gen = 5116, tg = 73.66 t/s, tg_3s = 99.42 t/s 11.30.365.338 I slot print_timing: id 0 | task 10227 | n_gen = 5309, tg = 73.27 t/s, tg_3s = 64.20 t/s 11.33.366.835 I slot print_timing: id 0 | task 10227 | n_gen = 5573, tg = 73.85 t/s, tg_3s = 87.96 t/s 11.36.390.528 I slot print_timing: id 0 | task 10227 | n_gen = 5835, tg = 74.35 t/s, tg_3s = 86.65 t/s 11.39.398.044 I slot print_timing: id 0 | task 10227 | n_gen = 6058, tg = 74.34 t/s, tg_3s = 74.15 t/s 11.42.424.685 I slot print_timing: id 0 | task 10227 | n_gen = 6355, tg = 75.19 t/s, tg_3s = 98.13 t/s 11.45.441.398 I slot print_timing: id 0 | task 10227 | n_gen = 6571, tg = 75.07 t/s, tg_3s = 71.60 t/s 11.48.465.392 I slot print_timing: id 0 | task 10227 | n_gen = 6789, tg = 74.97 t/s, tg_3s = 72.09 t/s 11.51.466.866 I slot print_timing: id 0 | task 10227 | n_gen = 6973, tg = 74.53 t/s, tg_3s = 61.30 t/s 11.54.494.631 I slot print_timing: id 0 | task 10227 | n_gen = 7163, tg = 74.16 t/s, tg_3s = 62.75 t/s 11.57.525.053 I slot print_timing: id 0 | task 10227 | n_gen = 7371, tg = 73.99 t/s, tg_3s = 68.64 t/s 12.00.558.694 I slot print_timing: id 0 | task 10227 | n_gen = 7606, tg = 74.10 t/s, tg_3s = 77.46 t/s 12.03.580.920 I slot print_timing: id 0 | task 10227 | n_gen = 7917, tg = 74.92 t/s, tg_3s = 102.90 t/s