양자화 전쟁에 또 다른 벤치마크가 등장함
Another benchmark has hit the quant war
핵심 요약
ISTA-DASLab의 양자화 성능 논란 속에서, Unsloth와 비교한 새로운 벤치마크 결과가 공유되어 커뮤니티의 주목을 받고 있습니다.
- 벤치마크 논란 — ISTA-DASLab의 양자화 수치 재현이 어렵다는 비판이 제기됨
- 성능 비교 — Basalt 제작자가 ISTA 대신 Unsloth 양자화를 사용해 성능을 비교함
- 데이터 시각화 — 공유된 차트의 가독성이 매우 나빠 비판을 받음
- 양자화 한계 — 극단적인 양자화가 모델의 지능을 저하시킨다는 우려가 나옴
If you have not been following https://huggingface.co/ISTA-DASLab[ISTA-DASLab](https://huggingface.co/ISTA-DASLab) the creators of the gsq-rco quants have received some criticism from unsloth regarding the inabilty to reproduce the benchmark numbers that ISTA-DASLab have quoted. The methodology used to produce their benchmark numbers has not been posted and their has been difficulty reproducing their numbers with unsloth finding in there testing unsloth quants outperforming the gsq-rco quants.
The numbers in this post here are from another creator the creator of basalt (a inference engine) previously his qwen-flash-next quants (his inference engine uses specially packed quants based on a existing model similar to ninfer ) used ISTA-DASLab's quants as the base but after doing some testing he replaced them with similar sized unsloth equivalents the performance numbers are in the screenshots
