int8 quantization matches float32 performance in text embeddings

The study found that 8-bit quantization performs statistically indistinguishably from float32 in pair classification and re-ranking tasks, while binary quantization is significantly worse. The benchmark evaluated 168 configurations across 38 tasks using four transformer models.

Read the original →