Model catalog

Everything we serve, with the numbers most providers leave out. Quantization is a spec, not a footnote — every model runs with ThousandML CQ (Coded Quant), and you can see the exact precision, latency, and price before you spend a cent.

4 models serving now

Llama 3.3 70B Instruct

Live

meta-llama/Llama-3.3-70B-Instruct

Parameters

70B

Max context

128K

Meta’s flagship dense model, tuned for instruction following, tool use, and multilingual chat. CQ 4-bit keeps it within a fraction of a point of full precision on reasoning benchmarks while cutting the memory it takes to serve.

ProviderThousandML

Context128K

P50 TTFT544ms

Throughput6.6 tok/s

Uptime99.99%

QuantizationCQ 4-bit

Input /M$0.08

Output /M$0.30

Cached input /M—

generaltool-usemultilingual

meta-llama/Llama-3.3-70B-Instruct

Llama 3.1 8B Instruct

Live

meta-llama/Llama-3.1-8B-Instruct

Parameters

Max context

128K

The fastest, cheapest model in the catalog. Ideal for classification, extraction, and high-volume routing where latency and price matter more than frontier reasoning.

ProviderThousandML

Context128K

P50 TTFT140ms

Throughput21.2 tok/s

Uptime99.99%

QuantizationCQ 4-bit

Input /M$0.018

Output /M$0.038

Cached input /M—

fastcheapclassification

meta-llama/Llama-3.1-8B-Instruct

Qwen3.8 27B

Live

Qwen/Qwen3.8-27B

Parameters

27B

Max context

262K

A balanced mid-tier model with strong coding and math performance and a long context window. The right default when 8B is too small and 70B is more than the task needs.

ProviderThousandML

Context262K

P50 TTFT393ms

Throughput21.3 tok/s

Uptime99.98%

QuantizationCQ 4-bit

Input /M$0.30

Output /M$2.50

Cached input /M$0.03

balancedcodinglong-context

Qwen/Qwen3.8-27B

Qwen3 Coder Next

Live

Qwen/Qwen3-Coder-Next

Parameters

—

Max context

262K

A coding-focused Qwen model optimized for software engineering, code generation, and agentic development workflows. Served with ThousandML CQ compression.

ProviderThousandML

Context262K

P50 TTFT—

Throughput—

Uptime—

QuantizationCQ 4-bit

Input /M$0.10

Output /M$0.78

Cached input /M$0.03

codingagentslong-context

Qwen/Qwen3-Coder-Next

DeepSeek V4 Flash

Coming soon

deepseek-ai/DeepSeek-V4-Flash-0731

Parameters

MoE (sparse)

Max context

A fast mixture-of-experts model tuned for reasoning and coding, activating only a fraction of its parameters per token. CQ 4-bit makes frontier-class reasoning practical to serve at open-weight prices. Coming soon — join the waitlist from your dashboard.

ProviderThousandML

Context1M

P50 TTFT—

Throughput—

Uptime—

QuantizationCQ 4-bit

Input /M$0.05

Output /M$0.12

Cached input /M$0.01

moereasoningcoding

deepseek-ai/DeepSeek-V4-Flash-0731

MiMo V2.5

Coming soon

XiaomiMiMo/MiMo-V2.5

Parameters

—

Max context

—

Xiaomi MiMo V2.5 served with ThousandML CQ compression. Coming soon.

ProviderThousandML

Context—

P50 TTFT—

Throughput—

Uptime—

QuantizationCQ 4-bit

Input /M$0.11

Output /M$0.23

Cached input /M$0.00255

reasoningcoming-soon

XiaomiMiMo/MiMo-V2.5

Prices per 1M tokens (USD). 100+ additional models available on request.

Quant Factory / early access

Quantize it. Run it. Decide for yourself.

Quant Factory is our open-source toolkit for turning Hugging Face models into ThousandML CQ 4-bit weights — without calibration data. Fine-tuning for quantized models is coming soon: we are building a secure platform for enterprises to adapt compact models to private data and production workloads.

Explore Quant Factory →Browse ready-made models →

Coming soon / enterprise fine-tuning

Use the open-source workflow today. Soon, enterprises will be able to fine-tune quantized models on private data through the ThousandML platform.

Start experimenting →

CQ 4-bit performance

Qwen3.8 27B, quantized without calibration

WikiText evaluation

92.4%

top-1 agreement with full precision

0.0282

mean KLD at CQ 4-bit

21.3 tok/s

measured throughput in production

No calibration dataset. No calibration forward passes. Quant Factory produces a compact CQ model directly from the weights, then lets you verify the result on your own GPU.

01 / quantize

./quantize-model meta-llama/Llama-3.1-8B-Instruct

02 / build

python build_convert_upload_tq_model_incremental_qwen4exp.py --model meta-llama/Llama-3.1-8B-Instruct --repo-id YOUR_USERNAME/Llama-3.1-8B-Instruct-TQ-4bit --no-upload

03 / install CQ support

cd inference
pip install .

04 / serve

vllm serve YOUR_USERNAME/Llama-3.1-8B-Instruct-TQ-4bit \
--quantization tq

Bring your own model with Quant Factory, or start with one of the ready-to-run commands below. Either way, you get an OpenAI-compatible endpoint at localhost:8000 for testing with your own prompts and workloads.

Qwen3 Coder Next

textclf/Qwen3-Coder-Next-TQ-4bit

docker run --rm --gpus all -p 8000:8000 docker.io/textclf/tq-quant:4bit-main vllm serve textclf/Qwen3-Coder-Next-TQ-4bit --quantization tq

Qwen3.8 27B

textclf/Qwen3.8-27B-TQ-4bit

docker run --rm --gpus all -p 8000:8000 docker.io/textclf/tq-quant:4bit-main vllm serve textclf/Qwen3.8-27B-TQ-4bit --quantization tq

Llama 3.3 70B Instruct

textclf/Llama-3.3-70B-Instruct-TQ-4bit

docker run --rm --gpus all -p 8000:8000 docker.io/textclf/tq-quant:4bit-main vllm serve textclf/Llama-3.3-70B-Instruct-TQ-4bit --quantization tq

Llama 3.1 8B Instruct

textclf/Llama-3.1-8B-Instruct-TQ-4bit

docker run --rm --gpus all -p 8000:8000 docker.io/textclf/tq-quant:4bit-main vllm serve textclf/Llama-3.1-8B-Instruct-TQ-4bit --quantization tq

These are the same CQ quantized weights used behind our hosted inference. Run them on your own GPU today; enterprise fine-tuning for quantized models is coming soon through the ThousandML platform.

Requires Docker, a CUDA-capable GPU, and a local Hugging Face cache. Each command serves an OpenAI-compatible endpoint on port 8000.