Skip to content

Model fine-tuning and quantization

A fine-tuned checkpoint is only useful if it fits your serving budget. We treat fine-tuning and quantization as one pipeline, and we publish the quantized results openly.

LoRA / QLoRA / Full

Methods

GGUF / FP8 / GPTQ / AWQ

Formats

IQ1_S → Q8_0

Levels

22 repositories

Published

A single node open on the bench under one hard lamp, accelerator cards seated in a row

From base model to a checkpoint you can actually serve

  • Dataset preparation review: formatting, deduplication and train/eval splits before a single GPU-hour is spent

  • LoRA, QLoRA or full-parameter fine-tuning, chosen against your budget and quality target

  • Quantization to the format your serving stack takes — imatrix GGUF for llama.cpp, FP8, GPTQ or AWQ for vLLM and SGLang — with a quality delta report, not just a smaller file

  • Evaluation against your task set so the regression is measured rather than assumed

  • A checkpoint packaged for the serving stack you actually run

Before you ask

Which quantization formats do you produce?
Four, and all four are published rather than described. GGUF with importance-matrix calibration is what llama.cpp-based local deployment needs, and calibrated quantization holds up far better at low bit widths than a naive conversion. FP8, GPTQ and AWQ cover GPU serving under vLLM and SGLang. Every Qwen3 size we publish exists in all four, so you can compare them on your own hardware. Every conversion ships with a measured quality delta against the source checkpoint.
Should I use LoRA or full-parameter fine-tuning?
LoRA and QLoRA cover most domain-adaptation work at a fraction of the cost and produce adapters you can swap at serving time. Full-parameter training is worth it when you are changing model behaviour broadly rather than adding domain knowledge. We size this against your eval targets before starting.
Do you publish the quantized models?
Yes, twenty-two repositories. Qwen3 at 4B, 8B, 14B, 30B-A3B and 32B in all four formats, with the GGUF ladder running seven levels from Q2_K to Q8_0. Kimi-K3 and DeepSeek-V4-Flash are published as GGUF down to IQ1_S. You can download any of them and check the work before commissioning any. Customer checkpoints stay private and are delivered directly to you.

Other workloads