Articles and technical reports from the projects I contribute to.
We study how to train LLMs to refuse harmful requests within a topic while still answering legitimate ones. Self-generated training data and harmful-benign prompt pairs help control that boundary, and evaluating both sides reveals the trade-off between safety and over-refusal.
Recovering compressed, 4-bit LLMs by distilling directly from the original model. We examine accuracy, training stability, and the cost of recovery.
How offline teacher logits and a fused KL loss make distillation more efficient in memory and compute, with benchmarks and an open-source implementation.
The training process behind FLUX.1 Krea, an open image-generation model developed with Black Forest Labs, with a focus on aesthetics and photorealism.
We present GoLLIE, a Large Language Model trained to follow annotation guidelines. It outperforms previous approaches on zero-shot IE and supports schemas defined on the fly.
We show popular benchmark datasets are already memorized by ChatGPT and that one can prompt ChatGPT to regenerate them.