Blog

Articles and technical reports from the projects I contribute to.

· This site

Optimizing KL loss: how to fit long context distillation into your VRAM

An interactive walkthrough of a KL loss kernel that computes exact distillation losses and gradients chunk by chunk, without storing the vocabulary-sized logits: 11.6× less memory and 3.3× faster at 256K tokens.

Iker García-Ferrero. Based on joint work with Bakbergen Ryskulov, David Montero, David Jansen, Ali Hashemi, Jezabel R. Garcia, Antonio Tiene, Román Orús.

· Krea

Releasing Open Weights for FLUX.1 Krea

The training process behind FLUX.1 Krea, an open image-generation model developed with Black Forest Labs, with a focus on aesthetics and photorealism.

Sangwu Lee, Titus Ebbecke, Erwann Millon, Will Beddow, Le Zhuo, Iker García-Ferrero, Liam Esparraguera, Mihai Petrescu, Gian Saß, Gabriel Menezes, Victor Perez

2023 · HiTZ

Did ChatGPT cheat on your test? (2023)

We show popular benchmark datasets are already memorized by ChatGPT and that one can prompt ChatGPT to regenerate them.

Oscar Sainz, Jon Ander Campos, Iker García-Ferrero, Julen Etxaniz, Eneko Agirre