When I was 13, I was still loading programs from cassette tapes and was happy if they managed to fit into 32 KB. In comparison, the availability of data today is enormous: there are now over a million datasets available on Hugging Face for a wide variety of applications. The “salt in the soup” has never been so cheap.
Data Quality and Benchmarks
Although the amount of data is gigantic, the question arises whether there is a benchmark for this data. I was lucky enough to find suitable data and expand it enough to train my first LoRA.
In the process, it became clear: training AI models costs not only time and GPU power, but above all, good data—specifically, data that the base model does not already know.
The Project: From Voice Message to Medical Report
I developed a project that takes audio files (a 1:1 conversation) and creates a transcribed and, ultimately, an anonymized medical report as a PDF. Anyone is welcome to try out the project for training purposes.
The project pipeline looks like this: A Telegram voice message is converted into text via WhisperX and Pyannote (including speaker diarization). This transcribed text is then converted into a structured PTV-3 JSON format using a Qwen2.5-7B model that has been fine-tuned with a LoRA (4-bit QLoRA). The final result is an A4 PDF report delivered via Telegram.
The project was implemented in three stages:
- Stage 1 (Telegram Bot + Transcription): A bot receives voice messages, transcribes them with WhisperX (large-v3), and performs speaker diarization with Pyannote to return a formatted transcript.
- Stage 2 (Synthetic Data Generation): 311 synthetic German samples (280 for training, 31 for validation) were created for the LoRA fine-tuning. This was done via a two-stage synthesis using a local Qwen2.5-27B: first, clinical profiles (diagnosis, demographics, therapy goals) were generated, from which simulated therapy transcripts and the corresponding structured PTV-3 JSON reports were derived.
- Stage 3 (QLoRA Fine-Tuning + Evaluation): A 155 MB LoRA adapter was trained using Unsloth (4-bit QLoRA) based on a Qwen2.5-7B-Instruct model. Evaluation was performed using ROUGE-L, structural checks, Cosine Similarity, and Pydantic.
The training data was trained on an RTX 5090 (32 GB VRAM) for 90 minutes over 3 epochs, achieving a final loss of 0.37 and a Cosine Similarity of 0.92.
The Evolution of Open Models
The world of AI is moving extremely fast. According to a Hugging Face report from the summer of 2026, public model repositories, datasets, and Spaces are growing massively every day. However, this shows an extremely uneven distribution: while the number of datasets has risen to over a million, 99.2% of all downloads are concentrated on just 1.5% of the repositories.
A striking trend in 2026 is the speed of development. While laboratories used to often start with small models and work their way up, Chinese labs often skip this process entirely. The most powerful open-source models from China significantly outperform US models in many months. While US models often remain under 130B parameters, the capacities of Chinese labs range between 754B and 2.78 trillion parameters.
The role of hardware manufacturers is also interesting: companies like NVIDIA and AMD release massive amounts of new models to promote their chips. An optimized, freely available model is the best proof that the hardware works.
An important distinguishing feature is the discrepancy between attention and actual usage: “Likes” reflect the excitement for new releases, while “Downloads” show what is actually built into developers’ pipelines. While large models often receive many likes, it is the small, stable models that generate the highest download numbers over many years.