BCP·classifying Dutch assignments
The AI side of an EdTech startup: a Dutch T5 that scores every candidate label by how likely it is to write it, fine-tuned on synthetic assignments generated from 386 real programme descriptions.
| Role | AI engineer and Scrum Master in a team of three |
| Built | a startup during the move from Firda to NHL Stenden, September 2024 to July 2025; the AI from May 2025 |
| Model | Dutch T5 v1.1 base, 248M parameters, labels scored by likelihood |
| Training data | 386 scraped programme descriptions and 2,198 generated assignments, 1,904 of them by Reynaerde-7B-Instruct in 4-bit |
| Served model | weight-difference merge of checkpoints 7.5 and 7.7, full precision, behind a Django REST API |
| Stack | Python, PyTorch, Hugging Face Transformers, Django REST Framework, React |
| Assessment | HBO competencies level 1, on the AI part, 11 June 2025 |
The Business Connectivity Portal was built to connect companies, teachers and students through real assignments: a company posts one, a teacher releases it to a class, a student applies. Three of us built it as a startup while moving from Firda to NHL Stenden. I was the Scrum Master and built the AI; my two teammates built the login and security, and the interface, database and school data. When a company types an assignment, a fine-tuned Dutch T5 fills in the education level, the programme, the learning phase and a ranked list of the programme’s learning outcomes. The model, its training data and its serving code are mine, and the AI part was the product for my HBO competencies assessment on 11 June 2025.
A startup between two schools, September 2024 to February 2025
We started in September 2024. The platform has three portals, one each for companies, teachers and students, and by late October we had mock-ups of the main screens. Between November 2024 and January 2025 we presented the idea at the offices of five companies, three of them on 13 November, 26 November and 10 December 2024, and asked each the same questions: how their contact with schools works, how they see MBO and HBO students, and which assignments they would post. As Scrum Master I ran the planning and the contact with the companies.
From November 2024 to February 2025 I wrote the research paper for the AI part. It weighed a hosted API such as OpenAI’s against a model of our own on cost, development time, maintenance and privacy, and recommended starting on a hosted API while building our own model alongside it. In the end the platform ran only our own model, so company texts never went to a third party.
Choosing the model and scoring labels, May 2025
I started building the AI in May 2025. The candidate base models were BERTje (109M parameters) and two 248M-parameter sequence-to-sequence models, the Dutch T5 v1.1 and FLAN-T5. I began with the Dutch T5, cased and pre-trained mainly on Dutch, and later fine-tuned FLAN-T5 alongside it.
The model has no classification head. For every candidate label it computes the loss of generating that label from the assignment text, and a softmax over the negated losses turns them into probabilities. Each candidate costs one forward pass. The candidates are read at request time, so a new programme or a new set of learning outcomes needs no retraining.
A training set that did not exist, May 2025
There were no labelled assignments to learn from. On 14 May I scraped the programme pages of both schools, 228 from Firda and 158 from NHL Stenden, each with its level and length in years. Two small generated sets came first: 180 Firda assignments, and 114 that GEITje-7B-ultra wrote for the Software developer learning outcomes. GEITje’s licence is non-commercial, so from 22 May the generator was Reynaerde-7B-Instruct, a Dutch fine-tune of Mistral-7B under Apache 2.0, loaded in 4-bit on my RTX 3070 with 8 GB. The prompt put it in the seat of a company looking for a student in year X of Y of one programme, gave it the programme description, and told it not to name the level or the programme. It wrote one assignment per programme, and in the later runs one per study year: 1,904 of the 2,198 assignments the model was trained on.
The generator named the answer in the text anyway. A cleaning function deletes every sentence that contains the programme name, the level or the year, strips the greeting and everything from “Met vriendelijke groet” on, and drops unfinished trailing lines, so the classifier has to learn from the content instead of reading the label off the text.
Fine-tuned against the base models, 23 May 2025
The test asked for the programme out of 30 candidates (chance 3.3%), on 64 generated NHL Stenden assignments kept apart from training. The untouched Dutch T5 picked the right programme 7.8% of the time and my fine-tune 7 34.4%: ten times chance and more than four times the base model. The FLAN-T5 fine-tune in the same test reached 21.9%.
From fine-tune 7 to the served model, late May 2025
From fine-tune 7 I trained a chain, 7.1 to 7.7, each run starting from the one before; 7.7 finished on 27 May. That run used 8,613 training examples, one per label of each record, from ten datasets (the two scraped catalogues and the eight generated sets), batch 16 with 8 steps of gradient accumulation, learning rate 3e-6, 3 epochs in bf16. Every fine-tuning run wrote its datasets and all its training arguments into a file beside the weights.
Alongside the chain I tried to make the model smaller. I wrote scripts for post-training int8 quantisation and for quantisation-aware training, and on 26 May I pruned attention heads from 7.6: in every layer, the two of twelve heads with the lowest attention entropy came out. The mean score fell from 68% to 28%. The entropy came from a single five-word sample sentence, far too little to judge a head on. None of it reached the served model, which runs in full precision, and pruning and quantisation went on the list of next steps in my improvement proposal.
The same day I tried extrapolating instead of training: pushing 7.6 one step further along the direction training had moved it from 7.5 gave no gain, and the plot of that run is saved as “Better is not better”. After 7.7 I applied the idea to the end of the chain, and that is the model the API loads: the weights of 7.7 plus the difference between 7.7 and 7.5.
I ranked the checkpoints on a benchmark of 100 questions per category, with up to ten candidates each, drawn from the training files. It measures recall of the training data rather than accuracy on new text, and it separates the checkpoints clearly: fine-tune 6 averaged 34.7%, 7.5 68.1% and 7.7 72.4%. The merge averaged 73.1% and 72.6% over two runs, against 72.4% and 72.3% for plain 7.7. At this sample size that is a tie, and the merge went into the product. A Fisher-weighted merge of 7.7 with it kept a mean of 71.4% only through 100% on the four categories with a few options; on programme and on learning outcomes it fell to 28%.
Into the product, June 2025
On 2 June the Django REST endpoint took the assignment text and a tree of candidates in one request and walked it downwards: level first, then a programme among those at that level, then one of three learning phases, and finally the programme’s learning outcomes, rescaled so the best scores 100% and the worst 0%. The company’s assignment form in React shows the predictions and pre-selects level, school, programme and year, so the company only corrects what is wrong. On the example request in the repository, 314 tokens of text against 25 candidates, a prediction takes about 1 s on my RTX 3070 and 6 to 7 s on a 12-core CPU.
The 4 June demo ran on a small hand-written tree with one programme per level. Seven pytest tests against the endpoint passed on 6 June, and on 10 June I moved the tree into the backend, built from the school database, with the model on the GPU when there is one.
On 11 June 2025 we demonstrated the platform at NHL Stenden in Leeuwarden to examiners, invited companies and guests: a company creates an assignment, the model fills in level and programme, a teacher releases it, a student applies. The same day my HBO competencies were assessed on the AI part, at first-year university-of-applied-sciences level while I was still finishing the MBO. All three of us passed the MBO cum laude, and the HBO certificate gave exemptions from some first-year HBO lessons.
Ready to launch, summer 2025
On 2 July I rewrote the endpoint as a fixed pipeline: the levels come from the schools in the database, the study year from the programme’s own record instead of a prediction, and the form shows a rolling average of the last ten prediction times while it waits. By the summer of 2025 the platform was hosted and ready to launch, with students and companies on the waitlist. It stopped there when the three of us started HBO-ICT at NHL Stenden that September.
This is the fine-tuning counterpart to my language model trained from scratch.