In early October, applications closed for a regional youth leadership summit that Biji-biji Initiative runs across ASEAN. Helen, the programme director, had planned for about 800 applicants. 869 applied.
A panel of 11 reviewers, one per ASEAN country, was waiting to make the final selection, and Helen needed a scored list by Wednesday to prepare it for them. On Monday afternoon, our tech team ran the eligible applicants through Agent-E, the applicant evaluation platform we built in-house. It scored all 652 against the programme’s criteria in 44.7 minutes, for USD 2.70 in AI model costs. Helen had the results that evening.
Here is how it worked, how it gave every applicant a fairer read, and where people stayed in charge.
Key takeaways
- Let AI do the first read, and keep the decision with people. The panel still chose every participant.
- Have a person approve the criteria before anything is scored, and test them on a handful of applicants first.
- Measure the cost of each run. At under a cent per applicant, cost is no reason to hold back.
- Score every applicant the same way, with the reasons written down. It is fairer to applicants and faster for the team.
- Compare the AI’s scores with your own reviewers. The gap shows what experience your team carries that the AI does not have yet.
The problem
Every Mereka and Biji-biji Initiative programme recruits by open application, and screening has always been done by hand. Programme managers export the form responses to a spreadsheet and read them one at a time. Adibah, who screens for another programme, put it simply in August:
“currently I’m the one doing the screening manually”Adibah, programme team
That works for a few dozen applicants. At several hundred it strains.
- Volume arrives late. Helen’s count went from 680 to 869 in the last 24 hours or so before closing, so the reading has to happen in a short window before the panel meets.
- Criteria drift. Each reviewer carries their own reading of the criteria, and it is hard to read the 600th application the way you read the first.
- Unclear answers get rejected. When an answer is ambiguous and time is short, the easy call is to drop the applicant. Those unclear answers can come from exactly the people a programme wants to reach, such as applicants writing in their second language.
That is a problem for applicants as much as for the team. Two people with similar answers can get different outcomes depending on who read them, and when. Applicants never see how they were judged, and the ones most likely to be dropped on a tired afternoon are the ones whose answers needed a closer read.
The solution
Agent-E is a platform our tech team built to give programme managers a first layer of evaluation. Gurpreet leads its engineering, and Miranda worked with Helen’s team on this cohort. The platform runs on OpenAI models for reading and scoring. The team built it with help from AI coding agents (Claude Code and Codex), which co-wrote about half of its code changes.
For this cohort, the work ran in six steps.
Start from the programme’s own documents
Miranda uploaded the summit proposal and the application strategy document. Agent-E drafted a scoring plan from them: the criteria, their weights and the eligibility rules.
A person approves the plan before anything is scored
The platform blocks all scoring until someone signs off the plan, and the plan went to Helen for approval. When an early draft could not read rules hidden inside Word form fields, the system stopped and said so rather than build a plan from half a document.
Handle the messy eligibility question by hand
One eligibility question asked about past programme participation, and applicants had answered it in many different ways. Helen filtered that question herself, which took the pool from 869 to 652. Agent-E checked the remaining eligibility rules, with citizenship as the deciding field for the country.
Calibrate on a handful first
Before the full run, Agent-E scored a small sample so the plan could be corrected cheaply if the scores looked wrong.
Score everyone the same way, with evidence
The full run scored all 652 applicants against every criterion. Each score comes with the reasoning behind it and the text from the application it drew on, so a reviewer can see why someone scored as they did. Every applicant gets the same careful read, whether they were the first in the pile or the 652nd.
Send unclear cases to a person
Agent-E does not guess. When it could not decide, it marked the applicant “undetermined” or flagged them for review instead of rejecting them. In this cohort that was 25 undetermined on eligibility and 7 flagged for review, all passed to Helen.
Helen received the results as a spreadsheet, with a tab grouping applicants by country for each panellist. The panel then made the selection, as it always had.
The results
| Metric | Result |
|---|---|
| Applications received | 869 |
| Applicants scored by Agent-E | 652, every one against every criterion |
| Time to score the full pool | 44.7 minutes |
| AI model cost for the full run | USD 2.70 (under half a US cent per applicant) |
| Applicants passed to a person instead of rejected | 32 (25 undetermined, 7 flagged) |
| When the results reached the programme director | The same evening, two days before her deadline |
| Final selection | Made by a panel of 11 country reviewers |
Source: Mereka’s tech team, November 2026
The whole project, including calibration and re-scores, cost USD 3.32.1
The bigger difference is what Helen’s time went on. Instead of reading 652 applications, she worked through the flagged cases and prepared the panel’s pack. Her check also caught two applicants that Agent-E had marked for verification and who were in fact eligible. We re-scored them the same day.
For applicants, the difference is fairness. All 652 were read against the same criteria, with the reasons written down, and the 32 whose answers were unclear went to a person instead of being dropped.
Helen’s verdict after the run was modest and honest:
“i think it’s going pretty smooth. or maybe because you’ve been doing it mostly for us.”Helen, programme director
For this first cohort, Miranda ran most of the platform for the programme team. That is the next thing to change.
Lessons and what’s next
Some of the work was not suited to AI, and that is fine. The free-text eligibility question was faster for Helen to sort by hand than to write rules for. We also added an AI-writing detector, but it was not reliable enough: it showed “unverified” for most applicants. So we moved it to a separate tab with a warning not to rely on it, and kept it out of the score.
Since this run, we have compared Agent-E’s scores with a manual review, to check what it scored and where it could improve. Helen’s team of seven reviewers went through the applications by hand, and it took them days. Agent-E had done its first read in under 45 minutes. The pattern was clear: Agent-E tends to score applicants higher than our reviewers do.
The reason is what each side knows. Agent-E judges only what is in the documents we give it: the programme proposal, the criteria and the application itself. Our reviewers have read applications for past programmes. They know the programme’s background, and they already know what a good answer looks like and what a weak one looks like. None of that is written down, so Agent-E cannot use it.
So the next step is to close that gap. We will work with our reviewers to capture what they know, such as examples of strong and weak applications from past cohorts, and refine the scoring until Agent-E’s results sit closer to how our reviewers read. Their judgement stays the final word; the aim is a first read that matches it.
We also want to take Agent-E further. Next, we plan to use it for our own company recruitment, and to connect it to our LMS so that every shortlisted applicant can be enrolled straight into their programme. And we still want programme managers to run evaluations themselves, without the tech team operating the platform. Other programme teams are now looking at it for their next intakes.
Sources and methods
1 Mereka’s tech team, November 2026
Figures come from Mereka’s tech team: Agent-E’s records of the October run (applicants scored, time taken and AI model costs), the programme team’s applicant counts and the later comparison with the reviewers’ manual scores. Quotes are from the teams’ working messages, as written.




