Programme operations

How our programme team used AI to score 652 applicants in under 45 minutes

Our own AI platform, Agent-E, gave 652 applicants to a regional youth summit the same careful first read, in an afternoon. People set the criteria and made every selection.

Published 13 November 2026 · 7 min read

652applicants scored against every criterion
44.7 minto score the full pool
USD 2.70in AI model costs, under half a US cent per applicant

In early October, applications closed for a regional youth leadership summit that Biji-biji Initiative runs across ASEAN. Helen, the programme director, had planned for about 800 applicants. 869 applied.

A panel of 11 reviewers, one per ASEAN country, was waiting to make the final selection, and Helen needed a scored list by Wednesday to prepare it for them. On Monday afternoon, our tech team ran the eligible applicants through Agent-E, the applicant evaluation platform we built in-house. It scored all 652 against the programme’s criteria in 44.7 minutes, for USD 2.70 in AI model costs. Helen had the results that evening.

Here is how it worked, how it gave every applicant a fairer read, and where people stayed in charge.

Key takeaways

  • Let AI do the first read, and keep the decision with people. The panel still chose every participant.
  • Have a person approve the criteria before anything is scored, and test them on a handful of applicants first.
  • Measure the cost of each run. At under a cent per applicant, cost is no reason to hold back.
  • Score every applicant the same way, with the reasons written down. It is fairer to applicants and faster for the team.
  • Compare the AI’s scores with your own reviewers. The gap shows what experience your team carries that the AI does not have yet.

The problem

Every Mereka and Biji-biji Initiative programme recruits by open application, and screening has always been done by hand. Programme managers export the form responses to a spreadsheet and read them one at a time. Adibah, who screens for another programme, put it simply in August:

“currently I’m the one doing the screening manually”Adibah, programme team

That works for a few dozen applicants. At several hundred it strains.

  • Volume arrives late. Helen’s count went from 680 to 869 in the last 24 hours or so before closing, so the reading has to happen in a short window before the panel meets.
  • Criteria drift. Each reviewer carries their own reading of the criteria, and it is hard to read the 600th application the way you read the first.
  • Unclear answers get rejected. When an answer is ambiguous and time is short, the easy call is to drop the applicant. Those unclear answers can come from exactly the people a programme wants to reach, such as applicants writing in their second language.

That is a problem for applicants as much as for the team. Two people with similar answers can get different outcomes depending on who read them, and when. Applicants never see how they were judged, and the ones most likely to be dropped on a tired afternoon are the ones whose answers needed a closer read.

The solution

Agent-E is a platform our tech team built to give programme managers a first layer of evaluation. Gurpreet leads its engineering, and Miranda worked with Helen’s team on this cohort. The platform runs on OpenAI models for reading and scoring. The team built it with help from AI coding agents (Claude Code and Codex), which co-wrote about half of its code changes.

For this cohort, the work ran in six steps.

  1. Start from the programme’s own documents

    Miranda uploaded the summit proposal and the application strategy document. Agent-E drafted a scoring plan from them: the criteria, their weights and the eligibility rules.

  2. A person approves the plan before anything is scored

    The platform blocks all scoring until someone signs off the plan, and the plan went to Helen for approval. When an early draft could not read rules hidden inside Word form fields, the system stopped and said so rather than build a plan from half a document.

  3. Handle the messy eligibility question by hand

    One eligibility question asked about past programme participation, and applicants had answered it in many different ways. Helen filtered that question herself, which took the pool from 869 to 652. Agent-E checked the remaining eligibility rules, with citizenship as the deciding field for the country.

  4. Calibrate on a handful first

    Before the full run, Agent-E scored a small sample so the plan could be corrected cheaply if the scores looked wrong.

  5. Score everyone the same way, with evidence

    The full run scored all 652 applicants against every criterion. Each score comes with the reasoning behind it and the text from the application it drew on, so a reviewer can see why someone scored as they did. Every applicant gets the same careful read, whether they were the first in the pile or the 652nd.

  6. Send unclear cases to a person

    Agent-E does not guess. When it could not decide, it marked the applicant “undetermined” or flagged them for review instead of rejecting them. In this cohort that was 25 undetermined on eligibility and 7 flagged for review, all passed to Helen.

Helen received the results as a spreadsheet, with a tab grouping applicants by country for each panellist. The panel then made the selection, as it always had.

The results

Exhibit 1The October run in numbers
MetricResult
Applications received869
Applicants scored by Agent-E652, every one against every criterion
Time to score the full pool44.7 minutes
AI model cost for the full runUSD 2.70 (under half a US cent per applicant)
Applicants passed to a person instead of rejected32 (25 undetermined, 7 flagged)
When the results reached the programme directorThe same evening, two days before her deadline
Final selectionMade by a panel of 11 country reviewers

Source: Mereka’s tech team, November 2026

The whole project, including calibration and re-scores, cost USD 3.32.1

The bigger difference is what Helen’s time went on. Instead of reading 652 applications, she worked through the flagged cases and prepared the panel’s pack. Her check also caught two applicants that Agent-E had marked for verification and who were in fact eligible. We re-scored them the same day.

For applicants, the difference is fairness. All 652 were read against the same criteria, with the reasons written down, and the 32 whose answers were unclear went to a person instead of being dropped.

Helen’s verdict after the run was modest and honest:

“i think it’s going pretty smooth. or maybe because you’ve been doing it mostly for us.”Helen, programme director

For this first cohort, Miranda ran most of the platform for the programme team. That is the next thing to change.

Lessons and what’s next

Some of the work was not suited to AI, and that is fine. The free-text eligibility question was faster for Helen to sort by hand than to write rules for. We also added an AI-writing detector, but it was not reliable enough: it showed “unverified” for most applicants. So we moved it to a separate tab with a warning not to rely on it, and kept it out of the score.

Since this run, we have compared Agent-E’s scores with a manual review, to check what it scored and where it could improve. Helen’s team of seven reviewers went through the applications by hand, and it took them days. Agent-E had done its first read in under 45 minutes. The pattern was clear: Agent-E tends to score applicants higher than our reviewers do.

The reason is what each side knows. Agent-E judges only what is in the documents we give it: the programme proposal, the criteria and the application itself. Our reviewers have read applications for past programmes. They know the programme’s background, and they already know what a good answer looks like and what a weak one looks like. None of that is written down, so Agent-E cannot use it.

So the next step is to close that gap. We will work with our reviewers to capture what they know, such as examples of strong and weak applications from past cohorts, and refine the scoring until Agent-E’s results sit closer to how our reviewers read. Their judgement stays the final word; the aim is a first read that matches it.

We also want to take Agent-E further. Next, we plan to use it for our own company recruitment, and to connect it to our LMS so that every shortlisted applicant can be enrolled straight into their programme. And we still want programme managers to run evaluations themselves, without the tech team operating the platform. Other programme teams are now looking at it for their next intakes.

Sources and methods

1 Mereka’s tech team, November 2026

Figures come from Mereka’s tech team: Agent-E’s records of the October run (applicants scored, time taken and AI model costs), the programme team’s applicant counts and the later comparison with the reviewers’ manual scores. Quotes are from the teams’ working messages, as written.

Get started

Book a free discovery call

  1. Step 1

    We learn your goals

    In 30 minutes: your team, your tools and what you want AI to change.

  2. Step 2

    We recommend and quote

    The right programme and format, in a tailored proposal.

  3. Step 3

    We handle HRD Corp

    We prepare the HRD Corp documents and claim from your levy, so there is no deposit. Paying another way? We set it up with you.

Prefer to message? Email ailab@mereka.io or WhatsApp +60 11-3011 6472

HRD Corp claimable