Hikari has launched a free, open library of AI skills for recruiters – and is publishing the results of its bias testing, including the skills that fail.
The Hikari Skills Lab was founded by recruiter Will Grady, who has worked in-house at companies including Google, Meta, Amazon, Shell and BP. The library is designed for use with ChatGPT and Claude, and Hikari says it will remain free for recruiters.
Testing AI with a “swap test”
Hikari’s method changes one characteristic at a time in a fictional candidate profile – such as name, gender or age – while keeping every other detail identical. The aim is to identify whether an AI tool’s scoring or output changes when it should not.
Across 220 swaps using 10 fictional CVs in ChatGPT and Claude, the average score change was 0.12 points and 83% of swaps produced no score movement, according to Hikari. The tests are published alongside the skills, together with the models used and dates of the runs.
“Everyone in AI hiring says their tool is fair. Almost none of them show you the test.”
Grady said: “I test mine in the open and publish the results, including the one that failed. If we want recruiters to trust AI, that’s the only way it works.”
Will Grady, Founder, Hikari Skills Lab
A failed test is published, not hidden
Two skills – an evidence-based CV screen and an inclusive job-advert check – passed the tests described by Hikari. A third, a shortlist write-up skill, did not receive a badge after failing a name-swap test in Claude.
In a larger test of 20 new CVs and 440 write-ups on 22 October, ChatGPT passed the stated tests. Claude passed the gender and age tests, but on identical CVs its name-swap result gave Siobhan O’Connell 4.55 out of 10 and Mohammed Al-Rashid 3.95 out of 10. Hikari says the difference was unlikely to be due to chance (p = 0.004), so the skill remains clearly marked without a badge.
The organisation stresses that a pass applies only to the CVs, models and date tested – not every possible model or candidate – and that a person should make every hiring decision.
Open testing for recruiter tools
Hikari says its approach is intended to make AI-assisted recruitment more transparent, giving recruiters access to usable methods and a clearer view of where they can and cannot rely on AI output.
Submitted by Will Grady, Hikari
Methodology
Hikari tested 10 fictional CVs through 220 swaps across ChatGPT and Claude for its initial screening test. Its published method uses a noise baseline, sets pass limits before testing and records results for each swap. The larger shortlist write-up retest used 20 new CVs and 440 write-ups.
