Quick Answer: Track five AI job portal metrics quarterly: quality of hire score, time-to-fill vs. time-to-hire, cost-per-hire, source of hire with candidate quality, and interview-to-shortlist conversion. Each number is only useful if it comes with a documented reason. A score without justification tells you what happened, not why, or whether to trust it.
A hiring manager in Pune pulls up her Q2 dashboard. 40 candidates screened, 12 shortlisted, 3 hired. Looks fine on its own. What it doesn't show is whether the AI tool that screened those 40 people actually predicted who'd perform well, or whether it just spat out a ranking she was told to trust and move on.
That's the real problem with most AI job portal metrics reporting today. The metrics exist. Nobody can explain what produced them. If you're posting roles through an AI-powered platform, here are the five worth reviewing every quarter, and each one needs a documented "why" attached, not just a number sitting on a dashboard looking authoritative.
Quality of Hire Score: The Metric Everyone Tracks Wrong
Quality of hire usually gets measured through 90-day retention, manager satisfaction ratings, or performance review scores months after someone joins. Useful, sure. But it's backward-looking. By the time you know a hire was poor quality, the role has already cost you a quarter or more of your time and theirs.
The earlier signal is the interview data itself. On Einstellen.AI, every interview produces a per-answer scored report with a full transcript, built on the MAGIC model — an evaluation framework designed to produce structured, justified scoring rather than an opaque number.
That means quality of hire can start getting measured at the interview stage, not just after 90 days. If a candidate scores well, a hiring manager can see exactly which answers drove that score. If a hire later underperforms despite a strong interview score, you can go back to the transcript and check whether the interview even asked the right questions, instead of shrugging and blaming "the algorithm."
Most AI recruitment vendors gesture at fairness and quality tracking without actually operationalizing it. Raghavan, Barocas, Kleinberg, and Levy's peer-reviewed study, presented at the ACM Conference on Fairness, Accountability, and Transparency (2020), found that bias-mitigation claims from commercial hiring algorithm vendors were often unsubstantiated, with vendors frequently failing to disclose meaningful technical documentation or empirical fairness evidence. Quality of hire tracking is only as good as the transparency behind the score that fed into it in the first place.
Read More: 7 Questions to Ask Before Choosing an AI Hiring Platform in India
Time to Fill vs. Time to Hire: Two Different Clocks
People use these interchangeably. That's a mistake, and it hides where your actual bottleneck sits. Time-to-fill is the full clock: the day the role opens to the day someone accepts. Time-to-hire is narrower: from when a candidate first applies or gets sourced to offer acceptance.
Long time-to-fill but short time-to-hire? Your bottleneck is upstream. The role sat unposted, or the JD wasn't pulling the right applicants. Long time-to-hire? The bottleneck is inside your process. Too many interview rounds, slow feedback loops, or a screening step that isn't actually filtering anyone out.
Adaptive AI interviewing compresses time-to-hire directly because it swaps multiple screening rounds for one conversation that adjusts to what the candidate actually says. humAIn, Einstellen.AI's interview engine, generates its next question based on the candidate's last answer rather than working through a fixed script no matter what was just said. A single adaptive interview round can pull out signal that would otherwise take two or three separate human screening calls to surface. Time-to-hire drops without cutting rigor.
Track both clocks separately, every quarter. A shrinking time-to-fill with a flat time-to-hire means your sourcing improved. The reverse means your interview process is the drag.
Cost Per Hire on an AI Job Portal: Why Per-Seat Pricing Distorts This Number
The standard formula is simple enough: total internal and external recruiting costs divided by the number of hires in the period. Where this falls apart for AI hiring tools specifically is that most vendors price by seat, subscription tier, or negotiated contract. That makes the "cost" side of the formula murky, and different for every company using the supposedly same tool.
Einstellen.AI runs flat, pay-per-use pricing: ₹249 per interview, same rate whether a client runs one interview or ten thousand, no subscription, no contract, no volume negotiation. The full report, video, transcript, skill scoring, and fraud detection are included, comes at that same rate with no add-on tier tacked on.
That flat rate is what makes cost-per-hire benchmarking genuinely comparable across quarters and across roles. The unit cost doesn't shift depending on how much you use the platform or what tier someone negotiated for you last year. Most AI interview competitors in this category are demo-gated with negotiate-only pricing, which means two companies using the "same" tool could be paying entirely different effective rates. Quarter-over-quarter cost tracking becomes guesswork.
Source of Hire and Candidate Quality Tracking
Source of hire tells you which channel a candidate came from. Source quality tells you whether that channel actually produced people worth interviewing. Most job boards can report the first number fine. Very few connect it to the second, because a static CV database has no idea which of its listed candidates are still actually looking.
Einstellen.AI is built around active candidates and active listings rather than a static repository. That means source-of-hire tracking reflects current job-seeking activity, not stale profiles nobody's touched in months. Pair this with structured, justified interview scores and a hiring manager sees more than "we hired 4 people from Source A this quarter" — she sees whether Source A's candidates consistently scored well on the specific skills the role needed, with the reasoning to back it up.
Review this every quarter, by role, not just in aggregate. A source that performs well for QA hiring might perform poorly for senior data engineering. Blend the two numbers, and you'll never notice.
Interview-to-Shortlist Conversion: The Portal-Specific Metric Nobody Tracks
Most competitor content skips this one entirely, since it's specific to portals rather than internal ATS reporting. The question it answers: of everyone who completes an AI interview on your job posting, what percentage actually makes it to a human shortlist review?
A low ratio might mean your screening bar is too loose, generating volume without quality behind it. A very high ratio might mean the AI interview isn't discriminating enough between candidates, especially if it's running a fixed script that never probes deeper on weak or vague answers. Adaptive interviewing addresses this head-on: if a candidate gives a vague answer about a specific project, the system probes further on that exact project rather than jumping to an unrelated scripted question. That produces real separation between genuinely strong and genuinely weak candidates, instead of everyone clustering in the middle.
Track this ratio quarterly alongside fraud/proxy detection flags, especially for high-volume hiring rounds like bulk campus recruitment, where interview integrity at scale matters just as much as the conversion number itself.
Metrics Reference Table
| Metric | What It Measures | Review Cadence |
|---|---|---|
| Quality of hire score | Interview-stage signal + post-hire performance | Quarterly + at 90 days |
| Time-to-fill vs. time-to-hire | Sourcing bottleneck vs. process bottleneck | Quarterly |
| Cost-per-hire | Total recruiting cost ÷ hires, at a comparable unit rate | Quarterly |
| Source of hire + quality | Which channels produce hires who actually score well | Quarterly, by role |
| Interview-to-shortlist ratio | Screening bar accuracy on the portal itself | Quarterly |
Proof point: Einstellen.AI's MAGIC model has produced 30,000+ AI interviews with per-answer scored reports and full transcripts, across 1,200+ institutions, so a quality-of-hire score can be traced back to the specific answer that drove it. Not just accepted as a number and moved on from.
FAQ
What metrics should I track when using an AI hiring platform?
Track quality of hire, time-to-fill separately from time-to-hire, cost-per-hire at a comparable unit rate, source of hire paired with candidate quality, and interview-to-shortlist conversion. The common failure isn't missing metrics. It's tracking numbers without the reasoning behind them, which makes quarter-over-quarter comparison basically meaningless.
How do you measure quality of hire with AI recruiting tools?
Combine post-hire signals (90-day retention, manager satisfaction) with interview-stage signals. A structured, justified interview score, one that shows exactly which answers drove which part of the result, lets you start measuring quality before the hire even happens. No need to wait three months to find out you got it wrong.
How can employers tell if an AI interview tool is biased?
Ask for the structured justification behind individual scores, not just an aggregate fairness statement someone's marketing team wrote. Raghavan et al.'s 2020 peer-reviewed study found that many vendors' bias-mitigation claims lack disclosed technical documentation or fairness evidence. A platform that shows the specific reasoning behind each score, rather than an unexplained ranking, actually gives you something to audit.
What is cost-per-hire and how does AI affect it?
Cost-per-hire is total internal and external recruiting spend divided by hires in the period. AI interview tools affect this two ways: they can cut cost by reducing screening rounds, but opaque per-seat or negotiated pricing can also distort the number, and comparisons across quarters stop being reliable unless the underlying rate is flat and public.
Post Your Role and Track Metrics That Explain Themselves
If your current quarterly report hands you numbers without reasoning, you're not tracking AI job portal metrics. You're just collecting them. Einstellen.AI pairs every interview score with a structured justification report and full transcript, at a flat ₹249 per interview, no contract, no volume negotiation, so your cost-per-hire and quality-of-hire numbers stay comparable quarter over quarter.
ATS integration, including Greenhouse, Lever, and Workday, is included at no additional cost. Your existing recruiter workflow doesn't need to change to start tracking metrics that actually explain themselves.
Post a role on Einstellen.AI and see the first quarterly scorecard for yourself.
Leave a Comment
Share your perspective. Comments are moderated before publishing.