CodeSignal's AI Interviewer: What It Records and Scores

A CodeSignal AI Interviewer session is a short structured interview conducted by a configurable software agent rather than a person. It asks predefined questions, probes with follow-ups that stay inside competencies the employer set in advance, and produces a transcript plus a rubric-scored report that lands in the employer's applicant tracking system. CodeSignal's own product page states the outcome plainly: "Humans review and make the decision—always" (CodeSignal AI Interviewer, retrieved 6 September 2026).

That is a different assessment from the CodeSignal most engineers have already met. If you were sent a link and are not sure whether a human, a timer, or an agent is waiting on the other side, start by identifying the surface — the preparation for each is genuinely different.

Which CodeSignal surface were you sent?

CodeSignal sells several candidate-facing products under one brand. The invitation, not the domain name, tells you which one you have.

The middle row is the one candidates most often misread. CodeSignal launched AI-assisted coding assessments on 28 May 2025, embedding its Cosmo assistant into the hiring product in either a full co-pilot mode or a guided mode limited to syntax and documentation, and hiring teams receive "session replays" and "transcripts of all AI interactions" (PR Newswire, 28 May 2025). Being handed a model inside the assessment is not a general license to bring your own; it is a narrower permission whose boundaries are set per assessment. If you are in that format, what AI-assisted coding interviews actually grade covers how the prompt transcript is read.

The first row is the classic timed test, covered separately in the CodeSignal General Coding Assessment format and rules guide. Everything below concerns the last row.

What the AI Interviewer actually is

CodeSignal announced Interviewer Agents on 12 August 2025 as agents that run role-specific interviews end to end, ask adaptive follow-ups, and convert the conversation into structured scorecards delivered to the ATS (CodeSignal press release, 12 August 2025). CEO Tigran Sloyan framed the motivation as volume: "With applications at record highs, recruiters need a way to evaluate more candidates without stretching their teams thin."

Several configuration details matter to a candidate, and they come from CodeSignal's administrator documentation rather than its marketing.

The employer chooses the interaction mode. CodeSignal's guide to creating a custom AI interviewer, updated 9 July 2026, describes selecting an agent type — Realtime, or speech-to-text-to-speech — choosing whether candidates respond by voice, text, or both, adjusting interruption sensitivity, and entering free-text instructions that steer the interviewer's behavior, with a separate evaluation panel where the response rubric is defined (CodeSignal Knowledge Base, updated 9 July 2026).

Two practical consequences follow. Interruption sensitivity means a pause of yours can be read as a finished answer, so a long silent think can end your turn early. And because the interviewer's behavior is written as instructions against a rubric, its curiosity is bounded: the product page says follow-ups "stay within the competencies you define." An interesting tangent that a human interviewer would chase is likely to be dropped.

The default session is short. CodeSignal states that its AI Interviewer runs "15 minutes, but fully configurable," and that candidates can take it by phone call or in a web browser. A session can also carry coding work: CodeSignal's setup documentation for AI Interviewer assessments, updated 30 April 2026, notes that coding questions are worth 300 points by default and recommends 2 or 3 coding questions within a 90-minute assessment (CodeSignal Knowledge Base, updated 30 April 2026). A "CodeSignal AI interview" can therefore be a 15-minute phone screen or a 90-minute technical session, and the invitation is the only reliable indicator of which you have.

CodeSignal also ships pre-built agents with human names and job specialisms — Matt Thompson for sales, Emily Prescott for recruiting, Simone Clarke for product, among others. The name in your invitation may be an agent, not a person.

What is recorded and who receives it

The transcript is the artifact. CodeSignal states that AI Interviewer reports contain a "full transcript with rationale" alongside an overall score, skill evaluation, and key insights, and that transcripts "let you see exactly what was asked." Read that from the candidate's side: every hedge, every restarted sentence, and every question you asked back is preserved verbatim for a reviewer who was not in the room.

Proctoring is a separate layer. CodeSignal's proctoring documentation, updated 3 September 2026, describes capturing the camera feed, microphone audio, and a screen recording for the duration of an evaluation, along with a facial photo and government-issued photo ID. It also states that the company requesting the assessment receives "a copy of your assessment result and score," that proctoring data itself is reviewed by CodeSignal's internal verification team rather than handed to the employer, and that identification and proctoring information is stored and deleted within 15 days (CodeSignal Knowledge Base, updated 3 September 2026).

That retention and separation claim is unusual and worth knowing, but note its scope: it describes CodeSignal's proctoring and verification process, and whether proctoring is enabled at all is a per-assessment setting. It does not tell you what a specific employer configured, and it says nothing about how long the transcript and scorecard live inside that employer's ATS. If retention matters to you, the employer — not the platform — is the party to ask, and ask before the session rather than after.

How the scoring is meant to work, and what remains a vendor claim

CodeSignal describes a calibration process in which client reviewers score the same interviews the agent scored, an industrial-organizational psychology team compares and tunes the two, and the rubric is then locked for consistency. The product page also claims interviews are "fairness-tested, and audit-ready," with logs kept for audit and support for adverse impact studies.

These are vendor descriptions of an internal process. They are plausible and specific, which is more than most of the category offers, but no published methodology, sample size, or independent audit accompanies them. The same caution applies to the adoption statistics CodeSignal publishes: its blog claims that more than 75% of candidates who used a leading AI interview solution said they prefer being interviewed by an AI recruiter, and reports a nearly 20% increase in 30-day retention, neither with a cited study or sample. Treat those as marketing until someone shows the data.

Its federal work is documented more concretely. CodeSignal's OPM customer story, describing work since 2025 on the government's Tech Force recruitment initiative, reports that roughly 80% of candidates passing its evaluations advanced to screening interviews and about 55% of those passed the subsequent technical skills interview before referral to hiring managers (CodeSignal, U.S. Office of Personnel Management customer story). Those figures describe skills assessments, not agent-run interviews.

The federal rollout that put this in the news

On 3 September 2026, CBS News reported that the federal government is beginning to use CodeSignal's AI agents to conduct interviews for Tech Force, the program launched roughly eight months earlier to recruit private-sector technologists into two-year government roles. According to that reporting, the agents run structured interviews with standardized questions and "will record and summarize the audio so hiring managers can view and evaluate later," with the Office of Personnel Management signaling that broader adoption across agencies is likely. OPM had issued guidance calling for human oversight in crucial personnel decisions, and a spokesperson did not immediately comment on the rollout (CBS News, 3 September 2026).

That is reporting from unnamed sources describing an imminent change, not a confirmed program with published rules. What it does establish is direction: the agent screen is moving from high-volume commercial roles into hiring pipelines that were previously conservative, including for engineering positions.

What preparation actually changes here

Most advice about "beating an AI interview" assumes the agent can be charmed. It cannot; it can only be given evidence that maps to a rubric it already holds. Four adjustments do more than anything else.

Answer in retrievable units. The scorer is a transcript reader. Name the situation, the action you personally took, and the measurable result in the first three sentences, then add detail. An answer whose point arrives at sentence nine may be scored on the first eight.

Finish sentences deliberately. Because interruption sensitivity governs turn-taking, a trailing pause mid-thought can hand the turn back. If you need to think, say that you are thinking rather than going silent.

Map to the job description, not to the agent. Follow-ups stay inside predefined competencies, which are usually drawn from the posting. Prepare two concrete examples for each named competency and you will have something to say to almost any probe the agent is permitted to make.

Test the channel before the session. Voice mode over a bad microphone becomes a bad transcript, and the transcript is the evidence. If the invitation offers a phone option and your audio setup is unreliable, take the phone.

Verify the tool rules before assuming anything about what you may have open. Availability is not permission, and an interview run by an agent is still a closed evaluation unless the employer says otherwise; how to check whether AI use is permitted in a technical interview sets out where that permission is actually documented.

Where this format is weak

An agent screen is bounded by its rubric in both directions. It will not recognize strong evidence outside the competencies it was given, it cannot tell you anything real about the team or the codebase, and it has no discretion to advance someone who is interesting for reasons nobody anticipated. Candidates whose strongest case is unusual — a career change, a domain transfer, an unconventional project — are structurally disadvantaged by a format that rewards legible matches to a predefined list.

The reasonable response is not to game the transcript but to make the legible case in the session and route the unusual one to a human, through the recruiter, in writing, where a person can weigh it.

If you want to practice before the session, the closest useful rehearsal is a scored one: running an AI mock interview against a written rubric reproduces the part that actually matters here, which is speaking a complete, evidence-bearing answer inside a short window without a human to read your pauses.