Building an Interview Scorecard That Catches a Bad Executive Hire
A scorecard with generic competencies like "strategic thinking" and "leadership" gives every interviewer a different idea of what a 3 out of 5 actually means. It feels rigorous and produces almost no useful signal.
A scorecard that actually catches a bad hire starts from the specific problems the role needs solved in its first ninety days, not from a generic list of executive virtues.
Start from the job's real first-90-days problems
Before writing a single competency, list the three or four specific things this executive needs to be able to do in their first quarter: rebuild a broken forecasting process, hire and ramp two senior reports, fix a specific customer relationship that's at risk. Each competency on the scorecard should map back to one of these, not to an abstract trait.
This also makes it far easier to write good interview questions, since "tell me about a time you rebuilt a broken process under pressure" produces a much more evaluable answer than "tell me about your leadership style."
Writing rubric levels with behavioral anchors, not adjectives
A scale that runs from "weak" to "strong" asks every interviewer to define those words themselves, which is exactly how two interviewers give the same answer wildly different scores. Write each level of a 1-to-4 scale with a specific behavior: a 2 might read "describes the process but can't explain why they made the specific tradeoffs they did," while a 4 reads "walks through the tradeoffs unprompted and names what they'd do differently now."
This takes real work to write well for each competency, but it's the single change that improves scorecard consistency the most.
Calibrating interviewers before the first candidate, not after a bad hire
Run a short calibration session where every interviewer scores the same sample answer against the rubric and compares notes on where they landed and why. Differences that show up here are far cheaper to resolve before a real candidate is in the process than after an offer's already been extended based on inconsistent scores.
Repeat this calibration for a new search, even with the same interview panel. People's calibration drifts over time, and a rubric that felt clear six months ago benefits from a quick refresh before it's used again.
A short calibration session usually runs like this:
- Have every interviewer score the same sample answer against the rubric before any real candidate enters the process.
- Compare where each interviewer landed and ask each person to describe the specific behavior that drove the score.
- Rewrite any rubric level that produced a split, replacing adjectives with a concrete, observable behavior.
- Repeat the session at the start of each new search, even with the same panel, because calibration drifts over time.
Weighting: why not every competency should count equally
A candidate who's exceptional at three of five competencies but weak on the one that matters most for this specific role shouldn't average out to a passing score. Assign real weights before the search starts, tied to which of the first-90-days problems is most urgent, so a strong overall average can't paper over a critical gap.
Decide in advance whether any single competency is a hard gate, a score below which disqualifies regardless of the overall average, for anything the role genuinely can't function without.
For example, suppose a search for a chief revenue officer comes down to two finalists. Candidate A scores well across all five competencies but only average on rebuilding the forecasting process, the first problem the role has to solve. Candidate B is strong on that one competency and merely adequate on the others. A simple average favors Candidate A, yet the weights set before the search say the forecasting problem matters most. Decide the rule in advance: if a gated competency falls below the agreed level, the candidate does not advance, whatever the average says. Write that rule into the scorecard before the first interview, so the debrief applies it instead of debating it after everyone has picked a favorite.
Aggregating scores without letting charisma win the room
Have every interviewer submit their scores before the debrief discussion starts, not after hearing everyone else's take first. Scores submitted after group discussion tend to converge toward whoever spoke first or most confidently, which defeats the purpose of collecting independent signal in the first place.
The median cost per hire for an executive role already runs more than $10,000 before onboarding even starts1, which is exactly why a scorecard that lets an unqualified but likable candidate slip through is an expensive failure mode to leave unguarded.
What to do when the scores and the reference checks disagree
A candidate who scores well on the scorecard but gets a lukewarm or vague reference deserves a second look before an offer goes out, not a decision to trust whichever signal is more convenient. Ask the reference specifically about the same competencies the scorecard measured, rather than a general "would you hire them again" question that's easy to answer politely without really engaging.
Treat a strong scorecard and a strong reference as two independent checks that both need to clear, not as redundant confirmations of the same thing. They tend to catch different kinds of problems.
What Good Looks Like
A working scorecard ties every competency to a specific first-90-days problem, has behaviorally anchored rubric levels, and collects every interviewer's score before the group debrief starts.
Building The Capability (5-Stage Skill Ladder)
How to Get Started
Frequently Asked Questions
How many competencies should an executive scorecard actually have?
Four or five, tied directly to the role's real first-quarter problems, works better than a longer list. A scorecard with ten competencies dilutes attention across all of them and makes it harder for any single gap to stand out clearly in the final discussion.
Should the CEO's gut feel about a candidate count for anything?
It's useful information, but keep it separate from the structured competency scores rather than letting it quietly override them. A strong personal impression that contradicts a weak competency score is worth discussing explicitly, not used to average away the concern.
Can the same scorecard be reused across different executive searches?
The structure and rubric-writing approach can be reused, but the specific competencies should be rebuilt for each search's actual first-90-days problems. A CFO search and a CMO search need different scorecards even at the same company.
What should happen when interviewers disagree sharply on a candidate?
Treat a sharp split as a signal to dig deeper, not a tie to be averaged away. Have the interviewers who disagree compare specifically what behavior they observed, since a real disagreement about facts usually points to something worth a follow-up conversation with the candidate.
Sources
Where we quote a benchmark, we show its source. Other figures in this guide are estimates or general guidance, so check them against your own numbers.
- Median cost-per-hire (SHRM 2025 Recruiting Executives Benchmarking). SHRM 2025 Recruiting Executives Benchmarking data brief (PDF), 2025.
Related Guides
Contingent vs Retained Executive Search: Which to Use
How contingent and retained executive search fees actually differ, and which fee structure fits a given senior hire based on urgency and difficulty.
What a Fractional Executive Bench Actually Costs
A worked example of what it costs to assemble a fractional CFO, CMO, and COO bench instead of hiring three full-time executives, and where savings vanish.
Running an Executive Team Split Across Time Zones
The specific habits that stop a distributed leadership team from letting whoever shares a time zone with headquarters make every real decision.
A 14-Day Onboarding Plan for Fractional Executives
A day-by-day plan for onboarding a fractional executive in their first two weeks, so context transfers fast without turning into a documentation project.
Interim vs Fractional Executive Contracts: Key Differences
How interim and fractional executive contracts actually differ in scope, duration, and termination terms, and which structure fits which situation.
Evaluating Executive Performance Beyond OKRs
A worksheet for evaluating senior executives on judgment and team health, not just OKR completion, and why OKRs alone miss the real signal.