Structured candidate assessment instead of gut feel

Same questions, same order, same scoring grid

We start with the application review. Set a 20 minute timer. For each applicant, score two criteria on a 1 to 5 scale: evidence of the core skill and clarity of written examples. Advance only those scoring 4 or above on both. This removes the urge to interview everyone who looks impressive. For a typical small team role, expect 8 to 12 applications for every 1 you advance.

Next, the phone screen. We run every candidate through the same 5 questions in the same order. Each call lasts 25 minutes. Questions ask for specific past actions, not hypotheticals. For example: describe a time you disagreed with a manager about a deadline. What did you say and what happened? Score each answer on a 1 to 5 scale using a written anchor. A score of 5 means the candidate described a concrete action, the outcome, and what they learned.

We never change the order. We never add a question because we like the candidate. The US Office of Personnel Management structured interviews guidance shows that consistency is what makes scores comparable. After all screens, rank by total phone screen score. Advance the top 3 to panel interviews. If fewer than 3 score above 3, restart the search.

Building a simple scoring rubric with weights

We build the rubric before the first interview. Take a blank sheet. List 4 to 6 criteria that map directly to the first 90 days of the job. For a customer support hire, criteria might be: written clarity, de escalation, product learning speed, and reliability. Assign each criterion a weight that sums to 100 percent. Illustrative weights: written clarity 30 percent, de escalation 30 percent, product learning speed 25 percent, reliability 15 percent.

For each criterion, write 3 anchor statements. A 5 means the candidate gave a detailed example with measurable results. A 3 means the example was vague or incomplete. A 1 means no example or a contradictory one. Print the rubric. Every panel member gets a copy. During the interview, each panelist scores independently on the same grid.

We use a simple weighted score. Multiply each criterion score by its weight. Add the results. The maximum is 5.0. We set the hiring threshold at 3.8. Any candidate below 3.8 does not advance. This number is illustrative, not a rule from research. We adjust it after 5 hires by comparing the grid score to the first 90 day performance rating.

We also keep a tiebreaker. If two candidates score within 0.2 points, we return to the phone screen notes and check for the strongest specific example. That keeps the decision on evidence, not on who spoke last.

Panel interviews without groupthink

We schedule panel interviews for 50 minutes. Three panelists maximum. One asks the structured questions, one takes notes on the rubric, one observes and scores. All three score independently before any discussion. We collect the grids and average the scores. Only then do we talk.

We use the same 6 questions for every candidate. Each question targets one rubric criterion. For example: tell us about a time you had to learn a new tool in under a week. What did you do on day one, day three, and day seven? We allow 5 minutes per answer. The panelist may ask one follow up: what would you do differently? That follow up is also scripted.

We forbid open discussion during the interview. No nodding, no smiling at a good answer, no follow up questions that only one panelist invents. After the interview, each panelist has 10 minutes alone to finalise scores. Then we compare. If scores differ by more than 1 point on any criterion, we discuss the evidence, not the person. We look at the notes. We ask: what did the candidate actually say? We do not change scores unless a panelist missed a direct quote.

We end the panel with a single decision. Hire, no hire, or hold for a second panel. A hold requires a written reason and a new 30 minute interview focused on the weak criterion. We do not hold more than one candidate per role.

References: what you may ask and what actually predicts performance

We run references after the panel, never before. We ask for 2 references from the candidate. We call each for 15 minutes. We use the same 4 questions. Question one: how did the candidate handle a missed deadline? Question two: describe a time they disagreed with you. Question three: what did they do when they did not know an answer? Question four: would you hire them again for the same role? Yes or no, and why.

We may ask about job performance, attendance, and specific skills. We do not ask about age, family, health, or anything unrelated to the job. The SHRM talent acquisition news regularly covers the legal risks of unstructured reference checks. We keep notes. We score each reference on the same 1 to 5 scale for reliability and self awareness. A reference score below 3 means we do not make an offer, even if the panel score was high.

What actually predicts performance? Structured interviews and work samples predict better than casual conversation. References predict modestly, mostly for reliability and teamwork. We combine them. The final decision uses 70 percent panel score and 30 percent reference score. That is an illustrative weight. We review it after every 3 hires.

Common bias traps and the check that removes them

We see five traps in small team hiring. First, the similarity trap: we like candidates who remind us of ourselves. Check: score before you discuss, and compare scores to the rubric anchors, not to your feeling. Second, the halo trap: one great answer lifts all scores. Check: score each criterion separately and never average until the end. Third, the urgency trap: we rush because a role is open. Check: set a minimum of 3 advanced candidates before any panel. Fourth, the confirmation trap: we look for evidence that supports our first impression. Check: assign one panelist to ask only about weaknesses. Fifth, the recency trap: the last candidate feels best. Check: review all grids side by side after a 24 hour cooling off period.

We run one final check before the offer. We ask: would we still hire this person if they had no personal connection to anyone here? If the answer is no, we return to the grid. We also check that every criterion has at least two pieces of evidence from different sources: phone screen, panel, or reference. If a criterion has only one source, we do not count it.

Here is the comparison we use to explain why structure matters. All numbers are illustrative.

We keep the table on the wall during hiring. It reminds us that a friendly conversation is not an assessment. A structured grid is.

Today, open your next job description. Write the first three criteria and their weights. Set a 20 minute timer. Score the next five applications against those criteria. That is your first structured step. We will be here when you need the next grid.

Sources

The substance of this page rests on the published material linked below. It is independent of this site and was read before writing.

• US OPM, Structured interviews
• SHRM, Talent acquisition news