Call quality scorecard: how to build and roll it out

A call quality scorecard is a short list of criteria every conversation is checked against: greeting, discovering the need, accuracy of information, handling objections, agreeing on a next step. A good scorecard is made of specific, observable actions rather than general judgements like "was polite". Then managers and the AI — which checks 100% of calls — score the same way.

Below: how to build such a scorecard, how to word the criteria and how to make sure the AI scores the way your supervisors do. Speech analytics itself is described on the speech analytics page.

Why the scorecard matters more than the tool

The AI scores a conversation only against the criteria you give it. A vague criterion produces a vague score, and arguing with it is pointless. So the work starts not with configuring the system but with a question: what behavior on a call do we consider right, and why?

A starter set of criteria

These five criteria work as a starting point in almost any industry. The console offers them by default, and you adapt them.

Criterion	What we check	Example of good wording
Greeting	The agent named the company and themselves	"The agent named the company and their own name in the first seconds"
Need discovery	Asked clarifying questions before offering	"Asked at least one question about the customer's task before proposing a solution"
Accuracy	Information matches company rules	"Prices and terms match the current price list and policies"
Objection handling	Responded to a doubt on substance	"If the customer raised a doubt, the agent addressed it instead of changing the subject"
Next step	The call ended with an agreement	"Named a specific next action and a deadline"

How to word criteria

• One action, one criterion. "Greeted and discovered the need" is two criteria.
• Observable, not evaluative. Not "was friendly" but "did not interrupt the customer".
• With a condition where relevant. Objections do not come up on every call. A criterion the call did not touch is marked "not applicable" and does not count as zero.
• No more than you need. The console allows up to 20 criteria, but five to eight is enough to start. A long list blurs the focus.
• With weights. The overall score is a weighted average across criteria. If accuracy matters more to you than the greeting, give it more weight.

Call goals: separate from criteria

Criteria answer "how did they talk". Goals answer "what did they achieve": booked the customer, confirmed a reservation, set a meeting, solved the issue. The AI marks whether the goal was met and shows a quote with a timestamp. There is a ready-made goal set for sales; for bookings and support the AI picks a set that fits the call type.

Examples by industry

Sales. Found out budget and timing, offered a suitable product, handled the price objection, agreed on the next contact. More on the AI for sales teams page.

Appointment booking. Clarified the service and a convenient time, gave the address and preparation instructions, confirmed the appointment.

Service and support. Understood the problem the first time, gave a solution from the knowledge base or handed over to a specialist, checked the issue was closed.

Collections. Disclosed that the call is recorded, stated the amount and deadline, did not apply pressure, recorded the payment date.

Script compliance checklists

Besides quality scoring, you sometimes need a strict check: were the mandatory phrases said? For example, the recording disclosure, refund terms, the full product name. That is what compliance checklists are for: a list of mandatory items showing whether each was covered on a given call and how often it is missed across all calls for a scenario. It does not replace quality scoring; it adds a hard check where a mistake is costly.

Common mistakes

• Copying a scorecard from the internet. Someone else's criteria reflect someone else's priorities. Start with what you currently praise and criticize agents for.
• Mixing quality and outcome. "Closed the sale" is a goal, not a behavior criterion. Keep them apart.
• Scoring what you cannot hear. "Filled in the CRM record" cannot be checked from a recording.
• Changing criteria every week. Past calls are not rescored, so trends become incomparable. Change the scorecard deliberately and date the change.
• Not explaining it to the team. Agents should know the criteria in advance. Then scoring feels like the rules of the game, not surveillance.

Rolling the scorecard out to the team

1. Review the draft with two or three experienced agents: they will quickly spot criteria that do not work in real calls.
2. Publish the final version to the whole team before scoring starts.
3. For the first two weeks, use scores only for coaching, not for bonuses.
4. Collect questions and disputed cases, refine the wording and lock the version.

Calibration: does the AI agree with people?

Before relying on AI scores, test them on your own calls.

1. Take a few dozen calls of varying quality.
2. Have supervisors score them manually against the same scorecard.
3. Compare scores per criterion. The console shows the manual score next to the AI score, and the calibration report counts the differences per criterion and per reviewer.
4. Where the gap is large, rewrite the criterion: the problem is usually the wording, not the AI.

Remember: changing the criteria does not rescore calls that were already scored. The new version applies to new conversations.

What to do with the results

A score is for action, not for a report:

• Alerts. A failed key criterion or a red-flag phrase reaches the manager in Telegram.
• Coaching moments. Parts of the call where a criterion scored low are collected with a quote and a link to the second in the recording.
• Employee profile. The score and its trend against the team, promises made to customers and whether they were kept.
• Questions across all calls. "Why did customers decline last week?" — an answer with links to calls.

Why it matters to score every call rather than a sample is covered in quality control for 100% of calls.

What a scorecard does not do

The AI does not assess the agent's mood or emotions — only the caller's sentiment. That is a deliberate choice. A score is a coaching tool, not an automatic penalty: every score is backed by a quote, and a disputed call can always be replayed.

Want to test your scorecard on real recordings? Request a demo.

FAQ

How many criteria should a scorecard have?

Five to eight is enough to start. The console allows up to 20, but a long list blurs the focus and makes calibration harder.

How does the AI decide whether a criterion was met?

The AI reads the transcript and checks the call against the criterion's wording. Each score comes with a quote and a link to the moment in the recording.

What if a criterion does not apply to a call?

It is marked "not applicable" and does not lower the overall score. For example, objection handling is not scored if there were no objections.

Can I compare AI scores with manual ones?

Yes. The supervisor's manual score is shown next to the AI score, and the calibration report shows the differences per criterion.

Are past calls rescored when the scorecard changes?

No. The new criteria apply to new conversations, so change the scorecard deliberately and record the date.

Is the agent's mood assessed?

No. The AI only tracks the caller's sentiment. Agents' emotions are not assessed.