01
Transcription layer
What did the user say?
Speech-to-text converts the turn into words so the agent can understand intent and continue the conversation.
audio → transcript
/voice-agent
Voice Agent Pronunciation Assessment & Speech Scoring
Speech evidence your voice agent can act on—across LiveKit, Pipecat, and custom voice runtimes.
Add a listening layer to LiveKit, Pipecat, or any voice-native agent stack. Chivox turns every spoken turn into structured evidence—pronunciation, fluency, tone, and audio quality—so your application keeps grading deterministic while the agent responds naturally.
Framework-neutral by design
Bring the transport and orchestration you already use.
LiveKitPipecatMCPREST APICustom runtime
/beyond-transcription
Transcription hears the words. Assessment hears how they were spoken.
A transcript is essential context, but it cannot reliably grade pronunciation or delivery. Chivox adds the missing speech-performance evidence without replacing your STT, model, or TTS provider.
02
Chivox assessment layer
How was it spoken?
Pronunciation, fluency, tone, completeness, and audio-quality evidence reveal performance that a transcript cannot show.
audio → scoring evidence
03
Agent layer
What should happen next?
Your application applies its rubric, then the agent can continue, clarify, correct, grade, or invite a focused retry.
approved result → response
/your-voice-stack
Agent-Native Speech Assessment Workflows
Use MCP or API wherever you can pass audio in and structured JSON back. Chivox handles assessment; your runtime still owns turn-taking, orchestration, and the user experience.
LiveKit agents
Score a spoken turn inside a real-time room and return pronunciation evidence without breaking the conversation.
Pipecat pipelines
Insert assessment alongside speech-to-text, model, and text-to-speech services in an existing pipeline.
Custom voice runtimes
Call Chivox through MCP or API from any orchestration layer that can send audio and accept structured JSON.
/decision-boundary
Real-time speech assessment integration for voice AI
Keep scoring, grade logic, and safety rules in your application. Give the model only the approved evidence it needs to guide the next turn.
01
Listen
Capture the relevant user turn from LiveKit, Pipecat, or your own voice runtime.
live turn → audio
02
Assess
Score pronunciation, fluency, tone, completeness, and recording quality.
audio → assessment
03
Grade
Apply deterministic thresholds and select the evidence the agent is allowed to use.
scores → approved result
04
Respond
Let the agent explain, clarify, continue, or request a focused retry in the same session.
result → agent turn
/technical-example
Adding Listening and Grading to Agents
This provider-neutral example shows the handoff: audio and product rules in, structured evidence out, then an approved response back into the conversation.
Voice-agent turn contractLiveKit, Pipecat, or custom stack · MCP and APIView detailsHide details
Live audio · structured evidence
The listening layer behind reliable voice agents
- Recording qualitySignal-to-noise, clipping, and volume indicate when a re-record is more useful than coaching.
- Pronunciation & fluencyAccuracy, integrity, rhythm, pace, pauses, and tone provide the high-level performance picture.
- Word-level evidenceTimestamps, word or character errors, stress, liaison, tones, and phonemes support precise explanations.
- Decision hooksStable fields let your application grade the turn, route the conversation, and constrain the agent response.
How a voice agent turns a live speech turn into an approved responseLive audio · en-US
INAudio turn + product rubricassess-turn.ts
const assessment = await chivox.assess({
audio: currentTurn.audio,
language: "en-US",
referenceText: currentTurn.expectedText,
});
const grade = applyRubric(assessment, {
pass: 80,
requestRetryBelow: 65,
checkAudioQualityFirst: true,
});OUTChivox returns — scoring evidencethinking…
The same structured evidence works with LiveKit, Pipecat, or any agent stack that accepts JSON. Keep thresholds, pass/fail decisions, and safety rules in application code.
/voice-agent-faq
Integration questions, answered.
Keep your current voice stack and add assessment only where the product needs reliable speech evidence.
Read all FAQsDoes Chivox replace speech-to-text?
No. Speech-to-text tells the agent what was said. Chivox adds evidence about how it was spoken, including pronunciation, fluency, tone, completeness, and recording quality. The two layers can work together in the same voice pipeline.
How does Chivox connect to LiveKit or Pipecat?
Capture the relevant user turn in your existing pipeline, send the audio and assessment context through MCP or API, then pass the approved scoring fields back to your agent. Your framework continues to control turn-taking, VAD, STT, the model, and TTS.
Can we keep grading rules outside the LLM?
Yes. Keep pass thresholds, retry rules, routing, and high-impact decisions in application code. The model can explain the approved result naturally without becoming the source of truth for the score.
Which speech signals can the agent use?
Depending on the assessment, the response can include pronunciation and fluency scores, audio-quality signals, completeness, word or character detail, phonemes, stress, liaison, Pinyin, and tones.
How should assessment fit into a live conversation?
Assess only the turns that need scoring. Your product can evaluate a completed turn before the next response, run selected checks alongside other pipeline work, or reserve detailed grading for explicit practice and verification moments.
/build-your-agent
Start with one speech turn, then scale your agent loop.
Connect Chivox to LiveKit, Pipecat, or your own voice-native stack. Assess the turns that matter, grade them with product-owned rules, and let the agent respond from approved evidence.
Real-time speech assessment MCP or API integration Deterministic grading rules