Everyone Can Build a Voice Agent. Almost Nobody Nails the Last 1%

https://video.wixstatic.com/video/190083_9178020f16b04dc58812aec3a6052407/1080p/mp4/file.mp4

Check out our solution in action!

The bit that gives it away

We build conversational AI agents that answer the phone for New Zealand businesses. We build them on ElevenLabs, where we're an accredited partner - which means we're in the platform every day, and we get to push on the bits that don't quite work yet.

Our agents talk to customers in English, all day, and they're good at it.

Right up until someone mentions where they live.

Ōmokoroa. Pāpāmoa. Whangārei. Taupō. That's the moment an otherwise excellent agent quietly announces it isn't from around here. Not offensively wrong - just off. A syllable landing in the wrong spot, a name stretched into a shape no local would recognise. And the caller hears it, because it's their town.

You can have a voice that sounds warm, natural and unmistakably Kiwi, and still lose the room on one word.

Why it happens

Purpose-built New Zealand voices don't exist at scale. A Kiwi phone agent starts life on a voice model trained on enormous volumes of American and British speech. Brilliant on English. But hand it a New Zealand place name and it does what the training data tells it to - which is guess.

It's the same problem plenty of newcomers to the country have, only the AI never gets the benefit of a colleague quietly correcting it after the meeting.

What we tried

We've got an agent live for a client in the Bay of Plenty. All day, every day, it's saying Ōmokoroa, Pāpāmoa, Tauranga. There's nowhere to hide.

So we had a choice: accept "close enough", or keep going.

We kept going. For months. Pronunciation dictionaries on top of the English engine helped somewhat. Phonetic respellings in prompts and knowledge bases helped somewhat. The real killer was inconsistency - a name would land perfectly on one call and drift back to something out of a US news bulletin on the next, for no reason we could pin down. You can't sell a product that works on a coin flip.

What finally worked

ElevenLabs' Conversational v3 model was the step change. Pronunciation got dramatically more accurate and, more importantly, stable call to call. Same name, same result, every time. Being close to the platform meant we were testing it on live New Zealand names the week it landed.

The last thing we chased was accent bleed - a call opening in a solid Kiwi accent and drifting British three turns in. That came down to the voice sample, so we recorded our own: a young New Zealand voice, professionally cloned. Best results we've had by a distance.

The final 1%

Plenty of providers can stand up a conversational AI agent. Getting one to sound like it actually belongs here - reliable, stable, right on every call, including the hard names - is the last 1%. It's also the only part your customers will notice.

If you're chasing that final 1%, we've already spent the months on it - on the best voice platform going, with the partner accreditation to prove we know our way around it.

Get in touch.