Video meetings have made international collaboration easier than ever. A team in Germany can work with partners in Japan, a startup in Ukraine can present its product to customers in South Korea, and companies can hire talent from almost anywhere.

But one fundamental problem remains: people still need a common language.

Most video conferencing tools assume that participants can communicate comfortably in English or another shared language. In reality, this assumption often creates limitations.

Participants may understand a foreign language but struggle to explain complex ideas, negotiate business terms, or respond naturally during conversations.

Live subtitles can help people understand what is being said, but reading translated text while participating in a meeting creates an additional cognitive burden.

We wanted to explore a different approach.

What if everyone in a video meeting could speak their own language and hear other participants translated into their preferred language?

That question became the foundation of Voicli, a real-time multilingual video meeting platform.

In this article, we'll share the architecture behind the platform, the technical challenges of combining video conferencing with AI voice translation, and some lessons from building the system.

1. Why Real-Time Voice Translation Is Technically Challenging

Traditional speech translation systems often follow a sequential architecture:

Speech → Speech Recognition → Text Translation → Speech Synthesis → Translated Audio

Each stage introduces processing time.

First, the system recognizes the speaker's words. Then it translates the resulting text into another language. Finally, a text-to-speech engine generates audio.

This architecture can produce useful translations, but latency becomes a serious challenge in interactive conversations.

Consider a business meeting where someone asks a simple question:

"Can we start the project next Monday?"

If the translation arrives several seconds later, the conversation becomes unnatural. Participants may interrupt each other, repeat questions, or wait for the system to finish processing.

The challenge is not simply translating words accurately.

A practical real-time translation platform needs to balance several competing requirements:

• Low latency: Deliver translated audio quickly enough to support natural turn-taking.

• Translation quality: Preserve the original meaning, including technical and business terminology.

• Audio quality: Produce speech that is understandable and comfortable to listen to.

• Multiple languages: Support participants with different language preferences.

• Synchronization: Coordinate translated speech with the original conversation.

• Scalability: Handle multiple speakers and listeners without making the meeting experience unstable.

Improving one of these factors can sometimes negatively affect another.

For example, waiting for a speaker to finish a complete sentence may improve translation context but increase latency.

Generating audio earlier can reduce waiting time, but short speech fragments may lack sufficient context.

These trade-offs influence the architecture of the entire system.

2. The Technology Stack Behind Voicli

We built Voicli as a web-based communication platform, combining conventional backend technologies with real-time media infrastructure and AI services.

Our main technology stack includes:

Each component solves a different part of the problem.

Django: Application Logic and Meeting Management

Django provides the foundation for the application backend.

It handles the business logic surrounding user accounts, meetings, participants, permissions, invitations, and other platform operations.

Django REST Framework provides APIs consumed by the React frontend.

We use Django Channels for real-time application communication, where traditional request-response APIs are not sufficient.

For example, participant and meeting state changes need to reach connected clients without constant polling.

The combination of Django, Redis, and WebSockets makes it possible to build these workflows around a familiar Python ecosystem.

React: The Real-Time User Experience

The frontend is built using React and TypeScript.

Video conferencing interfaces are particularly challenging because many elements change simultaneously.

A participant may join a meeting, switch languages, mute a microphone, enable subtitles, or change audio preferences while other participants continue speaking.

The interface must respond to these events without distracting the user.

For Voicli, language selection is especially important.

The language used to speak does not necessarily need to match the language used to listen.

This distinction is central to designing a multilingual meeting experience.

LiveKit: Real-Time Audio and Video

Real-time media transport is a specialized engineering problem.

Rather than implementing our own media infrastructure from scratch, we use LiveKit for real-time audio and video communication.

This allows us to focus more engineering effort on the translation experience and application logic.

However, introducing translation into a video meeting creates additional coordination challenges.

The system must handle original audio, translated audio, participant preferences, and the changing state of a live conversation.

Unlike a conventional video meeting, transmitting the original audio stream is only one part of the experience.

3. Designing Translation Around the Listener

One important design decision was to treat translation as a listener-specific experience.

Imagine three participants in a meeting:

• Anna speaks Ukrainian.

• Kenji speaks Japanese.

• Marie speaks French.

In a conventional meeting, the participants would probably need to agree on a common language.

In Voicli, the goal is different.

Anna should be able to continue speaking Ukrainian. Kenji should be able to listen in Japanese, while Marie listens in French.

The same original statement may therefore need to be translated into multiple target languages.

This changes how we think about meeting audio.

Translation is not simply a feature attached to the speaker. It is a communication preference associated with each listener.

This approach also creates interesting optimization opportunities.

If several participants prefer the same target language, a system can potentially reuse translation work instead of treating every listener as a completely independent translation request.

At the same time, participants may change their language preferences during the meeting.

The application must coordinate these changes while keeping the conversation active.

4. Managing Original and Translated Audio

Another challenge is deciding what participants should hear.

Simply playing translated speech on top of the original audio at full volume creates an unpleasant experience.

Two voices may compete for attention, especially when the speaker and translation overlap.

However, removing the original voice completely is not always ideal.

Some participants prefer to hear the original speaker because it provides emotional context, tone, and a sense of direct communication.

For this reason, Voicli supports different listening experiences, including original audio, translated audio alongside the original, and subtitles.

The important engineering lesson is that audio translation should be designed as part of the meeting experience rather than an isolated AI function.

Even an accurate translation can feel unusable if its timing, volume, or playback behavior interrupts the conversation.

5. Latency, Turn-Taking, and Speech Boundaries

Speech is not delivered in perfectly separated sentences.

People hesitate, interrupt themselves, change direction mid-sentence, and use short expressions that depend on previous context.

A translation system therefore needs to determine when it has enough information to produce a useful result.

Processing every tiny speech fragment independently can create fragmented translations.

Waiting too long can make participants feel disconnected from the conversation.

We have had to pay particular attention to speech boundaries, the timing of generated audio, and the transitions between speakers.

These problems become more complex in meetings with multiple participants.

For example, a translated audio segment may still be playing when another participant begins speaking.

Without careful coordination, listeners can hear overlapping translations or lose track of who is speaking.

Turn-taking is therefore not only a user interface consideration. It is also part of the real-time translation architecture.

6. Why Subtitles Still Matter

Voice translation is useful, but it does not eliminate the need for subtitles.

There are situations where users cannot listen to generated audio, prefer reading technical terminology, or simply want additional confirmation of what someone said.

Subtitles can also provide a practical fallback when audio translation is temporarily unavailable.

For these reasons, Voicli includes multilingual subtitle functionality alongside voice translation.

We see subtitles and voice translation as complementary capabilities.

The goal is not to force everyone to use a single communication mode.

Instead, participants should be able to select the experience that works best for their situation.

7. Building a Product Is Different From Building an AI Demo

It is relatively straightforward to create an impressive demonstration of AI translation using a short prerecorded audio clip.

A real product introduces a very different set of requirements.

Users need reliable authentication, meeting invitations, access control, participant management, stable connections, language preferences, and understandable error states.

Network connections can fail.

Participants can join late or reconnect.

Microphone permissions can change.

AI services can experience delays.

A translation session may need time to initialize before it becomes available.

These situations must be handled without making the entire meeting unusable.

One lesson from building Voicli is that the quality of the product depends as much on orchestration and recovery as on the AI model itself.

The surrounding engineering infrastructure determines whether powerful AI capabilities can actually be used in everyday communication.

8. The Future of Multilingual Collaboration

We believe multilingual collaboration will gradually move beyond conventional translated subtitles.

The long-term opportunity is to make language differences less disruptive during everyday conversations.

That does not mean AI translation is perfect.

Idioms, specialized terminology, accents, background noise, overlapping speech, and cultural context remain challenging.

Human interpreters also continue to play an important role in sensitive and high-stakes situations.

But real-time AI translation creates new possibilities for meetings that would otherwise require a shared foreign language or advance interpretation arrangements.

Some of the most interesting potential applications include international sales, supplier negotiations, global education, customer support, and collaboration between geographically distributed teams.

For smaller businesses in particular, lowering communication barriers may help make international relationships more accessible.

9. What We Learned

Building Voicli has reinforced several technical lessons:

1. Latency is a product feature. Translation quality alone does not determine whether people can have a natural conversation.

2. The listener matters as much as the speaker. Each participant may have different language and audio preferences.

3. Real-time AI requires careful orchestration. Speech recognition, translation, synthesis, media transport, and application state must work together.

4. The fallback experience matters. Subtitles, original audio, and clear connection states make communication more resilient.

5. Real-world conversations are unpredictable. A useful translation system needs to handle interruptions, pauses, reconnections, and multiple speakers.

These lessons continue to shape our engineering decisions.

Conclusion

Real-time multilingual video communication is not simply a matter of connecting an AI translation API to a video call.

It requires coordination between speech processing, media infrastructure, backend services, and an interface designed around how people actually communicate.

With Voicli, we are exploring how these technologies can work together to make international conversations more accessible.

The product is available at voicli.com, where users can explore multilingual video meetings with voice translation and subtitles.

We will continue sharing technical lessons from building and improving the platform.

If you are working on real-time AI, speech technologies, or communication infrastructure, we would love to hear about the challenges you have encountered.