You can set your chatbot Mode to Realtime (in the Chatbot section of the chatbot editor) to use OpenAI’s voice-to-voice models. Your visitors talk to the chatbot, and it answers with its voice.

Then open the AI Model section. A Realtime chatbot needs its own Environment (an OpenAI or Azure one, the Default environment is not enough) and a realtime model, such as GPT Realtime 2.1 or the cheaper GPT Realtime 2.1 Mini. If one of them is missing, the chatbot won’t work.

In the same section, you can pick the Voice of your chatbot and the Talk Mode: Hands-Free (the chatbot listens all the time and answers when you stop talking) or Hold to Talk (the visitor holds the button while speaking).

Your visitors start the conversation with the play button. Under it, small icons let them show the speakers, show captions (a written transcript of the conversation) and see some statistics. They can also type a message instead of speaking. Functions attached to the chatbot work in voice conversations too.

Text To Speech

With the regular chatbot, text-to-speech is not supported natively by AI Engine, but some freelancers have developed AI Engine addons that handle this functionality.

Speech To Text

A regular chatbot can support voice input to write messages instead of typing. Note that this does not rely on any AI transcription model, nor is this a feature of the plugin itself. This uses your web browser’s built-in Web Speech API, so the result and the supported languages you get from it depend on your current device.

You can enable this by going to Settings > Chatbot and checking Speech Recognition next to Web Speech API.

This will add a microphone icon in the message input section of your chatbot: