ChatGPT Can Now See, Hear and Talk: GPT-4V Arrives
OpenAI starts rolling out voice and image capabilities in ChatGPT. The assistant can now hold spoken conversations in five different voices and analyze photos thanks to GPT-4V.
OpenAI announced today that ChatGPT is no longer just a text-based chatbot. The company is rolling out two new capabilities: the ability to talk to the assistant by voice, and the ability to show it images for analysis, powered by GPT-4V, the vision-enabled version of its GPT-4 model.
An assistant that listens and talks back
The new voice feature lets users hold a spoken conversation with ChatGPT: the user talks, the app transcribes the question, and the model replies in audio, not just text. For transcription, OpenAI uses Whisper, its own open-source speech recognition system. For the spoken responses, it has built a new text-to-speech model capable of generating natural-sounding speech from just a few seconds of audio sample.
OpenAI said the voice was built in collaboration with professional voice actors, and the lineup includes five distinct voices for users to choose from. The company stressed that it does not allow cloning the voice of identifiable individuals without their consent — an explicit safeguard against the impersonation risks already stirring concern across the AI-generated audio space.
The feature is available through the ChatGPT mobile app on both iOS and Android, rolling out first to ChatGPT Plus subscribers and ChatGPT Enterprise customers.
GPT-4V: when the model can see, too
The second new capability is image analysis, powered by GPT-4V (the "V" stands for "vision"), the multimodal variant of GPT-4 that OpenAI has been documenting throughout the year alongside its safety system card. Users can now upload a photo or use their phone's camera and ask ChatGPT about what it shows: identifying an object, explaining a diagram, helping solve a handwritten math problem, or describing the contents of a document.
Until now, ChatGPT could only understand what was typed to it. With GPT-4V, the model processes text and images within the same conversation and can reason about both at once, significantly expanding the assistant's practical uses — from learning aids to accessibility tools for people with visual impairments.
A gradual rollout, limited for now
OpenAI said both voice and vision will roll out gradually over the next two weeks, and for now remain limited to paying subscribers on ChatGPT Plus and Enterprise. The company has not yet said whether or when the features will reach the free version of the service.
The move places ChatGPT in a different category than a simple text chatbot. Turning the assistant into something you can talk to and show images to brings it closer to the realm of traditional voice assistants, but with reasoning and visual understanding capabilities far beyond a classic Siri or Alexa.
It remains to be seen how this multimodality performs in everyday use, beyond controlled demos, and what limits OpenAI will impose to prevent misuse of a technology capable of generating realistic voices and interpreting any image shown to it.
