OpenAI introduces GPT-4o and takes giant leap forward

OpenAI is taking a giant leap forward with the introduction of GPT-4o, which will be rolled out in the coming weeks. This greatly improved version of GPT-4 can reason about audio, image and text in real time, is faster, more accessible and largely available for free to everyone!

The announcement was made during this livestream, in which several live demos were done with the new features. You can see a summary in the video below. 

An overview of GPT-4o

✔️ GPT-4o (the 'o' stands for 'omni') can process and generate combinations of text, audio and images. It responds to audio as quickly as a human responds, its performance for text in non-English languages is greatly improved, and it is 50% cheaper to use via its API.

✔️ Voice Interaction: You can talk to GPT-4o in real time and interrupt in the middle of a sentence. It responds instantly, without long silences. It can recognize emotions in your voice and respond in matching emotions, appearing increasingly human. And it can even sing. (This does make Apple's Siri look very old-fashioned.)

✔️ Visual recognition: It can now view the real world in real time. You can have live images, photos and screenshots analyzed. During the demo, they showed how GPT-4o can help solve a math problem if you write a sum on a piece of paper. It can also give hints first without immediately revealing the answer. Among other things, this makes it an even more powerful tool in education and homework.

✔️ Real-time translation: GPT-4o can translate spoken text in real time, as they showed during the demo with translation from English to Italian and vice versa. But you can also, for example, take a picture of a menu in another language and talk to GPT-4o to translate it and learn more about the dishes, and get recommendations.

✔️ It is possible for multiple people to play live games such as rock-paper-scissors, where ChatGPT can be the "showmaster" or "referee," for example.

✔️ GPTs can talk to each other and sing together!

✔️ Most current features of GPT-4 and the use of GPTs and custom GPTs are made available to all users free of charge. Also for anyone who does not have a paid subscription. This is a significant change and can start to give a huge acceleration to the use of ChatGPT and the development and use of custom GPTs.

This update will be rolled out with all features to paid ChatGPT users and ChatGPT Free users with usage limits in the coming weeks.

According to OpenAI, GPT-4o is just the beginning, with much more coming soon, especially for the paid versions.

🔗 View the entire demo here
🔗 View additional information including more impressive examples here