Can We Trust the AI Wizards? A Deep Dive into GPT-4o

ControlAI

May 16, 2024

See All Posts

If you go right this instant and log onto ChatGPT, you'll be greeted with the announcement: “You can now try our newest model, GPT‑4o. It’s faster than GPT‑4, better at understanding images, and speaks more languages.”

Sounds nice, doesn’t it?

GPT-4o (where the ‘o’ stands for ‘omni’, the Latin word for “all”) represents, in OpenAI’s own words, “a step towards much more natural human-computer interaction.” It accepts any combination of text, audio, image, and video as input and generates any combination of text, audio, and image as outputs. Significantly, GPT-4o can respond to audio inputs in as little as 232 milliseconds, averaging 320 milliseconds – this is genuinely comparable to conversational human response times.

GPT-4o demonstrates remarkable versatility, with capabilities ranging from interview preparation to playing Rock Paper Scissors, from solving mathematical problems to telling dad jokes. Before GPT-4o, Voice Mode in ChatGPT had average latencies of 2.8 seconds (GPT-3.5) and 5.4 seconds (GPT-4). This was due to a pipeline of three separate models for transcribing audio to text, processing text, and converting text back to audio. This setup limited the AI's ability to observe tone, multiple speakers and background noises and to express emotions like laughter or singing. GPT-4o pushes past these limitations by integrating text, vision, and audio processing within a single neural network.

Even though OpenAI states that “GPT-4o has also undergone extensive external red teaming with 70+ external experts in domains such as social psychology, bias and fairness” and guarantees that they will “continue to mitigate new risks as they’re discovered,” OpenAI’s new flagship model comes after a recent study that has found that when people are presented with two solutions to a moral dilemma, the majority prefer the answer provided by artificial intelligence over that of a human.

This study was conducted by Eyal Aharoni, an associate professor in Georgia State’s Psychology Department, inspired by the rapid rise of ChatGPT and similar AI large language models (LLMs). He noted that “people will interact with these tools in ways that have moral implications, like the environmental implications of asking for a list of recommendations for a new car. Some lawyers have already begun consulting these technologies for their cases, for better or worse. So, if we want to use these tools, we should understand how they operate, their limitations, and that they’re not necessarily operating in the way we think when we’re interacting with them.”

Aharoni’s test involved posing the same ethical questions to both undergraduate students and AI, then presenting their written responses to study participants, who rated the answers on various traits such as virtuousness, intelligence, and trustworthiness. “Instead of asking the participants to guess if the source was human or AI, we just presented the two sets of evaluations side by side, and we just let people assume that they were both from people,” Aharoni said.

Essentially, he conducted a version of the Turing test, which assesses a machine's ability to demonstrate intelligent behaviour that is equivalent to or indistinguishable from that of a human. If the human can’t tell the difference, then, according to Turing, the computer should be considered intelligent.

In this case, people could tell the difference, but not for the expected reason. “The twist is that the reason people could tell the difference appears to be because they rated ChatGPT’s responses as superior,” Aharoni noted. “If we had done this study five to ten years ago, then we might have predicted that people could identify the AI because of how inferior its responses were. But we found the opposite — that the AI, in a sense, performed too well.”

Currently, OpenAI is experiencing significant turnover among its safety researchers, losing five top talents in recent weeks. This includes four from the Superalignment team, responsible for ensuring control over AI systems much smarter than us, which should be a key part of OpenAI’s plan. Those who have left include:


OpenAI’s mission, and I quote, “is explicitly to create AGI”, a type of artificial intelligence which can outperform humans on a wide range of cognitive tasks (in contrast to narrow AI, which is designed for specific tasks); let us not forget. Meanwhile, OpenAI’s safety research is evidently unstable and in upheaval from within, precisely at the moment they’re leaping forward. Given their internal dysfunction, and given the growing moral salience of AI's influence on human decisions, the simple question grows louder and louder: do we trust them?

We need to do far more to stay in control of upcoming superintelligent AI. Those in the know understand the dangers of this technology, as highlighted in our new video, which you can watch here:

If you want to know more about the challenges of AI governance, the regulation of synthetic media, and the global security implications of AI advancements, join us on Discord at https://discord.gg/2fR2eZAQ4a. Here, we can collaborate, share insights, and contribute to shaping the future of AI in a manner that safeguards our security and democratic values and fosters responsible innovation.

If you go right this instant and log onto ChatGPT, you'll be greeted with the announcement: “You can now try our newest model, GPT‑4o. It’s faster than GPT‑4, better at understanding images, and speaks more languages.”

Sounds nice, doesn’t it?

GPT-4o (where the ‘o’ stands for ‘omni’, the Latin word for “all”) represents, in OpenAI’s own words, “a step towards much more natural human-computer interaction.” It accepts any combination of text, audio, image, and video as input and generates any combination of text, audio, and image as outputs. Significantly, GPT-4o can respond to audio inputs in as little as 232 milliseconds, averaging 320 milliseconds – this is genuinely comparable to conversational human response times.

GPT-4o demonstrates remarkable versatility, with capabilities ranging from interview preparation to playing Rock Paper Scissors, from solving mathematical problems to telling dad jokes. Before GPT-4o, Voice Mode in ChatGPT had average latencies of 2.8 seconds (GPT-3.5) and 5.4 seconds (GPT-4). This was due to a pipeline of three separate models for transcribing audio to text, processing text, and converting text back to audio. This setup limited the AI's ability to observe tone, multiple speakers and background noises and to express emotions like laughter or singing. GPT-4o pushes past these limitations by integrating text, vision, and audio processing within a single neural network.

Even though OpenAI states that “GPT-4o has also undergone extensive external red teaming with 70+ external experts in domains such as social psychology, bias and fairness” and guarantees that they will “continue to mitigate new risks as they’re discovered,” OpenAI’s new flagship model comes after a recent study that has found that when people are presented with two solutions to a moral dilemma, the majority prefer the answer provided by artificial intelligence over that of a human.

This study was conducted by Eyal Aharoni, an associate professor in Georgia State’s Psychology Department, inspired by the rapid rise of ChatGPT and similar AI large language models (LLMs). He noted that “people will interact with these tools in ways that have moral implications, like the environmental implications of asking for a list of recommendations for a new car. Some lawyers have already begun consulting these technologies for their cases, for better or worse. So, if we want to use these tools, we should understand how they operate, their limitations, and that they’re not necessarily operating in the way we think when we’re interacting with them.”

Aharoni’s test involved posing the same ethical questions to both undergraduate students and AI, then presenting their written responses to study participants, who rated the answers on various traits such as virtuousness, intelligence, and trustworthiness. “Instead of asking the participants to guess if the source was human or AI, we just presented the two sets of evaluations side by side, and we just let people assume that they were both from people,” Aharoni said.

Essentially, he conducted a version of the Turing test, which assesses a machine's ability to demonstrate intelligent behaviour that is equivalent to or indistinguishable from that of a human. If the human can’t tell the difference, then, according to Turing, the computer should be considered intelligent.

In this case, people could tell the difference, but not for the expected reason. “The twist is that the reason people could tell the difference appears to be because they rated ChatGPT’s responses as superior,” Aharoni noted. “If we had done this study five to ten years ago, then we might have predicted that people could identify the AI because of how inferior its responses were. But we found the opposite — that the AI, in a sense, performed too well.”

Currently, OpenAI is experiencing significant turnover among its safety researchers, losing five top talents in recent weeks. This includes four from the Superalignment team, responsible for ensuring control over AI systems much smarter than us, which should be a key part of OpenAI’s plan. Those who have left include:


OpenAI’s mission, and I quote, “is explicitly to create AGI”, a type of artificial intelligence which can outperform humans on a wide range of cognitive tasks (in contrast to narrow AI, which is designed for specific tasks); let us not forget. Meanwhile, OpenAI’s safety research is evidently unstable and in upheaval from within, precisely at the moment they’re leaping forward. Given their internal dysfunction, and given the growing moral salience of AI's influence on human decisions, the simple question grows louder and louder: do we trust them?

We need to do far more to stay in control of upcoming superintelligent AI. Those in the know understand the dangers of this technology, as highlighted in our new video, which you can watch here:

If you want to know more about the challenges of AI governance, the regulation of synthetic media, and the global security implications of AI advancements, join us on Discord at https://discord.gg/2fR2eZAQ4a. Here, we can collaborate, share insights, and contribute to shaping the future of AI in a manner that safeguards our security and democratic values and fosters responsible innovation.

If you go right this instant and log onto ChatGPT, you'll be greeted with the announcement: “You can now try our newest model, GPT‑4o. It’s faster than GPT‑4, better at understanding images, and speaks more languages.”

Sounds nice, doesn’t it?

GPT-4o (where the ‘o’ stands for ‘omni’, the Latin word for “all”) represents, in OpenAI’s own words, “a step towards much more natural human-computer interaction.” It accepts any combination of text, audio, image, and video as input and generates any combination of text, audio, and image as outputs. Significantly, GPT-4o can respond to audio inputs in as little as 232 milliseconds, averaging 320 milliseconds – this is genuinely comparable to conversational human response times.

GPT-4o demonstrates remarkable versatility, with capabilities ranging from interview preparation to playing Rock Paper Scissors, from solving mathematical problems to telling dad jokes. Before GPT-4o, Voice Mode in ChatGPT had average latencies of 2.8 seconds (GPT-3.5) and 5.4 seconds (GPT-4). This was due to a pipeline of three separate models for transcribing audio to text, processing text, and converting text back to audio. This setup limited the AI's ability to observe tone, multiple speakers and background noises and to express emotions like laughter or singing. GPT-4o pushes past these limitations by integrating text, vision, and audio processing within a single neural network.

Even though OpenAI states that “GPT-4o has also undergone extensive external red teaming with 70+ external experts in domains such as social psychology, bias and fairness” and guarantees that they will “continue to mitigate new risks as they’re discovered,” OpenAI’s new flagship model comes after a recent study that has found that when people are presented with two solutions to a moral dilemma, the majority prefer the answer provided by artificial intelligence over that of a human.

This study was conducted by Eyal Aharoni, an associate professor in Georgia State’s Psychology Department, inspired by the rapid rise of ChatGPT and similar AI large language models (LLMs). He noted that “people will interact with these tools in ways that have moral implications, like the environmental implications of asking for a list of recommendations for a new car. Some lawyers have already begun consulting these technologies for their cases, for better or worse. So, if we want to use these tools, we should understand how they operate, their limitations, and that they’re not necessarily operating in the way we think when we’re interacting with them.”

Aharoni’s test involved posing the same ethical questions to both undergraduate students and AI, then presenting their written responses to study participants, who rated the answers on various traits such as virtuousness, intelligence, and trustworthiness. “Instead of asking the participants to guess if the source was human or AI, we just presented the two sets of evaluations side by side, and we just let people assume that they were both from people,” Aharoni said.

Essentially, he conducted a version of the Turing test, which assesses a machine's ability to demonstrate intelligent behaviour that is equivalent to or indistinguishable from that of a human. If the human can’t tell the difference, then, according to Turing, the computer should be considered intelligent.

In this case, people could tell the difference, but not for the expected reason. “The twist is that the reason people could tell the difference appears to be because they rated ChatGPT’s responses as superior,” Aharoni noted. “If we had done this study five to ten years ago, then we might have predicted that people could identify the AI because of how inferior its responses were. But we found the opposite — that the AI, in a sense, performed too well.”

Currently, OpenAI is experiencing significant turnover among its safety researchers, losing five top talents in recent weeks. This includes four from the Superalignment team, responsible for ensuring control over AI systems much smarter than us, which should be a key part of OpenAI’s plan. Those who have left include:


OpenAI’s mission, and I quote, “is explicitly to create AGI”, a type of artificial intelligence which can outperform humans on a wide range of cognitive tasks (in contrast to narrow AI, which is designed for specific tasks); let us not forget. Meanwhile, OpenAI’s safety research is evidently unstable and in upheaval from within, precisely at the moment they’re leaping forward. Given their internal dysfunction, and given the growing moral salience of AI's influence on human decisions, the simple question grows louder and louder: do we trust them?

We need to do far more to stay in control of upcoming superintelligent AI. Those in the know understand the dangers of this technology, as highlighted in our new video, which you can watch here:

If you want to know more about the challenges of AI governance, the regulation of synthetic media, and the global security implications of AI advancements, join us on Discord at https://discord.gg/2fR2eZAQ4a. Here, we can collaborate, share insights, and contribute to shaping the future of AI in a manner that safeguards our security and democratic values and fosters responsible innovation.

Get Updates

Sign up to our newsletter if you'd like to stay updated on our work,
how you can get involved, and to receive a weekly roundup of the latest AI news.