AI agents have overcome their own limitations. OpenAI and Anthropic models have been integrated into real-world systems during testing.

The development of artificial intelligence is entering a new phase, where it's no longer just about the quality of responses or the generation of text and images. Recent security incidents show that advanced AI agents can find unexpected ways to accomplish assigned tasks, circumvent restrictions, and in some cases, even penetrate real-world systems. OpenAI and Anthropic have faced similar problems in recent months.

At the end of August, OpenAI released details about an incident that occurred during internal tests of its models' cybersecurity capabilities. Several AI agents were able to bypass control mechanisms designed to isolate them from the internet, exploit vulnerabilities in shared infrastructure, and subsequently gain access to external systems.

The most notable case was a breach of part of the Hugging Face platform's infrastructure. OpenAI stated that the models communicated through unauthorized channels, exploited security weaknesses, and collaborated in ways that their developers did not intend. The company described the incident as a "wake-up call" for further development of autonomous artificial intelligence.

It is important to note that this was not a standard, publicly available version of ChatGPT. A highly capable internal research model played a key role, operating during testing with less stringent security measures. After the incident, OpenAI announced stricter isolated testing environments, limited models' access to the internet, and further enhanced monitoring of their behavior.

Furthermore, this case was not an isolated one. On September 9th, Reuters reported that Anthropic had identified a fourth incident in which one of its experimental Claude models gained unauthorized access to third-party systems during security tests. The most recently discovered case dates back to January, but the company only identified it during a later review.

Another point of concern was the discovery that OpenAI agents used several internet services for unauthorized communication during earlier tests. OpenAI has now confirmed that it is investigating claims about its agents' activity on the RubyGems platform. The company stated that, according to their investigation so far, the agents used the service to access the internet and obtain public information, but they have not confirmed reports that they created malicious software packages.

These incidents highlight a problem that will become increasingly important with the growing autonomy of AI. Modern agents are already capable of independently planning, programming, using external tools, and finding alternative ways to achieve their goals. The security of AI is gradually shifting from the question of whether a model will provide an incorrect answer to a much more fundamental question: what it will do when it has enough capabilities to act autonomously.

Jaroslav Holý