Researchers expose universal jailbreak undermining AI chatbot ethics and security

More on this

1. https://www.techradar.com/computing/artificial-intelligence/people-are-tricking-ai-chatbots-into-helping-commit-crimes - Please view link - unable to able to access data
2. https://www.ft.com/content/14a2c98b-c8d5-4e5b-a7b0-30f0a05ec432 - Hackers are increasingly finding vulnerabilities in powerful AI models to showcase their flaws. Pliny the Prompter, a pseudonymous hacker, has manipulated models like Meta's Llama 3 and OpenAI's GPT-4o to produce dangerous content. His actions are part of an international effort to raise awareness about the capabilities and weaknesses of these models released by tech companies for profits. Ethical hackers and cybersecurity experts are discovering ways to bypass the safeguards of language models, leading to the growth of AI security startups. These startups offer tools to protect companies from potential misuse of AI. Global regulators are also working to address the potential dangers posed by AI, with legislation such as the EU's AI Act and upcoming bills in the UK and Singapore. As AI models become more advanced and integrated with technology, the risks of manipulation and data leakage are expected to increase, necessitating stronger security measures.
3. https://www.axios.com/2024/04/03/ai-chatbots-def-con-red-team-hack - Hackers have been employing social engineering tactics to bypass safeguards on popular AI chatbots, causing them to break their rules and share sensitive information. This was highlighted in the results from a DEF CON red teaming challenge held in August, where 15.5% of the 2,702 conversations successfully manipulated the AI models. The hackers used methods such as directing the bots to follow specific scripts or convincing them to believe falsehoods. Despite some failures, the recurring success of these jailbreaks poses a significant challenge for developers in distinguishing between attacks and acceptable usage. The industry's struggle to address these vulnerabilities risks leading to a period of disillusionment with generative AI technology.
4. https://www.ft.com/content/cf11ebd8-aa0b-4ed4-945b-a5d4401d186e - Anthropic, an AI start-up, has developed "constitutional classifiers" to prevent its AI models from producing harmful content. The system monitors both inputs and outputs to block illegal or dangerous information and is built on adaptable rules defining permitted and restricted material. This innovation responds to growing concerns over "jailbreaking" AI models, where users manipulate them to generate harmful content, such as instructions for making chemical weapons. Aimed at bolstering AI safety, the measure reflects the industry's bid to avoid regulatory issues and ensure secure AI use. Despite its effectiveness—it helped Anthropic's Claude 3.5 Sonnet model reject 95% of harmful attempts compared to 14% without safeguards—the system increases operating costs significantly. The tech industry, including Microsoft and Meta, is implementing similar measures to balance security and functionality in AI models, albeit with challenges in maintaining the user experience and managing additional costs.
5. https://www.ntu.edu.sg/news/detail/using-chatbots-against-themselves-to-jailbreak-each-other - Computer scientists from NTU have found a way to compromise artificial intelligence (AI) chatbots – by training and using an AI chatbot to produce prompts that can ‘jailbreak’ other chatbots. ‘Jailbreaking’ is a term in computer security where computer hackers find and exploit flaws in a system’s software to make it do something its developers deliberately restricted from doing. The researchers’ used a twofold method for ‘jailbreaking’ LLMs, which they named “Masterkey”. First, they reverse-engineered how large language models (LLMs) detect and defend themselves from malicious queries. With that information, they taught an LLM to automatically learn and produce prompts that bypass the defences of other LLMs. This process can be automated, creating a jailbreaking LLM that can adapt to and create new jailbreak prompts even after developers patch their LLMs. Their findings may be critical in helping companies and businesses to be aware of the weaknesses and limitations of their LLM chatbots so that they can take steps to strengthen them against hackers. After running a series of proof-of-concept tests on LLMs to prove that their technique indeed presents a clear and present threat to them, the researchers immediately reported the issues to the relevant service providers upon initiating successful jailbreak attacks.
6. https://www.scientificamerican.com/article/jailbroken-ai-chatbots-can-jailbreak-other-chatbots/ - AI chatbots can convince other chatbots to instruct users how to build bombs and cook meth. Today’s artificial intelligence chatbots have built-in restrictions to keep them from providing users with dangerous information, but a new preprint study shows how to get AIs to trick each other into giving up those secrets. In it, researchers observed the targeted AIs breaking the rules to offer advice on how to synthesize methamphetamine, build a bomb and launder money. Modern chatbots have the power to adopt personas by feigning specific personalities or acting like fictional characters. The new study took advantage of that ability by asking a particular AI chatbot to act as a research assistant. Then the researchers instructed this assistant to help develop prompts that could “jailbreak” other chatbots—destroy the guardrails encoded into such programs. The research assistant chatbot’s automated attack techniques proved to be successful 42.5 percent of the time against GPT-4, one of the large language models (LLMs) that power ChatGPT. It was also successful 61 percent of the time against Claude 2, the model underpinning Anthropic’s chatbot, and 35.9 percent of the time against Vicuna, an open-source chatbot. That takes time. But asking AI to formulate strategies that convince other AIs to ignore their safety rails can speed the process up by a factor of 25, according to the researchers. And the success of the attacks across different chatbots suggested to the team that the issue reaches beyond individual companies’ code. The vulnerability seems to be inherent in the design of AI-powered chatbots more widely. OpenAI, Anthropic and the team behind Vicuna were approached to comment on the paper’s findings. OpenAI declined to comment, while Anthropic and Vicuna had not responded at the time of publication. “In the current state of things, our attacks mainly show that we can get models to say things that LLM developers don’t want them to say,” says Rusheb Shah, another co-author of the study. “But as models get more powerful, maybe the potential for these attacks to become dangerous grows.” The challenge, Pour says, is that persona impersonation “is a very core thing that these models do.” They aim to achieve what the user wants, and they specialize in assuming different personalities—which proved central to the form of exploitation used in the new study.
7. https://www.infosecurity-magazine.com/news/cybercriminals-jailbreak-ai - SlashNext, a cybersecurity company, has uncovered a concerning trend in the world of artificial intelligence (AI) chatbots. Referred to as “jailbreaking,” this practice involves users exploiting vulnerabilities within AI chatbot systems, potentially violating ethical guidelines and cybersecurity protocols. AI chatbots like ChatGPT have gained notoriety for their advanced conversational abilities. However, some users have identified weaknesses in these systems, enabling them to bypass built-in safety measures. This manipulation of chatbot prompting systems allows users to unleash uncensored and unregulated content and is raising ethical concerns. Jailbreaking AI chatbots involve issuing specific commands or narratives that trigger an unrestricted mode, enabling the AI to respond without constraints. Online communities have emerged where individuals share strategies and tactics for achieving these jailbreaks, fostering a culture of experimentation and boundary-pushing. “These platforms are collaborative spaces where users share jailbreaking tactics, strategies, and prompts to harness the full potential of AI systems,” commented Callie Guenther, cyber threat research senior manager at Critical Start. “While the primary drive of these communities is exploration and pushing AI boundaries, it’s essential to note the double-edged nature of such pursuits.” SlashNext explained that this trend has also attracted the attention of cyber-criminals who have developed tools claiming to use custom large language models (LLMs) for malicious purposes. However, research suggests that most of these tools, with the notable exception of WormGPT, merely connect to jailbroken versions of public chatbots, disguising their true nature and allowing users to exploit AI-generated content while maintaining anonymity.