Technology

Agentic AI's rising autonomy prompts urgent governance and security reforms

Tuesday, 2 September 2025 4:14AM UTC

The rapid evolution of Agentic AI from passive assistants to autonomous decision-makers is transforming organisational operations but also exposing critical security vulnerabilities. As new standards like MCP emerge, businesses must implement robust governance frameworks to harness AI's productivity gains while managing escalating risks.

Since the public emergence of ChatGPT in late 2022, public perception of artificial intelligence (AI) largely framed it as a helpful, responsive chatbot capable of generating text and even computer code on demand. However, in under three years, the technology has evolved substantially, giving rise to what is now being termed Agentic AI — systems that transcend simple prompt responses to perform complex, multi-step tasks autonomously. These advanced AI agents are able to invoke APIs, execute commands, and write or deploy code independently, thus shifting from passive assistants to active decision-makers. While this evolution promises significant utility and productivity gains, it simultaneously introduces pronounced security risks and governance challenges for organisations.

The foundational risks of Agentic AI became apparent as early as 2023, highlighted in the OWASP Top 10 for Large Language Model Applications report, which coined the term "excessive agency" to describe the dangers of granting AI systems too much autonomy. This vulnerability arises when AI agents operate more like independent actors than controlled assistants, potentially resulting in unintended and harmful actions—ranging from innocuous mistakes such as mismanaging scheduling to severe consequences like unauthorized file deletions or the rogue provisioning of cloud infrastructure. Real-world demonstrations have shown that high-profile AI tools, including Microsoft Copilot and Salesforce's Slack-integrated agents, have been exploited to misuse escalated privileges and extract sensitive data.

Responding to these challenges, 2025 has seen the introduction of new standards and protocols aimed at safely managing the capabilities of AI agents. Among the most notable is Anthropic's Model Context Protocol (MCP), designed to maintain shared memory, task structures, and tool access during extended AI agent sessions. MCP provides a framework for defining explicit permissions and memory retention for agents, effectively operating as a kind of 'glue' that holds an agent’s operational context together over time and across functions. Despite these advances, MCP currently emphasises expanding an agent’s functional scope rather than constraining it, leaving critical issues like prompt injection resistance, command scoping controls, and protection against token abuse insufficiently addressed. Such gaps in security design have already been exposed through vulnerabilities involving memory poisoning and command misuse, especially where encryption and scoping of shared memory are lacking.

The implications of Agentic AI's ascent are profound for business operations. Coding assistants like Claude Code and Cursor have transcended their origins as mere code suggestion tools to become autonomous task executors, with internal studies revealing productivity lifts of over 50%. Anthropic noted that 79% of Claude Code’s usage now centres on automated task execution, not just code assistance. Moreover, MCP integration is expanding Agentic AI’s influence beyond programming, encompassing functions as diverse as email triage, sales planning, meeting preparation, and document summarisation. These developments mean that organisations cannot treat these tools as novelties but must consider them integral components of operational infrastructure, necessitating robust oversight from leadership including CIOs, CISOs, and Chief AI Officers.

To safely harness the benefits of Agentic AI, businesses need to implement comprehensive governance, risk management, and strategic planning frameworks from the outset. This includes initiating controlled pilot programmes, enforcing rigorous code reviews, restricting tool permissions, and employing sandboxing to isolate AI agent operations. Limiting agent autonomy to essential functions, avoiding unnecessary root access or long-term memory retention, and training developers on secure usage practices such as scope control and fallback protocols are vital. Failing to embed these safeguards risks outages, data breaches, and regulatory penalties. The emerging consensus is clear: organisations that proactively integrate AI agents as a core architectural element, rather than treating them as experimental add-ons, will be best positioned to leverage their substantial productivity advantages while mitigating associated risks.

📌 Reference Map:

Paragraph 1 – ^[1]
Paragraph 2 – ^[1], ^[2], ^[3], ^[4], ^[5], ^[6], ^[7]
Paragraph 3 – ^[1], ^[6]
Paragraph 4 – ^[1]

Source: Noah Wire Services

More on this

https://itbrief.com.au/story/the-business-benefits-and-challenges-of-agentic-ai - Please view link - unable to able to access data
https://owasp.org/www-project-top-10-for-large-language-model-applications/ - The OWASP Top 10 for Large Language Model Applications is a comprehensive list identifying the most critical security vulnerabilities in LLM applications. It encompasses various risks, including prompt injection, insecure output handling, training data poisoning, and excessive agency. The project aims to provide actionable guidance and tools to ensure the secure development, deployment, and governance of generative AI systems. The initiative has evolved into the OWASP GenAI Security Project, a global, open-source effort dedicated to addressing security and safety risks associated with generative AI technologies, including LLMs and agentic AI systems.
https://genai.owasp.org/llmrisk2023-24/llm08-excessive-agency/ - The 'Excessive Agency' vulnerability, identified as LLM08 in the OWASP Top 10 for Large Language Model Applications, refers to the risks associated with granting LLM-based systems too much autonomy. This can lead to unintended and potentially damaging actions in response to unexpected or ambiguous outputs. The root causes typically involve excessive functionality, permissions, or autonomy granted to the LLM. This vulnerability can impact the confidentiality, integrity, and availability of systems, depending on the interactions the LLM-based application can perform.
https://www.cloudflare.com/learning/ai/owasp-top-10-risks-for-llms/ - Cloudflare's article discusses the OWASP Top 10 risks for Large Language Models (LLMs), focusing on vulnerabilities such as prompt injection, insecure output handling, training data poisoning, and excessive agency. It highlights the importance of understanding these risks to develop secure LLM applications. The article provides insights into how granting LLMs excessive autonomy can lead to unintended consequences, emphasizing the need for developers to limit functionality, permissions, and autonomy to the minimum necessary levels to prevent potential security breaches.
https://www.hackerone.com/blog/hackerone-and-owasp-top-10-llm-powerful-alliance-secure-ai - HackerOne's blog post highlights the collaboration between HackerOne and the OWASP Top 10 for Large Language Models to enhance AI security. It delves into the 'Excessive Agency' vulnerability, explaining how granting LLMs too much autonomy can lead to unintended and potentially harmful actions. The article emphasizes the importance of limiting the tools, functions, and permissions granted to LLMs, implementing human-in-the-loop validation, and ensuring secure plugin design to mitigate these risks and maintain the integrity of AI systems.
https://www.fortanix.com/blog/top-10-security-risks-for-large-language-models-owasp - Fortanix's blog post outlines the top 10 security risks for Large Language Models (LLMs) as identified by OWASP. It provides an in-depth look at vulnerabilities such as prompt injection, insecure output handling, training data poisoning, and excessive agency. The article discusses how granting LLMs excessive permissions, particularly in agentic architectures or plugin settings, can lead to unintended, risky, or harmful actions without adequate human oversight or control. It also offers mitigation strategies to address these vulnerabilities and enhance the security of LLM applications.
https://www.tigera.io/learn/guides/llm-security/owasp-top-10-llm/ - Tigera's guide provides a comprehensive overview of the OWASP Top 10 risks for Large Language Models (LLMs), including prompt injection, insecure output handling, training data poisoning, and excessive agency. It explains how excessive agency occurs when an LLM-based system is granted too much autonomy, allowing it to perform damaging actions in response to unexpected or malicious inputs. The guide offers prevention strategies, such as limiting the plugins and tools that LLM agents are allowed to call, ensuring strict parameterized input, and enforcing least-privilege access control.

Noah Fact Check Pro

The draft above was created using the information available at the time the story first emerged. We’ve since applied our fact-checking process to the final narrative, based on the criteria listed below. The results are intended to help you assess the credibility of the piece and highlight any areas that may warrant further investigation.

Freshness check

Score: 8

Notes: The narrative introduces the term 'Agentic AI' and discusses its evolution, referencing the OWASP Top 10 for Large Language Model Applications report from 2023. The earliest known publication date of similar content is November 2024, with Anthropic's Model Context Protocol (MCP) being introduced in November 2024. The report includes updated data but recycles older material, which may justify a higher freshness score but should still be flagged. The persistent repetition in the notes indicates that the content may contain outdated or recycled information that impacts its freshness assessment.

Artificial Intelligence
Agentic AI
Security Risks