As AI agents, bots, and programmatic access become an increasingly significant portion of web traffic, organizations need better tools to understand, analyze, and manage this activity.
A few years ago, a spike in web traffic almost always meant one thing: more people finding a site. That assumption no longer holds. A growing and largely invisible share of the requests hitting servers today don’t come from people at all. They come from AI agents, autonomous programs that crawl content, call APIs, and gather data at a pace no human ever could.
For most businesses, this has been a quiet problem. The traffic is real, the infrastructure load is real, and the bill at the end of the month is real. What hasn’t been real, until now, is any meaningful visibility into what’s actually happening.
AWS has changed that with the launch of the AI Traffic Analysis Dashboard for AWS WAF. It’s a launch that sounds narrow on the surface, but the implications stretch well beyond the security team.
The Problem Nobody Was Talking About
According to AWS, AI agents now represent somewhere between 30 and 60 percent of total web traffic for many organizations. That’s not a rounding error. For some businesses, that’s the majority of their traffic, and most teams have had almost no tools to understand it.
Traditional bot management was built for a different era. It was designed to catch malicious scrapers, credential stuffers, and inventory bots. It wasn’t designed to help distinguish whether the agent hitting a research portal is a legitimate AI assistant from a trusted partner, or an unknown entity extracting proprietary content at scale.
The question was never really ‘are bots visiting the site?’ It was always ‘which ones, doing what, and should that matter?’ Most organizations simply didn’t have a clean way to answer it.
The default decision most teams make is to either block aggressively and hope for the best, or allow everything and deal with the consequences later. Neither is a real strategy. Both are ways of avoiding a question the organization doesn’t have the data to answer.
That’s the gap the AI Traffic Analysis Dashboard is designed to close.
What AWS Built, and Why it Matters
The dashboard lives inside the existing AWS WAF console, under the AI Traffic Analysis tab of a web ACL. There’s no separate product to buy or configure. Organizations already running WAF Bot Control can open the tab and see it start populating immediately.
What makes this significant isn’t just the data it surfaces. It’s the context it provides. The dashboard doesn’t simply indicate that bots are visiting. It shows:
- Who is visiting: Actual named agents with owning organizations and verification status. There’s a real difference between a verified crawler from a known AI company and an unverified agent hitting an API endpoint 10,000 times an hour.
- What they’re trying to do: AWS classifies AI bot behavior by intent: search indexing, research, training data collection. The right response to each is different, and now that distinction is visible.
- When and where the traffic spikes: Fourteen days of trend data, updated in near real time. A new agent hammering a pricing API at 2am becomes visible rather than disappearing into aggregate traffic stats.
- Which content they value most: URL level breakdowns show exactly which pages and endpoints carry the heaviest AI load, useful for security, but also for understanding what that content is worth to the AI ecosystem.
The diagram below shows how the three pieces fit together: from the challenge, through the dashboard capabilities, to the decisions it enables:
What the diagram makes clear is that this isn’t just a security tool. The four capability panels, identity, intent, access patterns, and trends, each answer a different question that a different team inside an organization cares about.
Three Things This Actually Changes
Security decisions get sharper
The biggest shift is moving from ‘block all unusual traffic’ to ‘block the right traffic.’ When an agent is verified, belongs to an organization with an existing relationship, and behaves consistently with established access policies, blocking it would be counterproductive. When something unverified is scraping an entire product catalogue every six hours, allowing it would be a mistake too. The dashboard provides the information needed to tell the difference. We applied this same verify then decide discipline to infrastructure itself in our Kubernetes migration case study.
Infrastructure costs become explainable
If AI agents are driving a significant share of server load, they’re also driving a significant share of the cloud bill. Knowing which endpoints carry the heaviest AI traffic allows engineering teams to make informed decisions about caching strategies, rate limits, and capacity planning, rather than discovering the problem after an unexpected invoice. See how this played out in our cloud cost optimization work, where the same visibility first approach cut cloud spend without a single code change.
There’s a revenue conversation worth having
This is the one most organizations haven’t gotten to yet. Content and APIs have real value. AI organizations are already accessing them. The open question is whether that value is being captured, or simply given away for free.
AWS has published a reference architecture combining WAF Bot Control, Amazon CloudFront, and the x402 payment protocol. In practical terms, this allows organizations to charge AI agents for access to premium content, settled at the network edge, with no changes to the existing application. It’s a pattern worth exploring wherever content monetization is on the roadmap.
Visibility is a precondition for strategy. Organizations can’t decide who should access their content, on what terms and at what cost, without first being able to see who’s accessing it.
Questions Worth Bringing to the Next Review
For teams responsible for cloud security, infrastructure, or digital strategy, here are the questions this capability is designed to help answer:
- Which AI organizations are accessing our applications right now, and is there a formal relationship with any of them?
- Are any AI agents consuming infrastructure at a scale that isn’t justified by the value they provide?
- Do current WAF rules accidentally block legitimate AI partners while letting unknown scrapers through?
- Which parts of the application are most valuable to the AI ecosystem, and should that access remain free?
- Are there anomalies in AI traffic that could indicate a policy violation or an emerging security risk?
None of these are purely technical questions. They sit at the intersection of security, operations, product, and finance. The fact that the data to answer them is now available in the AWS console, at no additional cost for most WAF customers, removes the main excuse for not having the conversation.
Getting Started
For organizations already using AWS WAF, the path forward is straightforward. Enabling AWS WAF Bot Control on the web ACL, if not already active, is the first step, followed by opening the AI Traffic Analysis tab. No additional configuration is needed. The dashboard populates automatically as AI traffic flows through the WAF.
From there, reviewing the top accessed endpoints and the organizations sending the most traffic is a useful next step. Surprises are worth flagging: agents that weren’t expected, content being accessed far more than assumed, usage patterns that don’t match any known business rationale.
Those findings are most useful when brought into a broader conversation, a security review, a product discussion, a cost optimization exercise. The data is a starting point, not an end in itself.
The Bigger Picture
The internet was built on the assumption that the entity on the other end of a request is a person. The protocols, the business models, the legal frameworks, almost all of it was designed with a human in the loop.
That assumption is quietly breaking down. AI agents are now a permanent fixture of web traffic. Organizations that treat this as purely a security problem will find themselves in a permanent defensive crouch. Those that treat it as a strategic reality to be understood and shaped will have a meaningful advantage.
Visibility is where that starts. AWS has made it considerably easier to get there. What organizations do with that view is the interesting part.