Did Anthropic Just Stab Test Automation Tools on the side?

Let’s talk about the ripple that Anthropic’s “Computer Use” feature is making—one that could turn into a full-on tidal wave in the test automation world. Imagine this: you tell a program to “go to Google and download 10 images,” and voilà, it takes the reins, operating your browser, clicking through search results, and saving those images, all without a single click from you. This isn’t just auto-mation; it’s auto-magic. And for those of us who’ve built careers around test automation tools (like me), it’s a bit like watching a plot twist where the supporting character suddenly takes center stage.

With Anthropic’s release, the question for traditional automation tools—those script-heavy, sometimes finicky, and often expensive solutions—isn’t just about relevance; it’s about survival. Are existing tools about to become relics, as action models take the wheel and drive us into a new era of intelligent, "real" testing?

A Glimpse in the Rear-View Mirror: The Origins of Test Automation

Before we get carried away with where test automation is headed, let’s look back.

I started my (software)career as a HP QC and QTP certified manual tester. Back in the day, test automation had its roots in software like Mercury Interactive’s WinRunner and LoadRunner, tools that transformed tedious, GUI-driven tests into scriptable routines. This was cutting edge stuff, and I still remember the spark in my eye, when I saw that a computer could be automated. It may sound old-school, but this was a game-changer for testers, taking them out of “manual labor” mode and giving them time to focus on strategy. These early tools automated repetitive actions, but there was no intelligence behind them—if you didn’t spell out every step, they’d get lost faster than a GPS on a cloudy day.

Fast forward a few years, and Selenium arrived, changing the game again. Developed as an internal tool at Thoughtworks , Selenium allowed testers to programmatically interact with web browsers, paving the way for cross-browser testing. Selenium quickly became a staple in the QA world, largely because it was open-source and customizable. But even with Selenium’s flexibility, it wasn’t a “smart” tool. It followed commands, yes, but like a loyal but unthinking assistant—it didn’t question, interpret, or adapt.

Cut to the Present (2024): So Many Tools, So Little Intelligence

Today, the test automation market is jam-packed with tools—everything from the legacy big shots like IBM 's Rational suite to newer players like Tricentis and SmartBear. In fact, Markets and Markets reports that the automation testing market is projected to grow from $20 billion in 2022 to $50 billion by 2030.

Clearly, the world wants automation.

The only problem? Most of these tools are great at following orders but clueless when it comes to actual “testing.” They check boxes but don’t ask questions. They follow scripts but don’t understand the “why” behind them.

Think of it this way: traditional automation tools are like actors in a play, perfectly executing lines but lacking any real understanding of the script. If a button moves or a label changes, the script breaks, and it’s back to the drawing board for testers. It’s a game of endless maintenance and patchwork fixes. We’re caught in this cycle where testers are so busy babysitting scripts that the idea of truly improving product quality takes a backseat.

Ive tried 17 "test" tools and frameworks(both free and paid), some of them running on this laptop as I type this article and sadly all of them break and fail, the moment you try something advanced on the UI. The salt on the wounds? None of these tools care about the quality of the software or probing the requirements for gaps and bugs.

Why "test" in paranthesis? Because IMHO none of them do real testing, and boil down quality to "Clicks on a browser".

Are all of the test automation tools, glorified browser automation wrappers ?

(ouch!)

Large Action Models: The Plot Thickens

For the first time, we’re seeing models that don’t just follow instructions—they interpret intent. Instead of scripting out every step, we tell it what we want, and it figures out the how. Anthropic has effectively introduced a system that understands commands like, “download the latest invoices, check them for errors, and report back,” and executes each step based on that higher-level instruction.

This shift is massive because it lets us move beyond rigid scripting into intent-driven automation. LAMs can handle multi-step processes, make on-the-fly adjustments, and interact across different systems. Imagine cutting test maintenance in half just because your tools get it.

These tools are not perfect, but this is the worst they'll ever be.

In the OSWorld benchmarking tests, which evaluate attempts by AI models to use computers, Claude 3.5 Sonnet scored a grade of 14.9%. Though that's far lower than the 70%-75% human-level skill, it's almost double the 7.7% acquired by the next best AI model in the same category. Thanks ZDNET for the report link.

By giving automation the ability to understand and respond, we’re stepping into an era where test tools aren’t just “tools” but true collaborators. This change allows QA teams to stop playing “whack-a-bug” and start focusing on the bigger picture—like innovating, strategizing, and improving product quality across the board.

Agents Assemble: The Move Towards Autonomy

So, if Large Action Models represent the dawn of intelligent automation, where does that lead us? To agents, of course—digital entities that go beyond following directions and can actively analyze, reason, and react. Agents can understand application flows, adapt to UI changes, detect edge cases, and ultimately function with a level of autonomy we’ve only dreamed about.

Imagine an agent that doesn’t just test a signup form but understands the entire user journey. It can recognize if the UX is inconsistent, if accessibility issues crop up, or if there’s a regulatory compliance risk. With this level of intelligence, agents can handle complex workflows without constant oversight.

If we’re really being ambitious, picture this: an agent network running 24/7 across all your environments, sniffing out bugs, suggesting improvements, and keeping your software robust.

Why Open Source Is the Future of Test Automation

If this shift to intelligent, autonomous agents is going to stick, we need the openness and collaboration that only the open-source community can bring. It’s no longer enough to build proprietary, siloed tools that only a few can customize, or pay for.

Quality software is everyone's right.

The future of testing demands a community-driven, democratized platform where anyone can contribute to, adapt, and improve agents. The limitations we experience today are simply milestones, points in a progression where each shortfall is an opportunity to improve.

The Red Hat 2023 report found that 82% of IT leaders believe open-source software will drive AI adoption because of transparency, innovation, and cost-efficiency. Imagine a shared platform where any company or tester can build custom agents for their unique needs. We’d see vertical agents for accessibility checks, security validations, regulatory compliance, and more—all shared with the community. The faster we can share these agents, the quicker we’ll accelerate testing innovation.

An open-source, agentic test automation ecosystem isn’t just a pipedream; it’s the next logical step. In short, it’s a way to crowdsource intelligence itself into our testing processes.

Let’s Forge a Smarter Path Together

In launching “Computer Use,” Anthropic has sent a signal. This isn’t just a “new feature”; it’s a challenge to rethink test automation from the ground up. It’s a reminder that our goal in QA is to assure quality, not just run scripts or tick boxes. We need tools—and agents—that aren’t just reactive but proactive, that can think, adapt, and even push us to improve.

The future of testing won’t belong to static paid tools.

It will belong to intelligent, autonomous agents that operate with an understanding of software quality, a grasp of user experience, and a "vision" for resilience (pun intended). As someone who’s lived in the trenches of automation, I’m more than ready for this future.

Are you ready to move from tools to true testing partners? Let’s roll up our sleeves and redefine what it means to “test.” The next generation of agents isn’t waiting; they’re already here.

In my unbiased opinion, an open-source, community-oriented vertical test automation platform is the way forward.

Are you ready for the "Gutenberg" moment in Test automation?