e/acc for Software Development

As a practitioner, it would be apt to say that I am going through a roller coaster ride. Its mostly fun, but it does get scary sometimes, when its too fast. The advent of AI-powered coding assistants like GitHub Copilot and SuperMaven has accelerated development speeds by up to 55%, pushing us into an era where software is being built faster than ever before. An engineer like me could take upto a week to code a basic CRM, which is possible in minutes now: 

Credits: Bolt.new

This acceleration is not just a quantitative change but a qualitative one, ushering in a new paradigm where software itself becomes probabilistic.

I was particularly struck by a statement from Jensen Huang, the CEO of NVIDIA, who predicted that

“every single pixel will be generated soon. Not rendered: generated.”

Let that sink in. 

I had heard about this a year ago, but dismissed it as too far-fetched. Lo and behold, Salesforce launched Generative Canvas (going GA in 2026) and Microsoft launched a similar one as well. So while we think the future is far away, it seems to be moving closer every second. 

Credits: Salesforce.

This encapsulates a fundamental shift in how we think about software and digital content. We're moving away from deterministic systems—where outputs are precisely predictable—to probabilistic systems that can produce a range of possible outcomes based on learned patterns and data inputs.

This shift poses a significant challenge for software testing. Traditional testing methodologies are built on deterministic principles, where a specific input should produce a specific output every time. But how do you test a system designed to generate varied outcomes? 

The Emergence of Probabilistic Software

Probabilistic software leverages AI and machine learning to generate outputs that are not strictly predetermined. This is evident in areas like natural language processing, image generation, and personalized user experiences. The software learns from data and makes predictions or generates content that can vary each time, even with the same input.

Think about this. Which one is easier to test (option A or B)?

Credits: From my slides at TrailblazerDX conference 2024.  

As development cycles shorten, the window for thorough testing narrows. Traditional testing methods, which are often time-consuming and require meticulous planning and execution, are becoming less feasible. 

This approach aligns with how humans interpret and interact with the world—we make decisions based on probabilities and past experiences rather than fixed rules. However, this introduces unpredictability into software behavior, making it challenging to test using traditional methods.

“Probably”: The future of testing

Testing probabilistic software requires a paradigm shift. Deterministic testing assumes a one-to-one relationship between input and output. But with probabilistic software, the same input might produce different, yet acceptable, outputs. This variability means that testers need to consider a range of possible outcomes and assess the software's performance across that spectrum. As my friend and cofounder-Shriyansh highlighted:

“We might need to test software in the future like how we test video games today. Where most of the moves are tested and a probabilistic simulation is created.”

This becomes more imperative, in verticals like CRM (Salesforce), eCommerce platforms, HealthTech, and BFSI (Banking, Financial Services, and Insurance), as the complexity of these systems could exponentially rise with every combination. 

Hence TestZeus and Hercules

Recognising these challenges, we set out to develop a solution that would bridge the gap between the new probabilistic nature of software and the need for robust testing methodologies. This led to the creation of TestZeus’ Hercules. 

Hercules, is an execution engine that balances dev agents in a maker/checker paradigm. It utilizes multi agentic systems to run extensive simulations efficiently, providing rapid feedback to developers.

The core vision behind TestZeus and Hercules is to redefine software testing for the modern era. We aim to provide agents that not only accommodate but embrace the probabilistic nature of contemporary software. By building in an AI native way, Hercules can use tools like browsers/APIs/databases to achieve a testing goal (“Test cart checkout on an ecommerce app”) by planning out the steps and executing them autonomously. This also elevates the user to perform higher order tests (application wide tests), and eliminates the need for costly tools. 

Interestingly, neither last decade's tools, nor last year's copilots can handle the probabilistic nature of GenUX. 

Looking Ahead

As we stand on the cusp of this new era in software development, it's clear that our tools and methodologies must evolve in tandem. TestZeus and Hercules represent our commitment to leading this evolution. By embracing the probabilistic nature of modern software, we can ensure that innovation continues unabated while maintaining the trust and reliability that users expect.

Hercules is free and open source under the AGPL v3 license because we believe in breaking down barriers to access. By sharing our source code, we give you the power to customize and extend Hercules to fit your unique testing needs—because let’s face it, testing is never one-size-fits-all. In a world where trust is everything, especially with AI and developer tools, open sourcing Hercules is our way of being transparent and building confidence in what we’ve created.

We are just getting started on this journey. Lets democratise and join hands to solve "software quality" together.

-Robin Gupta.

CoFounder at TestZeus.