Being shortlisted is not the same as winning
In October 2025, a1qa said it had been shortlisted in two categories: Best Test Automation Project (Functional) and AI Driven Automation. The company also credits its work with faster delivery, less manual effort, and lower costs. Those are the vendor’s claims about its own projects, not independently measured results from a controlled comparison.
The official North American Software Testing Awards 2025 winners page lists Infosys – Healthcare project as a winner of AI-Powered Quality Assurance, a category distinct from AI Driven Automation. a1qa was a finalist in the latter, not the winner of the former. Finalist is a recognition, but it is not equivalent to the award nor does it by itself validate a risk or cost reduction rate.
Awards, conferences and vendor cases show where AI is being tested: test design, defect analysis, visual automation, data preparation and evaluation of AI systems. The World Quality Report 2025–26, produced by Capgemini, Sogeti and OpenText, says that 89% of organizations surveyed were testing or using generative AI in software quality. Only 15% had extended its use to the entire company.
The report also indicates that organizations that reported results observed, on average, a 19% increase in productivity; a third said the improvements were small. As the survey was prepared by companies in the sector, it serves to know what these organizations report, but it does not guarantee the same result in all companies nor does it function as an independent comparison.
How to assess a promise of faster delivery
Ask for a clear comparison: which stages of delivery were made faster? How serious were the defects measured? Was review, test maintenance and incident time counted? How many projects were analyzed and for how long? Who collected the data? Ask for quantities and ratios, not just general percentages, and separate the time saved running tests from the cost of integrating and monitoring the tool.
A useful controlled test identifies a bounded task, prior success criteria, data the agent can process, stopping mechanism, independent review, and total cost of ownership. Without those elements, the case serves as inspiration, not a financial forecast for your team.
In a published case on regression testing for a retail company, a1qa describes a set of about five thousand tests and a first stage limited to a part of the product. According to the company, the time to decide if a version could be published dropped from about two weeks to five business days or less. It also reports that preparing some converted tests took more than 30% less time. AI helped propose tests from requirements, find gaps, prepare dummy data and classify bugs; a person approved the tests before incorporating them.
The case is more informative than an isolated percentage because it explains what part of the work was changed and how. Still, it is published by the vendor and is not an independent evaluation or a guarantee that other teams will get the same. To compare it with another project, you would have to know what tests were measured, how much review time was included, and what happened to defects that made it to production and subsequent maintenance.
A checklist for evaluating suppliers
Ask for five pieces of information to compare: what the project included and what it left out; what the initial situation was like and how long the measurement lasted; what “savings” and “default” mean; how much everything cost, including screening people; and what happened to security, stability and maintenance in the following versions. Choose a task with a clear expected outcome and compare the current process with the assisted one using equivalent examples. Decide in advance what result would cause you to extend, correct, or stop the initial test.
External recognition can help find experimenting teams and questions worth asking. The adoption decision, however, needs evidence from your own product: reproducible results, less relevant risk, or lower total cost without shifting invisible work to QA, development, or operations.
An award can demonstrate that a candidacy was evaluated and recognized within a specific category, edition and jury. It serves to discover suppliers, techniques or projects that are worth studying. It does not automatically prove that the solution is superior to all alternatives, that the result is repeatable on other computers, or that the return is worth the cost. The exact category matters: “AI Driven Automation”, “AI-Powered Quality Assurance” and “test automation” are not synonyms. Before using a badge as evidence, confirm whether the company was a finalist or winner, what the official category was, and what project it was awarded to.
Keep an award separate from a vendor’s account of its business results. A company may receive legitimate recognition and publish a case study with positive outcomes, but those outcomes still need a defined scope, time period, and measurement method. Published productivity figures may not say what work was counted, whether setup and review were included, or what happened to defects in production. Describe the figures as the vendor’s own claims unless there is independent evidence. That distinction helps readers assess the claim instead of repeating its headline.
Treat the a1qa case as a hypothesis to test
The case published by a1qa on regression for retail describes an inventory of thousands of tests and a limited initial phase. The company reports that the release decision went from about two weeks to five business days or less, and that preparing some converted cases required more than 30% less time. It also describes testing suggestions from requirements, gap finding, dummy data, and fault triage, with human approval. It's more informative than a generic phrase because it says which tasks you changed; it remains a case written by the provider and does not show an independent comparison group.
To convey that experience to your team, ask what types of tests came in, how many were transformed, how much human time was counted, and what proportion needed correction. Check if the improvement lasted several releases, if the equipment or acceptance rules changed, and if the escaped defect remained stable. Ask for examples of failed outcomes in addition to successes. If the provider cannot share data due to confidentiality, it can describe aggregate methodology or allow a controlled test on controlled data; there is no need to disclose sensitive code to clarify what was measured.
Agree on success criteria and stop conditions before looking at the results. A pilot can fall short because the tool is a poor fit, the test context is incomplete, security rules limit access, or integration is unreliable. Those causes need different responses. Faster execution is not a win if serious defects or maintenance work increase. An award can be a reason to look at a product; a purchase should rest on repeatable results in your own workflow, plus a clear way to export your data and leave the service.
Agree on success and stop thresholds before seeing the results. A controlled test can fail due to poor tool fit, insufficient context, security limits, or integration; These causes are not equivalent. If speed improves but serious defects or maintenance work increases, the balance is not positive. If a prize motivates you to look at a product, good; The purchase must depend on reproducible evidence in your own processes and an output that allows you to export the data and deactivate the solution without leaving the computer locked.
Questions to ask in a vendor demo
Apply the same care to announcements about awards. A finalist list confirms that entries were submitted and assessed under particular rules; it does not tell you how many entries there were or how every other candidate performed. The winners page confirms the published results, while the vendor can explain which product or project it entered. Each source answers a different question. Comparing them helps separate industry recognition from independent evidence for a vendor’s commercial claims.
Also check the output and portability. Request to export cases, results and evidence in formats that your team can retain; asks who has access to the data, how long it is kept, and how it is deleted at the end of the controlled trial. Review what happens if the provider changes or the license is canceled. A survey helps you find a company to talk to, but a helpful evaluation includes terms of use, operating limits, responsibilities, and an exit plan. The final decision should be able to be justified even if the award ceases to be in the news.
The same criterion is used to read market announcements. A list of finalists indicates that there were applications and an evaluation according to certain rules; It does not indicate how many solutions participated or how the candidates that did not receive a mention performed. The winners page allows you to verify the official result, while the supplier can clarify which part of their product or project they presented. Each document answers a different question. Contrasting them avoids confusing a company's reputation with the independent validation of all its commercial figures.