How AI-Assisted QA Responded to Security Vulnerabilities Before Launch

Imagine a client walks in already burned once, and now a security audit just handed them a second reason to worry.

Here is what happened. They had given months and spent thousands of dollars on a build with another agency, and the client still did not have what they paid for. Progress had stopped. The output did not match the invoices. They were done waiting and were ready for a fresh start.

There is nothing unusual here. Picking up someone else’s half-finished project is common for any agency that does custom software development. You learn the codebase and start fixing what is broken.

But this project had a twist: before we wrote a single line of code.

Before handing us anything, the client brought in a third-party security firm to audit the existing application. Smart move, especially after the last experience they had. The audit came back flagging multiple security vulnerabilities.

Suddenly, the client was facing two problems instead of one. They already suspected the product was not worth what they paid. Now they had proof it might not even be safe.

That is where we came in, to figure out just how bad it really was.

What We Were Actually Looking At

We want to be precise about our starting position, because it shapes everything that follows.

We had not written a line of this application. Nobody on our side had. We had no history with the architecture, no accumulated knowledge of which modules were fragile, no memory of why a particular decision had been made two years earlier. The people who could answer those questions were at a different company and were no longer involved.

An external audit had flagged security vulnerabilities in a codebase that was, to us, completely unfamiliar.

Our assignment was basically a complete security assessment of that application. Not a spot check on the audit findings. A full assessment, so the client could make an informed decision about the state of what they owned.

Why a Security Audit Lands Differently Than a Bug Report

A functional bug is easy to have a conversation about. It either happens or it does not. The client can usually see it, and the fix is scoped in an afternoon.

Security findings do not behave that way.

Functional QA asks whether the application does what it is supposed to do. Security testing in manual QA asks what else the application can be made to do. An application can pass every functional test it has ever been given and still carry gaps that nobody looked for, because nobody was assigned to look.

That is what makes late-stage findings so uncomfortable. The client had already spent 18 months and a real budget on this build. Being told there were security vulnerabilities in it did not just create a technical task. It raised a question about whether the whole thing was sound.

Answering that question was the job.

Getting Oriented in Code Nobody on Our Side Had Written

The hardest part of the first day was not testing. It was comprehension.

Before we could assess anything, we had to understand how the application was put together. How data moved through it. Where the sensitive operations lived. Which parts of the system had been built carefully and which had clearly been rushed.

Normally, that understanding gets built slowly, over months of working on a product. We did not have months.

This is where AI-assisted QA made the biggest practical difference, and it is not the part people usually talk about. Working with Claude, we could move through unfamiliar code far faster than we could reading it cold. We could ask what a section was doing, work through how it connected to the rest of the system, and get a usable mental model of the application in hours instead of weeks.

Where AI Genuinely Helped

Across the assessment, the AI assistance was useful for:

That rule existed for a reason. An AI assistant will follow a line of reasoning wherever it leads, and it will describe a theoretical problem in the same confident tone it uses for a real one. If we had delivered unverified output, we would have handed developers a list of maybes dressed up as findings. That damages trust faster than reporting nothing.

Two Days, and What That Number Actually Means

We identified and confirmed several security flaws within two days and wrote them down as part of a report that development teams could start working on at any moment.

We also decided to be careful with the numbers, as that part is most likely to be misunderstood by everyone.

It also did not replace what the third-party security company did. Their audit was independent, and independence is exactly what gives an external assessment its value. What the internal work did was respond to that audit properly, on an unfamiliar codebase, at a speed that kept the project moving.

The speed came from the combination. QA discipline plus AI assistance. Neither one alone would have produced it.

The Report Was the Real Deliverable

Finding a vulnerability is not the same as fixing one. A finding that a developer cannot locate is not resolved. It is queued.

So we put most of the effort into the report. Each finding included:

That combination removed the reproduction and location work from the developer’s plate entirely.

Anyone who has handed a vague security ticket to a development team knows the pattern that follows. The developer tries to reproduce it. Something does not match. A discussion starts about whether the finding is even real. Days pass before a line of code changes, and almost none of that time is spent fixing anything.

Evidence plus location plus root cause collapses that cycle. The developer opens the report, sees the affected code, understands why it is a problem, and starts working.

What Changed on the Developer Side

The detail in the report made it straightforward for developers to understand the exact location and cause of each vulnerability. That reduced debugging time and enabled faster implementation of security fixes.

There was a quieter effect too, and it is worth naming for anyone in an engineering management role.

Security findings can turn adversarial quickly. QA reports a risk, development questions the severity, and the conversation becomes about credibility instead of remediation. Well-evidenced findings take that argument off the table. When the evidence is sitting in front of everyone, the discussion moves straight to how to fix it.

Limits Worth Being Clear About

If you are considering a similar approach, these are the parts we would not skip.

Where the Project Landed

The assessment strengthened the application’s overall security posture. The project moved forward with significantly improved security standards, and the client acknowledged the quality of the QA and security testing work.

For a client who had just spent 18 months and a real budget getting an unsatisfactory result elsewhere, that mattered beyond the technical outcome. They had arrived with reasonable doubts about whether anyone was going to be straight with them about the state of their product. A report with evidence in it is a hard thing to argue with, in either direction.

What We Would Tell Another QA Engineer

AI does not replace QA expertise. We say that as someone who leaned on AI heavily during our first QA penetration testing assignment and still had to make every real decision ourselves.

What AI does, when used properly, is take friction out of the parts of the work that slow you down. It shortens the time between suspicion and conclusion. It makes an unfamiliar codebase readable in hours instead of weeks. It helps you produce better evidence and clearer documentation. It improves how your findings land with developers.

This project arrived with 18 months of someone else’s history attached to it and a security audit that had already found problems. What resolved it was not a longer timeline or a bigger budget. It was a QA engineer using AI deliberately, verifying everything by hand, and writing a report good enough that developers could act on it the same day.

That combination held up. It is worth building into how a team approaches application security testing rather than saving for the next emergency.

FAQs

What is AI-assisted penetration testing?

AI-assisted penetration testing is a security assessment carried out by a human tester who uses an AI assistant to speed up parts of the work. The AI supports code comprehension, reasoning about observed behavior, and drafting documentation. The tester still designs the assessment, verifies each finding manually, and owns the conclusions.

Can AI replace QA engineers or professional security testers?

No. AI can accelerate investigation and documentation, but it cannot confirm that a suspected issue is real in a specific application, judge whether it matters given the architecture, or take responsibility for the result. In this assessment, every finding was manually verified before it entered the report.

How does AI improve security testing speed?

Mostly by reducing comprehension time. On an unfamiliar codebase, understanding how the application is built normally takes weeks. AI assistance shortens that to hours, which leaves more of the schedule for actual testing, verification, and reporting.

What should a penetration testing report include?

At minimum: evidence of each vulnerability, the exact location in the source code, a clear explanation of the root cause, and specific recommendations for fixing it. Reports built this way reduce developer debugging time because the reproduction and location work is already done.

Should you run a security audit on an inherited application?

As general guidance, yes. When you take over an application built by another vendor, you inherit its security posture along with its code. Assessing it before further development means you find out what you are dealing with early, rather than after launch.

Is AI-assisted security testing safe to rely on?

It is safe as an accelerator and unsafe as an authority. AI-generated findings can be confidently wrong. Treat every output as a lead to investigate rather than a conclusion, verify manually, and make sure testing is properly authorized in writing before it begins.

Does AI-assisted QA work for functional testing too?

The same principle applies across manual QA testing and automation testing. AI is useful for generating coverage ideas, reading unfamiliar code, and drafting documentation. Deciding what to test and whether a result is acceptable stays with the engineer.