Ever since security automation became the norm in AppSec, we’ve seen seismic shifts in the way product engineering teams approach the development of their apps. Back when the only way to test an app for security was manual penetration testing, there was a huge gap between what an app actually needed for its security and what developers were able to provide in the limited time and bandwidth they had. It was so bad that vulnerabilities would continue to accumulate with each new release, giving developers a shrinking window in which to actually fix their bugs.
Even today, security debt still exists as one of the biggest symptoms of a development culture that de-emphasises the need for frequent and concerted security efforts. In a recent study, it was found that fixing a vulnerability later in the software development lifecycle (SDLC), as opposed to earlier incurred a cost increase of anywhere between four to ten times. And the older the vulnerability, the lower down it inevitably goes on the priority list, cutting its chances of being remediated in half month-on-month.
These are real numbers, and they indicate real patterns in how develop and secure our applications. You can check out our full infographic here. They also offer up a very simple truth: the more you scan your apps, the less time it takes you to fix them. So this snowball effect clearly works in the opposite direction, too. So why don’t more companies do it? Why don’t more product engineering teams just start running more automated security tests every week and use that data to start fixing bugs way faster than ever before?
Well, that’s exactly what this blog is about. Automation, although incredibly powerful and a total game-changer in the world of AppSec, is no magic wand or panacea to all our manual pentesting woes. What problems are solved by automation are simply replaced by new ones, and it’s a problem many security operations teams aren’t yet equipped to handle because of how little they understand.
False Positives, Noise, and the Automation Data Landslide
The biggest problem with security automation isn’t actually the process of setting up an automation pipeline. What really keeps most so many product engineering teams from adopting it are the mountains of vulnerability scan results that a security tool generates after running an automated scan. Scan tools find and report dozens, sometimes hundreds of vulnerabilities. It can get really bad, like one client of ours who was seeing upwards of 1300 vulnerabilities coming from just one tool.
Faced with such overwhelming numbers, developers who already have to stick to release schedules and deadlines simply won’t be able to keep up. At best, they might fix a few of the most critical vulnerabilities before moving on, leaving a majority of them completely unresolved. This landslide of automated scan data isn’t even particularly useful. Automated security scans are notorious for how full of false positives and ‘noisy’ they are.
It’s rather ironic that automation, which was supposed to be a solution to slow manual pentesting, has replaced that problem with one of its own: AppSec engineers and developers now have to spend a whole lot of time sifting through this tool-generated data and find the results that actually correspond to a real-world vulnerability and not just some false positive result. In the end, all the time you managed to save by automating your security scans has now been sunk into sorting through and organising the results from them.
As a security engineer, it should be your job not just to understand the security posture of your application, but make it as easy and smooth as possible for developers to figure how exactly how they need to go about remediating vulnerabilities. Here are 4 things experts say are major barriers to better collaboration between Security and DevOps teams.
1. Security reports full of false positives
“A good AppSec engineer should be able to weed out the false positives before they sit down with the developers,” says Mark Willis, security veteran and CISO of Bluescape. “You really want to be very efficient with [how you present your security findings] to the developers.”
Developers are not security experts, and more often than not, have only a passing knowledge of how many security scan tools work or how to decipher their reports. Add to that the overwhelming number of false positives plaguing your average security report, and you have a recipe for disaster. Not only does the list of bugs itself look bloated and intimidatingly long, but the developers are going to be wasting hours just figuring out which vulnerabilities even need fixing in the first place.
Check it out: Automated false positive management with Orchestron
The easier you make it for a product engineer to understand and effectively use your findings to remediate vulnerabilities, the more seriously they’re going to start taking you, because they know you’re not wasting their time.
2. AppSec engineers don’t understand code
Quite possibly the most popular bandwagon in AppSec these last few years has been the one calling for developers to learn and understand security. A developer that understand AppSec can write more secure code, and secure code means a more secure application. Now, there’s certainly nothing wrong with expecting developers to be familiar with secure coding practices. Everyone benefits when an app is built from the ground up with security in mind.
But equally important is for AppSec engineers to be familiar with the process of app development. When you don’t understand the app you’re trying to secure, you’re not going to be capable of analysing a security situation from the perspective of someone trying to break the application. That’s because you lack context. Not every vulnerability affects every app the same way, and having context for how your application works can dramatically change how you view a vulnerability. It helps you pinpoint with far greater accuracy what the source of the problem is, and arrive at a solution far quicker than otherwise possible. And that makes all the difference.
“I think the best way to do it is start off as a developer,” says Mr. Willis. “Make the transition like I did. Start off as a developer and get some serious code behind you, so when you go to that developer down the road as an AppSec professional, you’ve got that empathy. You know how their life is, and that they don’t have a lot of time to worry about fixing code.”
3. Not asking where your vulnerabilities come from
In a previous blog post, we45 Chief Evangelist Rahul Raghavan talked about how ‘Every Vulnerability Has a Story’. “If there’s one thing that all product engineers, developers and security professionals need to do, it’s to stop looking at vulnerabilities as a statistic,” he wrote. “Blindly trying to fix bugs in your apps is like treating the symptoms instead of the cause. You need to understand why the vulnerability is there in the first place, and what that means for your application. Vulnerability metadata can be far more useful than you think.”
Going off of the last point, it really helps to get some context for your vulnerabilities. It’s not enough to know the technical details of your OWASP Top 10 vulnerabilities or the most common ways to fix them. You need to look at security flaws in the context of your application as a whole. Why is your app seeing more vulnerabilities during dynamic testing than static testing? Why are there more architecture-related flaws than deployment-related ones?
When you understand not just what vulnerabilities you’re up against, but why you’re up against those specific ones, you’re finally looking at the bigger picture. In the long run, building automation pipelines and CI/CD workflows benefit from having a bird’s-eye view of your apps, and helps you ensure that you’re not squandering time and effort on the wrong things. Treat the cause, not the symptoms.
4. Not focusing on the vulnerabilities that matter
If there’s one really basic but really practical bit of advice from the AppSec gurus, it’s this: start with the Critical and High severity vulnerabilities. The rest can wait.
It can be tempting to bunch together all of your findings in a report and send them off to the developers to fix, but we’re going to have to be realistic here. The devs aren’t going to be able to fix all the bugs all the time. Hell, they’re probably not going to be able to fix all the bugs most of the time. The thing is, Medium and Low severity vulnerabilities just aren’t a huge priority, especially when your app already has Criticals and Highs to worry about.
In the vast majority of cases, it’s the more exploitable vulnerabilities that are ultimately, well, exploited because they have a big, red target painted on them. Once you’ve taken care of the bigger fish, the ones more capable and likely to sink your boat, then you can think about going after the small f