Pipeline Security Automation

Tech Deep Dive
Written By

This post talks about how we approach security automationin BitBucket Pipelines. It also introduces some new open source tools we built and use in the process.

Security In Pipelines

We’ve written beforeabout using GitHub Actionsand provided an Action friendly “workflow” with our Crush tool.

At a high level, Pipelines and Actions just do some computing workfor you in Atlassian or Github’s data center. Often that workis related to source code, testing, or deployment.

Both leverage containers heavily for sandboxing the work thathappens in the build and provide some level of abstraction that youcan use to build your own pipes or workflows.

On some level, we’re using them to do the same work we used touse Jenkins or CircleCI or Travis to do.

We like automating security in Pipelines or Actions because it makesit easy to build security into your natural development workflows.

Real World

So what are we really talking about? What do we want to have happen?

We want to be able to apply any automated tools flexibly in a controlledenvironment that already has our code artifacts and knows about the keyevents.

Triggers for actions and pipelines can be:

  • Push to a branch (we often set up a securityautomation/* branch to make this very deliberate at first)

  • Push to master or release (i.e. when we deploy)

  • A cron based schedule

The tools we want to run may range from:

  • Secrets detection

  • Vulnerable dependency detection

  • Code smells

  • Dynamic scanning (ZAP)

  • Cloud configuration checks

Of course, we can run security unit tests and trigger otherintegrations (eg. SonarCloud) as well.

Just remember:

Tools are necessary but must be used by skilled operators.

I can’t tell you how often I see security tools installed but not used effectively.

OK Show Us Code

Here is an example of how we have a pipeline configured:

pipelines:branches:'{securityautomation/*}':- parallel:- step:name: Run Crushimage: golang:1.16script:- go get -u github.com/jemurai/crush@v1.0.5- crush examine --threshold 7 . > crush.jsonartifacts:- crush.json- step:name: Dep Checkimage: openjdk:8script:- wget https://github.com/jeremylong/DependencyCheck/releases/download/v6.1.5/dependency-check-6.1.5-release.zip- unzip dependency-check-6.1.5-release.zip- rm dependency-check-6.1.5-release.zip- ./dependency-check/bin/dependency-check.sh --failOnCVSS 6 --exclude **/dependency-check/**/*.jar -f JSON --prettyPrint --scan .artifacts:- dependency-check-report.json- step:name: Reportimage: golang:1.16script:- go get -u github.com/jemurai/depcheck2off@v1.0.0- go get -u github.com/jemurai/off2jira@v1.0.0- depcheck2off ./dependency-check-report.json > ./depcheck.off.json- off2jira ./depcheck.off.json- off2jira ./crush.json

Let’s walk through it and talk about what is happening.

First, the branches part tells BitBucket when to run the pipeline. In this case, it will be on any push to a branch under securityautomation.

branches:'{securityautomation/*}':

We like doing this because it helps to isolate your security related changes and ensures that what you are finding doesn’t break other builds. In the long run, we want to have security tooling run more often.

Then we need to understand that there are three steps defined in the pipeline:

  • Crush

  • Dep Check

  • Report

Crush and Depenendency Check are both code analysis so theycan run in parallel. Hence the parallel: before theirstep: definitions.

To run Crush, we pull a base golang image image: golang:1.16, install Crush and run it. We drop the output into an artifact that means it will be available later.

- step:name: Run Crushimage: golang:1.16script:- go get -u github.com/jemurai/crush@v1.0.5- crush examine --threshold 7 . > crush.jsonartifacts:- crush.json

Running Dependency Check is similar. You can see that we’re pulling a release from GitHub and unzipping it. This is on an openjdk image. Then we invoke dependency check and put the report in an artifact.

- step:name: Dep Checkimage: openjdk:8script:- wget https://github.com/jeremylong/DependencyCheck/releases/download/v6.1.5/dependency-check-6.1.5-release.zip- unzip dependency-check-6.1.5-release.zip- rm dependency-check-6.1.5-release.zip- ./dependency-check/bin/dependency-check.sh --failOnCVSS 6 --exclude **/dependency-check/**/*.jar -f JSON --prettyPrint --scan .artifacts:- dependency-check-report.json

The next part “Report” is interesting and we’re going to put it in a whole new section.

Reporting and Rethinking Security Tooling

Once we have Crush and Dependency Check output, we want to do something with it. We could leave it in BitBucket as an artifact and refer to the plain text file. That is better than not running the tools, but we also want make these visible and integrate into our normal processes.

Here’s how that looks in the pipeline we defined where we’re pushing the issues identified by Crush and OWASP Dependency Check to JIRA:

- step:name: Reportimage: golang:1.16script:- go get -u github.com/jemurai/depcheck2off@v1.0.0- go get -u github.com/jemurai/off2jira@v1.0.0- depcheck2off ./dependency-check-report.json > ./depcheck.off.json- off2jira ./depcheck.off.json- off2jira ./crush.json

Here we are installing and using two new golang based tools:

  • depcheck2off - which takes dependency check output and puts it in OWASP off format.

  • off2jira - which takes a file of off findings and pushes them to JIRA.

The basic philosophy is to build small tools that do one thing and do it in a simple and predictable way. This goes directly against our own past approach with OWASP Glue which we retired.

With Glue, you could rundifferent tools and push the output to a variety of trackers. The problem was that you ended up with a JVM, Python, Ruby, Node and an ever growing docker image. That made it hard to incorporate into pipelines efficiently. We also had to maintain everything and keep everything working to get an update pushed. It was a monolith.

With the Jemurai autom8d set of tools, we’re taking more of a classic Unixphilosophy and building small purpose built utilities that can be put togetherin a number of ways.

So far we have:

  • fkit

  • depcheck2off

  • off2jira

We already have plans to build:

  • off2csv

  • zap2off

We also want to adapt and integrate some other code we have that does aninventory and metrics across repositories.

We’d love to hear from you about others that would be useful! We can helpwith this type of automation while doing so with open tooling you can leverage for the long term.

Leverage Pipelines

The great thing about pipelines (and actions) is that once you understand them,you can effectively push security tooling to a large number of projectsquite easily.

Note that there are compute charges associated with running pipelines (or actions).

We have also had good success helping companies who leverage BitBucket orGitHub cloud because we can actually help commit the code that starts theautomation project off. Combined with some training and a retained ongoingsupport setup - we can enable clients to very quickly improve their internalapp security posture.

References

Previous

Which Security Standard Should I Use?
Next

Pipeline Security Automation