DevSecOps

What is DevSecOps?

Devops with security baked in from the start

DevSecOps is the integrating of security practices into devops. A shorthand description is that security is “baked in” to the DevOps process, and not treated as another layer. Which means thinking about infrastructure and application security from the start and to include security as a key part of the application development lifecycle - using an operations-centric approach to vulnerability assessment via logging, event monitoring, configuration, patch, identity and privilege management.

A DevSecOps approach can be a driver of change in an organisation and because it applies to processes, people, and the tools that they use - it requires a change of attitude.

However, a key aim of applying a DevSecOps approach is not to slow down the DevOps process, the tension being that security and compliance should not be sacrificed for speed of development using continuous integration/continuous development tools.

DevOps organisations deploy code frequently, with shorter lead times, recovering from failures faster and spending less time addressing security issues. The implementation of security during development and delivery means less reliance on penetration testing and pre-production gate reviews.

It’s been reported that organisations that successfully embed security into DevOps experience a 50% drop in their production vulnerabilities and that their time to fix improves by 25%.

DevOps cycles through the following key phases:

• Pre-commit
• Commit / Continuous integration
• Acceptance
• Production
• Operations

DevSecOps tools

The SANS Institute have mapped a number of open source security tools to the DevOPs phases outlining a DevSecOps toolchain and DevSecOps process summarised in the this poster:

Eric Johnson also walked through the first two phases in this presentation to the Australian Cyber Conference in Oct 2018.

Pre-commit

Pre-commit checks are security activities used to find and fix common issues before changes are checked into source code repositories.

Threat modelling / attack mapping

• Attacker personas are based on user personas and they describe the characteristics, capabilities and resources displayed by attackers . A number of different attacker personas represent an attacker landscape as well as various attacker types and threats to defend against.

• Evil user stories an evil user story takes the viewpoint of a malicious adversary.

• Mozilla Rapid Risk Assessment (RRA) The purpose of a Rapid Risk Assessment is to understand the value and impact of a service to the reputation, finances, productivity of the project or business, based on the data processed, stored or accessible by services.

• OWASP Application Threat Modelling Threat modelling works to identify, communicate, and understand threats and mitigations within the context of protecting something of value. *OWASP ThreatDragon is An open source, online threat modelling tool from OWASP

Security and privacy stories

• OWASP Application Security Verification Standard Project The project provides a basis for testing web application technical security controls and also provides developers with a list of requirements for secure development.
• SAFECODE Practical Security Stories provides Agile practitioners with a list of security-focused stories and security tasks they can consume “as is” in their Agile-based development environments.

IDE security plugins

• DevSkim DevSkim is a framework of IDE plugins and language analyzers that provide inline security analysis in the dev environment as the developer writes code, for VSCode, Sublime and Visual Studio.
• Find Security Bugs is a plugin for security audits of Java web applications like Eclipse and IntelliJ.
• Puma Scan is a software security analyzer that provides real time, continuous source code analysis for C# applications like Visual Studio.
• SonarLint is an IDE extension that helps you detect and fix quality issues as code is written for Visual Studio, Intellij, and Eclipse.
• Brakeman is a static analysis security vulnerability scanner for Ruby on Rails applications.
• Veracode provides multiple security analysis technologies on a single platform.
• Fortify is a static code analyser.

IDE plugins are also available for most commercial Static Application Security Testing (SAST) products

Pre-commit security hooks

• git-hound is a plugin that helps prevent sensitive data from being committed into a repository by sniffing potential commits against Perl Compatible Regular Expressions (PCRE) regular expressions.
• git-secrets prevents you from committing secrets and credentials into git repositories.
• Repo-supervisor scans your code for security misconfiguration and searches for passwords and secrets.
• Thoughtworks Talisman By hooking into the pre-push hook provided by Git, Talisman validates the outgoing changeset for things that look suspicious - such as authorization tokens.

Secure coding standards

• CERT C Coding Standard - Wikipedia was developed by CERT as a standard for safe, reliable, secure coding in C.

• OWASP Proactive Controls - OWASP The OWASP Application Security Verification Standard (ASVS) Project provides a basis for testing web application technical security controls and also provides developers with a list of requirements for secure development.

  • Define security requirements
  • Leverage security frameworks and libraries
  • Secure database access
  • Encode and escape data
  • Validate all inputs
  • Implement digital identity
  • Enforce access controls
  • Protect data everywhere
  • Implement security logging and monitoring
  • Handle all errors and exceptions

Manual and peer reviews

OWASP in its secure code review guide says that “secure code review is probably the single-most effective technique for identifying security bugs early in the system development lifecycle.”

• Gerrit Code Review is a self-hosted pre-commit code review tool. It serves as a Git hosting server with option to comment incoming changes. It is highly configurable and extensible with default guarding policies, webhooks, project access control and more.
• GitLab offers git repository management, code reviews, issue tracking, activity feeds and wikis. Enterprises install GitLab on-premise and connect it with LDAP and Active Directory servers for secure authentication and authorization.
• Phabricator is a set of tools for developing software. It includes applications for code review.

Commit - continuous integration

Fast, automated security checks should be made during the build and continuous integration steps.

Static code analysis

Static code analysis examines source code without executing it using automated tools. It aims to find vulnerabilities and can also be used to ensure that code complies with coding guidelines like MISRA C and industry standards such as ISO 26262.

• Find Security Bugs is the SpotBugs plugin for security audits of Java web applications and Android applications. (Also work with Kotlin, Groovy and Scala projects).
• Brakeman is a static analysis security vulnerability scanner for Ruby on Rails applications.
• ESLint is an open source JavaScript linting utility which allows developers to discover problems without executing their code.
• Phan is a static analyzer for PHP that looks for common issues and will verify type compatibility on various operations when type information is available or can be deduced.
• Bandit is a tool designed to find common security issues in Python code.
• Flawfinder is a program that examines C/C++ source code and reports possible security weaknesses (“flaws”) sorted by risk level.
• puma-scan is a software security Visual Studio extension that provides real time, continuous source code analysis as development teams write code.
• gosec is a Golang security checker that inspects source code for security problems by scanning the Go AST.

Security unit tests

• JUnit is a simple, open source framework to write and run repeatable tests for Java.
• Mocha is a feature-rich JavaScript test framework running on node.js which makes asynchronous testing simple. Tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases.
• xUnit is a free, open source, community-focused unit testing tool for the .NET Framework.
• RSpec is a testing too for Ruby.
• PyUnit is an easy way to create unit testing programs and unit tests with Python.

Infrastructure as code analysis

• ansible-lint is a highly configurable best-practices linter for Ansible playbooks.
• Foodcritic is a lint tool that checks Chef cookbooks for common problems.
• puppet-lint will test modules and manifests against the recommended Puppet style guidelines.
• cfn_nag is an open source command-line tool that performs static analysis of AWS CloudFormation templates.

Dependency management

Dependency management is an automated method for declaring, resolving and using a projects dependencies - vulnerabilities can be inherited from open source libraries, docker images, and infrastructure templates.

• OWASP Dependency Check is a utility that identifies project dependencies and checks if there are any known, publicly disclosed, vulnerabilities. Currently, Java and .NET are supported; additional experimental support has been added for Ruby, Node.js, Python, and limited support for C/C++ build systems (autoconf and cmake).
• bundler-audit is a Ruby patch-level verification tool for Bundler.
• Retire.js is a scanner detecting the use of JavaScript libraries with known vulnerabilities.
• node security platform is a suite of continuous security products and services focused on helping to maintain secure node software.

Container security

NCC Group have published a useful white paper on Understanding and Hardening Linux Containers and the National Institute of Standards and Technology (NIST) published the Application Container Security Guide which explains the potential security concerns associated with the use of containers and provides recommendations for addressing them.

• Actuary Docker’s Actuary is an application that checks for dozens of common best-practices around deploying Docker containers in production. Actuary takes in a checklist of items to check, and automates the running, inspecting and aggregation of the results.
• Anchore allows developers, operations, and security teams to perform detailed analysis on container images, run queries, produce reports and define policies that can be used in CI/CD pipelines.
• clair is an open source project for the static analysis of vulnerabilities in application containers (currently including appc and docker).
• dagda is a tool to perform static analysis of known vulnerabilities, trojans, viruses, malware & other malicious threats in docker images/containers.
• The Docker Bench for Security is a script that checks for dozens of common best-practices around deploying Docker containers in production.
• Falco - Container Native Runtime Security is an open source project for intrusion and abnormality detection for Cloud Native platforms such as Kubernetes, Mesosphere, and Cloud Foundry.

Container hardening

• bane is an AppArmor profile generator for Docker containers.
• grsecurity is a security enhancement to the Linux kernel that defends against a security threats through intelligent access control, memory corruption-based exploit prevention, and other system hardening.

Accept - continuous delivery

Automated security acceptances, functional testing, and deep out-of-band scanning should be an essential component of Continuous Delivery.

Infrastructure as code

• Ansible
• Chef
• Puppet
• SaltStack
• Terraform
• Vagrant

Immutable infrastructure

• Docker
• rkt is a CLI for running pod-native containers on Linux. It’s composable, secure, and built on standards

Security scanning

• Arachni is a web application security scanner framework which helps users to evaluate the security of web applications.
• Nmap (“Network Mapper“) is an open source tool for network exploration and security auditing.
• sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers.
• sslyze is a Python tool that can analyze the SSL/TLS configuration of a server by connecting to it.
• ZAP The OWASP Zed Attack Proxy (ZAP) Scanner is a web application security scanner.
• ssh_scan is an SSH configuration and policy scanner for Linux and UNIX servers.

Cloud configuration management

• AWS CloudFormationprovides a common language which describes and provisions the infrastructure resources in a cloud environment, using a simple text file to automate secure provision.
• Azure Resource Manager enables you to repeatedly deploy an app in a consistent state by defining the infrastructure and dependencies in a single declarative template.
• Cloud Deployment Manager specifies the resources needed for an application in a declarative format using yaml, so configuration can be treated as code to perform repeatable deployments.

Security acceptance testing

• bdd-security is a security testing framework that uses Behaviour Driven Development concepts to create self-verifying security specifications.
• Gauntlt provides hooks to security tools so that security, dev and ops teams can collaborate to build secure software. It facilitates group testing and communication so that tests are part of your deploy and testing processes.
• Mittn provides a Python tool to test security use cases, and also code checks on previous security mistakes.

Infrastructure tests

• Serverspec tests verify the state of infrastructure such as bare-metal servers, virtual machines, cloud resources) ascertains that they are configured correctly.
• Test Kitchen provides a test harness to execute infrastructure code on one or more platforms in isolation. A driver plugin architecture is used to run code on various cloud providers and virtualization technologies such as Vagrant, Amazon EC2, and Docker.

Infrastructure compliance checks

• HubbleStack is a modular, open-source security compliance framework built on top of SaltStack.
• InSpec is an open source (OSS) automated testing tool for integration, compliance, security, and other policy requirements. Tests can be carried out on-site or on cloud platforms such as AWS, Azure, and Docker Containers.

Production - continuous deployment

Security checks should be made before, during and after code is deployed to production.

Security Smoke Tests

• ZAP Baseline Scan is a script that is available in the ZAP Stable and Weekly Docker images.
• nmap is a free, open-source tool for vulnerability scanning and network discovery. Nmap is used to identify devices are running on systems, discovers hosts that are available, finds open ports and detects security risks.
• ssllabs-scan is command-line reference-implementation client for SSL Labs APIs, designed for automated and/or bulk testing.

Configuration Safety Checks

• AWS Config is a service that enables you to assess, audit, and evaluate the configurations of your AWS resources.
• AWS Support – Trusted Advisor is an online resource to help reduce cost, increase performance, and improve security by optimizing an AWS environment, It provides real time guidance to help provision resources following AWS best practices.
• security_monkey monitors AWS, GCP, OpenStack, and GitHub orgs for assets and their changes over time.
• OSQuery Osquery uses basic SQL commands to leverage a relational data-model to describe a Linux, Windows, and macOS device. So, for example, tests can identify suspicious processes running without a binary on disk.

Secrets Management

• Ansible Vault is a feature of ansible that allows you to keep sensitive data such as passwords or keys in encrypted files, rather than as plain text in playbooks or roles. These vault files can then be distributed or placed in source control.
• blackbox allows safely storing secrets in a VCS repo.
• chef-vault allows the encryption of a data bag item by using the public keys of a list of nodes, allowing only those nodes to decrypt the encrypted values.
• Docker Secrets is a container first architecture designed to keep secrets safe and used only when needed by the exact container that needs that secret to operate.
• HashiCorp Vault allows secure storage and tightly controlled access to tokens, passwords, certificates, encryption keys for protecting secrets and other sensitive data using a UI, CLI, or HTTP API.
• Pinterest Knox is a service for storing and rotation of secrets, keys, and passwords used by other services.

Cloud Secrets Management

• AWS KMS Key Management Service (KMS) is a managed service that makes it easy for you to create and control the encryption keys used to encrypt your data, and uses FIPS 140-2 validated hardware security modules to protect the security of your keys.
• Microsoft Azure Key Vault safeguard cryptographic keys and other secrets used by cloud apps and services.
• Google Cloud KMS allows the management of encryption keys on the Google Cloud Platform

Cloud Security Testing

• CloudSploit provides automated AWS security and configuration monitoring.
• Nimbostratus is a tool for fingerprinting and exploiting Amazon cloud infrastructures.

Server Hardening

• DevSec Hardening Framework creates a common layer for operating system and services hardening.
• SIMP Compliance Automation The Systems Integrity Management Platform (SIMP) is a suite of systems management tools and automated compliance modules.

Host Intrusion Detection System (HIDS)

• Fail2ban scans log files and bans IPs that show the malicious signs – too many password failures, seeking for exploits, etc.
• OSSEC is an open-source host-based intrusion detection system. It performs log analysis and integrity checking.
• Samhain is an open source host-based intrusion detection system (HIDS) provides file integrity checking and log file monitoring/analysis, as well as rootkit detection, port monitoring, detection of rogue SUID executables, and hidden processes.

Operations - continuous security monitoring

Continuous security monitoring, testing, audit and compliance checks

Fault Injection

• Chaos Kong is a methodology built by Netflix which simulates the an extreme case of infrastructure failure eg killing an AWS region, and which follows the principles of Chaos Engineering.
• Chaos Monkey is a resiliency service which identifies groups of systems and randomly terminates one of the systems in a group. The service operates at a controlled time (does not run on weekends and holidays) and interval (only operates during business hours).
• Gremlin provides a framework to safely, securely, and easily simulate real outages with an ever-growing library of attacks.

Cyber simulations

• Game day exercises are run with the goal of increasing reliability by purposefully creating major failures on a regular basis.
• Tabletop scenarios is where a failure is injected into a system and the participants in the exercise go through a process of fixing it (i.e., identify, triage, fix and conduct a post-mortem).

Penetration testing

• Threat-based Defense involves building defences around real world attack patterns to gain an understanding of an attackers techniques, tactics and motivation.
• Bug bounties programs allow developers to discover and resolve bugs before the general public is aware of them.
• A Red team exercise is an all-out attempt to achieve the defined objectives by any methods available, and usually includes internal and external penetration testing, compromising wireless networks, physical access, and social engineering techniques.

Threat intelligence

• The Diamond Model details the fundamental aspects of malicious activity and the core analytic concepts used to discover, develop, track, group, and ultimately counter the activity and the adversary.
• Cyber Kill Chain® | Lockheed Martin framework is a model for identification and prevention of cyber intrusions activity, and it identifies what the adversaries must complete in order to achieve their objective.
• STIX is a structured language for cyber threat intelligence so it can be shared, stored, and analyzed in a consistent manner.
• TAXII Trusted Automated Exchange of Intelligence Information (TAXII) is an application layer protocol for the communication of cyber threat information in a simple and scalable manner. It’s used to exchange cyber threat intelligence (CTI) over HTTPS, enabling organizations to share CTI by defining an API that aligns with common sharing models.

Continous scanning

• OpenSCAP is an open source Security Compliance Solution that provides multiple tools to assist administrators and auditors with assessment, measurement, and enforcement of security baselines.
• OpenVAS Open Vulnerability Assessment System is an open source framework of several services and tools offering vulnerability scanning and vulnerability management.
• Prowler is an AWS security assessment and forensics tool.
• Scout2 is a security auditing tool for AWS environments.
• Vuls is an open-source, agent-less vulnerability scanner based on information from NVD, OVAL, etc.

Blameless postmortems

• Etsy morgue is an open source post-mortem tracker application storing outage data, fields for a timeline, IRC logs, graphs, images, remediation items etc.

Continous monitoring

• Grafana is an open source visualisation platform for analytics and monitoring.
• Graphite is an open source tool for analytics and monitoring graphing.
• seyren is an alerting dashboard for Graphite.
• SOF-ELK is a data analytics platform focused on the typical needs of computer forensic investigators/analysts and information security operations personnel.
• elastalert is a framework for alerting on anomalies, spikes, or other patterns of interest from data in Elasticsearch.
• 411 Etsy has created an open source alert management system, providing a framework for querying data sources and managing the generated alerts. It supports Elasticsearch-based alerts as well as other alert types.

Cloud monitoring

• Amazon CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, providing a unified view of AWS resources, applications and services that run on AWS, and on-premise servers.
• AWS CloudTrail provides event history of AWS account activity which helps to simplify security analysis, resource change tracking, and troubleshooting.
• reddalert is an AWS security monitoring/alerting tool built on top of Netflix’s EDDA project.

Cloud compliance

• Cloud Custodian is a tool that unifies the dozens of tools and scripts most organizations use for managing their AWS accounts into one open source tool.
• Forseti Security is a collection of community-driven, open-source tools to help improve the security of Google Cloud Platform (GCP) environments.