Tuesday, November 29, 2016

The Purple Team Pentest

It’s not particularly clear whether a marketing intern thought he was being clever or a fatigued pentester thought she was being cynical when the term “Purple Team Pentest” was first thrown around like spaghetti at the fridge door, but it appears we’re now stuck with the term for better or worse.

Just as the definition of penetration testing has broadened to the point that we commonly label a full-scope penetration of a target’s systems with the prospect of lateral compromise and social engineering as a Red Team Pentest – delivered by a “Red Team” entity operating from a sophisticated hacker’s playbook. We now often acknowledge the client’s vigilant security operations and incident response team as the “Blue Team” – charged with detecting and defending against security threats or intrusions on a 24x7 response cycle.

Requests for penetration tests (Black-box, Gray-box, White-box, etc.) are typically initiated and procured by a core information security team within an organization. This core security team tends to operate at a strategic level within the business – advising business leaders and stakeholders of new threats, reviewing security policies and practices, coordinating critical security responses, evaluating new technologies, and generally being the go-to-guys for out-of-ordinary security issues. When it comes to penetration testing, the odds are high that some members are proficient with common hacking techniques and understand the technical impact of threats upon the core business systems.

These are the folks that typically scope and eventually review the reports from a penetration test – they are however NOT the “Blue Team”, but they may help guide and at times provide third-line support to security operations people. No, the nucleus of a Blue Team are the front-line personnel watching over SIEM’s, reviewing logs, initiating and responding to support tickets, and generally swatting down each detected threat as it appears during their shift.

Blue Teams are defensively focused and typically proficient at their operational security tasks. The highly-focused nature of their role does however often mean that they lack what can best be described as a “hackers eye view” of the environment they’re tasked with defending.

Traditional penetration testing approaches are often adversarial. The Red Team must find flaws, compromise systems, and generally highlight the failures in the targets security posture. The Blue Team faces the losing proposition of having to had already secured and remediated all possible flaws prior to the pentest, and then reactively respond to each vulnerability they missed – typically without comprehension of the tools or techniques the Red Team leveraged in their attack. Is it any wonder that Blue Teams hate traditional pentests? Why aren’t the Red Team consultants surprised that the same tools and attack vectors work a year later against the same targets?

A Purple Team Pentest should be thought of as a dynamic amalgamation of Red Team and Blue Team members with the purpose of overcoming communication hurdles, facilitating knowledge transfer, and generally arming the Blue Team with newly practiced skills against a more sophisticated attacker or series of attack scenarios.

How to Orchestrate a Purple Team Pentest Engagement

Very few organizations have their own internal penetration testing team and even those that do regularly utilize external consulting companies to augment that internal team to ensure the appropriate skills are on hand and to tackle more sophisticated pentesting demands.

A Purple Team Pentest almost always utilizes the services of an external pentest team – ideally one that is accomplished and experienced in Red Team pentesting.

Bringing together two highly skilled security teams – one in attack, the other in defense – and having them not only work together, but to also achieve all the stated goals of a Purple Team pentest, requires planning and leadership.

To facilitate a successful Purple Team Pentest, the client organization should consider the following key elements:

  • Scope & Objectives - Before reaching out and engaging with a Red Team provider, carefully define the scope and objectives of the Purple Team Pentest. Be specific as to what the organizations primary goals are and what business applications or operational facilities will be within scope. Since a key objective of conducting a Purple Team Pentest is to educate and better arm the internal Blue Team and to maximize the return on a Red Team’s findings, identify and list the gaps that need to be addressed in order to define success.
  • Blue Team Selection - Be specific in defining which pieces of the organization and which personnel constitute the “Blue Team”. Go beyond merely informing various security operations staff that they are now part of a Blue Team. It is critical that the members feel they are a key component in the company’s new defensive strategy. Educate them about the roles and responsibilities of what the Blue Team entails. Prior to engaging with a Red Team provider and launching a Purple Team Pentest, socialize and refine the scope and objectives of the proposed Purple Teaming engagement with the team directly.
  • Red Team Selection - It is important that the client select a Red Team that consists of experienced penetration testers. The greater the skills and experience of the Red Team members, the more they will be able to contribute to the Purple Team Pentest objectives. Often, in pure Red Team Pentest engagements, the consulting team will contain a mix of experienced and junior consultants – with the junior consultants performing much of the tool-based activities under the supervision of the lead consultant. Since a critical component of a Purple Team Pentest lies in the ability to communicate and educate a Blue Team to the attacker’s methodologies and motivations, junior-level consultants add little value to that dialogue. Clients are actively encouraged to review the resumes of the consultants proposed to constitute the Red Team in advance of testing.
  • Playbook Definition - Both sides of the Purple Teaming exercise have unique objectives and methodologies. Creation of a playbook in advance of testing is encouraged and so too is the sharing and agreement between the teams. This playbook loosely defines the rules of the engagement and is largely focused on environment stability (e.g. rules for patch management and rollout during the testing period) and defining exceptions to standard Blue Team responses (e.g. identifying but not blocking the inbound IP addresses associated with the Red Team’s C&C).
  • Arbitrator or Referee - Someone must be the technical “Referee” for the Purple Team Pentest. They need to be able to speak both Red Team and Blue Team languages, interpret and bridge the gap between them, manage the security workshops that help define and resolve any critical threat discoveries, and generally arbitrate according to the playbook (often adding to the playbook throughout the engagement). Ideally the arbitrator or referee for the engagement is not directly associated with, or a member of, either the Red or Blue teams.
  • Daily Round-table Reviews - Daily round-table discussions and reviews of Red Team findings are the center-piece of a successful Purple Team Pentest. Best conducted at the start of each day (mitigating the prospect of long tired days and possible overflow of working hours – curtailing discussion), the Red Team lays out the successes and failures of the previous days testing, while the Blue Team responds with what they detected and how they responded. The review facilitates the discussion of “What and Why” the Red Team members targeted, explain the “How” they proceeded, and allows the Blue Team to query and understand what evidence they may have collected to detect and thwart such attacks. For example, daily discussions should include discussions covering what traffic did the tool or methodology generate, where could that evidence have been captured, how could that evidence be interpreted, what responses would pose the biggest hurdle to the attacker?
  • Pair-down Deep Dives - Allowing members of the teams to “pair down” after the morning review to dive deeper in to the technical details and projected responses to a particular attack vector or exploitation is highly encouraged.
  • Evaluate Attack and Defense Success in Real-time - Throughout the engagement the “Arbitrator” should engage with both teams and be constantly aware of what attacks are in play by the Red Team, and what responses are being undertaken by the Blue Team. In some attack scenarios it may be worthwhile allowing the Red Team to persist in an attack even if it has been detected and countered by the Blue Team, or is known to be unsuccessful and unlikely to lead to compromise. However, the overall efficiency can be increased and the cost of a Purple Team Pentest can be reduced by brokering conversations between the teams when attack vectors are stalled, irrelevant, already successful, or known to eventually become successful. For example, the Red Team are able to get a foothold on a compromised host and then proceed to bruteforce attack the credentials of an accessible internal database server. Once the Red Team have successfully started their brute-force attack it may be opportune to validate with the Blue Team that they have already been alerted to the attack in process and are initiating countermeasures. At that point in time, in order to speed up the testing and to progress with another approved attack scenario, a list of known credentials are passed directly to the Red Team and they may progress with a newly created test credential on that (newly) compromised host.
  • Triage and Finding Review - Most Red Team pentests will identify a number of security vulnerabilities and exploit paths that were missed by the Blue Team and will require vendor software patches or software development time to remediate. In a pure Red Team Pentest engagement, a “Final Report” would be created listing all findings – with a brief description of recommended and generic best practice fixes. In a Purple Team Pentest, rather than production of a vulnerability findings report, an end-of-pentest workshop should be held between the two teams. During this workshop each phase of the Red Team testing is reviewed – discoveries, detection, remediation, and mitigation – with an open Q&A dialogue between the teams and, at the conclusion of the workshop, a detailed remediation plan is created along with owner assignment.

The Future is Purple

While the methodologies used in Purple Team penetration testing are the same as those of a stand-alone Red Team Pentest, the business objectives and communication methods used are considerably different. Even though the Purple Team Pentest concept is relatively new, it is an increasingly important vehicle for increasing an organizations security stature and reducing overall costs.

The anticipated rewards from conducting a successful Purple Team pentest include increased Blue Team knowledge of threats and adversaries, muscle-memory threat response and mitigation, validation of playbook response to threats in motion, confidence in sophisticated attacker incident response, identification and enumeration of new vulnerabilities or attack vectors, and overall team-building.

As businesses become more aware of Purple Teaming concepts and develop an increased understanding of internal Blue Team capabilities and benefits, it is anticipated that many organizations will update their annual penetration testing requirements to incorporate Purple Team Pentest as a cornerstone of their overall information security and business continuity strategy.

-- Gunter Ollmann

Monday, November 28, 2016

Navigating the "Pentest" World

The demand for penetration testing and security assessment services worldwide has been growing year-on-year. Driven largely by Governance, Risk, and Compliance (GRC) concerns, plus an evolving pressure to be observed taking information security and customer privacy seriously, most CIO/CSO/CISO’s can expect to conduct regular “pentests” as a means of validating their organizations or product’s security.

An unfortunate circumstance of two decades of professional service oriented delivery of pentests is that the very term “penetration testing” now covers a broad range of security services and risk attributes – with most consulting firms provide a smorgasbord of differentiated service offerings – intermixing terms such as security assessment and pentest, and constructing hybrid testing methodologies.

For those newly tasked with having to find and retain a team capable of delivering a pentest, the prospect of having to decipher the lingo and identify the right service is often daunting – as failure to get it right is not only financially costly, but may also be career-ending if later proven to be inadequate.

What does today’s landscape of pentesting look like?

All penetration testing methodologies and delivery approaches are designed to factor-in and illustrate a threat represented by an attack vector or exploitation. A key differentiator between many testing methodologies lies in whether the scope is to identify the presence of a vulnerability, or to exploit and subsequently propagate an attack through that vulnerability. The former is generally bucketed in the assessment and audit taxonomy, while the latter is more commonly a definition for penetration testing (or an ethical hack).
The penetration testing market and categorization of services is divided by two primary factors – the level of detail that will be provided by the client, and the range of “hacker” tools and techniques that will be allowed as part of the testing. Depending upon the business drivers behind the pentest (e.g. compliance, risk reduction, or attack simulation), there is often a graduated-scale of services. Some of the most common terms used are:
  • Vulnerability Scanning
    The use of automated tools to identify hosts, devices, infrastructure, services, applications, and code snippets that may be vulnerable to known attack vectors or have a history of security issues and vulnerabilities.
  • Black-box Pentest
    The application of common attack tools and methodologies against a client-defined target or range of targets in which the pentester is tasked with identifying all the important security vulnerabilities and configuration failures of the scoped engagement. Typically, the penetration scope is limited to approved systems and windows of exploitation to minimize the potential for collateral damage. The client provides little information beyond the scope and expects the consultant to replicate the discovery and attack phases of an attacker who has zero insider knowledge of the environment. 
  • Gray-box Pentest
    Identical methodology to the Black-box Pentest, but with some degree of insider knowledge transfer. When an important vulnerability is uncovered the consultant will typically liaise with the client to obtain additional “insider information” which can be used to either establish an appropriate risk classification for the vulnerability, or initiate a transfer of additional information about the host or the data it contains (that could likely be gained by successfully exploiting the vulnerability), without having to risk collateral damage or downtime during the testing phase.
  • White-box Pentest (also referred to as Crystal-box Pentest)
    Identical tools and methodology to the Black-box Pentest, but the consultants are supplied with all networking documentation and details ahead of time. Often, as part of a White-box Pentest, the client will provide network diagrams and the results of vulnerability scanning tools and past pentest reports. The objective of this type of pentest is to maximize the consultants time on identifying new and previously undocumented security vulnerabilities and issues.
  • Architecture Review
    Armed with an understanding of common attack tools and exploitation vectors, the consultant reviews the underlying architecture of the environment. Methodologies often include active testing phases, such as network mapping and service identification, but may include third-party hosting and delivery capabilities (e.g. domain name registration, DNS, etc.) and resilience to business disruption attacks (e.g. DDoS, Ransomware, etc.). A sizable component of the methodology is often tied to the evaluation and configuration of existing network detection and protection technologies (e.g. firewall rules, network segmentation, etc.) – with configuration files and information being provided directly by the client.
  • Redteam Pentest
    Closely related to the Black-box pentest, the Redteam pentest mostly closely resembles a real attack. The scope of the engagement (targets and tools that can be used) is often greater than a Black-box pentest, and typically conducted in a manner to not alert the client’s security operations and incident response teams. The consultant will try to exploit any vulnerabilities they reasonably believe will provide access to client systems and, from a compromised device, attempt to move laterally within a compromised network – seeking to gain access to a specific (hidden) target, or deliver proof of control of the entire client network.
  • Code Review
    The consultant is provided access to all source code material and will use a mix of automated and manual code analysis processes to identify security issues, vulnerabilities, and weaknesses. Some methodologies will encompass the creation of proof-of-concept (PoC) exploitation code to manually confirm the exploitability of an uncovered vulnerability.
  • Controls Audit
    Typically delivered on-site, the consultant is provided access to all necessary systems, logs, policy-derived configuration files, reporting infrastructure, and data repositories, and performs an audit of existing security controls against a defined list of attack scenarios. Depending upon the scope of the engagement, this may include validation against multiple compliance standards and use a mix of automated, manual, and questionnaire-based evaluation techniques.
The Hybrid Pentest Landscape

In recent years the pentest landscape has evolved further with the addition of hybrid services and community-sourcing solutions. 
Overlapping the field of pentesting, there are three important additions:
  • Bug Bounty Programs
    Public bug bounty programs seek to crowdsource penetration testing skills and directly incentivize participants to identify vulnerabilities in the client’s online services or consumer products. The approach typically encompasses an amalgamation of Vulnerability Scanning and Black-box Pentest methodologies – but with very specific scope and limitations on exploitation depth. With (ideally) many crowdsourced testers, the majority of testing is repeated by each participant. The hope is that, over time, all low-hanging fruit vulnerabilities will be uncovered and later remediated. 
  • Purple Team Pentest
    This hybrid pentest combines Redteam and Blueteam (i.e. the client’s defense or incident response team) activities in to a single coordinated testing effort. The Redteam employs all the tools and tricks of a Redteam Pentest methodology, but each test is watch and responded to in real-time by the client’s Blueteam. As a collaborative pentest, there is regular communication between the teams (typically end of day calls) and synching of events. The objectives of Purple Team pentesting is both assess the capabilities of the Blueteam and to reduce the time typically taken to conduct a Redteam Pentest – by quickly validating the success or failure of various attack and exploitation techniques, and limiting the possibility of downtime failures of targeted and exploited systems.
  • Disaster Recovery Testing
    By combining a Whitebox Pentest with incident response preparedness testing and a scenario-based attack strategy, Disaster Recovery Testing is a hybrid pentest designed to review, assess, and actively test the organization's capability to respond and recover from common hacker-initiated threats and disaster scenarios.
Given the broad category of “pentest” and the different testing methodologies followed by security consulting groups around the globe, prospective clients of these services should ensure that they have a clear understanding of what their primary business objectives are. Compliance, risk reduction, and attack simulation are the most common defining characteristics driving the need for penetration testing – and can typically align with the breakdown of the various pentest service definitions.

[Update: First graph adapted from Patrick Thomas' tweet - https://twitter.com/coffeetocode/status/794593057282859008]