Testing Security Technology

There are many terms used to describe the technical review of security controls. Ethical hacking, penetration test, and security testing are often used interchangeably to describe a process that attempts to validate security configuration and vulnerabilities by exploiting them in a controlled manner to gain access to computer systems and networks. There are various ways that security testing can be conducted, and the choice of methods used ultimately comes down to the degree to which the test examines security as a system. There are generally two distinct levels of security testing commonly performed today:

  • Vulnerability assessment: This technical assessment is intended to identify as many potential weaknesses in a host, application, or entire network as possible based on the scope of the engagement. Configurations, policies, and best practices are all used to identify potential weaknesses in the deployment or design of the entity being tested. These types of assessments are notorious for finding an enormous amount of potential problems that require a security expert to prioritize and validate real issues that need to be addressed. Running vulnerability scanning software can result in hundreds of pages of items being flagged as vulnerable when in reality they are not exploitable.
  • Penetration test: The penetration test is intended to assess the prevention, detection, and correction controls of a network by attempting to exploit vulnerabilities and gain control of systems and services. Penetration testers (also known as pentesters) scan for vulnerabilities as part of the process just like a vulnerability assessment, but the primary difference between the two is that a pentester also attempts to exploit those vulnerabilities as a method of validating that there is an exploitable weakness. Successfully taking over a system does not show all possible vectors of entry into the network, but can identify where key controls fail. If someone is able to exploit a device without triggering any alarms, then detective controls need to be strengthened so that the organization can better monitor for anomalies.

Security control testing is an art form in addition to a technical security discipline. It takes a certain type of individual and mindset to figure out new vulnerabilities and exploits. Penetration testers usually fit this mold, and they must constantly research new attack techniques and tools. Auditors, on the other hand, might not test to that degree and will more than likely work with a penetration tester or team if a significant level of detailed knowledge in required for the audit. When performing these types of engagements, four classes of penetration tests can be conducted and are differentiated by how much prior knowledge the penetration tester has about the system. The four types are:

  • Whitebox: Whitebox testing is where the tester has complete information about the design, configuration, addressing, and even source code of the systems under test. This type of test is generally used to simulate a worst-possible scenario of an attacker who has intimate knowledge of the network and systems.
  • Blackbox: Blackbox testing is the classical penetration test in which the tester simulates an external hacker and is given no information about the subject under test, other than what he can glean from the testing methods. The concept of this type of test is to identify weaknesses that can be exploited based on publicly available information.
  • Graybox: This is a test that falls in the middle of the other two types in that some information is disclosed to the tester to “get him started.” Intended to simulate the insider threat, the penetration tester might be provided network diagrams, IP addressing, and user-level access to systems.
  • Red Team/Blue Team assessment: The terms Red and Blue Team come from the military where combat teams are tested to determine operational readiness. In the computer world, a Red and Blue Team assessment is like a war game, where the organization being tested is put to the test in as real a scenario as possible. Red Team assessments are intended to show all of the various methods an attacker can use to gain entry. It is the most comprehensive of all security tests. This assessment method tests policy and procedures, detection, incident handling, physical security, security awareness, and other areas that can be exploited. Every vector of attack is fair game in this type of assessment.

Auditors should have a base knowledge of testing tools and techniques. Using testing frameworks is a useful way to develop a technical testing planning. The next section introduces a couple of well known testing frameworks.