Penetration testing serves as a foundational part of your cybersecurity strategy. With regular pen test exercises, you can identify and resolve vulnerabilities before hackers exploit them. However, not all pen test vendors deliver the same results. As a result, they may not be as effective as you expect.
If you find inconsistencies or identify issues post-test that the security firm didn’t, it may be time to evaluate your partner. In this post, we’ll define some red flags that are signs you should seek a new pen test vendor.
The Basics: What You Should Expect from Penetration Testing
Before diving into those potential red flags, let’s talk about the basics of pen testing. At its core, a pen test is a simulated cyberattack carried out by ethical (white hat) hackers. A third party conducts these, and many organizations specialize in them.
Testers employ the same tools, techniques, and strategies as real cybercriminals. They can mimic a variety of attacks, depending on what they’re testing, which could include:
- Web applications: Assesses your overall security and risk by testing the architecture, design, and configurations of web applications, which consists of anything delivered over the internet via a browser interface.
- Network security: Identifies the exploitable issues on networks associated with routers, switches, or network hosts. The approach uses weak assets or misconfigured assets to breach.
- Cloud security: Confirms your cloud deployment security is accurate and evaluates overall risk and the likelihood that a breach could occur in cloud properties. You can test public, private, and hybrid clouds.
- IoT security: Leverages layered methodology to analyze IoT devices and their interactions. These assets are a preferred target for hackers, so assessing their security is critical.
- Social engineering: Uses phishing techniques to determine if your network can defend, detect, and react to these. It also gauges if employees are applying security training learnings.
In addition to what you can test, there are multiple methods:
- External testing: Testers target your visible assets (e.g., web applications, company website, email, and domain name servers) to attain access and extract data.
- Internal testing: This method occurs behind the firewall to simulate what could happen after an incident of human error, such as credentials stolen through phishing.
- Blind testing: A blind test only makes available the company name for the tester. In this scenario, your IT team experiences a real-time perspective of an application assault.
- Double-blind testing: In a double-blind test, internal security teams don’t know it’s happening. Your personnel would have to respond immediately to the threat.
- Targeted testing: Testers and internal security professionals work together. It provides a training experience for your team, and they get feedback from the hacker’s perspective.
Finally, there are different access levels in pen testing:
- Black Box Penetration Testing (Opaque Box): Testers have no prior knowledge of the target system’s internal structure. They operate like real hackers and look for any weaknesses to exploit.
- Gray Box Penetration Testing (Semi-Opaque Box): Ethical hackers have some information about the target system. Often, they receive insights on data structure, code, or algorithms. The penetration strategy is different and may include specific test cases.
- White Box Penetration Testing (Transparent Box): The third option involves giving pen testers access to systems and artifacts. They may also have the ability to enter servers running the system.
In reviewing pen test types, methods, and access levels, you’ll see that pen tests have many layers and options. You may not need every kind of pen test available, and not all firms will offer all these approaches. That’s the first potential red flag. If you’ve hired them in the past only to do web application pen tests and now want to execute a cloud security one, you may run into an obstacle here. They might not have the expertise for this, which could leave you vulnerable in the cloud.
More Red Flags: Proceed with Caution
Is it time for a new pen test vendor? These signs would point to yes:
1. They Only Use Automated Tools for Pen Tests
There’s nothing wrong with using automated tools in pen tests. They can be a great complement to human testers. Many organizations depend on them entirely to execute the test. The problem with relying solely on automation is that it is notorious for too many false positives and negatives. Scanning yields low observational findings, which are unlikely to be actionable. As a result, the accuracy of the tests falls into question.
It’s critical to ask providers how they pen test and what technology they employ. Again, automation can be a good start, but it will never equal the quality of human testers.
2. Their Experience Has Gaps
As discussed earlier, not all security firms can provide every type, method, or access level. Much of that has to do with their internal capabilities. Those testing need to have specific expertise and hold specific credentials like CISSP, CSSLP, OSWE, OSCP, ECSA, LPT (Master), CEH, etc.
These gaps may not have been apparent in an initial engagement. As you desire to go deeper and assess more parts of your cyber footprint, they may never have completed an IoT pen test, for example. If these assets are part of your network, it’s crucial to test their security posture. Such a need is very critical for medical device security. Threats are rising exponentially in IoT malware attacks, which hit 112 million in 2022.
In this case, along with others, your pen test vendor must be an expert in understanding the nuances. A standard pen test won’t accomplish your goals.
3. Complacency Can Create Blind Spots
Complacency is never a good thing in cybersecurity. The environment is too dynamic, with new threats occurring daily. Testers, however, can find themselves here, blind to vulnerabilities they overlooked. They aren’t purely inept. Rather, they’ve become comfortable about what they know and may not feel they need to brush up on skills. That puts you in danger.
One way to determine if that’s happening is to have a new potential vendor review the findings from your current provider. They won’t officially launch a new test, but they may point out some results they question.
4. Reports Are Confusing and Not Transparent
At the end of a pen test, you receive a report of what they found and recommendations for fixes. Ideally, the report should be concise and easy to understand without high levels of technical knowledge. It should also lay out the priority level for remediation and be an actionable list.
That’s the best-case scenario, but rarely what vendors deliver. Pen test reports can be overly complex and hard to understand. You’d need an “interpreter” to get any value from it.
Why do some vendors do this?
Several reasons can lead to this type of reporting. Some firms want to be too technical and complicated, so you have to depend solely on them for interpretation. Technical folks can be condescending and out of touch with what clients expect. If this is the exchange you’re receiving, it’s a big red flag.
5. They Don’t Provide RVTs
A remediation validation test (RVT) should be part of a firm’s deliverables. RVTs come after you’ve fixed the vulnerabilities identified in the pen test. It’s a final check to ensure you’ve resolved the issues properly. If your current provider doesn’t include this, you’ll have questions about remediation until they conduct the next round. During this time, you may be more susceptible to an attack.
6. They Aren’t Familiar with Regulatory Requirements
If you’ve been using pen tests to support regulatory requirements, your vendor must have experience with them. Some examples include:
HIPAA
HIPAA doesn’t explicitly require pen testing but does state in HIPAA Evaluation Standard § 164.308(a)(8) that a covered entity or business associate is required to “perform a periodic technical and nontechnical evaluation.”
A technical evaluation would be a pen test.
Information Access Management: § 164.308(a)(4)27 references the requirement to assess “security measures related to access control” and confirm how effective authentication practices are in preventing unauthorized access to PHI (protected health information) and other assets containing protected information.
NIST 800-66 for HIPAA cites this recommendation: “Conduct penetration testing (where trusted insiders attempt to compromise system security for the sole purpose of testing the effectiveness of security controls), if reasonable and appropriate.”
Thus, your pen test vendor must have deep knowledge of HIPAA and healthcare cybersecurity.
PCI DSS
PCI DSS (Payment Card Industry Data Security Standard) is an information security standard related to credit card information. The PCI SSC (Payment Card Industry Security Standards Council) oversees it, and card brands mandate it.
The standard includes four levels of compliance, depending on the transaction dollar amounts. All require a PCI scan.
SOC 2 Type 2
SOC 2 Type 2 is a System and Organization Controls (SOC) framework that provides a report about security, confidentiality, integrity, privacy, processing, and availability of data controls to organizations.
Any organization that uses or transmits protected data should undergo a SOC 2 Type 2 pen test. It applies to SaaS and tech companies that hold customer data. Conducting these ensures you comply with the scheme and have the correct security controls to safeguard data.
Is It Time to Find a New Pen Test Vendor?
If any of these red flags sound familiar, you should evaluate other options. Ask many questions about capabilities, expertise, certifications, and testing methods. Don’t let an ineffective vendor leave you at greater risk.
Contact our pen test experts today to learn more about our solutions.
