CISO Benchmark Study: Anticipating the Unknowns

See no evil, block no evil

Imagine if one could see deep into the future, and way back into the past – both at the same time. Imagine having visibility of everything that had ever happened and everything that was ever going to happen, everywhere, all at once.

And then imagine processing power strong enough to make sense of all this data in every language and in every dimension. Unless you’ve achieved that digital data nirvana (and you haven’t told the rest of us), you’re going to have some unknowns in your world.

In the world of security, unknown threats exist outside the enterprise in the form of malicious actors, state-sponsored attacks and malware that moves fast and destroys everything it touches. The unknown exists inside the enterprise in the form of insider threat from rogue employees or careless contractors – which was deemed by 24% of the survey respondents to pose the most serious risk to their organizations. The unknown exists in the form of new devices, new cloud applications and new data. The unknown is what keeps CISOs up at night.

This report sheds light on what actions are reaping results in strengthening organizational cyber health. For example, when asked, only 35% confirm that it is easy to determine the scope of a compromise, contain it and remediate from exploits. It suggests that visibility into the unknown clearly is a key challenge. It means 65% of CISOs in the survey have room to improve. 46% said that they have tools in place that enable them to review and provide feedback regarding the capabilities of their security practices. While the good fight is far from over, it’s also far from being all bad news. At least some respondents in the survey seem to be feeling good about their jobs. When asked about cyber fatigue, only 30% of respondents claimed to suffer from cyber fatigue this year. While almost a third seems like a high number to be tapping the mat and raising the white flag, the drop from last year’s figure of 46% is moving in the right direction and this is worth the fight.

State of the CISO

For some time now, threat hunters have talked about knowing the unknowns. It’s time to expand that to the entire spectrum of cybersecurity – to users, apps, data and clouds. You can’t protect what you can’t see.

You generally want to support the business, and not mire it down in bureaucracy. If you’re going to be a bit more open, how are you mitigating control? This is going to be different for everyone. CISOs must deal with that balance of organizational culture while combatting the most critical threats. Sometimes blocking everything and locking everything down doesn’t fit the culture of the enterprise. That might be right for a bank but not for a university.

The CISO faces several challenges managing cyber-risk – whatever their organizational model:

  • Breaches create adverse impacts to financial profitability, brand reputation, customer data security, customer satisfaction, and continuity of business.
  • Losses can be substantial and non-recoverable, creating a higher risk score for the organization on insurability.
  • Over the years, vendor point solutions looked promising; however, each generates their own set of alerts. Many point solutions competing on alerts makes it difficult to identify those threats posing the highest risk to the organization, and becomes a resource drain.
  • IT is usually siloed across the organization, making inte gration of securing the network, the cloud, and employee endpoints highly complex.
  • Aggressive tactics to hire security IT personnel are required, as the specialized pool of candidates cannot sustain the magnitude of the problem across global organizations. The talent shortage is, however, out of control and not solvable by trying to fill all jobs.
  • New threats such as Emotet, Olympic Destroyer and others appear daily, even hourly, and are employing more stealth and sophisticated methods. Threat response as a category has to evolve and there is a need for tools to consolidate information and centralize remediation of infections and other incidents.

Additional technologies and processes for the CISO to consider are:

  • AI and ML, and used right are essential to triage the volume of work.
  • The cost of a breach is falling – but don’t get too excited yet.
  • There is head room to realize obvious benefits in process improvement e.g., training.
  • There is more confidence in cloud-delivered security and in securing the cloud.

2019 findings

The findings from the Benchmark Study revealed several areas that are critical to strengthening organization’s security posture.

Set up for success?

What does it mean to be a CISO day-by-day? What is their charter? The present survey revealed multiple areas that together determine a organization’s cyber health including being practical about risk, setting criteria for budgeting, collaborating across divisions, educating staff, conducting drills, knowing how to track outcomes to inform investments, and being strategic on vendor and solution implementation.

Know your risk

Risk management is hardly table stakes. Understanding the risks of cyberattacks and the compliance landscape that encompasses security breaches is paramount to understanding how to defend and prepare for the worst. When asked who were very knowledgeable about risk and compliance, only 80% of respondents were very knowledgeable.

That leaves 20% of security professionals who could possibly use some of the discussed trainings.

How to spend budget

Almost half, or 47% are determining how to control security spending based on organizational security outcome objectives. Measuring outcomes against investments is the best data-driven approach. What’s more, 98% strongly or somewhat agree that their executive team has established clear metrics for assessing the effectiveness of their security program. 49% of respondents have metrics that are utilized by multiple areas of their companies to understand the risk- based decisions and improve processes to measure the security effectiveness throughout the organization. Back to the budget, and aside from outcome based measurement, there are some less healthy options. Controlling security spending on previous years’ budgets (46%) and percent of revenue respectively (42%) were both popular choices, but do not necessarily correlate with better security. The breach landscape changes year-to-year, and previous year’s budget or percent of revenue may have little to do with what it costs to defend against future threats.

The fourth most relied upon approach to determining security spending is cyber insurance – 40% are using cyber insurance, at least partly, to set the budgets. Taking this approach begins with a risk assessment to accurately identify security risks and ensure they can be mitigated by insurance or protected by controls. It may be for some companies that cyber insurance guidelines can play a role in technology selection and/ or budget setting. Either way this merits further investigation in subsequent reports.

Collaboration not isolation

In the survey, 95% judge themselves to be very or extremely collaborative between networking and security teams. They are not working in silos, and this has a tangible financial upside.

How much of a financial incentive? It turns out that 59% of those who were very/ extremely collaborative between networking and security experienced a financial impact of their most impactful breach of under $100K – the lowest category of breach cost.

This clearly merits further analysis and possibly points to greater need and possible development of more DevSecOps teams. The collaboration becomes not a matter of coinci dence, but a must, especially in the age of agile development, and this is recognized at the highest executive levels. According to a recent CIO study published by IDG – 82% of CIOs expect their IT and security strategy to be tightly integrated in the next 3 years.

Employee involvement: Drills and exercises

“ What if we train our people and they leave?” goes the question. “What if we don’t do it and they stay?” And the same applies from a security perspective. Yes, we focus on technology, but also we should spend equal time on process and on the people side of the business because our people are the front-line of help protecting our organizations. If people/ users are cited as the weakest link in security, having a process that starts with onboarding new employees is common sense. 51% rate themselves as doing an excellent job of managing human resources on security via comprehensive employee onboarding and appropriate processes for handling employee transfers and departures. It also seems counter-intuitive that the trend for training staff in the wake of an incident is flat year-on-year at only 39% of respondents. A disaster striking can be perilous without proper preparations. Potentially there is room for improvement in this area when 61% of organizations perform a drill or exercise every six months to test response plans to cybersecurity incidents drills can bolster the ability to have the proper controls in place to detect and respond as quickly as possible to mitigate damages.

Architecture: Navigating the vendor buffet

As the need for an all-encompassing approach to protect from cyber threats has grown, organizations have rushed to acquire multiple point solutions. In 2018, 21% of respondents had more than 20 vendors and 5% had more than 50. This year that has fallen to 14% and 3% respectively. The trend for number of vendors and solutions going down; but as multiple vendor solutions aren’t integrated, and therefore don’t share alert triage and prioritization on limited dashboards, the survey found that even those CISOs with fewer point solutions could better manage their alerts through an enterprise architecture approach. To better manage alerts, one best security practice is to reduce the number of vendors and point solutions. In 2018 there were 54% of respondents with 10 or fewer vendors in their environment, whereas now this number has risen to 63% This means more respondents have fewer vendors; vendor consolidation, for a variety of possible reasons, is real and measurable.

This multivendor approach (instead of an integrated approach) causes the persistent challenge of alerts to continue. 79% of respondents said it was somewhat or very challenging to orchestrate alerts from multiple vendor products, compared with 74% in 2018. Thus, while security professionals are attempting to address vendor sprawl and its attendant issues, managing it has not become easier and needs further improvement to optimize resources. This is where security analytics, machine learning, and AI can greatly help by automating the initial stages of alert prioritization and management. Bad adoption rates for these new technologies seem to have wobbled slightly this year.

Although the size of an organization may certainly contribute to the numbers of alerts and vendors, data tells that fewer vendors can make alert management more efficiently. At the top end of the funnel, 63% of organizations with only 1-5 vendors and 42% of organizations with 6-10 vendors saw fewer than 5,000 alerts per day. Of course it also might tell us they have alerts turned off, and reducing the number of vendors you have to manage helps your teams focus on more important work like remediation. Those with fewer than 10 vendors had a higher average response rate, remediating 44% of legitimate alerts, rather than 42%. You gain efficiency by lowering the number of security vendors.

Finally, 65% of organizations that are very up-to-date, and constantly upgrading with the best technologies available, more often experienced a lower count of daily security alerts (up to 10,000 per day). The next best option to replace or upgrade security technologies on a regular cadence (but not necessarily be equipped with the latest and greatest tools) – had a 60% chance of receiving up to 10,000 alerts per day.

Alert management challenge: You don’t know what you don’t know

Talking about too many alerts to anyone in security is like talking about the challenge of traffic to anyone in a major city. It’s a bad move on. But we generally do something about it – car pool, avoid rush hour, work from home etc. Moreover, alerts are also the key to the unknown and cannot be ignored. Buried in that pile of information are 1% of threats that get through even the best layered defense.

Here are five findings related to the alert landscape:

  1. There has been a year-on-year shift towards respondents seeing fewer alerts, which should mean less to manage, and in theory an easier time getting to the alerts that matter. The group with the lowest volume of alerts sees 10,000 or less per day, and 59% are in this group, versus 50% in our previous year’s survey.
  2. Ten thousand daily alerts are still too many? Sure, but when you consider that 41% see more than 10,000 and that some claim to see more than 500,000 alerts (admittedly only 1%), the figure of 10,000 is at least moving in the right direction.
  3. The good news is over. You’re responding to 50.7% of alerts compared with 55.6% in 2018. This suggests that while some of you are seeing fewer alerts, which feels like it should make the job easier, many are actually responding to fewer of them.
  4. Only 24.1% of alerts that were investigated turned out to be legitimate, down from 34% in 2018. This shows that the accuracy of the tools used to determine which alerts should be investigated are not doing their jobs.
  5. There’s worse news still when we look at alert remediation: There is a dramatic drop from the 2018 survey in the number of legitimate alerts that get remediated – from 50.5% to 42.8% this year.

Put another way as illustrated in Figure 7 – if an organization faces up to 10,000 alerts per day, it leaves 1,000 legitimate alerts unattended every single day, and that’s just the half (50.7%) of what is investigated. The case has never been stronger for security threat response tools that can ingest broad data sets, provide visibility into that big data, and provide a means to rapidly take action.

Manage what you measure

This drop in remediation is crucial given that many are moving towards remediation as a key indicator of security effectiveness. The number of respondents who use mean time to detection as a metric decreased from 61% (2018) to 51% (2019) on average. Time to patch has also dropped in focus from 57% (2018) to 40% (2019). The biggest shift is in respondents who focus on time to remediate (48%) as an indicator, which rose from 30% in 2018. This shows a new focus on remediation as a security professional’s KPI to measure their security posture. When you contrast this with the rise in the number of legitimate alerts not being remediated, a drop in investment in machine learning and a slow rise or steady rate in the amount of training, it appears we are in need of more innovation in alert management.

The survey data also revealed that 64% of those that collect data that allows them to measure their time to detect saw 10,000 or fewer daily alerts – the highest cohort in this matrix.

Breach readiness and response: When the unknowns comes calling

Attacks seen in the past year

For the first time this year, we got specific in asking about the types of attacks that CISOs have experienced and we asked about a set list of common attacks. While some have seen highly specific variants of malware such as WannaCry (11%) or threat categories such as wiper malware (15%) the most oft-cited attacks are malware and variants such as ransomware.

As shown in Figure 9, two of the top three are issues with email security remains the #1 threat vector. Whether you are investing in protecting the move to Microsoft Office 365 or trying to better protect against business email compromise (BEC) using DMARC, email is still an area to focus on. That two of the top 10 are insider threat issues (file sharing and stolen credentials), shows that you must look at what’s happening inside as much as outside, and be aware that some criminals can log in rather than break in. This drives the need for better multi-factor authentication (MFA). Nowhere is the need more apparent for balancing the need for security (letting the right people in) with supporting seamless business (not hindering the people you do let in with a clunky user authentication experience).

As concern with other areas remains high but manageable (such as the move to the cloud), concern about user behavior (e.g., clicking malicious links in email or websites) remains high and is now the top concern for CISOs. When asked about the challenge of defending various parts of their infrastructure, the highest concern was user behavior. This perception of vulnerability has held steady for the past three years between 56 to 57% of respondents.

We also asked which of these types of attacks resulted in some level of breach (loss of data), and received this priority of responses – Malware (20%), Data breach (19%), Spyware (14%), Phishing (13%), Ransomware (13%) and Malicious spam (13%)

Interestingly, the perceptions of risk varied among security-related roles. For instance, the risk and compliance officers consider the biggest vulnerability to be ‘Targeted Attacks’ – these executives are well-aware of the dire consequences a fatal attack could have on continuity of business.

Cost of a breach: More than just money

We’re all aware of the potential consequences of a breach financial loss (see Figure 10), brand and reputational setback or ruin, shaken stockholder confidence, loss of valuable data, regulatory and non-compliance penalties, and the list goes on. Looking at the year-on-year comparison of data, there is a clear shift towards issues of perception and sentiment; there’s no let up on the need to keep operations running, but customer experience and brand reputation both jumped up as key concerns.

What you are doing to address the threat of a breach

We asked our security professionals to what extent their organizations took precautions to put people, process, and products in place to safeguard their organizations. The results are shown in Figure 11. Further, we asked what approaches are taken to mitigate security risks, and those re sults are shown in Figure 12. And from even more data, we found that although 85% of respondents were very knowledgeable about policies and practices regarding infrastructure security and protection, only 74% were very knowledgeable about business continuity and disaster recovery. Only 75% percent of respondents were very knowledgeable about incident response. This is a problem. 100% of anyone involved in security should be knowledgeable about incident response; and in fact, this can be extended to all employees within an organization. This is where training becomes so vital, and its lack of prominence in this year’s results continues to stand out.

Dealing with the unknowns

The bridge between finding unknown threats and acting upon the right ones lies in an effective security posture. Here are practical recommendations we devised based on our survey findings for you to consider:

  • Base security budgeting on measured security outcomes with practical strategies coupled with cyber insurance and risk assessments to guide your procurement, strategy, and management decisions.
  • The only way to understand the underlying security needs of a business case is to collaborate across siloes – between IT, Networking, security, and compliance groups.
  • There are proven processes that organizations can employ to reduce their exposure and extent of breaches. Prepare with drills, employ rigorous investigative methods, and know the most expedient methods of recovery.
  • ML, AI and more automation should be able to boost security efforts exponentially – and next year we need to see more respondents in the ‘completely reliant’ phase of implementation and practice. Cisco employs machine learning technology in various security products including advanced malware protection, umbrella, stealthwatch and cisco threat response.
  • Build a security operations center (SOC) to manage breach response in organizations of all sizes.
  • Cloud security can help with the unknown. Ninety one percent agree that utilizing cloud security increased visibility into the network. Cisco Umbrella is cloud-delivered security that blocks users from connecting to known and suspected malicious domains, IPs and URLs, whether users are on or off the enterprise network.
  • Secure data centers and multi-cloud ecosystems with integrated solutions such as the Cisco Secure Data Center solution featuring visibility, segmentation, and threat detection from Tetration, Stealthwatch and the NGFW.
  • Address the number one threat vector with phishing protection, advanced spam filtering, and defense against business email compromised with DMARC – check out Cisco email security.
  • Endpoint security helps address unknown threats on user devices; try Cisco advanced malware protection for endpoints also available on our web, email, cloud and network security solutions, creating an environment of products that work together for more effective and efficient threat protection.
  • Gain fast threat detection, highly secure access, and software-defined segmentation with Cisco’s network visibility and segmentation which combines Cisco Stealthwatch Enterprise, the Cisco Identity Services Engine, and Cisco TrustSec technology.
  • Trusted access is a critical component of security. Duo verifies user trust (confirming a user is who they say they are) using its best-in-class adaptive multi-factor authentication (MFA) solution.


Leave a Comment