Security Operations Center: Challenges of SOC Teams

Fundamental weaknesses of a SOC (Security Operations Center) logo

Let me ask you a question: If big corporations, businesses and organizations around the world have SOC (Security Operations Center) monitoring their networks 24/7/365, how is it possible that data breaches still happen?

How is it possible that adversaries (APT groups, cyber criminals etc.) break their perimeter and exploit weaknesses in their networks, undetected?

In this article we will discuss what are some of the weaknesses and fundamental limitations of a SOC and why data breaches, security incidents and ransomware attacks will likely continue to happen in the future.

Table Of Contents
show

Introduction

In today’s world of increasing trends towards security “commoditization”, many organizations and business around the world are choosing managed (outsourced) SOC services (aka. SOCaaS – Security Operations Center as a Service) as their security front-liners.

This convenience, however, has its drawbacks when compared to an in-house SOC. But even the in-house SOC teams are facing issues, as the number of breaches only keeps rising and rising.

Let’s start drilling down a bit.

SOC requirements

In order for a SOC to be an effective defense mechanism, it needs the following things:

  • Good visibility (coverage) of the environment
  • Accurate information about the environment
  • Automation producing meaningful alerts
  • High quality threat intelligence feeds
  • Talented and skilled SOC analysts

All these things has to be there in order for a SOC to function well. If any of them is missing, it likely means a total failure of the SOC.

SOC or SIEM?

Most SOCs are built using one of the following SIEM (Security Information and Event Managenent) software products:

However, the issues mentioned here are not specific to any of them – rather they are applicable to a SOC in general.

Challenges and vulnerabilities of a SOC

The following sections provide closer look on what are some of the typical problems afflicting SOC teams around the world and why it is so hard to prevent security incidents from happening.

SOC is a reactive defense mechanism

SOC’s primary role is to passively monitor, observe and respond.

Although it is true that nowadays there is a Threat Hunting aspect to it which is somewhat proactive, most SOCs function mainly as a passive and reactive defense component that is triggered based on an alert of a suspicious or anomalous activity.

It is said that the best defense is offense and that’s why conducting penetration tests, red teaming and other offensive security exercises (phishing, vhishing etc.) should be vital for any organization.

Doing pentests regularly helps strengthening the overall security posture of the entire infrastructure and thus it should truly be the first line of defense.

But let’s get back to our topic.

Delayed events, alerts and workflows

In a SOC, everything relies on a quick response to events and alerts.

If an important event comes in, it is crucial that the SOC team can see it as soon as possible and that there are analysts ready to attend to it.

But this is an ideal case. In a real world, there are delays in the workflow caused by:

  • Late event arrival – event data coming delayed (tens of minutes late or even longer)
  • Backlog of unhandled incidents – alerts that are pending to be analyzed by the SOC

Larger the organization means larger attack surface, which means higher number of alerts to be processed and analyzed.

There is always a bottleneck somewhere and it’s extremely hard to diagnose or even detect a bottleneck in a SIEM. Are we getting real-time data? Aren’t there attackers poking into our systems right now? How can you be really sure?

This is one of the reasons why purple teaming can be extremely beneficial. Purple teaming allows close cooperation between the red team and the blue team allowing to also test the SIEM solution itself – e.g. test for precise correlation of events, test of alerts etc.

In most cases, getting a feedback and making sure that all SOC components are working correctly is a difficult problem. Most of the time, SOC analysts have no choice other than simply to trust the data.

Defending a castle with open gates

Nobody is perfect and people will always make mistakes.

No SOC can save you if your sysadmins keep leaving systems unpatched, misconfigured or protected by weak and default passwords (e.g. username: admin, password: admin).

The purpose of a SOC is not to go around and conduct pentests of the infrastructure nor do any kind of assessment. Their role is to monitor, observe and respond.

Now, if the environment is already broken and full of vulnerabilities, SOC cannot help much in defending it. Things will inevitably go south sooner or later.

For some examples, have a look on the list of top 10 vulnerabilities afflicting most organizations.

As already mentioned above, the best defense is to minimize the number of vulnerabilities in the first place. Keeping the environment harder to compromise will also increase the chances of attackers being spotted while they are doing reconnaissance, looking for vulnerabilities and so on.

Lack of information about the infrastructure

It is very common that managed SOC teams (SOCaaS) have to operate with limited information about the environment that they are monitoring.

This is because the clients generally do not share all the details about their infrastructure with the SOC. Things such as:

  • Updated list of important assets within the network
  • Correct naming scheme of devices and assets
  • Current up-to-date network diagrams
  • Stable list of firewall rules
  • List of important personnel

Failing to provide accurate and up-to-date information to the SOC team only makes their already hard job even harder.

In case of an event investigation, the SOC analysts have to spend precious time and effort figuring out additional details.

This can add unnecessary delays in the incident response, only making the client more vulnerable.

SOC analysts need to have the best information readily available about the environment that they are defending. Unfortunately, in many cases it is not the case.

Insufficient authority of the SOC

Another thing that a SOC requires for its proper function is to have appropriate authority within the organization.

For example, if the SOC team is not permitted to take a quick action without a signoff, how can they promptly respond to an incident and stop an ongoing intrusion?

Similarly, if they don’t have authority over other teams (e.g. network operations), how can they quickly isolate an actively compromised system?

Things can get difficult in such situations and every unnecessary delay can have serious consequences.

With a managed (outsourced) SOC monitoring, this is a even more difficult problem which often times prevents SOC from responding effectively to an incident in time.

Threat Intelligence gaps and lack of IOCs

A key component to every SOC is to have a high-quality threat intel feeds with a good curated list of IOCs (Indicators of Compromise).

Here are some of the most popular threat intel feeds known in the industry:

But for example just the EmergingThreats feed generates millions of alerts per day, so it’s actually a full-time job just to manage all the feeds and tune them down to reasonable levels.

Once tuned down, you essentially end up with a compromise between what you can handle and what you simply chose to ignore, because you don’t have all the resources (budget) in the world. You cannot respond to every (seemingly) benign alert that happens thousand times a day.

Now let me ask you: Is this going to stop sophisticated threat actors such as APT groups or other cyber criminals with nearly unlimited resources (and time)?

To fight against APTs and other advanced threat actors, the only solution is to involve yourself in a proactive threat hunting, in your specific and unique environment and produce IOCs in-house. This will then become yet another source, one of many.

In fact, there are cases of APTs having access to a network for years, only to be spotted after receiving a tip from a law enforcement agency with an actionable IOC. In such cases, there is no threat intel feed that would contain a matching signature for detecting such highly sophisticated threat actors.

Unfortunately, most SOCs don’t have resources for active threat hunting in their environments.

Limited visibility and blind spots

One of the things that hamper function of most SOCs is limited visibility.

For instance, SOCs don’t have visibility into TLS sessions and other encrypted communication that is going on within the network. Some organizations perform SSL inspection (aka. HTTPS inspection) on the border using solutions such as:

But this doesn’t cover all the encrypted communication that is happening internally between the systems and servers in the internal network. To achieve something like that is practically impossible.

Lateral movement is another problematic area which is very hard to take a grip of. Once the attackers are inside the organization, it is very hard to discern their activities from the massive amount of other internal communication that is going on within the organization.

Such a deep insights would require some sort of an EDR (Endpoint Detection and Response) solution running on every system, for instance:

In fact, endpoint visibility is another limitation most SOCs face. What they usually have at best is antivirus alerts from Windows-based endpoints. And this is not enough!

What they need are threat hunting capabilities on every machine including servers, network devices, appliances, mobile devices and so on. Needless to say, this is absolutely unachievable in the sheer technological complexity of the world we live in.

Available EDR solutions today are limited (platform coverage wise), expensive and integrations with SIEMs are problematic.

Adequate endpoint visibility is simply another blind spot for SOCs.

SOC alert fatigue / alert overload

Let’s face it, reviewing security alerts and looking on logs all day is not easy. In fact, it can be extremely exhausting and this a known phenomena in the cyber security industry known as a SOC alert fatigue.

Most SIEMs and threat management systems can log millions of events, producing thousands and thousands of alerts each day.

But even with all the automation efforts to filter out the noise, there is always a large number of alerts that you just can’t easily categorize and filter out by an automated tool.

Deciding whether an event is malicious or not is actually a really hard problem and not even AI (Artificial Intelligence) can really crack it. In fact, using AI can have detrimental effect!

For instance, when the AI is trained against a bad baseline (e.g. an already compromised or highly insecure environment), it can easily miss legitimate incidents.

So you can’t rely on automated tools or AI to do all the work. That’s why nothing can compare with a trained eye of a skilled cyber security / SOC analyst SME (Subject Matter Expert).

But people are not machines! Solving hundreds of security alerts every day, many of which are recurring and of a low importance, is extremely tiring, demotivating and stressful! Some even call it the biggest cyber threat of all.

Empirical data shows that the workflows implemented in most SOCs result in producing overwhelming number of alerts for the analysts to deal with.

Overwork, along with a lack of clear guidelines on what to do with so many recurring alerts, create environments prone to errors, only helping the attackers to accomplish their objectives.

Lack of training and no lab time

One of the key ingredient that can help with the aforementioned issues is to give the SOC members – SOC analysts and threat hunters – sufficient time off, away from dealing with alerts, and let them focus on their self-improvement.

Some of the things that every SOC analyst need for their work is to have:

  • Top quality training with focus on the latest trends
  • Time for practicing their skills in a lab environment

Ask yourself: How can the SOC respond to a sophisticated, possibly a novel (zero day) attack, if they don’t have the capabilities to recognize one? How can they fine-tune their SIEM to make the SOC more efficient, If they don’t have in-dept understanding of it?

SOC analysts and threat hunters need to have bleeding edge training, because their mission is to match (ideally exceed) the skills of the adversaries.

This is yet another important piece of a puzzle that is underestimated in many organizations.

Conclusion

When it comes to cyber security, defense is incomparably harder feat than offense. Defense has to protect against all threats – known and unknown (novel) ones, such as exploits, malware or zero day attacks.

Considering all the challenges that come with the job, defending an organization from cyber attacks is immensely difficult responsibility.

There are issues that cannot be easily solved and which require compromises, constant fine-tuning and adjustments on daily basis. This alone is something extremely difficult to do.

Now add a sophisticated threat actor (with unlimited resources) into the equation and you have the reality of what is happening today and what will likely continue to happen in the future.

But let’s not forget the rule number 1..

The best defense is..

Offense!

So let’s not give attackers a chance in the first place. The best defense is simply to keep the infrastructure up-to-date and secure. Conducting regular penetration tests and other offensive exercises greatly helps in this matter.

Consider the idea of purple teaming within your organization, because it really brings the best from both worlds. Blue team and red team cooperating together, learning from each other, sharpening their skills and tools at the same time, effectively doubling the value of every dollar spent.

You can learn more about purple teams here or here or here.

Thank you for reading! If you have enjoyed this article and you would like more like it, please subscribe to our mailing list and follow us on Twitter or Facebook to get notified about new additions.

SHARE THIS

TAGS | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

RECENT POSTS

MOST VIEWED POSTS

MOST VIEWED TOOLS

Leave a Comment

Your email address will not be published. Required fields are marked *