Solarwinds Cyber Hacking

Iqbal Singh
Technology Expert & Senior Corporate Executive in a European MNC
E: iqchucks@gmail.com

---

The recent SolarWinds Cyber hacking is deemed by many cyber security experts to be the biggest security breach ever in the history of cyber hacking. The attack was audacious, sophisticated, meticulous, stealthily executed, and the range of targets is said to be staggering – Fortune 500 companies, US Federal and State Departments including Defense, State, Treasury, US Cyber Command and the National Nuclear Security Administration (NNSA). The attack has shaken up the establishments and the corporate world across the globe. Such was the importance attached to the matter that US President Joe Biden allocated US$ 9 billion to improve cyber security infrastructure on Day 1 of taking office.

The full impact of the attack and the causes are still being ascertained as I write this. Being an extremely complex attack while a lot is spoken, written and talked about it, most people are not very clear about as to what exactly happened, and how and what preventive measures should one take in the future.

In this article I shall attempt to demystify the attack in as simple a manner as possible bereft of technical jargon, and in an easy to understand manner even for a non-technical layman. I must also insert here a disclaimer that the article is based on the current understanding of the issue as per the info available in the public domain, things can change as more unknown details unravel.

SolarWinds

SolarWinds is a company that makes IT monitoring and management software solutions. It counts 425 of the Fortune 500 companies and several key US Federal and State agencies amongst its customers. It has over 33000 customers globally. One of their products Orion had been infected and the same software was installed by around 18,000 of its customers.

I feel that for giving the readers an idea of the attack it would be best to begin with how the attack came to light. While the readers may not understand all the jargon I request them to hold on for a few moments as I would explain them later in the article.

 

What is different this time?

Cyber attacks are not new. Usually there’s a vulnerability that allows threat actors to get into the network. What’s unique about this case is that the initial vulnerability was in the vendor software, so it’s often now being referred to as a supply chain hack because the vulnerability was embedded as code.

Other differences are:

1. SolarWinds’ security products impacted. 18 known products and 18,000 customers were delivered with the malicious code.
2. Federal agencies. The exposure to federal agencies was a matter of grave concern. While targeting government agencies, they focused to access their emails.
3. FireEye red team tools. Sophisticated tools from FireEye got into the nefarious actors’ hands.
4. Post breach into the target network. The attackers settled in, sat there for a while, scanned the network, moved laterally in that environment and hunted for privileged access.
5. Orion software build and code signing infrastructure was compromised. The source code of the affected library was directly modified to include malicious back-door code, which was compiled, signed and delivered through the existing software patch release management system.
6. Attackers were very patient. They waited for a prolonged duration to extract the data and then cover their tracks.
7. SolarWinds operation is an intelligence gathering effort,’ rather than an operation looking to destroy or cause mayhem among US IT infrastructure.
8. SolarWinds customers. Look like the who’s who of the corporate world.
9. 1. 425/ 500 US Fortune 500 companies.
   2. Top 10 US telcos.
   3. Top five US accounting firms.
   4. All branches of the US Military, the Pentagon, and the State Department.
   5. Hundreds of universities and colleges worldwide.

This is not just an attack on specific targets but on the trust and reliability of the world’s critical infrastructure.

In that sense this attack would have profound implications not just in for cyber security teams, but software development, release, DevOps processes, automation and much more.

 

INTRO TO SOME TECH JARGON

a.  Attack vectors: Refers to the method or pathway that malicious code uses to infect systems.

b.  Virus: A computer virus attaches itself to a program or file so it can spread from one computer to another, leaving infections as it travels.

Almost all viruses are attached to an executable file, which means the virus may exist on your computer but it cannot infect your computer unless you run or open the malicious program.

It is important to note that a virus cannot be spread without a human action, (such as running an infected program) to keep it going.

People continue the spread of a computer virus, mostly unknowingly, by sharing infected files or sending e-mails with viruses as attachments in the e-mail.

c.  Worm: A worm is similar to a virus by its design and is considered to be a sub-class of a virus. Worms spread from computer to computer, but unlike a virus, it has the capability to travel without any help from a person.

A worm takes advantage of file or information transport features on your system, which allows it to travel unaided.

The biggest danger with a worm is its capability to replicate itself on your system, so rather than your computer sending out a single worm, it could send out hundreds or thousands of copies of itself, creating a huge devastating effect.

d.  Trojan horse: It is not a virus. It is a destructive program that looks as a genuine software application. Unlike viruses, Trojan horses do not replicate themselves but they can be just as destructive.

Trojans also open a backdoor entry to your computer which gives malicious users/ programs access to your system, allowing confidential and personal information to be stolen.

e.  Malware: This is malicious software which is used to attack information systems. Ransomware, spyware and Trojans are examples of malware. Depending on the type of malicious code, malware could be used by hackers to steal or secretly copy sensitive data, block access to files, disrupt system operations or make systems inoperable. The malware can carry out various functions like stealing data, encrypting files, deleting data, altering files or even adding those systems to a huge botnet and monitor these systems without the user ever knowing that their computers are infected.

To get an idea, according to the FBI, damages caused by different types of malware amounted to more than US$10 million just in 2019, and the most widely used form of malware spreading continues to be via email.

f.  Backdoor: A backdoor is a means to access a computer system or encrypted data that bypasses the system’s customary security mechanisms. Backdoors can vary widely. Some, for example, are put in place by legitimate vendors, while others are introduced inadvertently as a result of programming errors.

Developers sometimes use backdoors during the development process, which are then not removed from production code. A developer may create a backdoor so that an application or operating system can be accessed for troubleshooting or other purposes. However, attackers often use backdoors that they detect or install themselves as part of an exploit.

In some cases, a worm or virus is designed to take advantage of a backdoor created by an earlier attack. Backdoors are also commonly put into place through malware. A malware module may act as a backdoor itself, or it can act as a first-line backdoor, which means that it acts as a staging platform for downloading other malware modules that are designed to perform the actual attack.

g.  Command and control (C&C) server: A command-and-control [C&C] server is a computer controlled by an attacker or cybercriminal which is used to send commands to systems compromised by malware and receive stolen data from a target network. Many campaigns have been found using cloud-based services such as webmail and file-sharing services, as C&C servers blend in with normal traffic and avoid detection. C&C servers serve as command centers that malware related to targeted attacks use to store stolen data or download commands from. Establishing C&C communications is a vital step for attackers to move laterally inside a network.

C&C servers also serve as the headquarters for compromised machines in a botnet. It can be used to disseminate commands that can steal data, spread malware, disrupt web services, and more.

h.  A supply chain attack: A supply chain attack is a cyberattack that attempts to inflict damage to a company by exploiting vulnerabilities in its supply chain network.

A supply chain attack entails continuous network hacking or infiltration processes to gain access to a firm’s network in order to cause disruptions or outages, which ultimately harm the target company.

More than 60% of cyberattacks originate from the supply chain or from external parties exploiting security vulnerabilities within the supply chain, according to a 2016 survey by Accenture.

A supply chain attack seeks to infiltrate and disrupt the computer systems of a company’s supply chain in order to harm that target company.

The idea is that key suppliers or vendors of a company may be more vulnerable to attack than the primary target, making them weak links in the target’s overall network. Supply chain attacks can be more commonplace than attacks on primary targets, and originate via hacking attempts or through inserting malware. The supply chain network is a frequent target for cyber crimes, as a weak link in the supply chain can grant the cyber criminals access to the larger organization in custody of the data sought after.

Supply chain attacks expose a conundrum in a company’s supply network which discloses that an organization’s cyber security controls are only as strong as that of the weakest party on the chain.

i.  Understanding the software release methodology:

1. Continuous integration (CI): A software development practice in which all developers merge code changes in a central repository multiple times a day.
2. Continuous delivery (CD): Which on top of Continuous Integration adds the practice of automating the entire software release process.

A CI/ CD pipeline automates your software delivery process. The pipeline builds code, runs tests (CI), and safely deploys a new version of the application (CD). Automated pipelines remove manual errors, provide standardized feedback loops to developers, and enable fast product iterations.

HOW EXACTLY IT HAPPENED AT SOLARWINDS?

Let’s take a brief look at the timelines of the attack. It is now known that the attack started at least more than a year back.

CHRONOLOGY

1. 04-Sep-2019: Threat actor (TA) accessed the SolarWinds (SWI).
2. 12-Sep-2019: TA injects test code and begins trial run.
3. 04-Nov-2019: Test code injection ends.
4. 20-Feb-2020: Sunburst compiled and deployed.
5. 26-Mar-2020: Hot fix 5 DLL available to customers.
6. 04-Jun-2020: TA removes malware from the Build VMs.
7. 12-Dec-2020: SWI notified of Sunburst by FireEye.

MORE DETAILS

The attackers managed to modify an Orion platform software plug-in that is distributed as part of Orion platform updates. The trojanized component was digitally signed (as was the norm) and contained a backdoor that communicates with third-party servers controlled by the attackers.

Multiple trojanzied updates were digitally signed from March-May 2020 and posted to the SolarWinds updates website which were installed by their customers.

FireEye tracked this component as sunburst and released open-source detection rules for it on GitHub. After an initial dormant period of up to two weeks, it retrieved and executed commands, called ‘Jobs,’ that include the ability to transfer files, execute files, profile the system, reboot the machine, and disable system services.

The malware masqueraded its network traffic as the Orion Improvement Program (OIP) protocol and stored reconnaissance results within legitimate plugin configuration files allowing it to blend in with legitimate SolarWinds activity.

The backdoor used multiple obfuscated blocklists to identify forensic and anti-virus tools running as processes, services, and drivers. As a result it managed to evade them.

The attackers kept their malware footprint very low, preferring to steal and use credentials to perform lateral movement through the network and establish legitimate remote access.

MODUS OPERANDI

1. Teardrop: The backdoor was used to deliver a lightweight malware dropper (trojan) that had never been seen before; it has been dubbed Teardrop. This dropper loads directly in memory and does not leave traces on the disk.
2. Deployed customized penetration tool: Researchers believe it was used to deploy a customized version of the Cobalt Strike (a company’s product) Beacon payload. Cobalt Strike is a commercial penetration testing framework and post-exploitation agent designed for red teams that has also been adopted and used by hackers and sophisticated cybercriminal groups.
3. Temporary file replacement techniques. To avoid detection, attackers used temporary file replacement techniques to remotely execute their tools. This means they modified a legitimate utility on the targeted system with their malicious one, executed it, and then replaced it back with the legitimate one.
4. A similar technique involved the temporary modification of system scheduled tasks by updating a legitimate task to execute a malicious tool and then reverting the task back to its original configuration.

HOW IT MANAGED TO ACCESS THE SECURITY TOKENS

Once in the network, the intruder then uses the administrative permissions acquired through the on-premises compromise to gain access to the organization’s global administrator account and/ or trusted SAML (Security Assertion Markup Language) token signing certificate. This enabled the actor to forge SAML tokens that impersonate any of the organization’s existing users and accounts, including highly privileged accounts.

Anomalous logins using the SAML tokens created by the compromised token signing certificate could then be made against any on-premises resources as well as to any cloud environment (regardless of vendor) because they had been configured to trust the certificate.

Using the global administrator account and/ or the trusted certificate to impersonate highly privileged accounts, the actor may have added their own credentials to existing applications or service principals, enabling them to call APIs with the permission assigned to that application.

Because the SAML tokens were signed with their own trusted certificate, the anomalies might be missed by the organization.

SUGGESTED SAFETY MEASURES

Layered approach to security: This is the way to go in the future in light of this hack’s sophistication. There’s no silver bullet to stop a hack this sophisticated, though. No one strategy or approach could have prevented it.

This attack is pretty sophisticated and has multiple vectors to it and one has to assume there will be certain threat vectors compromised.

That initial vulnerability is there and you need those layers of security to prevent it, so you need to look at preventive controls, predictive controls and detective controls. All those need to be combined into a single, unified strategy.

Adopt standards:

1. The Federal Information Processing Standard 200, or FIPS 200, the standard offers excellent guidance, including discussing the different types of layers and controls available today.
2. ards and Technology (NIST) Special Publication 800-53, that gets a little deeper into the particular cyber controls you have in place. There is guidance available. You’re not out there on your own about what the layers should be and you can evaluate yourself against these standards.

Privileged Access Management (PAM): Investigations into this hack found specific evidence where they got in and created new accounts with elevated privileges to access data.

We typically state the Forrester statistics that 80% of hacks involve compromised privileged access. This SolarWinds example is no exception.

Passwords: Using the company’s name, followed by 123, is not a good idea. SolarWinds password was solarwinds123. The company was warned in 2019 by a cyber security expert to change its password.

Organizations can design preventive privileged access controls and detective controls and both are typically provided in Privileged Access Management solutions.

Best practices call for multiple preventive controls – strong passwords, multi-factor authentication, password rotation etc., maybe used as a federated credential and have privileged users log in as themselves for better auditing and accountability

Rethink cybersecurity strategy. From a preventive control perspective that includes least privileged access.

Preventive controls need to be strengthened with least privilege. The account creation process needs to provide as little privilege as possible to the server level.

Workflows to request additional access need to be used to provide resources for a predefined period. If these types of controls had been in place, malicious code disguised in executable files and dynamic linked libraries would not have traveled as far down the supply chain.

Detective controls. When even if threat actors get through or you don’t have enough of those layers in place, you want detective controls. PAM solutions should have audit capabilities that watch what privileged users do.

In the financial markets, there are things like the ‘foureye principle,’ where people are watching what other people are doing and so you can watch a privileged session in real-time and verify what users are doing.

Of course, all that’s audited in the recording. You can send that information off to a Security Info Event Management (SIEM) to be correlated with other data to look for compromise indicators.

Recent articles I’ve read pointed out the attackers were in the FireEye network for months before being detected. FireEye detected that they had been attacked thanks to detective controls.

Zero Trust model: Organizations can get started on their Zero Trust frameworks by reviewing the FIPS and NIST publications. Review the layers of your security stack with a Zero Trust mindset. Don’t configure your network to trust someone just because they gained access. That’s how these attackers got in, laying in the network for plenty of time. Zero Trust says, “Don’t trust that authenticated network access. That could still be a compromised credential or a threat actor,” and this is a perfect example of that. This is why Zero Trust is critical – just because they’re on your network doesn’t mean they’re trustworthy.

Segmentation. The concept of least privilege, of authenticating at each step, introduces segmentation. When I give access, it’s just to that machine or that service that I need access to and not broad access across the network. That’s how you prevent that lateral movement. A Zero Trust mindset is that Zero Trust philosophy of security which is critical in this case.

CONCLUSION

While some measures have been listed above these are by no means an exhaustive list. The hack or rather the espionage related details are still unfolding as on date. It may even take a couple of years before all the attacked systems can be detected, cleaned and sanitized. Cyber security is not easy, add to it the fact that so much of IT infrastructure is privately owned, it’s even harder. Every router, every software program, every server, IoT device may inadvertently offer a way for malicious actors to enter and compromise a network. It is no wonder that large organizations spend several hundreds of million dollars on their cyber security.

To conclude one may say that cyber security is complex, getting more significant by the day and despite all the efforts we may still have no ideas to what cyber dangers we face.

---