Redline is a free tool for investigation malicious activity through memory and file analysis. It has a lot of features for investigation but in this post, we will only mention searching for IoCs in the endpoint with Redline.
In previous post, we created an IoC to detect WinSCP.exe. Now, we will search it with Redline as the example.
We will go on with “Create an IOC Search Collector” menu in the main page of Redline. For doing this, we browse the folder that including IoCs we want to search in the PC. We have only one IoC here but if you have more IoCs in the folder, you will see all of them in “Indicators” tab.
Then we create a folder for IoC Collector and after clicking “Next” button, we show this folder. Redline creates the IoC Collector in this folder. We will now use RunRedlineAudit.bat file with the command line. Once the bat file finishes running, it will create a folder called “Sessions” and save outputs to this folder in the same directory.
Just run the “RunRedlineAudit.bat” file and wait for finishing. Then, open the “Sessions” folder. Each IoC sweep placed in its own folder calle “AnalysisSessionX”. This was our first sweep, so we click on “AnalysisSession1” folder. Our IoC report will be in “AnalysisSession1.mans” file. So, we click on this file, and it will take some time it generates the report.
When IoC report generated, we can see it on Redline tool, “IOC Reports” tab. As you can see in the screenshot, our WinSCP Indicator IoC got hits. When we click on it, we can see why this IoC got hit. Here, our IoC catch the file with its MD5 hash value and file name. With clicking on “View Details” button, we can see more details about the hit.
An effective threat hunting is critical because it is hard to think like attackers and to search for the unknown in an enterprise network. This post may help organizations for an effective and successful threat hunting.
Knowledge of Topology and Environment
The purpose of threat hunting is to find the anomalies and their sources in the network and endpoint. So, a threat hunter should know what is normal and so can understand what is not normal.
From the risk management point of view, critical assets – servers, applications, data – should be known to protect more effectively. With the knowledge of the environment, the threat hunter knows the critical assets and hunts according to this. If there is segmentation in network, it is also critical to know the network topology and networks – or vlans – of these critical assets.
It is also necessary to know which application is running on which operating system, so the threat hunter can know the weaknesses of the system and can search according to these weaknesses.
Effective Endpoint Management
For threat hunting, the most used tools are EDRs. Organizations should be sure that they installed endpoint security tools to all endpoints and detect when they removed or stopped. Asset management is something more than CMDB. It must be managed by security teams whose understanding the criticality of the lack of endpoint security tool in an endpoint.
Threat intelligence is one of the most important feeds of threat hunting. Threat hunters need to have most recent intelligence and IoCs so they cant perform hunting the latest threats. Many malicious are produced and detected every day. This situation causes a lot of noises about intel. To avoid this noise, hunters should get valuable intelligence about their organization’s sector and geolocation, and integrate these IoCs with SIEM, EDR, etc..
We all know that new tactics and techniques are created by attackers in general. The most important reason for this is while security professionals in organizations have to deal with so many different things, attackers can only focus on their target. Even if it so important to have valuable/experienced personnel, if they are dealing with different organizational missions while they are working, it will be difficult for them to think like an hacker and detect the unknown in the network. Threat hunters should focus to their mission to create their methodology and hunt. So, there should be dedicated personnels for threat hunting.
Coordination Across the Organization
Yes, threat hunters work in a strange mission, think like a hacker and search across the network but they must not work alone. A threat hunter should have a good relationships with key personnel in IT departments like network and system admins, help desk personnel and so on. With these relationships, they better understand the network, systems and more importantly the company’s and personnels’ way of doing business. For organization’s perspective, when a threat hunter finds a weaknesses during the hunting process, they inform critical IT personnel for remediation. This team working will result as a success in remediation phase of the incidents or weaknesses.
Intelligence is critical for hunting for the known threats but hunters should be familiar with the TTPs of the attackers against the zero day threats. Threat hunters also should be aware of the updated or newly TTPs. Only with this knowledge hunters can act like an attacker. TTPs are at the top of Pyramid of Pain (defined by David Bianco).
To disclose the anomaly or malicious activity, threat hunters should use advanced tools like EDR, NDR, SIEM, FIM, etc.. These tools will help hunter to find abnormal activities if configured properly. In different posts, we tried to explain why they should be used.
Although Threat Hunting is nothing new, it is a very hot topic lately. Even if you have perimeter and endpoint security devices and SIEM for collecting and correlating logs from them, it is not a good way to wait for incidents coming. Without Threat Hunting, dwell time is increasing more than 150 days, and this is not acceptable anymore. While attackers are working proactively and developing new techniques day by day, security teams need to be more proactive too. Threat Hunting is the most proactive approach in an organization’s security structure and improves its security posture.
Dwell time is the dirty metric nobody wants to talk about in cyber security. It signifies the amount of time threat actors go undetected in an environment, and the industry stats about it are staggering. Source: Extrahop
In its simplest definition, Threat Hunting is detecting abnormal activities on endpoints and network. But, what to look for hunting threats?
What to Hunt?
Threat Hunting is a continuous process. Hunters should check anything that could be an evidence of an incident.
Processes: Processes are important components of OSs. Adversaries may inject malicious code into hijacked processes. Therefore, hunters should check processes and child processes regularly.
Binaries: Hunters should check binaries with their checksum, name and other specifications.
Network: Network activities to specific destinations and anomalies in network should be checked.
Registery: Hunters should check registery key additions and modifications.
For a continuous hunting, organizations need to have threat hunters in their CSIRT. The difference between analysts and threat hunters is the proactive approach as mentioned before. Also in smaller organizations, SOC analysts may work on threat hunting but actually, a threat hunter may has more specifications than an analyst. In larger organizations, it is important to have a dedicated threat hunting leader and team. This team should has detailed knowledge about;
OSs: The Threat Hunting team should have knowledge about OSs that organization is using. This knowledge must include process structures, files, permissions, and registery depending on the OS. This is important because malicious files and attackers make changes in OS here. A threat hunter need to understand what is normal and what is not. Something is not normal could be a sign of an intrusion. For having this knowledge, baselines could be created for all critical systems. These baselines will help to know the normal and the anomalies.
Apps: Threat Hunters should have knowledge about the applications used in the organization. It is also important to know perimeter and endpoint security devices and applications used by organization.
Business: Threat Hunting team should have knowledge about the organization’s business so they need to follow adversaries working on the organization’s sector and geographical location. It is also important to know third party companies the organization works and communication ways with them.
Network: In a big and segmented network structure, it is important to know where the critical assets are.
TTPs: IoCs are important components for hunting but they provide to detect “known knowns”. TTPs are at the top of Pyramid of Pain (defined by David Bianco) and especially adversaries’ techniques and tactics should be known those are threatening the organization’s sector and location.
Threat Hunting Tools: In CSIRT plan, it needs to be included that which tools and techniques can be used for threat hunting. Threat hunters should have knowledge of these tools and techniques.
IR&H Plan: Threat hunting is only a step of proactive approach. If threat hunters successfully find an intrusion or anomaly in systems, they need to know the next step. Who should they inform? What should be done?..etc..
Threat Intelligence: Threat intelligence is one of the most important feeds of threat hunting. Threat hunters need to have most recent intelligence and IoCs so they cant perform hunting the latest threats.
EDR: Threat hunters need IoCs but also need to know how to use these IoCs. After gathering the most recent IoCs from TI platforms, an IoC sweep must be made on endpoints.
NDR: Just like endpoints, network traffic also need to be checked with the latest IoCs. For doing this, CSIRT need to collect all east-west and south-north network traffic. NDR devices those have AI capabilities also detect anomalies in the network.
SIEM: Depending on the hunt’s scope, the threat hunter may need to check IPS/IDS, proxy, DNS, firewall or some other tools’ logs. Because logs are coming from different sources, CSIRT need to collect and correlate these logs in SIEM and feeding SIEM with the latest IoCs, these logs will more meaningful.
FIM: We said that baselines must be created for critical systems. FIM solutions will help CSIRT to create baselines for OSs and alert analysts when an unauthorized transaction is made.
A new malicious called Prometei has been determined, that including Exchange servers have ProxyLogon vulnerability to cryptocurrency network. Prometei is a modular malicious code and has different features like credential dumping, usage of the system for cryptocurrency minning, and lateral movement. Prometei has two different versions for both Windows and GNU/Linux.
Prometei exploits the ProxyLogon vulnerabilities (CVE-2021-27065 and CVE-2021-26858) and uploads China Chopper web Shell. After uploading China Chopper, attackers downloads the zsvc.exe on the victim’s machine with PowerShell. After gaining persistence, another malicious file called sqhost.exe is downloade by attacker. With sqhost.exe, attackers can use the victim system for Monero minning with using XMRig open source code. However, Prometei uses Mimikatz for lateral movement.
A more detailed investigation of Prometei can be found on the cybereason blog page. It looks like threat actors still keep using Prometei. To avoid of this risk, exchange vulnerabilities need to be eliminated fastly, and these IoCs can be used to detect and prevent Prometei.
Last week, I hearth that an organization did not add antivirus agent to their PC image. They are formatting the PC with their image, then connecting to the network and waiting for the sccm installing the antivirus software to the PC. Also, for remote users working on the field, some contracted partners are formatting the PCs since these users cannot come to the company, they then join to the network via VPN after formatting and keeps working. Meanwhile, the IT team is waiting for the sccm install the antivirus software, but because of the VPN network, most of the time it fails. PC keeps working on the network for days.
While I was sharing this situation with some friends in the industry, some of them also said that it is a normal process for the organizations. So, I wanted to write this article.
A few months ago, I shared a post about falling of the AV. It is true that AV softwares are not very efficient in recent years. There are many other measures need to be taken to protect the endpoints. However, most of these measures are for APT attacks. As everyone says, and also I touch in the article, attackers’ profile and techniques has changed a lot, since the times AV was popular and successful. But, despite all these situations, nobody can say that AVs are not necessary anymore. Organizations does not face attackers that using highly advanced techniques and tools only. There are still many script kiddies and those trying to learn hacking. These people are always looking for easy vulnerabilities to hack. It is very great possibility they find you.
Another subject about AV, because of the hash databases downloaded, they can protect users for many of the malicious events, also while they are offline, or while they are not connected to the office.
Even, most of the AV softwares are improving themselves with behavioral and AI capabilities. So, these can also detect and stop some of the APT attack phases.
I am also curious your comments, but my opinion is an AV is still indispensable for all organizations. So, I want to some best (must) practices for using AV in an organization;
– An AV software should be installed on all devices. Clients should be periodically followed whether has AV on it or not. If it is possible, a NAC solution should be positioned and PCs that does not have AV should be blocked.
– AV solution should be centrally managed. So, updates can be managed centrally and out of date clients can be followed.
– Administrators should make sure all clients are sending logs properly. It is very important to response a suspicious situation quickly.
– AV software should be updated periodically. Meanwhile, administrators also should be sure that all clients are getting the latest updates properly.
– AV software should be included into the PC and servers’ regular images. When a PC formatted and re-installed, it should include AV before connecting the network.
– Users should not be able to disable the AV services and agent. Tamper protection and an uninstall password should be used and should be stored in a password management system.
– Malicious files should be blocked and quarantined to be analyzed by the administrators.
– Audit logs should be collected properly. Administrators should login to the software only by their own usernames. Generic usernames should not be used.
– Too many exceptions should not be given. If needed, exceptions should be given only as stated by the vendors.
– If including, host-based IDS should be enabled on the AV agent.
All IT Security experts surely faced with such situations that anyone who does irrelevant with security, know only AV about computer security. AVs was the hero of our security for long times.
Legendary Times AVs begin their advanture as signature based protection against known viruses and worms. With the development of the threats; fisrtly with script kiddies, than financially motivated hacker groups, it was enough to update signatures weekly or every few days. Today, things work a little differently. Actually, much more differently. As mentioned in the “A Guide to Choose EDR” blog before, with the explosion of the connectivity between PCs and mobile devices, usage of cloud more day by day, threats have also changed. Attackers now have the ability to bypass signature based detection and protection technologies. For dealing with these situations, heuristics detection skills have been added to AVs. Machine learning and behavior monitoring added for detecting and blocking suspicious behaviors. Also, AV vendors added host based IDS/IPS, hostbase firewall and device control skills, and these features become very useful for admins, to use all of these features within one agent, while this agent is already deployed in all PCs.
Fall of the Hero Despite all these new features, researches conducted after 2018 say that AV products misses more than %50 of attacks. Besides, false positives caused by constantly updates causes difficult situations fort he IT professionals. Everyone accepts that there is no any solution providing %100 security. With this approach, speed of response and visibility become the key features against threats. This is the reason SIEMs must be used to complement to AV. Yet there are also caveats that it is not enough and advanced tools like endpoint detection and response (EDR) solutions must be implemented alongside AV. That must be true, at least we see that AV vendors are also now developing such solutions beside their AV solutions. You can access to a more detailed review of EDR solutions; what they must include and how to choose them here.
What is next?
Now, AV vendors thich are also developing EDR solution, suggest that the customers must implement these solutions beside the AV. Meanwhile, the vendors developing only EDR solutions, or vendors which entered endpoint field with EDR, say that customers can change their AV with EDR solutions peace of mind. But is it so easy to replace AV with EDR? Or simply, is it easy to change any AV with something else. As mentioned before, companies are now using their AV agents for device control, host IDS/IPS, host firewall, application control and whitelisting. For replacing the AV, the new product must support these features, even if the solution is very success in detecting and responding. Meanwhile, event if the solution has these features, there are too many policies, rules and exceptions for all. I am sure all IT Professionals will be afraid of this replacement since these policies. Until overcoming these problems, it seems better to use EDR beside AV solution. For now, it is also important to use a vendor that have enough working experience with commonly used AVs.
As spoken in all security events in last decade, the attacker’s purposes and methods have changed greatly and become more complex. As if this is not enough, increase in the number of the mobile devices used in the organizations and moving some (or most of the) services to cloud made endpoints’ protection more difficult. With the expanded cloud usage and development of the mobile technologies, more users are coming less to the Office. This situation makes management, monitoring and protecting more difficult for the endpoints.
As said at the beginning, with the new advanced techniques of the attackers (like advanced malwares, fileless malwares or exploits), it is very difficult to protect endpoints with only traditional endpoint security solutions. Neverthless, as the subject of another discussion, this does not mean that signature based antivirus, host IPS, host firewall or other conventional endpoint security solutions meaningless anymore. Meanwhile, while talking about traditional security solutions, we have to touch what endpoint is. Because, attack surface increased against today’s organizations, since IoT and OT are parts of the endpoints’ of them. Now, we need to expand endpoint security solutions as covering mobile phones, POS, wearable devices, sensors, cameras, HVAC, and cars, since they can access both to internet – even if confined with the cloud – and organization’s network, wherever they are and whenever they want. According to this traditional endpoint security solutions, EDR solutions have malicious activity detection, containment of the endpoint, investigation of the incident and remediation capabilities fort he endpoint. With this capabilities, they can reduce the impact of an incident in the organization and provides intelligence for responding faster. EDR systems use an agent on each endpoint system. EDR vendors feed these agents with their intelligence services, global customer data, firewalls, network and/or e-mail based APT devices, etc. With these intelligence data, the agent provides deep and real-time monitoring on the endpoint, discovery and response. An EDR system must use at least a few monitoring methods such; IOC Detection, means that the agent is comparing the system changes with its Indicator of Compromises. This IOCs can be feed from other devices of the vendor in the same network, or global customers and intelligence services. Anolmaly Detection, means checking the system for anormal states. Behavior Detection, means checking the system for bad or malicious behaviours. Machine Learning and AI, means that the solution has the ability to determine the malicious activities without being explicitly programmed. For an effective tussle against threats, time is the most important thing. Your EDR solution should help you detect, investigate and response as quickly as possible. For doing this, first, your EDR should detect the threat as soon as possible. Right here, the power of the tools mentioned above shows the importance of their capabilities. Power of the intelligence services of your EDR solution’s vendor, shows the power of the IOCs. A vendor should feed customers’ EDR with fast and effective intelligences. Also, as community data, vendors can feed their customers with other customers’ known bad data. This means, bigger community helps you better. Also, integration with other security tools in the network is a key point. If an EDR solution can be fed with the other network tools, endpoints can be ready for the threats seen elsewhere, like in network traffic or in an e-mail. When we think that the malicious software reaches to the endpoint via e-mail or network channel, this feature becomes very important. With the integration and the advanced search capabilities in the EDR solution, a threat that seen in network anywhere can be catch quickly in the endpoint. From here, we also must see that an EDR solution must include an advanced search feature, searching the endpoint by many different options. These advanced searching options helps admin searching his clients against possible threats. An EDR solution must provides clear and meaningful explanations about the threats. Only determining a threat is not enough for admins. The solutions must help them responding to these threats. For responding quickly and correctly, admins must understand the content of the threat. Also, containment is an important feature fort he EDR solutions, a time-saving feature for the admins, while they are working on the threat. An endpoint has a malicious content should be contained during the analysis, so other endpoints prevented against spreading of this malicious content.
Also, for investigation, EDR solution must provide a full state output of the endpoint, for the timezone that the malicious thing happen. A full or specific memory dump information, states of the services, etc. An automatic creation of these information is critical during the investigation processes. From the experiences, I know that the endpoint is the most boring and difficult part of the security. Distribution problems, slowing machines after distributing, user complaints, etc. Most of the security admins don’t like to deal with the endpoint security. But as shown in most studies, thousands of threats are produced every day. Just protecting the network and a-mail channel is not enough for these new threats. We have to give the necessary importance to the endpoints. So, with all these problems of endpoints, choosing the right vendor becomes the most important thing. An endpoint security solution must not obstruct end users’ business. Even for security, business must go on. At this point, vendor’s experience is very important to choose. Also, getting a quick answer during a problem must be evaluated, during the selection process of the EDR solution.