Introduction

In the ever-evolving landscape of cybersecurity, keeping up with the latest technologies and solutions is crucial for protecting an organization’s digital assets. Among these technologies, detection and response solutions have become increasingly significant. Terms like EDR, XDR, NDR, and MDR are frequently used, but what do they actually mean, and how do they differ? This comprehensive guide will provide a detailed overview of these detection and response (DR) solutions, compare their features, and help you understand when to use each one.

Detailed Overview of Detection and Response Solutions

EDR (Endpoint Detection and Response)

Endpoint Detection and Response (EDR) focuses on detecting and investigating suspicious activities and threats on endpoints, such as desktops, laptops, and mobile devices. EDR solutions provide continuous monitoring, threat detection, and automated response capabilities. Essentially, EDR acts as a “Next-gen antivirus” because legacy antivirus (AV) solutions can be bypassed too easily. Key features include:

• Real-time monitoring of endpoint activities.

• Threat detection using behavioral analysis and machine learning.

• Incident response automation.

• Forensic capabilities for deep-dive investigations.

• Integration with other security tools for a holistic defense approach.

XDR (Extended Detection and Response)

Extended Detection and Response (XDR) is an evolution of EDR that integrates multiple security products into a cohesive system, providing broader threat detection and response capabilities across various environments. XDR solutions unify data from endpoints, networks, servers, and other sources. Key features include:

• Unified threat detection and response across multiple security layers.

• Enhanced visibility and correlation of security events.

• Centralized management and automated workflows.

• Improved threat detection with advanced analytics and machine learning.

• Seamless integration with existing security infrastructure.

NDR (Network Detection and Response)

Network Detection and Response (NDR) solutions focus on monitoring and analyzing network traffic to detect and respond to threats. NDR provides visibility into network activities, helping identify malicious behavior and potential breaches. Key features include:

• Continuous monitoring of network traffic.

• Detection of anomalies and suspicious activities.

• Real-time threat intelligence and analysis.

• Automated incident response and containment.

• Integration with other security tools for comprehensive protection.

MDR (Managed Detection and Response)

Managed Detection and Response (MDR) services extend beyond traditional DR solutions by adding a layer of expert management and monitoring. MDR providers offer 24/7 threat monitoring, detection, and response, leveraging a combination of technology, processes, and human expertise. MDR makes use of EDR, XDR, NDR technologies, and more. Additionally, MDR looks at multiple sources such as endpoints, networks, cloud environments, and other IT infrastructure. Another emerging term for MDR that reflects this multi-source capability is MXDR (Managed Extended Detection and Response). Key features include:

• Proactive threat hunting and monitoring.

• Expert analysis and incident response.

• Regular reporting and compliance assistance.

• Rapid threat containment and remediation.

• Access to a team of cybersecurity experts.

• Monitoring of cloud services and infrastructure.

Comparing EDR, XDR, NDR, and MDR

While all these solutions aim to enhance an organization’s security posture, they differ in scope, capabilities, and application. Here’s a detailed comparison:

• Scope:

  • EDR: Focuses on endpoints.

  • XDR: Integrates data from multiple security layers (endpoints, network, cloud, etc.).

  • NDR: Focuses on network traffic.

  • MDR (or MXDR): Managed service covering endpoints, networks, cloud services, and more.

• Capabilities:

  • EDR: Real-time endpoint monitoring, threat detection, automated response.

  • XDR: Unified threat detection, centralized management, advanced analytics.

  • NDR: Network traffic analysis, anomaly detection, real-time response.

  • MDR (or MXDR): Proactive threat hunting, expert analysis, managed response.

• Use of Resources:

  • EDR: Requires in-house security team.

  • XDR: Requires integration and management of multiple security tools.

  • NDR: Focuses on network security, often used alongside other DR solutions.

  • MDR (or MXDR): Managed by external experts.

When to Use Each Solution

Choosing the right detection and response solution depends on your organization’s specific needs, resources, and security posture. Here are some scenarios and use cases:

• EDR: Ideal for organizations with a dedicated security team that needs robust endpoint protection and response capabilities.

• XDR: Best for organizations seeking a unified security approach, integrating multiple data sources for comprehensive threat detection and response.

• NDR: Perfect for organizations prioritizing network security, needing detailed traffic analysis and rapid incident response.

• MDR (or MXDR): Suitable for organizations with limited in-house expertise or resources, benefiting from managed services and expert support.

SOC vs. MDR: Key Differences

While both a Security Operations Center (SOC) and Managed Detection and Response (MDR) are crucial for an organization’s cybersecurity, they differ significantly in their structure, scope, and functionality:

Security Operations Center (SOC)

• Definition: A centralized in-house function that continuously monitors and improves an organization’s security posture.

• Features: 24/7 monitoring, in-house team, incident response, threat intelligence, customizable.

• Pros: Complete control, tailored to organizational needs, direct communication.

• Cons: Resource-intensive, requires expertise, scalability challenges.

Managed Detection and Response (MDR)

• Definition: An outsourced service providing threat detection, response, and monitoring.

• Features: 24/7 threat monitoring, proactive threat hunting, expert analysis, incident response, regular reporting.

• Pros: Cost-effective, access to expertise, quick deployment, scalable.

• Cons: Less control, dependency on vendor, potential communication challenges.

Differences Between MDR and EDR, XDR, NDR

In EDR, XDR, or NDR solutions, detection and response are primarily automated with limited or no access to experts, no customization of response processes, no integration of custom sources, and no dedicated incident response team. Conversely, MDR (or MXDR) provides human expertise, tailored response processes, integration of custom data sources, and a dedicated incident response team.

Choosing Between SOC and MDR

The decision between an in-house SOC or MDR services depends on factors like budget, resources, control needs, scalability, and complexity. Organizations with limited budgets or expertise may benefit from MDR, while those needing full control and customization might prefer an in-house SOC.

Conclusion

Understanding the differences between EDR, XDR, NDR, MDR, and the distinction between SOC and MDR is crucial for making informed decisions about your organization’s cybersecurity strategy. Each solution offers unique benefits and capabilities, tailored to specific security needs. By selecting the right detection and response solution, you can enhance your organization’s ability to detect, respond to, and mitigate cyber threats effectively.

Introduction — The Case for Automated Response

If your SIEM only sends alerts, you're leaving value—and security—on the table.

In every incident I’ve investigated, there’s a window between detection and response that determines the scale of impact. In some cases, this delay is a matter of hours, sometimes minutes. But in fast-moving attacks like ransomware deployment, credential stuffing, or lateral movement, seconds might be all it takes to go from contained to compromised.

That’s where automated response comes in.

Rather than waiting for someone to read a Slack message or open their inbox, a well-placed automation can block an attacker IP, disable a vulnerable user, or trigger a workflow instantly. When configured right, automation gives defenders their time back—and gives attackers one chance less.

In this post, I’ll show you how to implement effective, low-risk automated actions in your SIEM. I’ll walk through examples like blocking brute force source IPs, sending adaptive notifications, and integrating webhooks. You don’t need a full SOAR platform to take meaningful action. You just need a reliable alert, a clear goal, and good guardrails.

Let’s explore what’s possible when your alerts don’t just end in a ticket—they end in action.

What You Can Automate (and Why It Helps)

Automated response in your SIEM doesn’t have to mean deploying a full SOAR platform. Even with basic webhook support, you can trigger meaningful actions when alerts match the right conditions. Here's a breakdown of the most common types of automated actions—each with practical benefits and varying levels of risk.

1. Send a Notification (Email, Slack, Teams)

🟢 Low-friction, low-risk

This is the simplest form of automation.

Your SIEM detects something unusual—like a user logging in from a suspicious IP—and sends a message to your Slack channel or a dedicated security inbox.

This doesn't stop anything yet, but it dramatically shortens response time by reaching humans faster (and in the right place).

👉 Example:

• Alert: Office login from TOR node

• Action: Send message to #sec-team on Slack with a link to verify user

2. Trigger a Webhook or API Call

🟡 Medium complexity, highly flexible

If your SIEM supports webhooks or external HTTP calls, you can trigger actions beyond notifications. For instance, you could call a small API you wrote that places an IP in your firewall’s blocklist, or notify a central webhook handler in your detection pipeline.

👉 Example:

• Alert: Multiple failed SSH attempts from a single IP

• Action: Send that IP to an internal API that appends it to iptables ban list via fail2ban

(This is the type I use most, and I’ll walk through it later in the blog.)

3. Execute a Script or Runbook

🟠 Higher risk, more automation

Some systems can run a script directly in response to an alert—especially if you’re using a SIEM combined with an EDR or agent on the machine. This could restart a process, disable a user account, or isolate a host.

While powerful, scripts need rigorous testing. Misfires can create outages.

👉 Example:

• Alert: PsExec lateral movement detected

• Action: Run PowerShell to isolate the machine from the domain

💡 Tip: These are best combined with tagging or human-in-the-loop review at first.

4. Push to a Ticketing System or IR Workflow Tool

🟢 Low-to-medium risk, includes humans

Another useful automation is pushing summarized alert data into your incident response or case management platform (like Jira, TheHive, PagerDuty, etc.). This ensures follow-up, adds context, and lets responders add manual judgment without missing the event.

👉 Example:

• Alert: Phishing link clicked

• Action: Open case in TheHive with log URLs, domain observed, user’s IP

Real-World Use Cases (That Aren’t Too Risky)

Getting started with SIEM automation doesn’t mean plugging your alert pipeline straight into production-blocking scripts. That’s a great way to lock yourself out of an important service—or worse, cause a false positive outage. I like to start with low-friction, high-confidence alerts with clear outcomes.

Here are some use cases I’ve implemented or advised others to adopt first:

🚫 Block Brute-Force IPs Automatically

Brute force attempts are a perfect starter for automation because they’re noisy, repetitive, and usually easy to fingerprint.

🧠 Trigger:

An alert that detects 10+ failed SSH or Windows logon attempts from the same IP within 5 minutes.

⚙️ Action:

Send a webhook to a lightweight API receiving bad IPs, which in turn updates your firewall, fail2ban, or a zero-trust ACL.

✅ Why It Works:

• You can whitelist trusted IP ranges up front

• Blocking via firewall doesn’t affect internal systems

• These attacks are usually automated and follow patterns

🌍 GeoIP + Impossible Travel Notifications

User logs in from Toronto. Three minutes later, they try accessing a sensitive application from Singapore. Unless they have a teleportation pod, something's wrong.

🧠 Trigger:

Alert when two logins are detected from geographically distant locations within an impossible timeframe.

⚙️ Action:

• Automatically send a Slack message or email to the affected user’s manager

• Optionally open a ticket in the incident response system

✅ Why It Works:

• No impact, just visibility

• Helps build context if ongoing investigation is needed

🧪 Tagging or Labeling High-Risk Hosts

Not all automations need to stop attackers directly. You can enrich telemetry by tagging things as risky the moment they're detected.

🧠 Trigger:

Host exhibits lateral movement behavior (e.g., PsExec or WMIExec detection via Sysmon)

Or: Endpoint shows signs of crypto-ransomware behavior

⚙️ Action:

Attach a tag to the host in your log system or asset tracker

Or: write an enrichment flag in Sentinel/Humio to surface in dashboards

✅ Why It Works:

• Non-destructive

• Makes hunting and triage faster

• Builds starting points for weekly reviews or ML training

These ideas are deliberately safe to start with. As you implement these and trust their accuracy, you can progressively increase risk tolerance and expand to more disruptive actions (like user disabling or host isolation).

Hands-On Example: Blocking an IP from a Humio Alert

Let’s walk through a real-world example: a brute force detection alert that automatically blocks the source IP by sending it to a small script via webhook.

This approach works with any SIEM that supports HTTP-based alert actions—Humio makes it simple, but you could do the same with Logz.io, Graylog, or even a homegrown setup.

🎯 The Goal

• Detect failed login attempts coming from a single IP more than 10 times in 5 minutes.

• When that happens, send the IP to a webhook endpoint.

• The webhook triggers a script to block the IP on the firewall.

• The action is logged, monitored, and reversible.

Low risk, high reward.

🗂 Step 1: Create the Detection Query in Humio

In Humio or your SIEM, build the alert like this.

Query Example (Sysmon/Winlogbeat logs):

event.code=4625 
| groupby([beat_agent.hostname, winlog.event_data.IpAddress], function=count()) 
| where([count > 10]) 

This detects 10+ failed login attempts from the same source IP on the same host.

🔍 Tips:

• Avoid internal IPs or your own jump-boxes by filtering with where(not winlog.event_data.IpAddress in (your allowlist))

• Tune down to 5 attempts in testing, tune up to 20+ in prod

⚠️ Step 2: Configure the Alert Action

In Humio:

• Go to Alerts

• Choose your query, click “Create Alert”

• Set threshold = 1 (fires whenever there’s a matching group)

• Throttle: >5 mins per host+IP tuple

Action Setup (Webhook):

• Type: “HTTP POST”

• URL: https://your-automation-host/api/block-ip

• Payload:

{
  "hostname": "{{beat_agent.hostname}}",
  "ip": "{{winlog.event_data.IpAddress}}"
}

✔️ Set content-type as application/json

✔️ Secure it with an API key in the header, or only allow network-restricted source IPs

🛠 Step 3: Write the Webhook Script

Here’s a very simple example in Python (Flask-based API):

from flask import Flask, request, jsonify
import subprocess
import logging

app = Flask(__name__)
logging.basicConfig(filename='ipblocks.log', level=logging.INFO)

@app.route('/api/block-ip', methods=['POST'])
def block_ip():
    data = request.json
    ip = data.get('ip')

    # Validate the IP (simple check)
    # This is not enough to safeguards against RCE.
    # TODO: add more checks!
    # This is only an example!
    if not ip or '..' in ip or '/' in ip or ';' in ip or '&' in ip or '|' in ip:
        return jsonify({"error": "invalid IP"}), 400

    # Block the IP using iptables (Linux)
    # One would imagine a CTF challenge here!
    cmd = f"iptables -A INPUT -s {ip} -j DROP"
    subprocess.run(cmd, shell=True)

    logging.info(f"Blocked IP {ip} via automation")
    return jsonify({"status": "blocked", "ip": ip})

💡 Save and run this on a secured automation server. Rotate IPs out after X days or add removal logic.

Alternatives:

• Use UFW, fail2ban custom chains, or cloud firewall/blocklist APIs (e.g., AWS, Cloudflare)

• If you don’t want actual blocking: just log or simulate the response action!

🔎 Step 4: Monitor & Tune

Now that you’ve automated your first response, be sure to:

• Log every action (separate audit file or SIEM event)

• Monitor alerts for false positives—adjust thresholds

• Use alert “Preview” and dry runs to test

• Sanitize inputs and control webhook access

Optional additions:

• Store blocked IPs in SQLite or Redis to allow removal later

• Add Slack/Email alerts from the script for visibility

🔐 Bonus: Add a Safety Switch

Build a toggle in the script (“MAINTENANCE_MODE=True”) that allows you to disable enforcement instantly. Or only allow from specific alert sources (e.g., trusted Humio instance UUIDs).

🎉 Boom — you’ve now built an early warning system that defends itself.

Up next, we’ll talk about how to scale this approach and avoid common mistakes.

Subscribe now

Best Practices & Guardrails

Before you get too deep into automating blockade systems and forensic scripts, let’s cover the most important part of any automation effort: treating it like production code.

Automated actions affect your systems—sometimes permanently. Without proper guardrails, a bug in detection or a misfiring rule could put your security team in the spotlight for all the wrong reasons. Here's how to do it right.

✅ 1. Start with Notifications, Then Actions

The temptation to go straight from alert to action is real—but don’t skip the dry run phase.

Always begin by triggering a notification or logging-only action, and monitor the alert fidelity over time. Only after verifying that your false positive rate is low should you consider moving to blocking or remediation automation.

📝 Pro tip:

“Log + Notify + Review” for a week before enabling the final blocking logic.

🛡 2. Whitelist Known Good Behavior

Always protect critical systems, known testers, and internal IPs from being impacted by automated actions unless absolutely required. You’ll want to include a whitelist—either inline or backend-controlled—which the automation must check before enforcement.

Examples:

• Don’t block your VPN endpoint

• Don’t isolate your own monitoring agent

• Don’t trigger changes on development or staging logs

🧪 3. Log Everything the Automation Does

If your automation blocks an IP, disables a user, or restarts a service, you need a clear audit trail of:

• What triggered the action

• When it ran

• What was actually done

• Who or what performed it

• Whether it succeeded or failed

📁 Tip: Log all of this to your SIEM itself.

Bonus: Create a dashboard of recent automation actions to catch errors early.

🔁 4. Add Rate Limiting and Throttling

Just because 50 alerts fire at once doesn’t mean you want 50 API calls and actions firing simultaneously.

Add throttling—or use your SIEM’s built-in alert throttle—to avoid alert storms. This ensures you don’t overwhelm your firewall with too many rule changes or bounce through rate limits on an external system.

📏 Example:

Only execute the action once per unique IP per hour, or once per system per 10 minutes.

🔂 5. Make It Reversible (and Pauseable)

You WILL make a mistake. Your automation might react to a misparsed event or a data ingestion bug.

Plan for rollback. Here’s how much:

• Can you remove a blocked IP with a single call?

• Is there a safe list to re-allow access?

• Can you “pause” automation with one toggle if you’re under pressure?

Most mistakes in security are made in fear. Don’t let your automation increase panic—build kill switches and clear recovery steps.

Automating threat response means trusting your SIEM’s alerts with real power. These best practices are like a flight checklist: they ensure it flies where and when you want it to—and lands safely when it’s done.

Up next, let’s look at the typical automation pitfalls I see in the wild—and how you can avoid them.

Common Pitfalls to Avoid

When I talk to teams about SIEM automation, they’re usually excited. And with good reason—automation turns alerts into actions and allows teams to scale. But I’ve also seen well-meaning automation wreak havoc because a few basic rules were overlooked.

Here are the most common mistakes I see when teams launch automated responses too early—or without thinking through the consequences.

🔁 Acting on Noisy Alerts

This is the classic mistake: building automations on top of alerts that still generate false positives. If your brute force alert is misfiring, feeding it into an auto-blocking script will just create chaos.

You might end up:

• Blocking your own security scanner

• Disabling perfectly legitimate logins

• Filling your firewall ruleset unnecessarily

🛑 Fix it first: Make sure your detection logic is solid, tuned, and meaningful before giving it power.

💡 Pro tip: Review fired alerts over 7–14 days before moving to response.

🌪 Forgetting Alert Throttling

Without rate limiting, a flood of log-ins from one device, a SIEM parsing bug, or log ingestion issue could trigger hundreds of simultaneous actions.

I once saw a mis-parsed Linux event trigger 300+ alerts in under 60 seconds—which then triggered an overload in the iptables automation service.

✔ Add guardrails:

• Alert throttling (native in many SIEMs)

• IP block counters (rate-limits per source)

• Timeout queues on your action scripts

👁️ No Visibility on What’s Happening

If you're not logging the actions taken—or you’re only logging success—you’re working blind.

Imagine:

• You blocked 42 IPs… but which ones?

• A script failed to run… was it retried?

• Someone paused your webhook host… and now no action is running

🌟 Always log:

• What triggered the action (alert details)

• What was attempted (API call, command)

• The result (success/failure)

Put this in a shared log store, or even feed “meta events” to the SIEM itself.

🔒 No Authentication or Input Validation

If your webhook is listening for JSON from anywhere—make sure it's authentic.

Imagine an attacker figures out your webhook URL… and submits their own IP with your same automation call. Or worse, they inject shell code as part of a manipulated payload.

Secure your endpoints:

• API keys or HMAC validation

• Restrict source IP to the SIEM only

• Sanitize all fields before running commands

If you're passing variables into a shell script, validate and escape everything.

🧨 Over-Automating Detonation

Some teams go all-in: "If this alert fires, disconnect the user, disable the account, and terminate the EC2 instance."

That’s tempting, but only safe at maturity. Without escalation logic, human review, or an understanding of business impact, automation might react more aggressively than intended.

Instead:

• Automate gently (alert, log, escalate)

• Use tagging, enrichment, and visibility automation

• Add human-in-the-loop steps before destructive actions

You can scale trust with automation—but only if everyone trusts it over time.

TL;DR — Don’t Put a Bazooka on an Alert Until it Knows What It’s Aiming At

Automation is powerful—but only when it obeys the rules of reliability, auditability, and control. Treat your response logic like a developer treats production code: observability, rollback, and safety nets should always be included.

Where to Go From Here

You’ve built one automation, logged its behavior, and avoided tripping your own firewalls—great work. Now what?

One of the strengths of building lean, scriptable SIEM automations is that you can grow them gradually. The goal isn’t to automate everything. It’s to automate the right things consistently and recoverably.

Here’s how to take your setup further:

🧰 1. Expand Your Webhook Handler

Don’t let your blocking script do just one thing. You can support more actions using conditional logic:

• Block an IP

• Put a tag in a shared asset tracker

• Write a note into a shared Google Sheet or case system

• Notify an MDR or response team externally

• Populate a blocklist file or feed

Small automations like this scale well and create glue between tools.

🛠 Tip: Build modularly — write clean, multi-source-capable input handlers and you won’t need to rebuild later.

🎩 2. Integrate with Your Ticketing or Workflow Tool

If you’re not ready to block things automatically, pipe your alert context into your response platform.

Examples:

• Create a Jira ticket for phishing-link clicks

• Trigger a case in TheHive for suspicious process chains

• Send markdown-formatted incident summaries to Mattermost

This acts as “automated assurance”—you won’t forget the incident, and responders can add human judgment if needed.

🔄 3. Enrich Alerts — Not Just Respond

Your SIEM should grow more informed over time.

When an alert fires, automation doesn’t have to only take action. It can:

• Append GeoIP context based on source IP

• Mark an asset’s risk score in the SIEM

• Add domain reputation or VirusTotal score

• Link to previous similar alerts

Automation = context at machine speed.

🧠 4. Create a Lightweight “SOAR Without SOAR”

If you’re not buying a full commercial SOAR (Security Orchestration, Automation and Response) platform, you’re not alone.

With the right stack, you can build your own SOAR-lite with:

• Your SIEM alerting engine (Humio, Elasticsearch, etc.)

• A webhook layer / action handler (Flask, Lambda, etc.)

• Basic storage (SQLite, Google Sheets, JSON-logs)

• Dashboards or markdown playbooks

This gives you enough control and extensibility for most use cases—especially in a small or mid-size team.

📝My blog used this model before the move to Substack. Works well. Cost? $0.

🔭 5. Experiment, Measure, and Refine

Great SIEM automations are never built in one pass. I recommend:

• Logging every action, even if it “does nothing”

• Holding monthly reviews of automation performance (e.g. top 10 triggered rules)

• Keeping a chart of false positives vs. enforcement success

• Versioning your detection + automation rules together

This turns security automation into a feedback loop—not just a trigger.

Conclusion

Security automation isn’t reserved for large enterprises or teams with a SOAR budget. You don’t need to over-invest, overbuild, or over-trust untested systems.

With just a few hours and the right SIEM alert, you can build useful automations that:

• Respond faster to obvious threats (like brute-force attempts)

• Help defenders focus on high-value work

• Reduce fatigue from repetitive incidents

Whether you're defending your home lab, managing detection pipelines for clients, or running SecOps in a growing organization, your SIEM can do more than notify—it can act.

Start with one use case. Get it right. Monitor its behavior. And when you’re confident, scale it up to your next alert. You’ll be amazed at how quickly “just an alert” becomes something that actually helps you sleep at night.

If you’d like to explore more, I’ve also written about detecting PsExec lateral movement, configuring alert thresholds, or tuning SIEM correlation rules to reduce false positives.

☑ Build one automation. Review. Repeat.

🎯 Want more hands-on detection walkthroughs and security strategies?

Subscribe now

to get the next blog posts as I publish them!

—

💬 Did you try an automation and learn something from it? Share it in the comments! I’d love to hear it.

Leave a comment

—

📬 Need help building detection + action pipelines for your team? I run detection engineering for MDR platforms and help teams implement lightweight but effective automation. Feel free to reach out.

🧡 Donation

If you like my blog and my posts, please consider donating!

Imagine this: each minute of reading a post takes about 1–2 hours to create!

Donate 🧡

Benefits of Elasticsearch

Customers and the open-source community enjoyed a variety of services offered by Elastic through its products (like Elasticsearch). Elasticsearch is a distributed search engine providing a number of features that make data processing more efficient and effective while maintaining its ability of scalability, speed and resilience. Elasticsearch through its partner product Kibana, Beats and Logstash provide a simplified way of data visualization, ingest and reporting. The other very reliable feature is its distributable nature which allows it to handle a huge size of data without compromising performance. It provides a real-time search platform which provide the shortest time possible when a user makes a search.

The open-source community contributes much on this product through various ways like providing codes and documentation, bugs report, write of tutorial, blogs etc. Elasticsearch was providing the code under the Apache v2 license. All this makes everyone feel the meaning of "open" in "open-source". A report by macrotrends shows that the company net worth from 2018-2020 is estimated as $14.33B. This is an indication of how big the customer base is, and the revenue generated help contribute to the open-source community.

The context

On February 27’th 2018, the Elastic company CEO, Shay Banon, delivered a message on their website that mealy read as an announcement for the release of the new repository for their products on the version 6.3. A new folder in the code, named X-Pack, was created and all the proprietary product code moved to that folder under the elastic license which many customer and open-source code community believed that was a way of facilitating greater collaboration.

He affirmed that their products had to be open source hence providing the opportunity to reach so many people who can provide their contribution through various ways and even enable a push for what was seen, at the time, as a pipe dream. He even committed himself that as a company they will ever maintain to be open as a way of building their business.

However, on January 14'th 2021, Shay Banon announced that they were moving away from the truly open-source license (Apache v2) to the newer Server Side Public License (SSPL). The open-source community was shocked to realize that in order to use the Elastic products, you now had to agree to the terms and conditions of the SSPL or the elastic license. The former as new as it looks deprive the organizations of their intellectual property rights when observed from the spirit of "open source" in the business environment:

By using an SSPL project in your code, you are agreeing that if you provide an online service using that code then you will release not only that code but also the code for every supporting piece of software, all under the SSPL. It’s not a stretch to interpret the wording of the license as requiring users of the SSPL’d software therefore to release the code for everything straight down to the bare metal.

https://anonymoushash.vmbrasseur.com/2021/01/14/elasticsearch-and-kibana-are-now-business-risks

From Elastic's website:

Why this move to the new SSPL Elasticsearch license

The elastic company felt that “enough is enough” and that they should protect its product from cloud service providers whose intention was to capture the value of the same product while contributing nothing to the open-source community, as expressed by Eliot Horowitz, CTO and co-founder of MongoDB. Together with elastic’s CEO, they felt that companies like Amazon that are providing this product as a Software as a Service, without the knowledge or the consent of the Elastic company while misleading the community that they are, is "not ok". They tried and didn't succeed to resolve this issue through courts.

A new hope

In response, AWS announced that it will create and maintain an Apache v2 licensed fork of Elastisearch and Kibana. They further expressed that the choice to fork a project was the right path forward when the community needs are diverging. To emphasize it, they stated that they are in the game for a long haul and ready to work in a way that will create a healthy and sustainable open-source practice.

Logz.io joined the fray and expressed their concern as to why they think the change of license from Elastic, at the expense of the open-source community, is “not OK”. In a retaliatory tone, they gave their objective to have projects driven by multiple organizations rather than a single commercial entity and plans to collaborate with such organizations (like Amazon) that believe that those products, Elasticsearch and Kibana, need to stay open source.

References

• https://anonymoushash.vmbrasseur.com/2021/01/14/elasticsearch-and-kibana-are-now-business-risks

• https://www.elastic.co/blog/doubling-down-on-open

• https://medium.com/@AIMDekTech/what-is-elasticsearch-why-elasticsearch-advantages-of-elasticsearch-47b81b549f4d

• https://www.elastic.co/blog/why-license-change-AWS

• https://aws.amazon.com/blogs/opensource/stepping-up-for-a-truly-open-source-elasticsearch/

• https://logz.io/blog/open-source-elasticsearch-doubling-down/

• https://mjg59.dreamwidth.org/51230.html

2021-04-18 Update

AWS announced they will be actively developing OpenSearch and OpenSearch Dashboard to replace Elasticsearch and Kibana, respectively. Both these new products will be fully opensource under the Apache 2.0 License.
Source: https://aws.amazon.com/blogs/opensource/introducing-opensearch/

🧡Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Donate 🧡

In this post I’ll explain the CIS controls and how they can be used to start securing an enterprise.

When you’re in a medium or big environment and the maturity level of the InfoSec is around zer0, it’s hard to decide where to start in securing your environment. Do we need to update the systems, software and hardware first? Do we need to focus on awareness with the employees? The list is long; it can seem like there is no possible way to reduce the global risk.

Funny note: since I started writing this post, the controls got an update both in the number (previously 20, now 18) and in how they are organized! You can find an overview of what changed here: https://www.sans.org/blog/cis-controls-v8/

Context

There are multiple framework that can be used to help decide what to do and in what order. The first that comes to mind is the NIST Cybersecurity Framework. If you’re already an expert in InfoSec, you probably know about this framework and understand how it’s an excellent framework. But as an expert or as a newly appointed CISO, this framework is not ideal when you’re starting from near zer0. That’s because it’s complete, too complete; and complex. There is no prioritization and it addresses all that needs to be addressed at some point. That’s why the 18 CIS controls are ideal to start.

These 18 controls are ordered by logical priority. For example, there is no need to choose the best Antivirus if your systems are not updated. But you can’t update your systems if you don’t know what needs to be updated, hence why an inventory is crucial.

The 18 CIS Controls

You can find the controls here: https://www.cisecurity.org/controls/cis-controls-list/

You can also download a PDF or Excel version here:
Latest version: https://learn.cisecurity.org/control-download
V7.1: https://learn.cisecurity.org/control-download-v7.1

Here you can find a detailed explanation of all the controls: https://controls-assessment-specification.readthedocs.io/en/stable/

The controls were grouped in a logical manner in the V7, but this grouping has been removed in the V8. For an easier reading of my post, I’ll keep the original grouping and reorganize based on the mapping changes, however, this shouldn’t be understood as a categorization of what you should do. To categorize, they developed “Implementation Groups” designed to help organization of different size focus on the right safeguards. Here are the original groups: basic controls, foundational controls and organizational controls.

Here is why they changed the way they organized the controls:

Historically, the CIS Controls were ordered in sequence to focus an enterprise’s cybersecurity activities, with a subset of the first six CIS Controls referred to as “cyber hygiene.” However, this proved to be too simplistic. Enterprises, especially small ones, could struggle with some of the early Safeguards and never get around to implementing later CIS Controls (for example, having a backup strategy to help recover from ransomware). As a result, starting with Version 7.1, we created CIS Controls Implementation Groups (IGs) as our recommended new guidance to prioritize implementation.

CIS_Controls_v8_Guide.pdf

Basic Controls

Here is a brief explanation of each basic controls:

• Inventory and Control of Enterprise Assets
  – It’s important to always have an up to date list of all the hardware in our possession. That includes the servers, workstations, switches, etc. In our cloud world, I would include all the virtual hardware in the cloud also, like load balancers, VMs, SaaS, FaaS, etc.
  Once you have this list in your possession, it’s simply a matter of prioritization. You can make sure no rogue devices are allowed in your environment. They can be updated as needed. You can remove unneeded devices, etc.
  Another great advantage of this list is when you are responding to an incident: it will allow you to better understand what happened, where and how. Once the incident is resolved, you’ll know what to restore from previously well designed backup plans.
  There are many other advantages to lists like this, I’ll let your imagination run free!

• Inventory and Control of Software Assets
  – Like the hardware assets, it’s important to know and track all the software used in our environment. That will allow us to continuously validate for outdated software or if some of them have known vulnerabilities. We can also use this inventory to detect suspicious behavior. For example, if a tool like PsExec is installed on a HR computer. Another use would be to detect unlicensed software or software licensed with bad licenses that could be a threat to the enterprise (like a license requiring you to opensource all the tools developed with it’s software, like the Elastic License or the GNU GPL license).

• Data Protection
  – The wording “Data Protection” is voluntarily broad. The official definition of this control is “Develop processes and technical controls to identify, classify, securely handle, retain, and dispose of data“. As you can see, it means nothing and everything at the same time. The idea of this control is that a systematic process should be put in place around data manipulation. We need to analyze how data should be handled when a new data set is added to a software we use of develop, or a new software is adopted. Most company will first acquire new data without thinking too much about it. For example, who asked if and how should Google Analytics be implemented before it was done? But there is an inherent responsibility to all this data handling. This responsibility should be taken seriously and the data should be:
  – Identified, meaning it should be crystal clear what the data will be;
  – Classified, meaning some tags should be added to this data like the type of data and the sensitivity for customers and for the internal business;
  – Securely handled, meaning the process around the data should be clear: encryption at rest, encryption in transit, who has access, is it sanitized, etc.;
  – Retained, meaning the process on how long the data should be kept and what triggers the need for it’s destruction should be clear;
  – Disposed, meaning the process for disposing securely of the data should be clear;

• Secure Configuration of Enterprise Assets and Software
  – When you use a software, most of the time it’s not secure by default but can, and should, be hardened. Of course, this is a never ending process and every time a software received a major upgrade, the process should be revisited. This point should be applied to everything and anything that can be configured; mail server, FTP server, Windows, Linux, Android, iOS, a switch, etc. And the possibility to securely configure a software should be in it’s selection criteria when deciding which software to use to fill a need.

• Account Management
  – There should be a well defined account management process and architecture. For example, if you are using Windows, you should follow the 3-tier architecture, or it’s newer version, for the administrators account: https://docs.microsoft.com/en-us/security/compass/privileged-access-access-model. You should also make use of LAPS for local administrators (as opposed to having always the same user with the same password for all computers in the company). Accounts for services should have a strong password managed in a password vault, each service having it’s own account with it’s own password and permissions.

• Access Control Management
  – “Access Control Management” applies to both physical and technological accesses. It is important to have a clear and well defined process to manage access to users and service accounts, grant them privileges or revoke access. We need to be able to easily grant or remove all access to systems and enterprise assets to users, whether they are new or not an employee anymore. The worst case scenario is forgetting to remove some or all accesses to an employee with bad intentions. But there is also a lost in productivity and/or security risks if it’s hard to grant accesses with the least privileges possible.

• Continuous Vulnerability Management
  – As we are all too aware, new vulnerabilities are found weekly, probably event daily. After they are found, it is a matter of time before they are exploited. Because there is a continuous flow of vulnerability, it is important to evaluate the risks, track and prioritize them. Of course, this point is directly dependent of the first 2 CIS Controls.

• Audit Log Management
  – When an incident is detected, it is often too late to start adding tools that could help understand how the attack occurred. It is not rare to see that the logs have simply been deleted by the attacker. Because of this, it is important to continuously collect the logs in a centralized location, like a SIEM (see my posts on the subject: https://www.tristandostaler.com/category/information-technology/infosec/siem/). Once we collect all these logs, we can start to detect attacks as they occur and stop the attacker in it’s track, instead of waiting for a true incident to occur. These logs can also be useful to understand how an attack happened and what can be done so it doesn’t happen again.

Foundational Controls

Here is a brief explanation of each foundational controls:

• Email Web Browser and Protections
  – Probably the most prevalent initial access tactic, phishing, through email or other means, can have a devastating impact if successful. This is why it is important to have protections in place to detect and block these attacks. There are many layers that can be implemented to protect us. We can start from the human aspect by providing training to detect phishing and other forms of attacks. We can also have an email filter, use DNS servers like Quad9, a modern browser that can detect typical attacks, an antivirus (or it’s new form: an EDR), etc.

• Malware Defenses
  – This 2 words category is one of the most complex technically speaking. The idea of this category is to put in place defenses against malware. One of the first layer to do this is to use an antivirus (or it’s new form: an EDR), but it is nowhere near sufficient. New EDRs are really hard to bypass, and get harder by the day, but they are far from perfect. This is why it is important to have other defenses in place. But configuring these defenses so they detect enough, but not too much, potentially malicious activity is hard. For example, if you want to restrict what executables can run on your computers, you need to know what are the complete list of executables and understand that there will be a lot of demands to ask for new software to be added to the exception list. And what do you do with developers?
  This category is difficult but really important and the strategy should be revised at least yearly.

• Data Recovery
  – When an incident occurs, be it a ransomware, a deleted database table in production because of a bad manipulation or any other cause, it is important to be able to recover the lost data in a timely manner. There are multiple strategies for this, and it is not rare to see multiple strategies being used at the same time for deadly crucial data. A “3–2–1 backup” strategy* of important folders is an example. Or an “Active/Active” architecture, combined with a constant (ex: every hour) duplication of the data to a cloud storage solution is another example where multiple strategies are combined.
  * I found this nice article talking about different backup strategies. If backup strategies is something useful for you, I think you’ll find it interesting.

• Network Infrastructure Management
  – It’s important to keep a list of all the enterprise assets. But it’s equally important to make sure the network they are connected to is secure. This is done, partly, by continuously tracking (and/or scanning) the devices in the network and then making sure they are correctly configured. It happens really often to induce a vulnerability in the network by installing a new device without changing the default vulnerable configurations. These devices can be switches, firewalls, wireless access points, etc.

• Network Monitoring and Defense
  – While it’s important to continuously maintain the network devices, nothing is perfect. This is why it’s important to “Operate processes and tooling to establish and maintain comprehensive network monitoring and defense against security threats across the enterprise’s network infrastructure and user base”. To get more information on this, see my series on SIEMs: https://www.tristandostaler.com/category/information-technology/infosec/siem/

Organizational Controls

Here is a brief explanation of each organizational controls:

• Security Awareness and Skills Training
  – Once you cover the basics, the most probable way in the network is through human. Bad actors will use techniques like Phishing to get a foothold for further attacks. This is why it’s important to have a security awareness program. With this, you can raise the level of your employees awareness so they better recognize these type of attacks. This can reduce the risk to the enterprise. Once you have an awareness program, it’s important and useful to keep statistics on the results so the recidivists can be better trained.

• Service Provider Management
  – You probably of the famous Supply Chain Attack, an old concept that got popularized in 2020 because of the SolarWinds attack. To reduce the risk of getting caught off-guard by these types of attacks, it’s important to evaluate service providers and ensure these providers have proper processes to protect our data.

• Application Software Security
  – If your enterprise develops software of some kind, it’s important to have a process to prevent, detect and remediate issues in these software. Many tools and techniques can be used for this, from code review with a security point of view to having a SAST, DAST, IAST or RASP. If you don’t have security specialist that can do code review, multiple security firms can help you with this.

• Incident Response Management
  – Even if you do everything described in this post, hackers can still get through. This is why it is important to plan before it happens and have a well defined Incident Response process:

1. What would you do if an incident happened?

2. Who would you call for help?

3. Who would be implicated and/or informed?

4. Who is responsible to handle the technical aspects of the incident response?

5. What and where are the response playbooks?

6. Etc.

• Penetration Testing
  – It is always recommended to periodically do a Penetration Test (en français on dit “Test d’intrusion” 😉 ). I have some clients that do them yearly and other monthly or even weekly (targeting different things (internal, websites, external, etc.)). It is important to regularly do this because security is hard and it’s really easy to miss something, even when we do everything right.

My opinion on these controls

Although the order I explain these controls make sense somewhat, these controls shouldn’t be addressed purely in order, but rather in parallel by carefully analyzing our current stance and addressing the point with the bigger gaps first.

As I explained, there are other, more complete framework to work with. But I personally think the CIS controls are an excellent starting point as well as a cheat sheet to keep handy. In this post I mentioned the controls at a high level, but you’ll find that the complete version has a lot more details.

I often work on Incident Responses for clients where they ask me for some resources to help them make sure they can work to secure their environment. I always recommend the CIS Controls because they are useful for technical people and understandable by the management.

The Excel version is especially useful to organize and prioritize the controls as we work on them. Here’s a picture of how it looks:

Feel free to leave your comment down here for any questions or comments.

🧡Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Donate 🧡

You’ve hardened all you could on your servers or computers, you have a strong password with MFA, you update regularly or automatically everything and are wondering what can be done next?

Like you probably know, you can never be 100% protected and there is always a way criminals can find to access private information. For those who tells me they are 100% secure, here is an example:

You have a secure WordPress website at www.mysite.com which has nice plugins to secure it, you maintain it up to date and you have MFA. You then go in a public cafe to make some changes to your website. You enter your WordPress URL in your browser, get the login window, enter your password + your MFA and get access to your website. You then make some changes and everything works as normal. Then a few days later your site gets defaced, or worse. What happened?

What you missed here is that an attacker in the same cafe as you was able to change the IP returned when you did a DNS query for your domain (either through MITM or by changing the DNS Server in the router config). He then redirected you to a new domain: www.rnysite.com. This new domain is owned by the attacker and directs you to a sever that runs Modlishka. This attacking tool essentially proxies all your traffic to your real website www.mysite.com, even validating your password + MFA and making the changes you ask for, but he also makes a copy of all the traffic, taking your cookies and password. Although he doesn’t have your MFA, he now has your cookies. Oh and if that’s not enough, it’s possible now to brute-force the TOTP seed with Hashcat with only 2 TOTP token. (Here is a definition of Brute-force attack)

So now what?

Well, here comes logging, aggregating, correlating and alerting — also known as a Security Information and Event Management, or SIEM.

But what is a SIEM? As Varonis puts it:

Security Information and Event Management (SIEM) is a software solution that aggregates and analyzes activity from many different resources across your entire IT infrastructure.

SIEM collects security data from network devices, servers, domain controllers, and more. SIEM stores, normalizes, aggregates, and applies analytics to that data to discover trends, detect threats, and enable organizations to investigate any alerts.

Source: https://www.varonis.com/blog/what-is-siem/

…

Now that we established what is a SIEM and why it is needed, how to we get started with this?

Well, to get started using a SIEM, you can read my blog posts on the subject! My goal is to make it easy to understand and accessible to anyone with some IT experience.

Next post: SIEM 101 — Introduction

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

In the following days, I’ll write a few blog posts explaining how to easily learn to use a Security Information and Event Management, or SIEM.

Note: In this post I use Logz.io for my examples, but I recently switched to Humio. For more details: https://www.tristandostaler.com/why-i-switched-from-logz-io-to-humio/

But what is a SIEM? As Varonis puts it:

Security Information and Event Management (SIEM) is a software solution that aggregates and analyzes activity from many different resources across your entire IT infrastructure.

SIEM collects security data from network devices, servers, domain controllers, and more. SIEM stores, normalizes, aggregates, and applies analytics to that data to discover trends, detect threats, and enable organizations to investigate any alerts.

Source: https://www.varonis.com/blog/what-is-siem/

To do this, I wrote the post SIEM 101 — Initial setup where I explain how to create an account on Logz.io; they offer a free account where you can send up to 1 GB / day of logs to try their platform. Time needed: around 2 minutes.

Still in the post SIEM 101 — Initial setup, I then explain how to send your logs there, whether you’re on Windows or on Linux. Time needed: around 5–10 minutes.

After that, I’ll explain the basic usage of Logz.io, how to search logs, how to create an alarm, etc. Time needed: around 30 minutes.

When all this is done, the scene will be in place to start writing blogs on detection cases, for example: receive an alert if Windows is brute forced, receive an alert if a failed logon happened on your WordPress server, etc.

I’ll eventually talk about the scripts I developed that uses Logz.io to automatically block bad IPs on my servers.

For the most advanced users, you’ll then be able to discover some useful tools the community provides, like the Sigma rules.

Follow me in my next post: SIEM 101 — Initial setup

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

A few years ago, I was on the defense team when a pentest was ongoing on one of our website. This website, coded in C#, was hosted on Windows and had Windows Defender activated.

During the pentest, the attacker had access to the CMS. He discovered that he could upload his own plugins, giving him Remote Code Execution (RCE) on the server. Happy with this discovery, he started his C&C server (it was PowerShell Empire 2 at the time), crafted a payload and launched the result in a C# wrapped plugin and then… nothing.

As some of you may know, C#, PowerShell and Defender all love to work together. That’s because the .NET framework is now tightly integrated with Windows Defender. When sending the payload, Defender could see and block the attack.

So all the attacker had to do was to modify his payload until it passed all the checks Defender does. After a few hours, he had a complete reverse shell (although restricted to the IIS user).

From the defense point of view, the fact that Defender did his job is perfect. But the fact that we didn’t know Defender blocked a file-less attack and we only learned it when we receive the report should be considered a disaster. It means your AV is useless because an attacker has all the time he needs to craft his attack.

From this point, the next steps where clear, we needed to receive an alert when a virus was detected. And maybe on other suspicious behavior? That’s when I discovered what is a SIEM!

If you wan’t to know more about what is a SIEM and how to use it, you can read my blog posts on the subject starting with: SIEM 101 — Introduction

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Note: In this post I use Logz.io for my examples, but I recently switched to Humio. For more details: https://www.tristandostaler.com/why-i-switched-from-logz-io-to-humio/

This is the second post of the series “SIEM 101″ where I explain the basics of a SIEM, from installation to simple usage. You can see the previous post SIEM 101 — Introduction.

I decided to write the series using Logz.io because they offer an easy to use platform with a free 1GB/day trial with a smooth learning curve. Of course they don’t offer all the features in the free version and some crucial features for a SIEM are missing, but this is perfect to get started with a SIEM! For the Splunk fans, I’ll probably write a blog post about why I don’t recommend it often and why I recommend an Elasticsearch based SIEM most of the time.

First step: open an account

The first thing to do is to open an account on Logz.io. They offer to open an account using your google account, or to register with an email. For this demo I’ll use the google SSO option. So head over to Logz.io and register: 

https://app.logz.io/#/login

Next, click on “Sign in with Google”: 

You’ll get redirected to the Google SSO. Select your existing account, login or create an account. I’ll let you figure this part out.

Then, once back on the Logz.io website, you get a Sign up page. Fill out the details and click “Create Account”:

Note: It is smart to choose a region close to where you live to get better bandwidth when sending logs.

That’s it, your account is ready to receive logs! When you login, you can follow their tutorial if you want to. The next step in this post is pretty much the same thing as their tutorial.

Second step: start sending logs

Now that we have an account, we can finally start sending logs. Make sure you are logged in your account for the next steps. I divided this section in two, one for Windows and one for Linux (I use Ubuntu).

Once logged in, click on “Send Your Data” in the top navigation: 

The next steps are starting to get interesting. We’ll configure the log shipper agents on Windows and Ubuntu.

Windows

On the left navigation, close “Log Shippers” that should be opened, and open “Operating Systems”:

Select “Windows” under the “Operating Systems” menu. Your page will refresh to the URL 

https://app.logz.io/#/dashboard/data-sources/Windows

They propose 2 options; we are interested in the option 1: Winlogbeat.

As they explain in the page, you first need to install Winlogbeat. So that’s what we’ll do.

Head over to https://www.elastic.co/downloads/beats/winlogbeat-oss and download the right MSI version for your computer (32 bits or 64 bits, if unsure use the 32 bits). For me, that’s the 64 bits MSI:

Note: here, my link is slightly different than the one they propose on their website. This is because I recommend using the OpenSource version (oss).

When the download is finished, click the downloaded file, a window will open.

Check the “I accept the terms in the License Agreement” checkbox and click “Install”:

When prompted, accept to run as Administrator (UAC). When the installation is done, click Finish (we leave the checkbox “Open winlogbeat data directory in Windows Explorer” checked.

In the Windows Exporer window that opened at the location “C:\ProgramData\Elastic\Beats\winlogbeat”, rename the file “winlogbeat.example.yml” to “winlogbeat.yml”.

Open your favorite text editor (it can be Notepad, I use VSCode) as Administrator (right click, Run As Administrator) and then open the file “winlogbeat.yml”.

Replace all the content of the file with the content from this gist:

In this content, you’ll find the text “YOURTOKEN_PLACEHOLDER”; replace this with the token they provide in the Logz.io settings page in the section “Account settings”:

Becareful, this token is secret! If someone get his/her hand on this token, they can send fake data to your account and overload it with garbage data.

Now, download the Logz.io public certificate and place it in “C:\ProgramData\Winlogbeat\COMODORSADomainValidationSecureServerCA.crt“. This certificate is available at https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt. You’ll need to do this using your text editor as Administrator.

When this is done, run the following command in a PowerShell window as Administrator: Restart-Service winlogbeat

That’s it, you should now receive your Windows logs in Logz.io! Head over to “Kibana” to see them come in (click “Refresh” if you don’t see them right away).

Note: You might have noticed that in the winlogbeat.yml I enabled logging of Sysmon. I plan to make a blog post to explain what is Sysmon and how to use it. For the moment, it doesn’t mather if Sysmon is not installed, Winlogbeat will simply ignore this config line.

Hint: You can enable the Dark Theme for Kibana in the Logz.io settings page

Ubuntu

Note: There is a TL;DR; section at the end. For this section, I assume you know how to use the Command Line Interface (CLI). For Ubuntu, we will be using Filebeat. By default, if you send logs using rsyslog, the data will not be encrypted over the internet, which is not recomended. By using filebeat, we can send encrypted data easily. On the left navigation, in “Log Shippers” that should be opened (if not, open it), click on “Filebeat”. As they explain in the page, you first need to install Filebeat. So that’s what we’ll do. Run the following commands: Note: here, my link is slightly different than the one they propose on their website. This is because I recommend using the OpenSource version (oss).

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.10.0-amd64.deb
sudo dpkg -i filebeat-oss-7.10.0-amd64.deb
rm filebeat-oss-7.10.0-amd64.deb

We also need the Logz.io public certificate. Run the following command:

sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt

Next, we’ll configure filebeat to ship the logs to Logz.io using my default template and some bash-foo. When asked for the token, provide the token available in the Logz.io settings page in the section “Account settings”:

Becareful, this token is secret! If someone get his/her hand on this token, they can send fake data to your account and overload it with garbage data.

Run the following commands:

sudo curl https://gist.githubusercontent.com/tristandostaler/7a212a0ba23b97c97a5e2c6925bde240/raw/72c90076cf4aa16179c123316b3ef8cec775d576/filebeat.yml --create-dirs -o /etc/filebeat/filebeat.yml
echo -n "Please enter your logz.io Token: " && read -s LOGZ_TOKEN && sudo sed -i "s|YOURTOKEN_PLACEHOLDER|$LOGZ_TOKEN|g" /etc/filebeat/filebeat.yml

Now we only need to start FIlebeat and the logs will start shipping! Run the following command:

sudo systemctl start filebeat || sudo service filebeat start
(sudo systemctl enable filebeat) || (sudo update-rc.d filebeat defaults && sudo update-rc.d filebeat enable)

That’s it, you should now receive your Ubuntu logs in Logz.io! Head over to “Kibana” to see them come in (click “Refresh” if you don’t see them right away).

Note: You might have noticed that in the filebeat.yml I enabled logging of many systems that are not installed by default, like auditd. I plan to make a blog post to explain what is auditd and how to use it. For the moment, it doesn’t mather if auditd is not installed, filebeat will simply ignore this config line.

Hint: You can enable the Dark Theme for Kibana in the Logz.io settings page

TL;DR; putting all the commands together:

curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-oss-7.10.0-amd64.deb
sudo dpkg -i filebeat-oss-7.10.0-amd64.deb
rm filebeat-oss-7.10.0-amd64.deb
sudo curl https://raw.githubusercontent.com/logzio/public-certificates/master/AAACertificateServices.crt --create-dirs -o /etc/pki/tls/certs/COMODORSADomainValidationSecureServerCA.crt
sudo curl https://gist.githubusercontent.com/tristandostaler/7a212a0ba23b97c97a5e2c6925bde240/raw/72c90076cf4aa16179c123316b3ef8cec775d576/filebeat.yml --create-dirs -o /etc/filebeat/filebeat.yml
echo -n "Please enter your logz.io Token: " && read -s LOGZ_TOKEN && sudo sed -i "s|YOURTOKEN_PLACEHOLDER|$LOGZ_TOKEN|g" /etc/filebeat/filebeat.yml
sudo systemctl start filebeat || sudo service filebeat start
(sudo systemctl enable filebeat) || (sudo update-rc.d filebeat defaults && sudo update-rc.d filebeat enable)

Conclusion

That’s it, you now have your initial setup for a SIEM, with logs being sent! Of course, at the moment there are no alerts and the platform we are using don’t offer all the features we would like from a SIEM. But this is perfect to start learning to use a SIEM! In the next blog post SIEM 101 — Basic usage, I explain the basic usage of Logz.io, how to search logs, how to create an alert, etc.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Note: In this post I use Logz.io for my examples, but I recently switched to Humio. For more details: https://www.tristandostaler.com/why-i-switched-from-logz-io-to-humio/

For this post, I assume you’ve read my SIEM 100 series where I explain the basics of Logz.io. I’ll be using the stack I created in this series to demonstrate what to do.

PsExec is a tool provided by Microsoft in the Sysinternals suite: https://docs.microsoft.com/en-us/sysinternals/downloads/psexec
Like they explain:
 

PsExec is a light-weight telnet-replacement that lets you execute processes on other systems, complete with full interactivity for console applications, without having to manually install client software. PsExec’s most powerful uses include launching interactive command-prompts on remote systems and remote-enabling tools like IpConfig that otherwise do not have the ability to show information about remote systems.

https://docs.microsoft.com/en-us/sysinternals/downloads/psexec

PsExec can be use as an adversary to pivot between computers and/or elevate privileges. It is rather easy to detect when executed on a monitored endpoint, but what happens when a remote computer not monitored use it to connect to a monitored endpoint?

It turns out it can be detected by looking for a chain of 3 events, distinctive of PsExec behavior when remote connecting.

Note: Sysmon is needed to detect this behavior. You can read my post SIEM 201 — What is Sysmon on the subject.

The 3 events are:

1. An incoming connection is made on the port 445 or 135 (Sysmon event ID 3 AND destination port: 445 OR 135)

2. An executable file is created in C:\Windows\ (Sysmon event ID 11 AND target filename: C:\Windows\[^\]+\.exe)

3. The executable from the step 2 is executed (Sysmon event ID 1 AND image: C:\Windows\[^\]+\.exe. The image should be the same as the target filename of the step 2)

The result should have no false positive (it would detect the normal usage of PsExec, so make sure to whitelist your allowed users).

In a Lucene query format, it should give something like this (adapt the field names to your context):

(event.code: 3 AND winlog.event_data.DestinationPort: (135 OR 445)) OR (event.code: 11 AND winlog.event_data.TargetFilename: /C:\\Windows\\[^\\]+\.exe/) OR (event.code: 1 AND winlog.event_data.Image: /C:\\Windows\\[^\\]+\.exe/)

Then, you can correlate the winlog.event_data.Image with the winlog.event_data.TargetFilename, but with a simple hack that don’t give a lot of false positive, that’s not necessary: you can simply make sure that the number of unique event.code in your results is equal to 3. This is what I’ll be doing. Head over to Logz.io and login to your account.

The same logic can be applied to detect SCShell, WMIExec and SMBExec, but the detection details are a bit different; I might write another blog for these ones.

In practice

The first thing to do is to simulate a remote PsExec execution so we can detect it. First download and extract PsExec which is bundled with a few tools called the PsTools: https://download.sysinternals.com/files/PSTools.zip

In the location of the extraction, open PowerShell as Administrator (File -> Open Windows PowerShell -> Open Windows PowerShell as administrator). For me the working directory of PowerShell is “C:\Users\User\Downloads\PSTools”. Then execute the following command:

(Get-NetIPAddress).IPAddress | Where-Object {($_ -ne "127.0.0.1") -and ($_ -ne "::1") -and ($_ -notlike "*:*")} | %{.\PsExec.exe \\$_ cmd}

After running this command, you will be in a cmd.exe context. You can simply write “exit” to go back to PowerShell. At this point you generated a remote PsExec execution, we now need to find it in the logs.

Now in Kibana search the query (it’s the same as above):

(event.code: 3 AND winlog.event_data.DestinationPort: (135 OR 445)) OR (event.code: 11 AND winlog.event_data.TargetFilename: /C:\\Windows\\[^\\]+\.exe/) OR (event.code: 1 AND winlog.event_data.Image: /C:\\Windows\\[^\\]+\.exe/)

We can see that we get the pattern I described above:

Now click on Create alert.

I recommend using a Group by on “beat_agent.hostname” because if you have more than 1 computer or server, the event could get mixed between computers and you could get a lot of false positives.

Now, to use the small hack I described above, so we don’t have to correlate events, in the “Trigger if…” section, check that the unique count of event.code is equal to 3. That way, the alert will trigger if and only if the 3 events are met in a small period of time:

We need this little hack because on Logz.io we can only do 2 step correlations and we would need 3 steps for this specific detection case. There are other platforms that allow to correlate more events, so you could do the correlation logic, but there are also some platform that don’t allow correlation so this hack could come in handy.

Conclusion

That’s it, you can now detect when a hacker uses PsExec to move laterally. Of course, you might need to whitelist some source IP or source IP range if you use PsExec legitimately in your context to allow known Administrators VLAN to use it.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Note: In this post I use Logz.io for my examples, but I recently switched to Humio. For more details: https://www.tristandostaler.com/why-i-switched-from-logz-io-to-humio/

For this post, I assume you’ve read my SIEM 100 series where I explain the basics of Logz.io. I’ll be using the stack I created in this series to demonstrate what to do.

Why we need to detect Windows bruteforce

When we have a lot of Windows machine in our environment, it can be useful to be able to detect a bruteforce on a machine, be it over RDP or not. For example, with this Use Case, we could detect an attacker trying to login on a sensitive computer or trying to pivot between machine.

This Use Case is really simple so it’s rather easy to develop and test. But it’s also prone to false positive, so it’s a good Use Case to learn to reduce the false positives.

How

As I explained, this use case is really really simple. We’ll send an email if the event for a failed logon is triggered at least 5 times in a 5 minutes period. The Event ID for a failed logon in Windows is 4625 so the query is pretty simple: event.code: 4625. When creating the alert, we simply need to Group by beat_agent.hostname and winlog.event_data.TargetUserName and check if the number of logs in the last 5 minutes is ≥ than 5.

The group by beat_agent.hostname is necessary because if we have more than 1 Windows in our SIEM, as soon as 5 computers or less do 5 failed logon in the same 5 minutes block, the alert would trigger, and this is not a bruteforce and happens pretty often. The group by winlog.event_data.TargetUserName is necessary if you have some user that connects from time to time to different machines. If you were to not group by winlog.event_data.TargetUserName, you would detect a password spray attack which is an interesting Use Case but it’s not the purpose of this post.

For the sensitivity (5 failed logon in 5 minutes), I’ll explain how we can tweak it and reduce the false positive later in this post.

For the next part, head over to Logz.io and login to your account.

In practice

The first thing to do is simulate a bruteforce.

If you don’t have winrm enable on your computer, like it was the case for my test machine, open a PowerShell window as Administrator, and enter the following command:

Enable-PSRemoting -SkipNetworkProfileCheck -Force
Set-Item wsman:\localhost\Client\TrustedHosts -Value * -Force
Restart-Service WinRM

Next, enter this command to generate 5 failed logon on your computer:

[pscredential]$credObject = New-Object System.Management.Automation.PSCredential ("Non existent", (ConvertTo-SecureString "Bad password" -AsPlainText -Force))
for ($i=0; $i -lt 5; $i++) { Enter-PSSession -ComputerName localhost -Credential $credObject }

Now in Kibana, search the query:

event.code: 4625

You can add the following fields as columns: winlog.event_data.TargetUserName and beat_agent.hostname. You will get something similar to this:

Now click on Create alert and enter the alert name “Windows bruteforce”.

In the Group by field, add winlog.event_data.TargetUserName and beat_agent.hostname.

In the “Trigger if” section, change the values to match the following image:

In the Notify section, add your email and then save your new alert.

Managing false positives

When we first create this alert, the level of false positive we’ll receive is directly proportional to the size of the environment. The more Windows machines we have linked to the SIEM (inherently linked to the number of users) , the more likely false positives will arise.

There are a few strategies we can use to reduce the level of false positives:

1. We can reduce the sensitivity. Increase the threshold by reducing the time frame we look at, or increase the number of failed attempt.

2. Exclude some users from this specific alert and create a new alert less sensitive for these users.

3. Look only at bruteforce over RDP (query: “winlog.event_data.LogonType: 10”). And/or only at bruteforce over the nerwork (like in this example) (query: “winlog.event_data.LogonType: 3”). By doing this, we won’t alert when someone fails to enter his password on his own computer, which is the most frequent case of false positive we would get.

4. Whatever fits your environment. When creating an alert, it’s important to be creative and understand the usage patterns in our environments to reduce the level of false positive.

Note: you can see the logon types here: https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4624

Edit: since I originally wrote this post, I found better ways to minimize False Positives. I summed them up in this post: https://www.tristandostaler.com/siem-103-detect-windows-bruteforce-part-2/

Conclusion

That’s it, you can now detect a Windows bruteforce and know how to manage false positives!

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Note: In this post I use Logz.io for my examples, but I recently switched to Humio. For more details: https://www.tristandostaler.com/why-i-switched-from-logz-io-to-humio/

For this post, I assume you’ve read my SIEM 100 series where I explain the basics of Logz.io. I’ll be using the stack I created in this series to demonstate what to do.

Sysmon is a tool provided by Microsoft in the Sysinternals suite: https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon.  

Like they explain:

System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time. By collecting the events it generates using Windows Event Collection or SIEM agents and subsequently analyzing them, you can identify malicious or anomalous activity and understand how intruders and malware operate on your network.

Sysmon can be used to detect suspicious and malicious events that would normaly not be visible in the default Windows logs. You can see all it can do and it’s corresponding event IDs here: https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/default.aspx?i=j and select Sysmon from the list.  

Installation

Note: There is a TL;DR; section at the end of this section. 

First download Sysmon here: https://download.sysinternals.com/files/Sysmon.zip Once downloaded, extract the content. In the Explorer window that opened after the extraction, open a PowerShell window as Administrator (FIle -> Open Windows PowerShell -> Open Windows PowerShell as administrator). For me, the working directory is “C:\Users\User\Downloads\Sysmon”. Next, we’ll use a config file that reduces the noise sent to the SIEM and that adds some useful logs not enabled by default. The detail of the config file we’ll use can be found at https://github.com/SwiftOnSecurity/sysmon-config. In the PowerShell window opened as Administrator, run the command:

Invoke-WebRequest -URI https://raw.githubusercontent.com/SwiftOnSecurity/sysmon-config/master/sysmonconfig-export.xml -OutFile sysmonconfig-export.xml

Now we simply need to install Sysmon giving it this config file. Always in the PowerShell window opened as Administrator, run the command:

./sysmon.exe -accepteula -i .\sysmonconfig-export.xml

If you already had Sysmon installed, you can update the configuration with the following command:

./sysmon.exe -c .\sysmonconfig-export.xml

Now that we installed Sysmon, if you didn’t have it installed before, we need to restart Winlogbeat so it now sends the logs from Sysmon. Always in the PowerShell window opened as Administrator, run the following command:

Restart-Service winlogbeat

That’s it, you should now be receiving the Sysmon logs.

TL;DR;

Putting all the commands together for easy installation, in a PowerShell window as Administrator, run the following commands:

cd C:\Windows\Temp\
Invoke-WebRequest -URI https://download.sysinternals.com/files/Sysmon.zip -OutFile Sysmon.zip
Expand-Archive .\Sysmon.zip
cd .\Sysmon\
Invoke-WebRequest -URI https://raw.githubusercontent.com/SwiftOnSecurity/sysmon-config/master/sysmonconfig-export.xml -OutFile sysmonconfig-export.xml
./sysmon.exe -accepteula -i .\sysmonconfig-export.xml
./sysmon.exe -c .\sysmonconfig-export.xml
Restart-Service winlogbeat

How it can be used

The number of detection patterns that Sysmon allows are to numerous to talk about in one post, so I’ll only do a quick summary. I’ll use this site as a reference (select “Sysmon” once on the page): ultimatewindowssecurity.com 

Sysmon can log every process creation that happens in Windows. The processes that actually get logged depends on the config file we gave. With the config we gave in this post, we only exclude some known noisy processes so most of the activity we do can be logged. The event ID for a process creation is 1, as we can see in the ultimatewindowssecurity.com website. So if we go in our Logz.io account and search for “event.code: 1” (without the quotes), we can see all the process creation logs:

In this picture, I added relevant fields in the columns, here is what they mean:

• winlog.event_data.Image: This is the name of the process created

• winlog.event_data.CommandLine: This is the command line that was used when creating the process

• winlog.event_data.ParentImage: This is the Parent process; the process that created the current process creation log

• winlog.event_data.User: This is the user that created the process

If we look at all the available fields, we can see some interesting one:

• winlog.event_data.Hash.MD5: This is the MD5 of the current process that was created. This is useful to compare against a list of known malicious MD5. For example, we can look at this file tested in Virus Total: https://www.virustotal.com/gui/file/fb55414848281f804858ce188c3dc659d129e283bd62d58d34f6e6f568feab37/details. The process here is mimikatz, which is a really popular tool used by hackers. It’s MD5 is “2c527d980eb30daa789492283f9bf69e”, so if we were to see this hash in a process creation log, we would know it’s this specific mimikatz process.

• winlog.event_data.Hash.SHA256: This is the same as the MD5, but instead of a MD5, it uses SHA256 to hash the process. It can be used the same way.

• winlog.event_data.Hash.IMPHASH: This is similar to the last 2 fields, but a bit different. This is more advance so I won’t get in the details, but if you’re curious, there is a good article here: https://medium.com/@malcomvetter/defeating-imphash-fb7cf0183ac

Hint: Remember the website virustotal.com, it’s really useful! I might write a blog about it one day, but there is plenty of information online if you want to know more.Note: A process hash can be changed really easily, so don’t rely solely on these to detect bad behavior. Although they can be really useful for basics attacks, they are not perfect. 

So at this point, we could create an alert to detect specific process name or parent process name, or look at the hashes of the process that was created. For example, you could alert when you detect the creation of PsExec by looking at the process name, or when PsExec is used with a specific utility (like PowerShell), etc.

Conclusion

Sysmon is one of the best tool for a Blue Teamer. It’s usage will come back a lot. Altough it can’t do everything, it’s one of the best tool in your toolset. In my next blog posts, I’ll often reference to it. I suggest you read more on Sysmon and the patterns you can detect with it. You can also read on it’s configuration file format; I gave you a decent but basic configuration file and it could be enhanced and adapted to your context.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Note: In this post I use Logz.io for my examples, but I recently switched to Humio. For more details: https://www.tristandostaler.com/why-i-switched-from-logz-io-to-humio/

This is the third post of the series “SIEM 101” where I explain the basics of a SIEM, from installation to simple usage. You can see the previous post SIEM 101 — Initial setup.

Now that we receive our logs in our Logz.io account, it’s time to learn how we can navigate the logs, how they are organized and how to search for specific patterns. Once we know how to search patterns, we’ll be able to start creating alerts on specific pattern, like SSH bruteforce or Windows bruteforce.

The technology used by this SIEM

Let’s start with the technology. Logz.io is based on Elasticsearch as a database and uses Kibana to interact and visualize the data in the database. They added some custom features around these two tools to provide alerting and other things.

Elasticsearch – an ideal SIEM database

Elasticsearch is a “NoSQL” database that stores, indexes and provides analytics tools for JSON formatted data. The official description is:

Elasticsearch is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured. Elasticsearch is built on Apache Lucene and was first released in 2010 by Elasticsearch N.V. (now known as Elastic). Known for its simple REST APIs, distributed nature, speed, and scalability, Elasticsearch is the central component of the Elastic Stack, a set of open source tools for data ingestion, enrichment, storage, analysis, and visualization. Commonly referred to as the ELK Stack (after Elasticsearch, Logstash, and Kibana), the Elastic Stack now includes a rich collection of lightweight shipping agents known as Beats for sending data to Elasticsearch.

Source: https://www.elastic.co/what-is/elasticsearch

Kibana – a basic SIEM interface

Kibana is a web-based user interface to visualize the data stored in Elasticsearch. It was designed by the same team that developed Elasticsearch and built specifically to interact with it’s data. The official description is:

Kibana is an open source frontend application that sits on top of the Elastic Stack, providing search and data visualization capabilities for data indexed in Elasticsearch. Commonly known as the charting tool for the Elastic Stack (previously referred to as the ELK Stack after Elasticsearch, Logstash, and Kibana), Kibana also acts as the user interface for monitoring, managing, and securing an Elastic Stack cluster — as well as the centralized hub for built-in solutions developed on the Elastic Stack. Developed in 2013 from within the Elasticsearch community, Kibana has grown to become the window into the Elastic Stack itself, offering a portal for users and companies.

Source: https://www.elastic.co/what-is/kibana

The interface

For the next steps, login in your Logz.io account and head over to Kibana:

As you can see, there are a lot of information available in a compact manner.

The top section, like in the picture below, allows to navigate in the available Logz.io features / sections

We’ll talk about each of them in details when the time comes, but in a few words:

• Metrics is used to have an easy and centralized view of your server(s) metrics, like CPU usage, memory usage, etc.

• Live Tail is used to look at the logs as they come in the platform, in real-time.

• Send Your Data is where you can get details on how to connect many platforms and technologies to Logz.io.

• Alerts & Events is where we go to create and modify alerts.

• Insights provide useful hints on what problems might be occurring in your server(s).

• ELK Apps is a marketplace of Dashboard, Alerts and other useful tools provided by Logz.io and the community.

For the left section, you can press the button to the bottom to expand the section:

Once expanded you get the following :

The Discover section is the one you see as you come on the site. It allows to search and browse the data. We’ll have a look later in this post.

The Visualize section is used to generate aggregated graphs (Pie Chart, Data Table, Heat Map, Region Map, Line Graph, etc.). This is really useful to easily make sense of a lot of data. For example, you could search for the most common processes started on a single server or on all your servers. Or you could draw the total traffic logs over time for your website to better understand your peak hours.

The Dashboard is useful to create, you guess it, dashboards. They are a centralization of multiple saved Visualize graphs. They allow to quickly see different pattern that interest us in a centralized place. Also, any filter you apply on the dashboard will be applied to all the Visualization in the dashboard. For example, if you have a dashboard for your website traffic, you can isolate a single IP and see all the Visualization graphs automatically updated to display info on that single IP.

Timelion is useful to overlay and compare different time-based graphs. You could, for example, compare the traffic between 2 servers by having them in a single graph.

The Management section is for, well, management (of Kibana). But this is for advanced users and we don’t have to go there often, so I won’t go in details.

In the top of the center section we have the search tools:

Here we can use a Lucene query or a Kibana Query Language (KQL) query to search for the data. We can also apply filters (for example: “IP: 127.0.0.1”).

On the right we have the time period over which we apply our search (in the picture here the data displayed is from the 15 minutes before the moment the search was launched).

To refresh the data, we can press the “Refresh” button. It is possible to Save a search so they can be Opened at another time, or Shared.

The big “Create alert” button is useful to quickly jump to the section to create an alert from the currently applied query and filters. This is what we’ll be using later to create an alert.

In the left of the center section we have all the fields available from our logs. We can interact with these fields (filter, visualize, etc.):

The Selected fields are the one used as columns in the main center section. When you hover on a field, the “add” button appears. You can add columns to the main center section by clicking on the “add” button. You can also click a field to get a quick view of the top values for that field.

Note: If you get the button “Field not indexed”, simply click on it. The page will refresh and the field will be available.

In the center of the center section we have the main section:

At the top of the main section we have a timeline for the count of messages over a period or time.

In my picture, we can see I have around 3 messages per period of 20 seconds, for a total of 68 messages (hits). You can use your mouse to select a time period and change your time frame (here being the “Last 15 minutes”).

Note: in my blog, “messages” and “logs” are interchangeable and are used to describe a single event that happened on the server or computer, like a logon.

Then we have the logs. This is where all the magic happens:

Here I only have 2 columns selected: Time and message. If you where to add more fields as column, they would be visible here.

When you hover on a field (like “message”) you get an X to remove the field from the displayed columns.

If you click on the small arrow to the left of the row, you can expand the specific log to see all the fields and their values:

Here, on the left is the name of the field, like “@metadata.beat” and on the right is the value, like “winlogbeat”. If you hover on the field, you get 4 icons:

The + and – are used to add a filter to specifically search for the value or to remove the value from the results, respectively. The next icon to the right is used to add the field as a column and the last one is to add a filter to make sure the field is present in the log (for example, exclude the logs that don’t have an IP).

Navigating the logs in a SIEM

To learn how to navigate the logs, we’ll use a useful context in windows environment: we’ll search for the last user that logged in.

A quick google for “windows view the logon events” give me many results, including https://docs.microsoft.com/en-us/windows/security/threat-protection/auditing/basic-audit-logon-events.

By reading we can understand that Windows logs the logon events. By scrolling a little, we can see the “Logon events” number and their matching description. Here “4624” looks like what we are looking for: “A user successfully logged on to a computer.”.

Hint: learn the event IDs 4624 and 4625, they are used really often in a Windows environment!

So back in Logz, we need to find the field that contains the 4624. What happens if I search for “4624”?

No results… Hum… Ohhh the time frame! I didn’t login in the last 15 minutes. Let’s change that to 60 minutes ago and click the Update button.

14 hits! But wait, I didn’t login 14 times in the last hour? And what are these users with a dollar “$” sign I can see in the “message” column? Let’s ask Google. A search for “windows event id 4624 user with dollar sign”, first result: https://www.ultimatewindowssecurity.com/securitylog/book/page.aspx?spid=chapter5

Hint: you can save this site for later, ultimatewindowssecurity.com is filled with info on Windows event details, including Sysmon!

This page has a lot of info. Let’s jump to the explanation of the “$” sign with CTRL+F (search in page). We get this description:

DC Security logs contain many Logon/Logoff events that are generated by computer accounts as opposed to user accounts. As with computer-generated account logon events, you can recognize these logon and logoff events because the Account Name field in the New Logon section will list a computer name followed by a dollar sign ($). You can ignore computer-generated Logon/Logoff events.

So the events with these accounts are computer-generated Logon events. But they happen on a Domain Controller (DC). In my case, I am testing on a Windows instance not joined to a domain so this is not what I am looking for.

Let’s keep digging. The name of the field was “winlog.event_data.SubjectUserName”, I wonder if another field contains “UserName”. Looks like it: “winlog.event_data.TargetUserName”.

Hint: When DC joined, when you have more than one DC sending logs, you’ll get more than one user. You can enter the following query to filter our all the users with a dollar sign: “winlog.event_data.SubjectUserName: /.?$.?/” (without the quotes), the total query at this point being: “4624 AND NOT winlog.event_data.SubjectUserName: /.?$.?/” (without the quotes)

For simplicity, I’ll add the field “winlog.event_data.TargetUserName” as a column:

There you go, I now have useful details!

So it looks like my most recent logon was on “Nov 26, 2020 @ 21:13:04.405”.

Note: You might have noticed that the logs timestamp change from picture to picture, this is because as I write this post I update and refine my search so I can get useful images.

Creating an Alert – the best feature of a SIEM

Now we know how to search the logs, the only thing left is to create an alert and send an email. I’ll use the same context and create an Alert to email me every time I logon.

Note: this is used as an example only. If you create an alert like this, you’ll receive a lot of emails!

So, starting from my research for the most recent logon for my user named “User” (I know, it’s original!), I’ll add a filter specifically for my user so I don’t receive an email every time my computer generates an event:


We are now filtering specifically for the user “User”, but I don’t like the “4624” in the search bar, without specifying it’s field name; it’s comparing the value to all the values of all the fields! Now that we know we only have the real logon events for my user, let’s find the field name for the event ID 4624 with a CTRL+F “4624”:

Note: don’t forget to open a log so we can see all it’s field

Note: Kibana display the thousands with a comma so we need to search for “4,624”… I am Canadian, I hate this format 

Let’s filter for the value “4624” for the field “event.code” and remove the “4624” from the search bar and click the “Update” button:

Hint: For performance reasons, it’s better to use filters than the search bar, but that only work for exact matches.

Now that we have exactly the results we want, we know our query is good. Let’s click the big blue “Create alert” button. We are redirected to the “Alerts & Events” section of Logz.io.

We give the alert a name: “The user User logged on”

Leave everything as is, up to the section “Notify” (the query in the section “Search for…” is pre-filled with the filters we prepared before, thanks to the big blue button).

You can give a useful description if you want, I don’t as my alert title is clear enough.

Then in the field “Who to send it to”, simply write your email and select it.

When you’re satisfied with everything, click the “Save” button on the top right:

The only thing left is to test the alert. Simply login on your computer and you should receive an email in the next 15 minutes (that’s the time I defined in the “Trigger if…” section). You will now receive an alert every time you login on your computer!

Note: If you add more computers, because we didn’t filter for the source, you would get an alert every time the user “User” logon on any computer.

Conclusion

That’s it! You now know the basics of Logz.io, how to search logs and how to create alerts, which is the basis of using a SIEM. I will leave to you the pleasure of discovering how to create Visualize graphs and Dashboards. There are a lot more things to learn, which we’ll do in time. Everything is now in place for me to start writing blogs on detection patterns! I’ll do simple patterns in the “SIEM 102” series, and more complex detection patterns in the “SIEM 201” series. I’ll eventually talk about how we can automate responses and transform and bonify the data.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Information Security, Cybersecurity, IT security. We’ve all heard and read them in interchangeable ways, these synonyms all aim to define what is security. But most of them imply assumptions harmful for the Information Security domain. Let me explain.

What we try to do

Generally, when we, or often the media, use the term “Cybersecurity”, we want to describe the idea that we want to be secure against some kind of “hacker”. Most people who use this term imagine a “hacker”, in a black hoodie, in front of a screen full of green scribble. But this idea misses the point of what really is Security, or as I prefer to say, Information Security, or InfoSec.

What a bad actor, often named “hacker”, really want to do is to gain access to information that he/she wouldn’t normally have access otherwise to reach a goal.

In a ransomware attack, the goal is to keep you from having access to this information and extort money from you. In a phishing attack, the goal could be to steal credentials, which is a kind of information, to use them or sell them.

So once we understand what the bad actor wants, we understand what we need to protect. At the highest level, the goal is Information. Hence the word “information” in “Information Security”. The security part imply the idea that we want to protect this information against all kind of attacks.

Why it’s a problem

The problem with the words “Cybersecurity”, “IT security” and other similar terms is that they imply that the security must be done at the technology level: the computers, the servers, the firewalls, the switches, etc. While it is important to secure these technologies, the global strategy needs to be way larger.

For example, no technology can help us against the CEO fraud if an untrain employee falls for it. The problem in this specific case is the lack of training. Training is not a technological solution but it is one of the most effective strategy to protect the business.

If we try to reduce the security problem to only the technology, we’ll fail miserably.

The reason these terms are harmful to the domain is that they create this idea in the mind of the leaders that security is an IT problem. So it often falls in the hands of an IT officer, like the CTO. Most businesses don’t have a CISO, and those who do have him/her under the IT department. I could write a complete post on why this is a bad idea.

What should we do

This is a business culture problem and this is where we need to start the work. Businesses that invest in a Security culture often thrives even more than the ones that put millions on technologies.

So it is more than time that we start talking about this domain by naming it “Information Security“!

Note: I wrote “hacker” between quotes when talking about bad actors because this term became pejorative. Initially, a hacker is only someone who finds a way around some limitations, with good or bad intentions; mostly goods. For example, my reading light is too bright and I can’t dim it enough; so I put black tape on it to cover about 80% so I get just enough light. That’s hacking applied to something else than technology. It’s a ridiculous example to demonstrate my idea.

As explained in wikipedia:

The controversy is usually based on the assertion that the term originally meant someone messing about with something in a positive sense, that is, using playful cleverness to achieve a goal. But then, it is supposed, the meaning of the term shifted over the decades and came to refer to computer criminals

Source: https://en.wikipedia.org/wiki/Hacker

Note 2: because of the way SEO works, I’ll hack my way to the top results by placing terms like “Cybersecurity” in my blog from time to time 😉

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

What is SolarWinds Orion

SolarWinds is a US based company. They have clients worldwide. One of the products they offer is Orion. This tool does a lot of things, but the main feature that interest us is that it provides the ability to monitor your infrastructure. To do this, you need to install the Orion agent on one of your servers. This agent is installed with high privileges and is able to login on most of your infrastructure to gather metrics like CPU, RAM, etc.

What’s the hack

At the moment I am writing these lines, it is unclear how SolarWinds got hacked. What we know is that it was perpetrated by an APT (Advanced Persistent Threat, a term used to describe high profile malicious hacker groups) probably backed by Russia. In other words, they got hacked by a highly skilled group of hackers with deep pockets.

Once inside SolarWinds network, the APT added a malware inside the Orion software that was then pushed to all the agents world wide. The malware was now alive inside many thousands (an estimate says 18 000+) public and private enterprises in all continents. This is what we call a “supply chain attack“.

SolarWinds provided an update that removes the malware from the code. You should make sure to apply it ASAP if you don’t already have it.

Why it matters

This impressive hack matters because now all companies that uses Orion needs to verify if they received the bad update. And if so, they need to check if the malware was used maliciously in their network.

To phrase it differently, because of a single hack, the malicious actor was able to infiltrate multiple thousands companies. Quite impressive and scary.

The good news

The good news is that at this point this hack was used by the Russian APT only to attack specific targets (like FIreEye). In other words, you probably were affected but the malicious actor probably did not use the malware to attack you (unless you are a high profile target, in which case you’re probably aware of this threat? At least I hope…).

The other good news is that the firm FireEye published IOCs that can be used to verify if the malware was used in your network. The IOCs are available here: https://github.com/fireeye/sunburst_countermeasures

Sources:
https://www.zdnet.com/article/microsoft-fireeye-confirm-solarwinds-supply-chain-attack/
https://www.crn.com/news/security/fireeye-us-federal-agencies-hacked-through-solarwinds-report
And many threads on Twitter

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

What is a Zero Trust Architecture

A Zero Trust Architecture (ZTA) is an Information Security architecture based on the idea that a network shouldn’t have inherent trust on the internal boundary. Many enterprise networks are designed like an egg where they have a rather strong and robust external shell, but once inside the shell, everything is lax; it needs only a little needle that is able to break through the shell to wreak havoc inside.

By designing a network using the ZTA, you can make sure that if an attacker is able to get a foothold in the internal network, he doesn’t have access to anything. Of course, if the attacker stole credentials, he would still have access to what the user he impersonate has access to. But with a ZTA, there is a dynamic and constant evaluation of the global risk and current status of the whole enterprise assets, meaning that an access that was given to a user could be removed at any point based on many criteria.

For example, if the global risk of the enterprises reaches a critical point when a new CVE with a 9.9/10 CVSS score appears within it’s assets, the algorithm could block access to sensitive information to users until the CVE is patched, or could enforce MFA verification to get access to the data where there would normally be only a single factor.

Another interesting example is that an access could be denied if it’s discovered that some credentials of the user trying to get access were leaked recently (for example by using

https://haveibeenpwned.com/

).

Why it matters

A Zero Trust Architecture is a key element in modern architecture, cloud or not. For example, it is one of the main elements of the new architecture recommendation by Microsoft: https://docs.microsoft.com/en-us/security/compass/privileged-access-strategy.

This model significantly reduces the risk that an attacker could escalate it’s privileges, maintain an access for a long period and it makes it harder to use elevated privileges when they are available to the attacker (for example: through phishing).

How is a Zero Trust Architecture implemented

As I explained, the official documentation is dense so I won’t be able to summarize how to design and implement a ZTA here. For that, I would recommend you read the official document. But I will try to make the intent of the architecture clear enough so that you can decide if it’s something you should do in your enterprise.

The whole principle of the architecture is summarized in the section 3 of the document at page 9: Logical components of Zero Trust Architecture. Without going to much in the details, we can get a pretty good idea of what’s going on with this picture:

In this graph, we can see many components:

Subject: The subject is the user using the System (a computer) to get access to some Enterprise Resource (some piece of information)

System: This is simply the computer, or something similar, being used by the Subject (user) to get access to something.

Policy Enforcement Point: This is one of the core components of a ZTA. It’s purpose is to receive access requests from Systems (users) and ask the Policy Decision Point if an access is granted. It is important to note that the access can be revoked at any moment as everything goes through it, even once an access is granted. A Policy Enforcement Point can be a single system, but can also be split in 2: the Client (like a computer) and the Resource (like a website allowing access to the user).

Policy Decision Point: This is a logical component encompassing the other 2 main components of a ZTA: the Policy Engine and the Policy Administrator.

Policy Administrator: This is the component that handles all the session information, like the authentication token. It is also responsible of allowing or stopping the communication between the System and the Resource depending on the Policy Engines decision.

Policy Engine: This is the component that makes the ultimate access decision based on the policies defined by the enterprise. When the Policy Enforcement Point asks if a System is allowed to communicate to a resource, the Policy Administrator communicates with the Policy Engine. The Policy Engine makes a decision using multiple decision points (including external resources like Threat Intelligence) and the policy for the resource. The Policy Administrator will then execute the decision by creating the token for the sessions and other related things. If at any point the Policy Engine decides that the communication should be stopped, the Policy Administrator will revoke the validity of the token and the communication between the System and the Resource won’t be allowed anymore.

So, to summarize the flow, the Subject (user) using a System (computer) will try to connect to a Resource. To establish the connection, the System will ask access to the Policy Enforcement Point. The Policy Enforcement Point will ask how to proceed to the Policy Administrator which validates with the Policy Engine if the access should be allowed or denied. If access is granted, the Policy Administrator generated all the required information for the session to be authenticated and authorized and returns the session details to the Policy Enforcement Point which grants access to the System to the Resource.

How can we implement a ZTA

The previous section explains in an architectural concept how a ZTA can be implemented but it doesn’t explain how to actually install and use it. Because of the complexity of the architecture, I won’t go into the details on how this can be achieved, but I’ll direct you towards great resources.

If you’re in a Windows environment, the best way is to follow the guidelines and steps provided by Microsoft: https://docs.microsoft.com/en-us/security/compass/privileged-access-strategy

If you’re in a Linux environment, there are some providers that can help implement this strategy, one of them being Okta: https://www.okta.com/resources/datasheet-advanced-server-access-a-zero-trust-approach-to-linux-and-windows-server-access-via-ssh-and-rdp/

What I think of the ZTA

Because of the current context with the COVID-19 and the fact that a lot of enterprises now have to support people working from home, the idea of a ZTA starts to get traction. Since I started reading about it I have seen more and more blog posts talking about it.

In an ideal world, everyone would implement a ZTA. That would make most enterprise really resilient to attacks and reduce the number of successful hacks we see weeks after weeks. But like it is for everyone, budgets are limited and technical human resources that can make these changes are scarce.

The main problem is that, like most secure features, it is not enabled by default. This implies that we need to assign a skilled resources fulltime for a significant amount of time on this project, slowing us on all the other important things we need to do.

If you go to the link for Microsoft Windows, you’ll quickly realize that this architecture is no simple task. And if you know Microsoft, you know everything won’t be perfectly documented and some feature won’t work like you think it should, meaning you’ll have to do some trial and error to make it work.

I think a ZTA should be applied to everyone, but because of the effort needed, it will be applied mostly by some medium and large enterprises that value Security as a core business need.

Sources

• https://csrc.nist.gov/publications/detail/sp/800-207/final

• https://docs.microsoft.com/en-us/security/compass/privileged-access-strategy

• https://docs.microsoft.com/en-us/security/zero-trust/identity

• https://docs.microsoft.com/en-us/security/compass/esae-retirement

• https://www.techrepublic.com/article/zero-trust-security-a-cheat-sheet/

• https://www.helpnetsecurity.com/2020/03/02/building-zero-trust/

• https://www.okta.com/resources/datasheet-advanced-server-access-a-zero-trust-approach-to-linux-and-windows-server-access-via-ssh-and-rdp/

• http://www.digitaljournal.com/tech-and-science/technology/a-zero-trust-framework-is-no-longer-optional-for-enterprises/article/582718?s=09

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

WordPress is one of the most popular CMS and website server in the world. It handles “more than 30%” of the websites on the internet. Because of this, bad actors are really interested in finding ways to get control of them.

One of the most easy way for them to achieve this is by bruteforcing the login for common username and password combinaisons, including from past leaks. This is a working strategy because by default WordPress doesn’t protect against bruteforce attacks.

How do we detect a WordPress bruteforce attack

In the WordPress world, there are many ways to handle bruteforce attacks, as this is rather simple to detect. But because we are in the SIEM series, I’ll talk about how to detect the attack using a SIEM.

For this post, I assume you have WordPress installed on linux using Apache and that you are sending Apache’s logs, as defined in the filebeat configuration I provided in the “Initial Setup” post.
(Hint: LAMP)

Some background

To understand the next section, we need to understand how WordPress works when someone tries to login and how we can see this in the logs.

There are two ways to login to wordpress:

1. Through the standard login page (wp-login.php)

2. Through xmlrpc when not disabled

After a successful logon, we get redirected (302 Found) to the WordPress dashboard. When a failed logon occures, the response is different (we get a 200 OK and the page contains an error message).

The detection pattern

Now that we know how WordPress behaves when logging in, we know what to look for to detect a bruteforce attack.

The idea is to count the number of times the same IP sent a request to one of the login pages and got something else than a successful logon response. When the count reaches a defined threshold (like 10) in a defined time window (like 5 minutes), we consider it as being a bruteforce attack. In our case, we’ll simply send an email, but we could automatically block the attacking IP.

Here are the useful elements:

The query: request: (/.*wp-login.*/ OR /.*xmlrpc.*/) AND NOT response: 302
The field “request” is the field, after parsing, that stores the page being accessed. The field “response” is the field, after parsing, that stores the response code of the request.

First group by: beat_agent.hostname
This group by is necessary if we plan, or do, have more than 1 WordPress website, or any website with these login pages.

Second group by: geoip.ip
(or simply “ip”, depending on your config)

This group by is necessary because we want to know what IP is attacking us so we can eventually take an action. This info could also be usefull in an investigation.

For the sensitivity (10 failed login in 5 minutes), I find it to be sensitive enough for most attacks and it has a low level of false positives. But you can always tweak it to fit your context.

For those of you who tells me that “an attacker could make 9 attemps every 5 minutes to bypass your measure”, that is true, but a bruteforce attacks that does 9 attemps every 5 minutes will probably never succeed and the attacker will go elsewhere, to a more juicy target.

For the next part, head over to Logz.io and login to your account.

In practice

The first thing to do is simulate a bruteforce. Because we are only looking for 10 tries in 5 minutes, it’s feasible manually. If you had a more agressive threshold, you would need to develop a script to send login requests.

Head over to the login page of your WordPress website. For me that would be https://www.tristandostaler.com/wp-login.php.
Note: because I have some security plugins, this link is not normally working. I disabled the plugin while writing this post.

You can write anything in the fields Username and Password, then press “Log In”. Do this 10 times. You should now have 10 failed logon visible in the logs.

In Kibana, search the query: request: (/.wp-login./ OR /.xmlrpc./) AND NOT response: 302

You can add the following fields as columns: beat_agent.hostname, geoip.ip (or simply ip) and response (this one is not useful for the alert but we’ll add it to see the behavior I described). You will get something similar to this:

Here, we can notice a few things:

1. My hostname is “novutech.ca” and not “tristandostaler.com”. This is because I am hosting my site on the same server as the website I use to advertise my hosting services (the website needs to be updated, I know!)

2. The responses are “200”. This is, as I explained, the behavior that happens when a bad logon occurs.

3. We can see the IP 43.250.249.5 trying to bruteforce my site at the same time!

You can now click on Create alert and enter the alert name “WordPress bruteforce”.

In the Group by field, add geoip.ip (or ip) and beat_agent.hostname.

In the Trigger if section, change the values to match the following image:

In the Notify section, add your email and then save your new alert.

Managing false positives

After creating the alert, the number of false positive should be pretty low. But if you happen to get too much false positives in your environment, there are a few strategies we can use to reduce the level of false positives:

1. We can reduce the sensitivity. Increase the threshold by reducing the time frame we look at, or increase the number of failed attempt. A smaller timeframe will detect aggressive bruteforce but can be bypassed more easily. A higher number of failed attempts will increase the risk that an attacker guesses the right username / password combination before being locked out.

2. If you do have geoip enabled, you can have more sensitive alerts for some countries and less sensitive alerts for other countries.

3. Whatever fits your environment. When creating an alert, it’s important to be creative and understand the usage patterns in our environments to reduce the level of false positive.

Conclusion

That’s it, you can now detect a WordPress bruteforce and know how to manage false positives!

Don’t hesitate to write to me if you have any questions. This could also help me better understand how to write a good blog that answers most questions.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Source: https://twitter.com/OffenseTeacher/status/1351927187180572673

Some background

Some background are required to understand his response.

First, it is important to know that Guillaume is an excellent attacker / hacker (Red Team). He works at the same company as me (Okiok) on the offense side. He is hired by companies, through Okiok, to test their defenses. It’s his job to know the best attacking tools available. His job is to simulate an attack by a highly skilled bad actor.

Second, Fireprox is a tool that uses some features of Amazon Web Services (AWS) to send requests from AWS’s IPs. AWS is a cloud provider; they offer many cloud services like storage, computing, etc. They own a huge amount of IP addresses so Fireprox leverages this feature to send requests to website using a new IP on almost every request (with some limitations).

In my post, I explain that we can detect a simple bruteforce if the same IP does 10 tries in 5 minutes. Because Fireprox uses a different IP (almost) every request, it can bypass this detection pattern.

My answer: a good InfoSec Strategy

The short answer is: you need a good InfoSec Strategy. There are a few key elements to answer this comment:

1. A SIEM, and any detection use cases, is not a silver bullet. One use case is designed to detect one pattern of attack, but there are always new attacks. A SIEM is something that keeps evolving. It’s important to add new detection use cases as needed, and remove the old ones that are out of date.

2. A defense in depth approach is key. It’s important to have many layers of security measures.

3. Understand your Threat Model. The idea is to understand who might target you, from an unskilled bad actor to an APT.

The lifecycle of a Use Case

When using a SIEM, we’ll develop more and more detection use cases as we learn of new attacks, we introduce new software and systems, etc. The problem with this is that the more use cases we have, the more resources are needed to run and operate the SIEM, without counting the false positive rate and time to maintain them.

As we add people, devices, systems and software in our environment, the existing use cases might start to generate more and more false positives. Because of this, it’s important to revisit the existing use cases and evaluate what strategy can be used to reduce the level of false positives. To get some ideas on false positive mitigation strategies, you can review the following posts:

1. SIEM 102 — Detect WordPress bruteforce

2. SIEM 102 — Detect Windows bruteforce

3. Be creative! Try to find new ways to reduce the level of false positives without increasing the level of false negatives

As a use case ages, it might become obsolete if, for example, the technology we try to protect is not used anymore. For example, if I were to change from WordPress to another CMS for my blog, any use case base on WordPress would become obsolete, but they would continue to impact the SIEM’s performance and it’s maintenance. For this reason, it’s important to periodically revisit and question if the use case is still useful.

When I say it would impact maintenance, this is because when you have a lot of use cases, it clusters the view of the SIEM and/or reports and affects any statistics you have (ex: what’s the percentage of use cases that trigger every weeks)

A defense in depth strategy

I couldn’t presume to produce a better definition of what is Defense in Depth than the CIS:

Defense in Depth (DiD) refers to an information security approach in which a series of security mechanisms and controls are thoughtfully layered throughout a computer network to protect the confidentiality, integrity, and availability of the network and the data within. While no individual mitigation can stop all cyber threats, together they provide mitigations against a wide variety of threats while incorporating redundancy in the event one mechanism fails. When successful, this approach significantly bolsters network security against many attack vectors. An effective DiD strategy may include these (and other) security best practices, tools, and policies.

https://www.cisecurity.org/spotlight/cybersecurity-spotlight-defense-in-depth-did/

With an analogy, it’s like a car and it’s many defense mechanism to reduce injuries and deaths: Collision avoidance system, Anti-lock braking system, air bags, seatbelt, etc. Like with a car, it’s not completely risk proof. But the more layer you have, the less risk you have.

Using my WordPress server as an example, I have many layers to secure it as much as possible according to my Threat Model:
Note: this list is incomplete as I do not want this post to be too big, but it give a good idea of what I’ve done

1. WordPress and all the plugins are always up to date

2. I have a few security plugins, some of them provide bruteforce protection

3. I have MFA enabled for the login

4. The login page is unavailable (it is, but you need to know how. A little bit of security through obscurity can help sometime!)
   Note: this point changed since I did this post. I am now using another security plugin and other tools and I needed to keep the normal URL for the login page. But I added other mitigation strategies around the login page.

5. The web server logs are sent to my logz.io stack with a few detection use cases and automated actions to block attacking IPs

6. My server has fail2ban

7. My server provider offers some basic protection, like DDOS.

8. I have write-only backups (you need special access to read and restore)

9. Etc.

Understand your Threat Model

A Threat Model analysis is where you carefully analyze what are your attack vectors, what bad actors might be interested in your assets and to which ones. There are many ways to analyze this and I will dedicate a post for this at some point.

I didn’t do a complete and thorough Threat Model analysis for my blog, but I am basing mine on my gut feeling and my experience (if you are doing this for an enterprise, don’t! You are way more interesting to bad actors than me!).

In my case, I don’t have much asset of value and I have backups in case of a ransomware or a DOS of some kind (like if I do an error in a config). The most important thing I try to protect is my time because restoring a backups or something like that is time consuming. From the bad actors point of view, my server could also be useful in a botnet. Since I started publicizing my blog, I know some script kiddies might try to hack my website in some way, including with a DDOS.

Based on this, I think that the protection layers I have are more than enough. If I look at the attacks I block almost everyday, I know I am protected against the most probable attacks on my server, including a distributed bruteforce, like my fiend was proposing.

Conclusion

Like I thought, it took me more than 280 characters to answer this question 😉
I know I touched some point I didn’t address yet in my blog, but I plan on talking in depth about most of them at some point. Don’t forget that the initial post name was “SIEM 102 (…)”. For the moment I am only talking about the base and I already have plans for some “SIEM 401” posts!

I am glad Guillaume asked me this question though, as this allowed me to see what subject I should address in the future. Thanks Guillaume!

Don’t hesitate to write to me, like Guillaume did, if you have any questions. This could also help me better understand how to write a good blog that answers most questions.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Note: I used “Cybersecurity paths” , “where to start in cybersecurity” and “Cybersecurity jobs” just so I get good SEO results. To know why, you can read my post Information Security Synonyms.

If you Google “Cybersecurity paths”, “Cybersecurity career”, “Cybersecurity jobs” and other similar terms, you’ll find an array of results with all kind of advices all addressed at people already in the field. In this post, I’ll try to explain my take on this subject, but destined for new comers that have no idea where to start.

The Information Security domain is wide, really wide. I used to surprise people in meetings because I would be interested in what they where doing and how they where doing it. In their mind, security was about hackers and viruses. It’s not the case.

A good way to see this, is to search on Google for “cybersecurity mind map”. You’ll find a lot of results like these ones:

It’s important to know that these are incomplete, but they give a good idea on the size of the domain.

But, when looking at all this, it’s normal to ask “where do I start?”. Let’s answer this question.

Where do I start?

This is a big question because there are multiple paths possible. I’ll explain some possible paths based on my experience, but in reality, there is no clear way to get in Information Security. I think the most important thing to have is passion!
Note: notice I wrote “some” paths and not “the” paths!

First I’ll explain the most frequents roles we see in cybersecurity jobs offers, based on the website https://www.cyberseek.org/pathway.html. Then I’ll explain how these entry roles can lead to the more advanced roles. These parts will be based on my comprehension of the roles and aimed at new comers. It’s important to note that I focus mainly on the technical roles as they are the ones I know, but there are other non-technical roles that exists. For example, you can be a lawyer expert in cybersecurity. I’ll be writing about common roles but it’s important to know that a lot of roles are missing from this post.

I’ll split the explanation by paths so you can jump ahead to the section of your interest:

• First step

  • Networking

  • Software Development

  • Systems Engineering

  • Financial and Risk Analysis

  • Security Intelligence

• How to get one of these jobs

• The main roles

  • Entry-level roles

  • Mid-level roles

  • Advanced-level roles

The principals Information Security roles

As we can see in https://www.cyberseek.org/pathway.html, the InfoSec roles all assume some kind of basic experience in IT. It can be in networking, software development, etc. The idea is that in order to protect systems you must have a good understanding of how it works. The following roles can lead to cybersecurity jobs.

First step

So the first step to start in InfoSec is to get some experience or follow a recognized course in one of the feeder role: Networking, Software Development, Systems Engineering, Financial and Risk Analysis, Security Intelligence. Some people are able to get a job without a degree in some of these roles when they have enough experience through personal experimentation and they are able to demonstrate these abilities. Here is a quick summarize of what these jobs are:

Networking

A network analyst will configure devices and computers to make sure they can talk to each others. For example, when you are reading this post through your device, there are multiple devices in-between that need to be well configured so that the communication can flow flawlessly. To get this job, you need to have a good understanding of network protocols. Some experience with vendor devices (like Cisco) will definitely open doors. You will also need to learn to debug network communications.

Software Development

A software developer is someone who know how to code. Coding is the act of writing, in some form, instructions to a computer that it will then execute. For example: when I was writing this post, I was telling my server (a computer dedicated to serve websites) what text to display and how to display it. To get this job, you need to know at least one programming language. The first one I learned was Java but my favorite ones nowadays are Python and C#. Onece you learn one, the other ones are a lot easier to learn as they all share similarities.

Systems Engineering

A systems analyst is someone who has a good understanding of code, networking and hardware but do not work specifically in one of these field. His or her job would be to oversee the development of a system that involves one or more of these fields. It could be to oversee the cost, design, staff and timeline implication of a project. Wikipedia has a good and more complete description. To get this job, you need to have a good general understanding of the fields previously mentioned. In other words, you need to know a little bit of everything related to computers and IT.

Financial and Risk Analysis

A financial and risk analyst is someone who analyses what risks faces a company but on the financial point of view. This can be based on the market volatility, laws, environment, within the corporation, etc. I have to say I have no experience on this so the input I can provide is limited. You can find more information here: https://www.investopedia.com/terms/r/risk-analysis.asp. This risk analysis process can be used in Information Security to analyze an enterprise risk.

Security Intelligence

A security Intelligence analyst is someone who analyzes the threats and malicious actors that threaten a company. Because these change based on the size and field of the company, that person needs to understand how a bad actor thinks. This can be done by monitoring different trends and/or reviewing reports and analysis of companies renowned in this field (there are a few). This analysis can then be used to make informed decision on the priorities of the company.

How to get one of these jobs

For these jobs, you can be lucky and get a cybersecurity job without a degree if you have some experience and you can demonstrate your understanding, but a good way to start would be a degree of some kind in IT or software development. It can be at a university, and that would definitely open a lot of doors, but it doesn’t have to be. There are a lot of schools that offer good formations. I personally did a bachelor degree in Software Engineering.

When I said there are multiple paths possible, I know people that came from all sorts of background. For example, I know lawyers that now work in Information Security! I mentioned a formation in IT or software development because this is what I know.

If you have questions regarding where to start, if a program is good, or anything like that, don’t hesitate to write me, I can definitely help you find your way in this field!

The main roles

Once you have your foot in the door from one of the feeder roles, there are some key entry-level roles you can have. I’ll write about the ones mentioned on https://www.cyberseek.org/pathway.html but these are mainly technically focused. In reality there are some other non-technical roles where you “only” need global knowledge of the Information Security principles which can translate to cybersecurity jobs. If you look at the following sites, you’ll see that there are a lot of roles in the field (though there are some overlaps in the roles mentioned in these sites):

• https://www.cs.seas.gwu.edu/cybersecurity-roles-and-job-titles

  • This one is technically focused.

• https://www.snhu.edu/about-us/newsroom/2020/07/cyber-security-roles

  • This one is interesting as they mention some interesting roles like in management. They also have good advices

• https://cybersecurityventures.com/50-cybersecurity-titles-that-every-job-seeker-should-know-about/

  • This one demonstrate the number of roles (or different name of roles) that exists, but there is a lot of overlap, meaning that some of these are different name for the same role.

• https://www.cyberdb.co/3-non-technical-positions-in-high-demand-in-the-cybersecurity-industry/

  • This one is clearly incomplete, but they mention 3 non-technical roles

• https://www.educause.edu/focus-areas-and-initiatives/policy-and-security/cybersecurity-program/resources/information-security-guide/toolkits/information-security-governance

  • This one is a detailed view of what is InfoSec governance and how to do it. This role is non-technical but requires to work partly with technical people.

Entry-level roles

The roles in this section require basic knowledge of Information Security principles but they allow beginners to start gaining real hands-on experience. Once you get enough experience in these roles, you can aspire to get in a mid-level role. Generally, these roles require 1-5 year in IT and/or a bachelor degree. The idea is that you need to have basic knowledge in IT and InfoSec.

Cybersecurity Technician

A cybersecurity technician is someone that work at the operation level. They will be responsible to maintain software and devices used by the cybersecurity team, assist users and manage basic alerts. They are the first line of defense. Example of what they maintain are:

• Firewall

• Web application firewall

• Antivirus or it’s more advanced brother: the EDR

• SIEM and it’s alerts

This role is really interesting because you touch a lot of different systems and software. If they see an alert and think it could be an incident, they would call an incident responder to help them.

Cyber Crime Analyst

This role and it’s derivatives are not my expertise, so I’ll rely on someone else’s definition:

Investigates a number of crimes, ranging from recovering file systems on computers that have been hacked or damaged to investigating crimes against children. Recovers data from computers that can be used in prosecuting crimes in court. Computer crime investigators must also write reports for and testify in court. Also may work for large corporations to test security systems that are currently in place. Investigators do this by trying various ways to hack into the corporation’s computer networks. At corporations, computer crime investigators also maximize optimal computer system performance levels.

https://online-distance.ncsu.edu/career/cyber-crime-analyst-investigator/

In other words, someone in this role will investigate a cyber incident but will do so with a potential optimal goal of going to court. This involves careful analysis and evidence collection and handling.

Incident Responder

An incident responder is a technical person with a deep understanding of how a typical environment behaves that will be able to analyze events when an incident occurs. He/she will then have to understand if it’s a false alarm or if a complete response needs to be done. Because of the amount of knowledge required to be an excellent responder, I would divide this role in two and have one of the two as a mid-level role; that person would be the second line of defense to solve harder incidents.

In my opinion, this role is one that provides a lot of thrill as you are directly in the fire, trying to extinguish it. But because of this, it can also be stressful. This role requires some level of organization because there are reports to be done on what happened, what action was taken to remediate the incident, etc.

IT Auditor

In the modern era that we live in, enterprises must legally meet certain security criteria, depending on what information you handle. For example, an enterprise that handles credit cards must meet the PCI-DSS requirements. To make sure these requirements are met, an IT Auditor will do an audit (a PCI audit, in the case of PCI-DSS) of the policies, processes and procedures in place to make sure they meet the minimum criteria.

Mid-level roles

These roles assume you have good experience in InfoSec. They typically require 3-5 years of experience in InfoSec. This experience is mostly gained through the entry-level roles, but that’s not the only way.

Cybersecurity Analyst

A cybersecurity analyst is a mid-level expert in InfoSec that is able to do all (or most) of the entry-level jobs. A cybersecurity analyst can be asked to handle devices, respond to incidents, etc. They are the second line of defense for an incident. This role is really fun because you get to touch to pretty much all aspects of InfoSec for a single company. I had this role before the one I have right now and I enjoyed it because of the amount of things I learned in a short amount of time.

A cybersecurity analyst will also be asked to manage user awareness campaign (like phishing), choose the best fit for the company between multiple products (like an antivirus, a password manager, etc.), and many other tasks.

Cybersecurity Consultant

A cybersecurity consultant is a mid-level expert on a wide array of domains to help and advise clients (like the cybersecurity analysts in other enterprises) on cybersecurity subjects. A consultant can help find a strategy to mitigate a problem, review configurations, mitigate a vulnerability, select a product between multiple providers, etc. In this role you’re not part of an enterprise team but you provide your diverse expertise to your clients. The positive point is that you get to see many way of doing things and learn from it. The negative point is that, because you don’t go in the details as much as the cybersecurity analyst, you need to compensate on your own time or assume you won’t gain deep knowledge as quickly as an analyst on some specific subjects. In other words, a cybersecurity consultant jobs is to covers more broad but in less depth.

In my opinion, it is a great role if you’re curious to learn as much as possible on many different subjects. You’re essentially paid to learn and then use this knowledge to advise your clients.

Penetration Tester

A penetration tester also known as “pentester”, is someone that’s hired by enterprises to test the security of their network, applications, implementations, websites, etc. They use techniques used by hackers and then provide their clients with a report that explain where and how they were able to exploit a vulnerability, and they explain how to fix them. Enterprises use their services so they can patch the vulnerabilities before an malicious attacker can exploit them.

This job is really interesting and fun because you get paid to hack things. I had this job a few years back and I really liked it! It’s important to know that a good pentester is highly technical. That’s because to understand how to bypass or hack something, you need to understand how it works and how you can modify the intended behavior to make it do something unintended. The natural evolution of a pentester is to be a Red Teamer, which isn’t discussed in this post.

Advanced-level roles

Cybersecurity Manager

A cybersecurity manager is someone with manager skills but adapted to the cybersecurity context. This last part is important because that person could be asked to make decisions that would have an impact on the security of the enterprise. For example, that person could be asked to choose between accepting a risk or displacing resources towards a problem that needs to be resolved timely; and accepting a risk is not something that should be done lightly. The cybersecurity manager is the link between the technical teams and the upper management.

In order to get this role, you need a lot of experience in management and a lot of experience in cybersecurity. It is frequent to see technical persons from the mid-level roles move to this job at some point. This job is one of the jobs that could lead you to the Chief Information Security Officer (CISO) role, although in some enterprises where cybersecurity is not represented in the upper management, this is the highest role.

Cybersecurity Engineer

A cybersecurity engineer is the natural evolution of the cybersecurity analyst or consultant. It is a senior role that entails a lot of experience in cybersecurity, focused mainly on the technical aspects. In that role, you get to be the main advisor for less experienced teams to guide them towards the best solutions. Because of the changing nature of technology, you need to keep learning and stay ahead of the news so that role is ideal for curious people.

The name “engineer” is important in this role title: because of your experience, you could be asked to help developer teams and operation teams find the ideal solution to a problem and how to do it securely. This role is the “Sec” in “DevSecOps“.

Cybersecurity Architect

A cybersecurity architect is a senior role that requires a lot of knowledge. An architect needs to have an in-depth understanding of a wide area of domains, both technical and non-technical, to be able to design secure systems, evaluate where these systems could fail and mitigate these risks. In other words, a cybersecurity architect’s knowledge is both broad and in depth. That role also requires continuous learning of the new trends and technologies. The persons in this role need to be able to explain complex concept in a simple manner because they need to communicate the architecture to the management.

This is one of the most challenging cybersecurity jobs because security is hard. But it’s also one of the most interesting and rewarding for a technical person.

Conclusion

As you can see, the cybersecurity field is large, challenging and super interesting. As we can see in the news, this field is recognized as being more and more important so we will see an increase in demand for these roles and cybersecurity jobs offers.

I understand that it can be daunting to choose this path but I can reassure you, as long as you have passion, there is nothing to fear. Once you start in this path, you’ll discover that the community can be generous.

Note: If you find some people in the community to be judgmental, like someone saying something along the line of “RTFM“, don’t mind them. Some people like to flatter themselves by being elitist but in reality they mostly lack self confidence. If it happens to you, you can always write to me, I’ll be more than happy to help you.

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Lockbit is a “Ransomware as a Service” which means you can buy a powerful ransomware, launch it in your target’s network and Lockbit will take care of the rest. They even have support teams to assist victims into the payment and recovery steps. So I was proud that I could recover the data from a file encrypted with the Lockbit ransomware.

It is important to know that they are using the double extorsion tactic, which means that they exfiltrate the data they encrypt and then threaten to publish this data of the ransom is not paid.

The context

I was recently working on a Lockbit ransomware incident and I was able to recover the data from encrypted VMDK and VHDX files (virtual machines hard drives). I am really proud because this allowed us to get the backups back and we will be able to restore everything!

Note: Your mileage may vary! Lockbit is evolving quickly. But I think it is worth a try! If unsure, write to me. I would be really curious to know if it works everywhere!

Everything started when I was reading the blog post “LockBit 2.0: How This RaaS Operates and How to Protect Against It” looking for IOCs for my investigation. In this post, we can read:

The LockBit group claimed that LockBit 2.0 is “the fastest encryption software all over the world” and provided a comparative table showing the encryption speed of various ransomware samples.

Source:https://unit42.paloaltonetworks.com/lockbit-2-ransomware/#:~:text=LockBit%202.0%20is%20ransomware%20as,first%20observed%20in%20September%202019.

When I saw this, I immediately realized that the data might be only partially encrypted. Enough to effectively destroy most of the files, but allowing the ransomware to be more rapid. They are not the first to use this strategy. I had this realization because in cryptography, you can’t simply “have more speed”. So either they were using symmetric encryption (which means the key is in memory) because it’s faster, or they only partially encrypt files. The second option makes more sense because the first one would be a really bad design choice, and let’s admit: these groups are smart!

As soon I as realized this, I ran the “strings” command on an encrypted VHDX. And guess what? I could see a lot of known strings like “This program cannot be run in DOS mode”. This means that some part of the file is still intact and could maybe be restored!

The path to victory

Knowing this, I started looking for hints and ideas online to help me understand how we could get access to the intact data. This is when I found this video:

Youtube video: Recovery for vhdx file, encrypted with ransomware “lucky”

Even though this video is talking about the “lucky” ransomware, the idea is the same. What we need to understand from this video is that only the header of the VHDX file was encrypted (using Lockbit 2.0). This header is important because it holds information about the virtual drive (partitions, block locations, etc.) and without it, most software can’t read the data. So what they show in the video is how to essentially fake a corrupted but working VHDX file, allowing normal tools like 7-Zip to access the data. They do it by essentially replacing the first 2048 sectors of the drive with the data from an empty but working VHDX file using UFS Explorer Professional Recovery.

After this is done, you can save the “RAID” generated with the tool to a new VHDX file. This file won’t be readable with Hyper-V, but you’ll be able to open it with 7-Zip and get your data back.

I tried to use the same technique on a VMDK file, but it was not working. I believe this is because it was encrypted with Lockbit 3.0 which, I think, encrypts more data. Also, I think a VMDK file is less permissive than a VHDX file because it is more complex.

For the VMDK file, I could still see readable information but I could not see the magic bytes for a NTFS partition. I was looking for something similar to this:

For the VMDK file, I tried the “Scan for lost data” feature of the software and it worked! It could see all the files on the drive. It was not able to repair the partition so I was not able to rebuild a virtual drive, but using the paid version, I was able to extract all the files!

Note: the VHDX technique can be done without the paid version.

At the time of writing this post, I am not sure how Lockbit does this. Do they encrypt a percentage of every files, or do they encrypt the X first bytes of every files. My gut feeling (based on a few files I tested) tells me they are only encrypting the first bytes of every files. This could mean that any large file could be recovered, at least partially!

I do need more data to better understand how they operate, so please leave comments below or contact me if you have information!

Victory

The Lockbit ransomware is aggressive and I was hoping we could recover the data because they encrypted the backups.

To prove the technique was effective, I then proceeded to recover the data from 2 VHDX and 2 VMDK files to prove it was not a fluke. And it worked!

So next time I am working on a ransomware incident, I’ll remember to try the simple but effective “strings” command on the encrypted files!

Feel free to leave your comment down here for any questions or comments.

Donation

If you like my blog and my posts, please consider donating! Imagine this: each minute of reading a post takes about 1-2 hours to create!

Today I’m announcing a brand new addition to my Substack publication: Tristan’s Cybersecurity Substack subscriber chat.

This is a conversation space exclusively for subscribers—kind of like a group chat or live hangout. I’ll post questions and updates that come my way, and you can jump into the discussion.

Join chat

How to get started

1. Get the Substack app by clicking this link or the button below. New chat threads won’t be sent sent via email, so turn on push notifications so you don’t miss conversation as it happens. You can also access chat on the web.

Get app

2. Open the app and tap the Chat icon. It looks like two bubbles in the bottom bar, and you’ll see a row for my chat inside.

3. That’s it! Jump into my thread to say hi, and if you have any issues, check out Substack’s FAQ.