Attending the Network Security Monitoring Operator: A Journey Through Network Security and Incident Response

Recently, I had the privilege of attending the Elasticsearch Security Operations training, an in-depth course designed to strengthen understanding of network security monitoring, incident detection, and analysis using tools like Suricata, Zeek, and Kibana. This five-day course offered a comprehensive look at the mechanisms behind network security and how to effectively detect and respond to security incidents. Here’s a breakdown of the week:

Day 1: Network Security Fundamentals and Storage Considerations

The training kicked off with an extensive overview of network security, with a special focus on how to monitor traffic efficiently and the storage requirements involved. We explored how different levels of network monitoring—like PCAP (Packet Capture), IDS alerts, and session metadata—affect the amount of storage needed.

PCAP vs IDS vs Session Info:

PCAP (Packet Capture): Offers the most detailed level of information, capturing every packet sent over the network, but requires substantial storage, especially on busy networks.
IDS Alerts (Intrusion Detection System): Captures security-related events based on predefined rules, like those generated by Suricata. This uses less storage than PCAP but only retains data for events triggered by the rules.
Session Info: Retains metadata about connections (like what you’d find in Zeek’s conn.log), which offers valuable insight into traffic patterns without the heavy storage overhead of PCAP.

Network Interfaces, Protocols, and Devices

We also dug into the fundamentals of network interfaces and how different protocols and devices contribute to overall network security. Understanding the various network layers and how they communicate—especially through routers, switches, and firewalls—is key to setting up an effective monitoring environment.

OSINT and Wireshark Filtering

Later in the day, we delved into the OSINT (Open Source Intelligence) layer, discussing how publicly available information can enhance network monitoring by enriching logs with external context (e.g., domain reputation, threat intelligence).

We also touched on Wireshark and how to use its powerful syntax to filter traffic effectively. A few examples included:

ip.addr == 192.168.1.1 (Filter by IP address)
tcp.port == 443 (Filter by protocol)
ip.src == 172.16.0.0/12 (Display only traffic from a specific subnet)

Day 2: Diving Into Suricata Rules

Day 2 was all about Suricata, a powerful open-source intrusion detection and prevention system (IDS/IPS). We learned how to write custom Suricata rules for detecting specific patterns of network traffic. Writing Suricata rules involves specifying conditions that trigger alerts when traffic matches certain patterns.

Example Suricata Rules:


1. alert tcp any any -> any 22 (msg:"SSH Connection Detected"; sid:1000001; rev:1;)

2. alert http any any -> any any (msg:"HTTP GET Request Detected"; content:"GET"; http_method; sid:1000002; rev:1;)

3. alert dns any any -> any any (msg:"DNS Query for Malicious Domain"; content:"badsite.com"; sid:1000003; rev:1;)

Available Options for Suricata Rules:

Option	Description
msg	Message displayed when the rule is triggered.
sid	Unique identifier for the rule.
rev	Revision number of the rule.
content	Specifies the string or pattern to match in the payload.
http_method	Matches HTTP methods (e.g., GET, POST).
pcre	Use Perl-compatible regular expressions for pattern matching.
flow	Specifies the direction of the traffic (e.g., to_client).

Day 3: The Anatomy of Zeek Rules and Efficient Log Filtering Techniques

On Day 3, we turned our attention to Zeek (formerly Bro), a powerful network analysis tool. Zeek logs provide rich data about network events, and we explored several key logs that offer deep insight into network traffic:

conn.log: Logs all network connections, including details like source and destination IPs, ports, and the status of the connection.
http.log: Logs HTTP requests and responses, providing insights into web traffic, including methods, URIs, and user-agents.
files.log: Logs information about files seen on the network, including details like MIME type and MD5 hashes, which can be crucial for detecting malware.

Understanding Zeek Rules

Zeek rules function differently from traditional intrusion detection systems like Suricata. Instead of triggering specific alerts, Zeek rules act more like scripts, processing network traffic and generating rich logs. Zeek’s flexibility allows us to create custom scripts to monitor various network activities, such as detecting anomalies, extracting files, or logging unusual connection patterns.

We looked at different types of Zeek rules, such as:

Connection-based rules: These rules focus on logging specific types of network connections. For example, monitoring SSH traffic by analyzing ports and connection duration.
File-based rules: These are geared toward identifying and tracking file transfers across the network. This is especially useful when monitoring potential data exfiltration or malware distribution.

With Zeek, rules are highly customizable, enabling analysts to collect meaningful data from traffic flows. But, once you have vast amounts of log data, the next challenge is how to sift through it efficiently.

Log Field Information:

conn.log: Fields like id.orig_h (originating host), id.resp_h (responding host), proto (protocol), and conn_state (connection state) are key to analyzing connection behavior.
http.log: Fields like host, uri, status_code, and method help analysts understand web-based interactions.
files.log: Fields like md5, source, and total_bytes are essential for understanding file transfers over the network.

Day 4: The Anatomy of Kibana

On Day 4 of the Elasticsearch Security Operations training, we dove deep into Kibana dashboards and the new Lens functionality and discussed how to build them effectively to enhance our monitoring capabilities. But it wasn’t just about dashboard creation; we also explored the importance of using simple yet powerful command-line tools like awk, cut, sort, and uniq to quickly parse and analyze log files from both Zeek and Suricata.

Efficient Log Filtering Using AWK, CUT, SORT, and UNIQ

Once we have captured network data using Zeek or Suricata, analyzing the logs effectively is key to identifying security incidents quickly. That’s where simple UNIX command-line utilities come into play. We learned how to use awk, cut, sort, and uniq in combination to create a powerful “sort sandwich” to extract meaningful information from logs fast.

Here’s how each of these commands can be used to speed up log analysis:

awk: A text processing tool that allows you to extract specific fields from a log file.
cut: Used to cut out sections from each line of a file, often by specifying delimiters.
sort: Sorts lines of text alphabetically or numerically, an essential step before using uniq.
uniq: Removes or reports duplicate lines from a file, which is incredibly useful when you need to see unique entries, such as distinct IP addresses or URLs.

Example: Using a Sort Sandwich to Filter Zeek Logs

Let’s say you want to extract all unique IP addresses from Zeek’s conn.log file to identify all IPs involved in network connections. Here’s how you can create a “sort sandwich” to do it efficiently:

Extract the IPs using awk:
```
awk '{print $3}' conn.log
```
This command extracts the third column, which typically holds the source IP address in conn.log.
Pipe the results through cut (if you need to remove unwanted portions of the output):
```
awk '{print $3}' conn.log | cut -d' ' -f1
```
This command helps clean up the output by removing extra fields or delimiters if necessary.
Sort the extracted IP addresses:
```
awk '{print $3}' conn.log | cut -d' ' -f1 | sort
```
Sorting is crucial because uniq only removes consecutive duplicate lines, so sorting ensures that all duplicates are adjacent.
Remove duplicate IP addresses with uniq:
```
awk '{print $3}' conn.log | cut -d' ' -f1 | sort | uniq
```
Now you have a list of unique IP addresses involved in network connections.

Example: Finding the Most Common Source IP

Sometimes, you need to find the most frequent source IP in the log. Here’s how you can do that:

Extract source IPs:
```
awk '{print $3}' conn.log
```
Sort the IPs:
```
awk '{print $3}' conn.log | sort
```

Use uniq -c to count occurrences:

awk '{print $3}' conn.log | sort | uniq -c

Sort the results numerically to find the most common IP:
```
awk '{print $3}' conn.log | sort | uniq -c | sort -nr
```
The -n flag sorts numerically, and -r reverses the order, showing the most frequent IP addresses at the top.

Why Use This Approach?

Using awk, cut, sort, and uniq in this structured way allows you to filter and analyze large log files in seconds, helping you get quick answers from your Zeek and Suricata logs without the need for heavy graphical interfaces. The simplicity and speed of these tools make them a must-have for any security analyst dealing with large volumes of data.

Day 5: Capture the Flag (CTF)

The final day was a hands-on Capture the Flag (CTF) challenge. We were presented with four different attack scenarios, each simulating a real-world incident. Using Kibana and the logs from Zeek and Suricata, we had to analyze the attacks and answer a series of questions.

Each scenario tested our ability to:

Investigate network anomalies.
Identify specific attack vectors.
Correlate logs from different sources to uncover the attack details.

Participants who scored over 3,000 points successfully completed the CTF and passed the course. I’m happy to report that I passed, having honed my skills in identifying and mitigating network security threats!

Final Thoughts

The Elasticsearch Security Operations training was an enriching experience. I gained practical skills in using tools like Suricata, Zeek, and Kibana to monitor network security and respond to incidents effectively. Whether crafting custom rules or analyzing network logs, this course provided me with a strong foundation in network security operations.

Looking forward to applying these new skills to real-world scenarios!

Course instructor: Roger Galobardes