SIEM on Elastic Security: first steps with detection rules

SIEM (Security Information and Event Management) collects security events from across your environment, compares them with detection rules and raises alerts for the team to handle. Elastic Security does this on the same Elastic Stack where many companies already keep their logs: data goes into Elasticsearch, rules run in the detection engine and the analyst works in Kibana. Below we describe where to begin so that after a few weeks you have a working system rather than thousands of alerts nobody reads.

Start with questions, not with the tool

Before installing anything, settle three things:

1. What you want to detect. Account takeover, lateral movement in the network, unusual administrator logins, malware on workstations? The list of first scenarios should fit on one sheet of paper.
2. Where your data comes from. A rule without data detects nothing. To detect account takeover you need authentication logs (Active Directory, Microsoft Entra ID, VPN); for lateral movement you need Windows system logs and firewall logs.
3. Who responds to an alert. An alert with no owner and no procedure is noise. Decide who sees it, how quickly they respond and what they do first. If you do not have a team that does this around the clock, agree the scope and response times with your service provider.

Step 1: collect data and normalise its format

Data reaches Elastic Security through Elastic Agent managed from Fleet, together with integrations (for example for Windows, Linux, firewalls, cloud platforms and Microsoft services). Integrations write events in the Elastic Common Schema (ECS), which means shared field names such as source.ip, user.name and event.action. This matters because a single rule can then work on data from different sources.

For endpoint events there is also Elastic Defend, which detects and blocks malicious activity on workstations and servers, and whose alerts land in the same view as alerts from rules. Start with the sources of highest value: domain controllers, the edge firewall, email and cloud identity. Add others gradually, because each one also means storage cost and rule-tuning work.

Step 2: enable prebuilt rules for the data you have

Elastic maintains a library of prebuilt detection rules mapped to ATT&CK tactics and techniques (a public knowledge base of attacker techniques compiled by MITRE). Install them and enable only those for which you have data: a rule for Windows logs will not work without those logs, and a rule enabled “just in case” only burdens the cluster. Elastic advises starting with the tactics most important to you, validating a rule’s logic against historical data before enabling it, and then monitoring rule health (whether rules run without errors) and whether alerts are actually generated.

Step 3: know the rule types

Elastic Security has several rule types, and it helps to know what each is for:

Rule type	What it is for
Custom query	Matching events with a KQL or Lucene query. The simplest and most flexible type, good for known values and patterns.
Threshold	An alert when the number of matching events (grouped by fields) exceeds a limit. Fits password-guessing attempts.
Event correlation (EQL)	Detecting sequences of events over time that share fields. This is correlation: only several steps together make an alert.
Indicator match	Comparing event fields with an index of threat indicators (IP addresses, file hashes) and enriching the alert.
New terms	An alert when a value (for example a login country or a process) appears for the first time in a history window.
Machine learning	Deviations from normal behaviour found by machine learning jobs, which you must create first.
ES|QL	Rules based on piped queries that aggregate data before alerting.

The availability of some features depends on your Elastic licence (subscription). Before you plan a specific rule type, check in the Elastic documentation that your subscription covers it.

Step 4: write a few custom rules

Prebuilt rules do not know your company. For your first custom rules, choose scenarios where context matters:

• Password-guessing attempts. A threshold rule: more than a set number of failed logins to the same account from one address in a short time window.
• Successful login after a failure. An EQL correlation rule that links a failed and a successful authentication of the same user within one time window (a query sketch follows the list).
• First administrator login from a new country. A new terms rule on the country field for a group of privileged accounts.
• Contact with a list of known malicious addresses. An indicator match rule on firewall traffic against an index of threat indicators.

An EQL sketch for the second scenario:

sequence by user.name with maxspan=10m
  [ authentication where event.outcome == "failure" ]
  [ authentication where event.outcome == "success" ]

This is only a skeleton. Without a threshold for the number of failures and an exclusion for known service accounts, it will generate many false alarms, so tuning is part of the job.

Give each rule a severity, a description and a short note on what to do after an alert, and map it to an ATT&CK technique. After a few months you can then see which areas are covered and which are not.

Step 5: reduce noise

The most common reason people abandon a SIEM is too many false alarms. Elastic Security has several tools for this: exceptions, which exclude specific known behaviour from a rule; alert suppression, which groups repeated alerts; and snoozing. The rule of thumb: add exceptions narrowly (a specific host and process), not broadly (a whole subnet). After the first two weeks, review all alerts, judge which were justified and fix the rules. Repeat this regularly, because the environment changes.

Step 6: set up a process for handling alerts

In Kibana, alerts land in a shared view from which the analyst opens details, builds a timeline of events (Timeline) and creates a case assigned to a person. Agree a simple process: who reviews alerts and how often, how you classify an alert (true, false, to watch), when you escalate and how you record conclusions. After an incident, add or fix the rule that would have caught it sooner.

What a SIEM will not replace

A SIEM detects but does not protect on its own. It does not replace updates, backups, multi-factor authentication or network segmentation. It is also only as good as its data and the people who look at it. That is why the cycle described above (data, rules, tuning, process) matters more than the number of enabled rules.

We describe collecting data from applications and infrastructure with the OpenTelemetry standard in our article on OpenTelemetry alongside Elastic and Grafana. We deploy and run SIEM on Elastic Security, including the choice of data sources and rule tuning, as part of our SIEM and security service.