8 Writing rules
Robin edited this page 2026-08-22 23:15:50 +00:00

Writing rules

Two concepts do the work: a parser recognises something in a line, a rule counts what parsers find and decides. Everything else parameterises them.

Start from the line

Take a real line out of your logs and write the parser against it. The literal prefilter first — it is mandatory, and it is what keeps the agent cheap:

parser: wp-login-post
log: http                       # the kind of line it reads — the word the machine's logs declare
prefilter: "/wp-login.php"
regex: '^(?P<ip>\S+) \S+ \S+ \[[^\]]+\] "POST /wp-login\.php'

Every line without /wp-login.php in it is discarded by a substring check. Only the survivors meet the regex. On a busy web server that is the difference between one core and ten.

Several parsers may match the same line, by design. The base parser of a log matches every line and extracts its fields; more specific parsers recognise particular behaviour on top. Ports are not a parser's business: the machine declares them on its logs (agent.yaml, logs: [{path, log, ports}]), where they are true.

Count with a rule

rule: wp-bruteforce
category: web            # where it is filed, for people
parsers: wp-login-post   # what it recognises
threshold: 10/5m         # from how much: ten hits in five minutes
ban: standard            # what it does: a ban policy
report: wp-bruteforce    # what it says (default when absent)

Ten posts to the login form from one address inside five minutes. The window is a sliding one over event times, not a bucket. That is the whole rule — five blocks in the order things happen — and it is the bar: what a person writes by hand in six lines.

count is what "one offender" means: address by default, or network to count a whole /24 together — useful against an attack spread thinly across many addresses of one prefix. asn and country are accepted so the same files carry to the controller era, but load inactive today: the validator, the startup log and a dry run all say so.

Every scoring parser must capture the grouping field — a rule grouped by ip whose parser has no (?P<ip>…) group could never fire, and the validator refuses it rather than blessing a dead rule.

distinct: <field> counts distinct values instead of hits, which is how you say "tried ten different usernames" rather than "failed ten times".

Scoring: several behaviours, one rule

parsers also takes a list, and each entry carries a weight. This is the same model as the simple case — a bare name is weight 1 — and it is what lets one rule accumulate varied behaviour:

rule: http-scanner
parsers:
  - parser: env-probe
  - parser: xmlrpc-post
  - parser: plugin-scan
    weight: 2
threshold: 5/10m
ban: standard

Any mix reaching five points convicts. What matters is the shape of the visit, not one path.

Negative weights argue against a ban

rule: mail-auth-balance
parsers:
  - parser: mail-auth-fail
  - parser: mail-auth-success
    weight: -3
threshold: 10/2h
ban: mail-standard

A successful login subtracts three failures' worth. Someone with a broken client who eventually gets in is not an attacker, and this says so in the rule rather than in an exception list.

Negative weights change the rule's timing: because late evidence could turn a verdict around, such a rule's crossing waits out the lateness allowance before it becomes a sanction. Positive-only rules fire at once.

Try it before you arm it

This is the part worth the habit. Ask what the rule would have done to the logs you already have:

shieldlist-agent -dry-run 24h -rule wp-bruteforce

It prints every crossing with the lines that caused it, and enforces nothing:

read	48210 lines from 2 files
recognised	1204
would ban	3
too old to matter	11

2026-08-13T21:30:16Z  wp-bruteforce  203.0.113.9  would ban until 2026-08-14T03:30:16Z
  | 203.0.113.9 - - [13/Aug/2026:21:29:16 +0000] "POST /wp-login.php HTTP/1.1" 200 1500 "-" "Mozilla/5.0"

Reading the past needs the parser to declare time_field — otherwise every line claims to have happened now. The dry run says how many lines it had to skip for that reason.

Then run it as state: test for a while. It records everything and sanctions nothing, so you see it against live traffic before it can hurt anyone.

The regex in the rule

A rule with no reason to share its regex keeps regex and rule in one file: an entry of parsers writes regex and prefilter instead of naming a parser, and log on the rule says which log those regexes read. Shared parsers mix in freely.

rule: wp-xmlrpc-flood
category: web
log: http
parsers:
  - regex: '^(?P<ip>\S+) .* "POST /xmlrpc\.php'
    prefilter: xmlrpc
    weight: 2
  - http-404
threshold: 10/1m
ban: web

The loader makes a parser of each such entry, named <rule>#<n>, and that is the name shield why and the console show. Everything a parser file says — prefilter, (?P<ip>…) — holds here too.

Packs

packs/<name>.yaml names a set of rules together — pack: plesk, description, rules: [wp-login, wp-xmlrpc, …] — the set for one type of server. On a machine of its own every rule in the tree runs and a pack is documentation; on a controller a pack is what a server is given, as a whole (see the controller wiki). A rule is never copied, only listed.

Levels: one rule, several kinds of machine

The same failed-login rule wants three attempts on a hardened box and ten on a shared-hosting server, and writing it twice is how the copies drift. So a rule writes its numbers once, and the machine's level scales them — the same way for every rule, in a way anyone can predict:

level what it does to every threshold
instant the first hit convicts (one point, the shortest window)
strict half the hits (rounded up, never below one)
standard as written — the default
lenient twice the hits

level: strict in a machine's agent.yaml, or its declaration on the controller (there, per pack and per rule too). Enrolled, the controller's level is the one that counts: the numbers arrive already scaled. Mind instant on rules with negative weights: one positive hit is enough there too.

Thresholds worth thinking about

  • A rule that never fires holds everything. If the threshold is far above what the traffic produces, every matching line stays buffered for the window plus the lateness allowance. That is memory. Prefer a threshold the traffic actually reaches.
  • A short first rung beats a long one. The ladder escalates on repeat; five minutes stops a scanner and costs a mistaken ban almost nothing.
  • Match narrowly, ban broadly. A precise parser and a generous policy is safer than a vague parser and a timid one.

What a rule may not do

Rules are data. There is no path from a rule to a command, a shell, a plugin or a file path — sanctions are a fixed set compiled into the binary, and a rule selects and parameterises one. This is what makes a rule safe to accept from somewhere else.

By target — the site or domain a hit was aimed at

Every hit can carry a target: the site whose log file it was read from (a {target} segment in the log input's path — see Configuration), or the domain of the user it named (user=jo@example.org). A rule may then count only hits aimed at some targets, or leave some out:

rule: http-probing

except_targets: [cloud.*, webmail.*]     # Nextcloud and webmail instances make their own 404s — not probing there
rule: shop-checkout-flood

targets: shop.example.org

Names, or patterns with * (a * does not cross a dot). One of the two keys, never both. A hit with no target counts for except_targets (nothing says it was aimed at an exception) and not for targets (nothing says it was aimed at one of them). The target is also a field: distinct: target, and the controller counts the most attacked sites.