Shieldlist/shieldlist-agent
Watch
1
0
Fork
You've already forked shieldlist-agent
 0
Code Issues Pull requests Projects Releases 4 Packages Wiki Activity Actions
Page: Configuration
Pages
Abuse reporting Commands Configuration Enforcement Enrolment Getting Started Home Replay Sizing and limits Troubleshooting Writing rules
13 Configuration
Robin edited this page 2026-08-22 23:15:50 +00:00
Table of contents

Configuration

YAML, one format, hand-written as the primary interface. Everything that has a sensible default has one. Loading is atomic: a configuration is valid whole or refused whole, and shieldlist-agent -t reports every problem at once with its file and line.

The configuration reference in the controller repository is the authority for the format. This page is the working guide.

agent.yaml

What this machine watches and what it allows itself. Every key is optional; an absent file means a standalone agent with defaults that watch nothing.

mode: local                  # local (default) or enrolled — see Enrolment
controller: https://...      # required when enrolled, forbidden otherwise
state: on                    # on (default) or test
level: standard              # scales every rule's threshold here: instant | strict | standard | lenient
log_file: /var/log/shieldlist/agent.log   # as well as the journal

logs:
  - path: /var/log/nginx/access.log
    log: http                # the kind of line
    ports: [80, 443]         # the service's ports here — what `ports: service` in a policy bans
  - path: /var/www/vhosts/system/*/logs/proxy_access_ssl_log
    log: http
  - unit: ssh.service
    log: ssh
    ports: [22]

limits:
  memory: 50%                # or "512MB", "2GB", "1TB"

replay_on_start: 6h          # off unless set

state — test (older files say detect-only: read as the same) puts every rule in test and keeps the agent off the data plane entirely: no nftables table, no restore, nothing added or removed. It decides, records and reports exactly as it would otherwise, with every sanction marked unenforced.

That is how this gets adopted. Run it for a day beside fail2ban, CrowdSec or whatever already guards the machine, read what it would have done, then set on and restart. Doing the same by editing every rule's own state is how one gets missed.

While it is set, shield status and shield stats answer enforcement DETECT-ONLY before any count, and the figures they show are the records the agent made rather than bans it holds — otherwise every number would read zero beside a log full of decisions. shieldlist-agent -t still checks that this kernel could enforce, which is the only way an agent in this mode ever finds out.

log_file — where the agent writes its own log, as well as to standard error, which under systemd is the journal. Unset means the journal alone; journalctl -u shieldlist-agent -f has the same lines either way.

It is set in the shipped configuration because /var/log is the first place anyone looks after starting a security daemon, and finding nothing there reads as "it did not start". The lines written before the configuration was read — the version, what loaded, the memory ceiling — are replayed into the file when it opens, so it holds the whole run.

There is no logrotate snippet to install. The agent identifies the file by (device, inode) like every other file it follows, notices when logrotate has renamed it away, and reopens. The shipped unit sets LogsDirectory=shieldlist, which is what makes the directory writable under ProtectSystem=strict — point log_file somewhere else and you must open that path yourself in the unit.

logs — each entry names a file (path) or a journald unit (unit), never both, plus the kind of line it holds (log: the word whose parsers apply to it) and, optionally, the service's ports on this machine — the one place they are true. One source per kind per machine: reading the same lines from a file and from journald counts everything twice, and the validator refuses it.

A path may be a pattern (*, ?, […]), which is how a machine hosting sites is watched at all: a panel gives every site its own log directory, and sites are added and removed by people who are not thinking about this agent. A hand-written list of paths goes stale in silence, which is the one failure mode this program must not have.

The pattern is re-read every fifteen seconds. A file it starts matching is read whole — it is all present, like a rotation's successor — and one it stops matching, once the file is really gone, is let go. Two patterns matching the same file is fine: it is read once. shieldlist-agent -t prints what each pattern matches on this machine right now, and says so when a pattern matches nothing.

{target} — what a file is about. A pattern may name one segment {target}: it globs like *, and the text it stood for becomes the target of every line read from that file — the site, the domain the attack was aimed at:

logs:
  - {path: "/var/www/vhosts/system/{target}/logs/access_log", log: http, ports: [80, 443]}
  - {path: "/var/log/nginx/{target}.access.log", log: http, ports: [80, 443]}

(Quote the path when the entry is written in flow style — braces inside {…} are YAML's, not the pattern's.) Every hit then carries target=example.org beside what the parser captured: the controller counts the most attacked sites, the console filters on one, a rule may distinct: target. A line that names a user with a domain (user=jo@example.org in a mail or webmail log) gets target=example.org without any pattern — a password guessed on a mailbox is an attack on that domain. A parser that captures a group named target itself wins over both.

limits.memory — the most the agent may ever hold, as a share of the machine or an absolute size. Half the machine by default, which is a lot for a guest program and is deliberate: evidence it cannot hold is an attacker it cannot convict.

A ceiling and never a reservation — it takes only what its rules need — and a fixed one: the agent does not watch how much the machine has free and does not shrink when something else grows. See Sizing and limits.

replay_on_start — read this much of the logs' past at startup and enforce what still applies. Off unless asked for; see Replay.

parsers/*.yaml

A parser recognises something in one kind of log line. One per file; sub-directories (parsers/nginx/, parsers/sshd/) are for people.

parser: ssh-auth-fail
log: ssh
prefilter: "Failed password"
regex: 'Failed password for (?:invalid user )?(?P<user>\S+) from (?P<ip>\S+)'
time_field: time             # optional, with time_format
time_format: syslog

prefilter is mandatory. It is a literal substring that must be present before the regex is allowed to run, and it is the whole performance contract: the overwhelming majority of lines are rejected by a substring check that never touches the regexp engine. A parser without one is refused (prefilter_any: [a, b] for a list of literals).

regex is RE2 — Go's regexp, never a backtracking engine, so a regex cannot be made to hang. Named groups become the parsed fields. (?P<ip>…) is the one every rule expects: whom to ban.

time_field / time_format say where the line carries its own timestamp. Both or neither. A live agent never uses them — it stamps a line with the moment it read it, which is the same instant — but a replay does, and without them a week-old file claims to have happened now. Formats: nginx, apache, syslog, rfc3339, unix, or a Go layout. syslog prints no year, so the reader supplies one and a line dated ahead is read as last year's.

rules/*.yaml

One rule, one file, five blocks in the order things happen:

rule: ssh-bruteforce
description: "Repeated password failures from one address"
category: ssh                # where it is filed — for people and the console
parsers: ssh-auth-fail       # what it recognises: a name, a list, weighted entries, or a regex written here
threshold: 5/10m             # from how much: five hits in ten minutes — or [5/1m, 20/1h], the first crossed fires
ban: standard                # what it does: a ban policy — and/or notify: warning
report: ssh-intrusion        # what it says: a report preset (the built-in default when absent)
count: address               # per address (default) or network (/24)
state: on                    # on | test | off

ban and notify combine: a rule may ban, notify, or both; with neither it only records. test records everything and sanctions nothing — the way to watch a rule in production before arming it. A rule writes its numbers once; the machine's level scales them (instant, strict, standard, lenient). See Writing rules.

policies/*.yaml

What a ban is, named once and reused rather than written into every rule.

policy: standard
durations: [5m, 1h, 24h, 30d]   # first offence, second, third, fourth — the last repeats; or one: 24h
ports: all                      # all (default) | service (the machine's ports for the rule's log) | [80, 443]
memory: 30d                     # how long the machine remembers an address after its ban ended
tighten: 2x                     # optional: each return within memory divides the hits needed
reputation: {above: 90, duration: 30d}   # optional: the controller lengthens the ban when AbuseIPDB agrees
duration_start: application     # application (default) or evidence

durations is the ladder, one rung per repeat offence. The last rung repeats for ever after.

memory is how long an offence counts towards the ladder, measured from the moment the last ban ended — an offender cannot reoffend while banned, so a long ban does not consume its own memory. Past a full quiet period, a returning offender starts at the first rung again; a repeat is judged per address whatever the rule. No memory means the machine never forgets.

ports is one scope for every rung: everything (the usual), the service's ports as the machine declares them on its logs, or a list.

duration_start decides when the clock starts: at the moment the ban is applied, or at the evidence that earned it. The second matters when the agent is behind, or when the ban comes from a replay.

tighten (optional, e.g. 2x) makes a returning offender easier to convict: each past episode within memory divides every rule's threshold count by the factor, never below one hit — ten hits the first time, five the second, three the third.

reputation is the controller's: after a ban, it asks the provider and, when the address's confidence score is at or above above, orders the machine the longer duration — after the fact, never in the path of a sanction.

trust.yaml

A graduated weight on where traffic comes from, between exemption and neutral. Trust adjusts how much evidence is needed; it never removes logging.

- origin: FR                 # a country, an ASN, or a range
  require: 3x                # three times the usual evidence
  range_ban: never           # never widen a ban inside this origin

- origin: AS64500
  require: 0.5x              # half the usual evidence

Country and ASN origins need the enrichment databases and are inactive until then; range origins work today.

reports/*.yaml and secrets.yaml

Abuse reporting: what a rule's bans say to AbuseIPDB, and the key that signs it. One preset per file in reports/, referenced by a rule's report:; the key and your daily_limit live in secrets.yaml (0600). The whole subject has its own page.