18 Configuration
Robin edited this page 2026-08-24 10:08:43 +00:00
This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Configuration

The controller's tree is an agent's tree plus two things. Every key, with its default, is in the configuration reference; this page is the shape of it.

/etc/shieldlist/controller/
  controller.yaml         the service itself
  parsers/<source>/ rules/ policies/ reports/   the LIBRARY: every parser, rule, policy, preset — once
  packs/<name>.yaml       a PACK: a named list of rules — what a server is given
  agents/<hostname>.yaml  one machine: its packs, the rules it adds, what it changes, its logs
  trust.yaml exemptions.yaml exemptions.d/      the fleet's, to every machine
  exemptions-cli.yaml     what the console, the API and shield exempt
  secrets.yaml            the controller's own provider credentials

Packs: which rules run where

A pack is a list of rules for a type of server — plesk, infra, docker, mysql — and the unit a server is given: packs: [plesk, infra] in its declaration. A rule is never copied, only listed: a server runs the union of its packs (and the rules its declaration names on their own), each rule once, and receives only what those rules need — their parsers, their policies, their presets. A server given nothing runs nothing. In the console: Rules → Packs is a grid of packs × servers to tick, and a server's Rules tab has its packs (each on or test, at the server's level or its own), the rules added on their own, and everything it runs with the threshold it crosses on there.

Where addresses are from

geoip:
  source: dbip            # the free DB-IP lite country + ASN databases, fetched monthly under <state>/geoip
# or files of your own (GeoLite2 or any MaxMind-DB provider):
#  country: /var/lib/geoip/GeoLite2-Country.mmdb
#  asn: /var/lib/geoip/GeoLite2-ASN.mmdb

Off unless configured — a fetch is outbound traffic. Sanctions are enriched as they arrive (country, AS number and name) and rows from before are backfilled; the console shows a flag and the network, filters and counts by them; the API answers country, asn, as_name. DB-IP's licence (CC BY 4.0) asks for attribution: the console shows "IP Geolocation by DB-IP".

controller.yaml

listen:
  agents: 0.0.0.0:17453     # the agent transport
  api: 127.0.0.1:17454      # the API and the console
state: /var/lib/shieldlist/controller
log_file: /var/log/shieldlist-controller/controller.log   # its own log (the unit's LogsDirectory), as well as the journal; notices logrotate by itself
tls:                        # absent: self-signed, pinned by agents through the join token
  cert: /path/fullchain.pem #   the operator's own certificate…
  key: /path/privkey.pem
  off: false                # …or plain HTTP behind a proxy on a private address
level: standard             # the fleet's default level
reporting: agent            # who reports a machine's sanctions: agent | controller | off
lift: owners                # who may lift a machine's bans: owners | operator
sources: [googlebot, bingbot, cloudflare]   # published exemption lists to fetch
sources_refresh: 24h        # on this schedule; unset = on command only

Everything is optional; an absent file is a controller on one machine.

agents/.yaml

The file is named after the machine's hostname, its identity — the hostname as the machine reports it (hostname there: web1.example.net or web1; the console's Servers page shows it). A machine without a file runs nothing until it is given packs.

agent: srv00
name: srv00                      # what the console calls it — default: the short hostname when no other machine shares it
description: the infrastructure host
level: strict                    # scales every rule here: instant | strict | standard | lenient
packs: [openssh, plesk, srv00]   # the packs it runs — or, per pack, how:
# packs:
#   plesk: {}
#   infra: {state: test, level: strict}
state: on                        # on | test — relaxes the machine, never arms it
fleet:                           # the pack is the unit of sharing: a ban travels
                                 # only to servers running a pack that holds its rule
  offer: no                      # its bans go to the servers sharing its packs (default yes)
  enforce: no                    # it applies the bans of the servers sharing its packs (default yes; lifts and orders always arrive)
lift: operator                   # only the operator may lift its bans
reporting: agent                 # agent | controller | off
rules:                           # what this machine changes about a rule — or adds
  ssh-bruteforce: {level: lenient}
  http-bad-ua: {threshold: [3/1m, 10/1h]}
  wp-scan: {state: off}
  mail-auth: {ban: mail-hardcore}
  root-login:                    # a rule outside its packs: runs here as written
logs:                            # where its logs are (pushed; empty: its own agent.yaml says)
  - {path: /var/log/nginx/*.log, log: http, ports: [80, 443]}

An entry under rules: changes what one machine runs of one rule — its level, its threshold outright, its state, its policy — and, for a rule outside the machine's packs, adds it. Parsers, policies and presets are shared, never overridden: a machine that needs a different regex needs a different rule.

Who reports

reporting: — the fleet's default in controller.yaml, each machine's own in its declaration — says who submits a machine's enforced sanctions to the abuse provider: agent (the machine itself, with the key in its own secrets.yaml), controller (the controller on its behalf, with the key and daily_limit in the tree's secrets.yaml — one account and one reporting address for the fleet), or off. A machine told controller pauses its own sender; the controller then reports each of its sanctions once, as it arrives, composed from the rule's report preset and the fields the convicting lines named — the same words the agent would have used — and keeps the outcome on the sanction (the console says "reported by the controller"). What was not enforced is never submitted, whoever would report; a report the provider refused or the allowance could not cover is not retried; a lift "with withdraw" takes back the controller's own report where it was the controller that reported.

# secrets.yaml — beside controller.yaml, 0600
abuseipdb:
  key: your-api-key
  daily_limit: auto           # your plan, or a number below it; auto learns it from the provider
  # enabled: false            # keep the key, send nothing (Settings → Reporting writes this)

Levels

A rule writes its numbers once; the machine's level scales them, the same way for every rule: instant — the first hit convicts; strict — half the hits; standard — as written; lenient — twice the hits. Set for the fleet, per server, per pack on a server, per rule on a server. See the agent wiki's Writing rules.

Assembly

For each machine the controller assembles the tree it will run: the rules of its packs and the ones it adds, the parsers they score, the policies and presets they name, the fleet's exemptions — the rules that differ rewritten in place (comments kept) with the resolved threshold, state and policy — validated with the agent's own loader before it leaves. An agent older than 0.1.0 (the format's first release) still syncs but is refused a bundle by name and keeps running what it last received. shieldlist-controller -t shows every machine's assembly and what differs. A tree written in the 0.0.x format is named by the loader: shieldlist-controller -config DIR -convert NEWDIR rewrites it, comments kept.

Lists: block or exempt a network, a country, prefixes

lists/<name>.yaml names a set of addresses and says what it does where:

list: as398781
description: Oculus Networks — proxy network, flood of 2026-08-21
asn: 398781              # or country: CN, or prefixes: [203.0.113.0/24, 198.51.100.7]
block: fleet             # a preventive firewall ban of every prefix — the fleet, or [host01, srv02]
exempt: [ns1, pack:web]  # never sanctioned on these machines and on every machine running the web pack (the key was `allow` once; still read)
duration: permanent      # the blocks' duration: permanent (default), 30d, 1y…

A list may instead name a published list:

list: malicious-ip
url: https://raw.githubusercontent.com/romainmarcoux/malicious-ip/main/full-40k.txt   # or urls: [a, b]
format: lines            # lines | cidr-lines | json-prefixes | ips-anywhere
refresh: 1h              # 10m at the least; 24h when absent
attribution: Aggregated by Romain Marcoux — MIT licence
block: [pack:web]

It is fetched over https only, on its cadence and on Refresh now, read into prefixes and nothing else, bounded (a million entries, nothing wider than a /8 or an IPv6 /20, a sixteenth of the internet in all, public addresses only — a line that is not an address refuses the whole fetch, and a failed fetch keeps the last good file under <state>/lists/), then pushed to the machines in its block scope as datalists/<name>.set in their bundle, held in sets of their own on the agent (see the agent's Enforcement page): no orders, no sanctions, a diff at every refresh. With an exempt scope it is an allow list (its prefixes travel as exemptions; at most 20 000). A catalogue ships — preset: malicious-ip in the file, or the console's Published blocklists — each entry with its attribution.

A scope is fleet, or a list mixing machine names and pack:<name> items — the machines running that pack, the ones given it later too. Search engines exempted on the web hosts and never on the DNS servers is exempt: [pack:web].

Block travels as DATA, never as orders: the list's resolved prefixes are published as one nftables set (lists/<name>.set in the bundle) that each machine in the scope loads whole — seconds for tens of thousands of prefixes, one named drop counter per list, no sanction, no expiry; removing the block (or the list) withdraws the set. The machine's own exemptions are carved out of the set before it loads, so an exemption always wins. An autonomous system is resolved to its prefixes from the ASN database, a country from the country database (geoip in controller.yaml), at most 100 000 prefixes per list. Exempt travels in the bundle as exemptions. A machine's own exemptions always win over a block. The console's Lists page is the remote control; any network or country opened from the rail offers Block…. The feed leaves the list-driven bans out (copies-like: GET /v1/sanctions?list_blocks=include shows them).

Published allow lists (sources:)

Search engines and CDNs publish their address ranges. Naming a source in controller.yaml lets the controller fetch it, on the schedule and on command (shield refresh cloudflare, the console's Refresh now). The catalogue is compiled in — googlebot, google-special-crawlers, bingbot, applebot, duckduckbot, openai-searchbot, openai-chatgpt-user, openai-gptbot, cloudflare, fastly, jetpack, ahrefs, uptimerobot, pingdom, stripe-webhooks, and the presets private-ipv4, private-ipv6, carrier-nat — nothing is fetched unless named, the result is a generated file under exemptions.d/ (never hand-edit it), a list that looks like nothing a crawler publishes is refused, and the last good file stays when a fetch fails. geoip: {source: dbip, city: on} adds DB-IP City Lite beside the country and ASN editions (Settings → About → Cities): the city and region under every address and in the search — a reading aid, never a decision. Costs about 60 MB downloaded a month and 130 MB of memory on the controller; the sanctions on file are completed in the background the first time. With files of your own, city: is the path of a city database. console_session: 24h is how long a console session lasts after the last activity (12h unless set; 30 days since sign-in at most). A source applies everywhere unless sources_on narrows it, by source, to a scope:

sources: [googlebot, bingbot, cloudflare, uptimerobot]
sources_on:
  googlebot: [pack:web]        # the servers running the web pack — not the DNS servers
  uptimerobot: [host01, srv02]

An allow list of your own (custom_sources) may use the format ips-anywhere: every address or range found in the document, whatever its shape (JSON of any layout, text, HTML).

Retention (retention: in controller.yaml)

What the store keeps, applied once a day; Settings → Maintenance shows the sizes and can flush sooner.

retention:
  evidence: 180d     # the convicting lines (and what they named)
  samples: 400d      # the agents' hits/bans-in-force minute lines (the charts)
  # sanctions: 730d  # unset: the sanctions themselves are kept for good

Bans in force are never removed, whatever the spans say.

Update checks (updates: in controller.yaml)

updates: off stops the daily look at the forge's releases; on (the default), Settings → About says what the latest controller and agent releases are, the topbar shows an Update available pill, and GET /v1/updates answers the same.

Reporting switch (secrets.yaml)

abuseipdb.enabled: false keeps the key and sends nothing — the console's Settings → Reporting switch writes it; absent means on. The same key works in the agent's own secrets.yaml for machines that report with their own key (reporting: agent).