Shieldlist/shieldlist-agent
Watch
1
0
Fork
You've already forked shieldlist-agent
 0
Code Issues Pull requests Projects Releases 4 Packages Wiki Activity Actions
4 releases 4 tags
  • v0.0.9 0b05b0fbcd

    v0.0.9 Stable

    Robin released this 2026-08-26 21:35:03 +02:00 | 1 commits to main since this release

    Shieldlist Agent 0.0.9

    What it is

    The agent is what actually protects a machine: it tails its logs, scores what it reads against your rules, and blocks the offenders in the kernel — by itself, with no database and no daemon beside it. It works alone with local files, and it can be enrolled with a controller to be driven with the rest of a fleet.

    What you gain over 0.0.3

    Mostly speed and steadiness: a large ban set no longer costs you the loop, a flood no longer stalls detection while the firewall is being written, and a restart's restore takes seconds instead of minutes.

    Then what came with the fleet: enrolled mode, a simpler configuration format, rules that count by site or by origin, lists that block whole networks, and a test mode that decides like production and bans nobody.

    Everything that changed

    Faster, and steadier under attack

    • Single addresses live in plain firewall sets and only ranges in interval sets: listing an interval set dumps all of it before any change, so with a hundred thousand bans every call cost the best part of a second, stalled each round and turned a restore into minutes.
    • A refused batch is retried without the element the kernel names, then bisected — never one process per element — and the reason it was refused is logged.
    • Orders and lifts from a controller are applied as one pass: one transaction, one firewall batch, one log line. Thousands applied one at a time used to freeze evaluation for minutes.
    • The sync sends what changed instead of scanning the store, and the machine's own counters are published rather than recomputed.
    • A query that walked the whole sanctions table on every sync was split into two indexed halves merged in order.
    • The bans in force come from the published aggregates, not a table scan on every sync.
    • WatchKeep in two passes: count every group, build lines and progress only for the survivors.
    • The control loop's questions are answered the moment they arrive, not on the next snapshot.
    • A permanent ban's timeout is written in days — the kernel refuses nine digits of seconds, and long bans were silently refused.
    • Lifting keeps a kernel element another live sanction still needs; every value entering the store is canonical, so two rows can never fight over one element.
    • The unit asks for a CPU weight instead of a hard quota: a quota rationed the restore and the enforcement exactly under attack.
    • The restore says where its time went — store, kernel, waiting for the loop.
    • One named drop counter read by reset, instead of listings that dump the sets.
    • Aho-Corasick prefilters for the rules whose patterns are long lists of literals.

    Detection

    • Hits survive a restart: the rules' windows are snapshotted to disk and come back, and they cross an in-place reload.
    • {target} in a log input's path names the site every line of that file is about — a field, a filter and a facet, without allocating; a fixed file can declare its target.
    • Rules count by origin: countries, asns and their exceptions, with the geo databases pulled from the controller or named locally.
    • targets / except_targets: count only what was aimed at a site, or everything but.
    • A severer rule's conviction extends and renames an existing sanction instead of stacking a second one.
    • tighten: a returning offender needs less evidence per past episode.
    • Evidence lines keep what the parser captured, by name — the username tried, the path asked for, the site aimed at.
    • Lines two web servers write for one request are counted once.
    • A hit is a line some rule scores against the offender: a successful login weighed against a ban is not counted as one.
    • Several thresholds per rule, and a verdict restarts every threshold's counter for that offender.
    • The machine is never an offender: loopback and its own interface addresses are exempt in the core, and a non-routable address is never reported.

    Enrolled mode

    • shield enrol <token> attaches the machine: the bundle is validated before it is swapped in and reloaded in place — the pipeline is rebuilt between rounds, the store and the kernel untouched.
    • A received configuration that no longer loads is set aside at start; the machine runs its own files until a good one arrives.
    • The sync carries what it decided, who it is counting and how close they are, its running view (memory, lines and hits per minute, pending verdicts, shed, errors) and its clock.
    • It is nudged the moment a round bans or hits, so the console shows this second's decisions, and falls back to its interval on a quiet machine.
    • Suspects travel as a delta, with their lines; a suspect new to the list sends its last lines whole, so the controller always holds lines for what it shows.
    • Bans the operator orders are applied like any sanction — same checks, same enforcement, recorded as manual with the reason as evidence, reported when a preset is named.
    • Blocked lists arrive as data and load into their own firewall sets off the main loop, carved around the exemptions; shield why and shield status say when an address is covered by one.
    • The controller's log inputs replace the local ones; exemptions by country or AS are assembled for the machine.
    • Detect-only from the controller is remembered across a restart, and arming says when a restart is needed.
    • Nothing executable ever comes down that link: a controller sends configuration, never code.

    The configuration made simple

    • One rule, one file, five blocks; parsers named or written inline; thresholds as 5/10m, as a list, or scaled by level (instant, strict, standard, lenient).
    • Packs group rules; -convert rewrites a 0.0.x tree once.
    • state: test on a rule or a machine: decides like production, bans nothing.
    • share: no keeps a rule's bans off the fleet.
    • An old-format file is named once and dropped, not half-validated.

    Reporting

    • The comment ends with the episode's minute, first report and renewal alike, so a repeat offence is actually stored by the provider.
    • Sends may run in parallel; an unreachable provider is no longer read as a spent daily allowance; reporting can be switched off without losing the key.
    • The rule is unchanged: one chance at the moment of the decision, and shield report -last <duration> is the manual catch-up.

    Commands and diagnostics

    • shield status <key> prints one value alone; new_today counts from midnight in the machine's own zone.
    • shield tail follows decisions and their evidence live; shield why explains one address; shield ranges shows the busiest buckets.
    • SHIELDLIST_DEBUG=1 writes a debug log with each sync's cost; SHIELDLIST_PPROF=<addr> opens Go's profiler on a loopback address.
    • Per-parser and per-rule cost sampling, published in health and shown by shield stats — a rule at zero hits is visible.

    Operations

    • Debian packages for amd64, arm64, armhf and riscv64 from our APT repository; a hardened unit; logrotate; man pages; third-party licence notices.
    • The unit never wants nftables.service: Debian's own unit flushes the whole ruleset when it starts, taking every other firewall's rules with it.

    Security

    • A hardening pass on the geolocation database reader against crafted files.
    • https:// required for a controller address.
    • The packaged unit no longer blocks shield exempt.

    Early days

    Formats can still move between releases and an upgrade can ask for a re-enrolment. Start on test servers, keep what protects you today, and tell us what breaks.

    Credits

    Geolocation: DB-IP Lite (CC BY 4.0). Go modules: THIRD_PARTY_LICENSES. Detection rules translated from public sources keep their attribution in the rule files.

    Downloads