3 Sizing and limits
Robin edited this page 2026-08-22 10:20:41 +00:00

Sizing and limits

The agent runs on a server bought to run something else. This page is what it takes, what bounds it, and the one thing that nothing here can.

The measured figures live in docs/RESOURCES.md; this is how to think about them.

The one setting

limits:
  memory: 50%        # or "512MB", "2GB", "1TB"

Half the machine by default. That is a lot for a guest program and it is deliberate: this is the program that stops the attack, and evidence it cannot hold is an attacker it cannot convict.

It is a ceiling, never a reservation. The agent takes only what its rules actually need, which on a quiet machine is a few tens of megabytes.

It is also a fixed ceiling. The agent does not read how much memory the machine has free, and does not shrink when something else grows.

That is a deliberate reversal. An earlier version subtracted a reserve from available memory so the agent would "give room back" — and on a hosting node with a database and twenty PHP pools, a nightly job pushed available below the reserve and the budget collapsed from 929 446 buffered events to 16 384 while the process was holding 16 MB. It shed a capacity it had never taken, because of what something else was doing at midnight.

A program whose behaviour depends on the rest of the box is one whose behaviour cannot be reproduced, and one that competes with every other program for the same slack. Tell this one what it may have. Dividing the rest is your job, and you have better tools for it — MemoryMax in the unit, which the shipped one already sets as a backstop.

What happens at the ceiling

The agent does not stop reading. It remembers less far back — uniformly across every rule, so no offender is forgotten whole while others are kept intact — and it says so once, with what you can do about it:

WARN  memory budget reached — holding less evidence than the rules ask for
      remembering="41% of each rule's window"
      remedy="shorten the rules' windows, narrow what they match,
              raise limits.memory, or give the machine more memory"

shield status counts the evidence that went, under shed. A run that sheds is telling you the ceiling is below what these rules need.

Under the hood the bound is on buffered events, not on the heap: the number of events a rule may hold is arithmetic the agent controls, whereas watching its own heap and reacting would be a feedback loop around the garbage collector, which under a memory limit is already holding the heap near that limit by design.

What one buffered event costs is measured, not assumed — it is mostly the log line, and lines differ by an order of magnitude between services, so the agent times itself and adjusts.

The rule that costs the most memory

A rule whose threshold is rarely reached holds everything it matches for the window plus the lateness allowance. A flood rule counting 300 requests a minute per address, on traffic where most addresses sit just under 300, buffers every request of every address. Prefer thresholds the traffic actually reaches, or a shorter window.

Processor

One core is plenty for a busy server: tens of thousands of log lines a second on a two-core virtual machine with the rules a real web server needs. Under a flood the agent degrades by falling behind on reads — never by making the protected service wait. read_lag in shield status is how far behind it is, and the log says so once when it starts falling behind and once when it catches up.

The shipped systemd unit gives it a processor share (CPUWeight=50, half the default), not a cap: under contention it yields to the service it protects; when the processor is idle it takes what it needs — a restart restoring hundreds of thousands of bans, a flood being parsed and enforced. A hard CPUQuota rations exactly those moments: if you set one, size it against a restart of your own store, not against idle.

Disk

Sanctions and their evidence are durable — evidence is what lets shield why tell you the story. It is capped at twenty lines a sanction, but the sanctions themselves are not: five million bans with ten convicting lines each measured about 4 GB. Watch state_size in shield status.

The budget nothing here bounds

A ban lives in the kernel, and kernel memory is not the agent's to limit. Roughly a hundred bytes per address, unswappable:

active bans kernel memory
100 000 ~10 MB
1 000 000 ~100 MB
5 000 000 ~500 MB

Neither limits.memory nor systemd's MemoryMax covers it — those bound the agent's process, not the kernel's tables. On a small machine facing a flood from millions of distinct addresses, this is the limit you meet first.

Watch it in the Slab: line of /proc/meminfo, or with slabtop. It grows with the active set and falls as bans expire.

If that is your risk, the levers are:

  • Ban ranges rather than addresses where the evidence supports it: one element covers a whole prefix.
  • Shorter first rungs, so a one-off scanner leaves the set sooner. Elements expire on their own; the set drains with nobody doing anything.
  • Kernel guardrails — connection and rate caps, which are dynamic sets the kernel bounds itself — to absorb volume that would otherwise become one element per source.
  • More memory, honestly, if the fleet's traffic warrants holding millions of bans at once.

The systemd unit

The shipped unit confines the agent to what it needs — one capability, one writable path, no new privileges, a filtered syscall set — and sets MemoryMax well above limits.memory. That is on purpose: the kernel's limit is a backstop for a runaway, and must never be what bounds normal operation. Move the two together if you move either.