mir-sentinel

The log toolkit behind MIR Sentinel, carved out for reuse:

1. A tolerant, zero-dependency parser (parseLine) for the kind of heterogeneous web/access log lines you get when many sites and formats share one stream. Instead of matching every format exactly, it extracts the handful of fields that matter — source, method, path, status, ip — plus a few request-class flags (attack / bot / asset / proxy), and degrades gracefully on lines that carry none of them (heartbeats, debug dumps, alerts).
2. A power-user log shipper (mir-sentinel-shipper) — a small bash daemon that tails a directory of .log files and POSTs each line to an ingest endpoint, with the filename as the source tag. Linux/macOS only (bash).

• The parser is ESM, Node >=18, zero runtime dependencies, and runs anywhere Node does (Linux, macOS, Windows). The shipper is the only POSIX/bash-only piece.
• Handles an optional [source] line prefix, Apache combined, quoted "GET …", bare request lines, IPv4 + IPv6, and CDN/reverse-proxy edge detection.

Install

npm install mir-sentinel

Parser

import { parseLine, isProxyIp } from 'mir-sentinel';

parseLine('1.2.3.4 - - [10/Oct/2026:12:00:00] "GET /wp-login.php HTTP/1.1" 404 200');
// {
//   raw: '…',
//   source: 'mir',
//   method: 'GET',
//   path: '/wp-login.php',
//   status: 404,
//   statusClass: '4xx',
//   ip: '1.2.3.4',
//   isRequest: true,
//   isAttack: true,     // matches a known probe fingerprint
//   isAsset: false,
//   isBot: false,
//   isProxy: false,
//   isAlert: false,
// }

parseLine('[curio] 9.9.9.9 "GET /css/app.css HTTP/1.1" 200');
// { source: 'curio', path: '/css/app.css', status: 200, statusClass: '2xx', isAsset: true, … }

isProxyIp('173.245.48.1'); // true for known CDN/reverse-proxy edge ranges

Return shape

parseLine(raw) always returns an object. Fields are null/false when the line doesn't carry them, so it's safe to call on any string (or non-string — you get the empty record back).

| field | meaning | | --- | --- | | raw | the original line | | source | [source] prefix if present, else "mir" | | method | HTTP verb, or null | | path | request path (query/fragment stripped, capped at 80 chars), or null | | status / statusClass | numeric status and "2xx"…"5xx" / "other" | | ip | first IP-looking token (v4 or v6), or null | | isRequest | a request line was recognised | | isAttack | path matches a known probe/scanner fingerprint | | isAsset | static asset (image/css/js/font/…) | | isBot | UA carries a crawler marker | | isProxy | ip is a known CDN/reverse-proxy edge (don't blame it) | | isAlert | line is an alert banner |

Parser configuration (env)

| var | default | effect | | --- | --- | --- | | FLAG_PHP_PROBES | off | when on, treat any .php request as a probe (for sites that serve no PHP) |

Power users — shipping logs from a box

Linux/macOS only. The shipper is a bash script; it won't run on Windows (the parser above does). Needs bash >= 4, tail, curl, jq, awk, mktemp.

mir-sentinel-shipper watches a directory of *.log files and ships each new line to an ingest endpoint. The basename of each file (minus .log) becomes the source tag; new files dropped into the directory are picked up automatically. Run one per remote box (not one per site).

# installed as a bin:
MIR_INGEST_URL=https://example.com/internal/logs/ingest mir-sentinel-shipper /var/log/sites

# or straight from the repo:
./bin/mir-sentinel-shipper.sh /var/log/sites

| var | default | effect | | --- | --- | --- | | MIR_INGEST_URL | https://mirregistry.com/internal/logs/ingest | where lines are POSTed ({source, lines[]}) | | BATCH_INTERVAL | 1 | flush interval, seconds | | BATCH_MAX | 100 | max lines per flush per source | | RESCAN_INTERVAL | 30 | how often to scan for new .log files, seconds |

It frames each record as source\tline over a shared FIFO and reads with a blocking read driven by a __tick__ sentinel for the periodic flush — a timed read would consume-then-discard a partial record at its deadline and drop the first byte of the next source tag (nafco → afco). Lines are capped at 4000 bytes so every write stays under PIPE_BUF and can't interleave across the concurrent tail writers.

License

MIT