add readme and AGENTS slop
This commit is contained in:
182
AGENTS.md
Normal file
182
AGENTS.md
Normal file
@@ -0,0 +1,182 @@
|
||||
# AGENTS.md
|
||||
|
||||
This file provides guidance for AI coding assistants working on the Panoptikon project.
|
||||
|
||||
## Project Overview
|
||||
|
||||
Panoptikon is a NixOS module that monitors website content and command output changes. It periodically runs scripts, compares outputs, and reports changes to various destinations.
|
||||
|
||||
**Key Components:**
|
||||
- `flake.nix` - Flake definition with modules, overlays, and VM app
|
||||
- `nix/module.nix` - Main NixOS module implementation
|
||||
- `nix/overlay.nix` - Overlay providing `panoptikonWatchers` and `panoptikonReporters` helpers
|
||||
- `examples/` - Example configurations
|
||||
- `README.md` - User-facing documentation
|
||||
|
||||
## Coding Conventions
|
||||
|
||||
### Nix Code Style
|
||||
|
||||
- Use Nix expression language (not NixOS modules have `lib` available)
|
||||
- Prefer `lib.attrsets.mapAttrs'` with `lib.nameValuePair` for transforming attrsets
|
||||
- Use `lib.strings.concatMapStringsSep` for joining strings with separators
|
||||
- Maintain consistent formatting (2-space indentation)
|
||||
|
||||
### Systemd Configuration
|
||||
|
||||
- Services run as dedicated `panoptikon` system user
|
||||
- Use `RuntimeDirectory` for temporary per-run isolation
|
||||
- Use `ReadWritePaths` to restrict filesystem access
|
||||
- Enable hardening flags: `ProtectSystem`, `ProtectHome`, `PrivateTmp`, `NoNewPrivileges`
|
||||
|
||||
### State Management
|
||||
|
||||
- State files stored in `/var/lib/panoptikon/`
|
||||
- Each watcher uses `${watcherName}` and `${watcherName}.old`
|
||||
- Scripts ensure `.old` exists before diff
|
||||
- Rotate state after comparing
|
||||
|
||||
### Security Practices
|
||||
|
||||
- Never log sensitive data (tokens, credentials)
|
||||
- Use `set -efu` in shell scripts (avoid `-x`)
|
||||
- Use `LoadCredential=` for passing secrets
|
||||
- Validate file existence before reading tokens
|
||||
- Escape shell arguments with `lib.escapeShellArg`
|
||||
|
||||
## Testing
|
||||
|
||||
### Manual Testing
|
||||
|
||||
1. Start the VM with all examples:
|
||||
```bash
|
||||
nix run .#panoptikon-vm
|
||||
```
|
||||
|
||||
2. Inside the VM, check service status:
|
||||
```bash
|
||||
systemctl status panoptikon-*
|
||||
journalctl -u panoptikon-bitcoin-price -f
|
||||
```
|
||||
|
||||
3. Trigger a manual run:
|
||||
```bash
|
||||
systemctl start panoptikon-bitcoin-price
|
||||
```
|
||||
|
||||
### Linting and Formatting
|
||||
|
||||
```bash
|
||||
# Check Nix syntax
|
||||
nix flake check
|
||||
|
||||
# Format Nix code
|
||||
nix fmt
|
||||
```
|
||||
|
||||
## Debugging
|
||||
|
||||
### Service Issues
|
||||
|
||||
1. Check if service is enabled and running:
|
||||
```bash
|
||||
systemctl status panoptikon-<watcher-name>
|
||||
```
|
||||
|
||||
2. View logs:
|
||||
```bash
|
||||
journalctl -u panoptikon-<watcher-name> -f
|
||||
```
|
||||
|
||||
3. Test scripts manually (as panoptikon user):
|
||||
```bash
|
||||
sudo -u panoptikon /nix/store/...-script
|
||||
```
|
||||
|
||||
### State File Issues
|
||||
|
||||
- Check state directory: `/var/lib/panoptikon/`
|
||||
- State files persist across reboots
|
||||
- Delete state file to force change detection on next run
|
||||
|
||||
### Timer Issues
|
||||
|
||||
- Timer uses `RandomizedDelaySec` of 1 hour to prevent thundering herd
|
||||
- This means services may be delayed up to 1 hour from the scheduled time
|
||||
- Check active timers: `systemctl list-timers "panoptikon-*"`
|
||||
|
||||
## Architecture Notes
|
||||
|
||||
### Data Flow
|
||||
|
||||
1. systemd timer triggers service
|
||||
2. Script runs, stdout saved to `${watcherName}`
|
||||
3. Diff against `${watcherName}.old`
|
||||
4. If changes detected: pipe diff to each reporter
|
||||
5. Rotate: current → old
|
||||
|
||||
### Reporter Contract
|
||||
|
||||
- Reporters receive the diff via stdin
|
||||
- Can access watcher name via `$PANOPTIKON_WATCHER`
|
||||
- Should handle errors gracefully (exit code ignored in main script)
|
||||
- Run in isolated empty `/run/panoptikon/<watcher-name>` directory
|
||||
|
||||
### Watcher Contract
|
||||
|
||||
- Must be an executable path (script or binary)
|
||||
- Output is stored verbatim for diffing
|
||||
- Should be idempotent (running twice produces same output)
|
||||
- Avoid side effects; output only relevant data
|
||||
|
||||
## Extension Points
|
||||
|
||||
### Adding New Watcher Helpers
|
||||
|
||||
Add to `panoptikonWatchers` in `overlay.nix`:
|
||||
|
||||
```nix
|
||||
panoptikonWatchers = (prev.panoptikonWatchers or { }) // {
|
||||
myWatcher =
|
||||
args:
|
||||
prev.writers.writeDash "watch-my" ''
|
||||
${prev.somePackage}/bin/tool --option ${lib.escapeShellArg args}
|
||||
'';
|
||||
};
|
||||
```
|
||||
|
||||
### Adding New Reporter Helpers
|
||||
|
||||
Similar pattern in `panoptikonReporters`:
|
||||
|
||||
```nix
|
||||
panoptikonReporters = (prev.panoptikonReporters or { }) // {
|
||||
myReporter =
|
||||
{ requiredArg, optionalArg ? "default" }:
|
||||
prev.writers.writeDash "report-my" ''
|
||||
cat | ${prev.tool}/bin/send --to ''${requiredArg}
|
||||
'';
|
||||
};
|
||||
```
|
||||
|
||||
### Common Dependencies
|
||||
|
||||
- `writers.writeDash` - Write shell scripts
|
||||
- `curl` - HTTP requests
|
||||
- `jq` - JSON processing
|
||||
- `htmlq` - HTML selectors
|
||||
- `diffutils` - Compare outputs
|
||||
|
||||
## Known Limitations
|
||||
|
||||
- Large outputs may cause memory pressure in diff
|
||||
- No built-in rate limiting for external APIs
|
||||
- Reporters cannot access the original script output, only the diff
|
||||
- State stored in plain files (no compression/rotation)
|
||||
|
||||
## Future Considerations
|
||||
|
||||
- Adding watchdog timeouts
|
||||
- Metrics collection (execution time, success rate)
|
||||
- Backoff strategies for failing watchers
|
||||
- Support for structured output formats (JSON events)
|
||||
358
README.md
358
README.md
@@ -5,137 +5,305 @@ A NixOS module for monitoring website content and command output changes.
|
||||
|
||||
## Overview
|
||||
|
||||
Panoptikon is a generic command output and website watcher that periodically runs scripts and reports changes. It's designed to be flexible and can monitor anything from API endpoints to system metrics.
|
||||
Panoptikon is a flexible, secure, and modular system for monitoring changes to websites, API endpoints, and command outputs. It runs scripts at configurable intervals, detects changes by comparing outputs, and reports differences to various destinations.
|
||||
|
||||
**Perfect for:**
|
||||
- Monitoring service status pages (GitHub, cloud providers)
|
||||
- Tracking website content changes (blogs, news)
|
||||
- Watching API endpoints (cryptocurrency prices, weather)
|
||||
- System metrics monitoring (disk space, load, processes)
|
||||
- Security canaries and breach detection
|
||||
|
||||
## Features
|
||||
|
||||
- **Flexible Watchers**: Monitor any command output or website content
|
||||
- **Custom Frequencies**: Run scripts at any interval using systemd.timer syntax
|
||||
- **Multiple Reporters**: Report changes to various destinations (IRC, Telegram, Prometheus, etc.)
|
||||
- **Secret Support**: Securely pass credentials to scripts without exposing them in the Nix store
|
||||
- **Stateful Tracking**: Automatically tracks previous output and reports only changes
|
||||
- **Modular Design**: Easy to extend with custom watchers and reporters
|
||||
- **Flexible Watchers**: Execute any script or command; monitor HTTP endpoints with built-in helpers
|
||||
- **Custom Frequencies**: Use systemd timer syntax for any schedule (*/5 minutes, daily, weekly, etc.)
|
||||
- **Multiple Reporters**: Notify via IRC, Telegram, Matrix, email, desktop notifications, wall, or custom scripts
|
||||
- **Secret Support**: Securely pass credentials using `LoadCredential=` without exposing them in the Nix store
|
||||
- **Stateful Tracking**: Automatic diffing; only reports actual changes
|
||||
- **Modular Design**: Built-in helpers for HTML, JSON, and plain text; easy to create custom ones
|
||||
- **Security Hardened**: Dedicated system user, filesystem isolation, and process protection
|
||||
- **systemd Native**: Full integration with systemd for reliable scheduling and logging
|
||||
|
||||
## Installation
|
||||
## Quick Start
|
||||
|
||||
Add Panoptikon to your NixOS configuration:
|
||||
1. **Enable Panoptikon** in your NixOS configuration:
|
||||
|
||||
```nix
|
||||
{ config, pkgs, ... }:
|
||||
|
||||
{
|
||||
# Enable Panoptikon service
|
||||
services.panoptikon.enable = true;
|
||||
|
||||
# Configure your watchers
|
||||
services.panoptikon.watchers = {
|
||||
# Your watcher configurations go here
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Basic Watcher Configuration
|
||||
2. **Add a simple watcher**:
|
||||
|
||||
```nix
|
||||
{
|
||||
services.panoptikon.enable = true;
|
||||
|
||||
services.panoptikon.watchers = {
|
||||
# Monitor GitHub metadata
|
||||
github-meta = {
|
||||
script = pkgs.writers.writeDash "github-meta" ''
|
||||
${pkgs.curl}/bin/curl -sSL https://api.github.com/meta | ${pkgs.jq}/bin/jq
|
||||
'';
|
||||
frequency = "*:0/5"; # Every 5 minutes
|
||||
reporters = [
|
||||
# Report changes to Telegram
|
||||
(pkgs.writers.writeDash "telegram-reporter" ''
|
||||
${pkgs.curl}/bin/curl -X POST https://api.telegram.org/bot''${TOKEN}/sendMessage \
|
||||
-d chat_id=123456 \
|
||||
-d text="$(cat)"
|
||||
'')
|
||||
# Also show desktop notifications
|
||||
(pkgs.writers.writeDash "notify" ''
|
||||
${pkgs.libnotify}/bin/notify-send "$PANOPTIKON_WATCHER has changed."
|
||||
'')
|
||||
];
|
||||
example = {
|
||||
script = pkgs.panoptikonWatchers.plain "https://example.com";
|
||||
frequency = "hourly";
|
||||
reporters = [ (pkgs.panoptikonReporters.wall { }) ];
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
# Monitor a website for specific content
|
||||
nixos-updates = {
|
||||
script = pkgs.panoptikon.urlSelector "#news h2" "https://nixos.org/blog/";
|
||||
frequency = "daily";
|
||||
reporters = [
|
||||
# Report to IRC
|
||||
(pkgs.panoptikon.kpaste-irc {
|
||||
target = "#nixos";
|
||||
3. **Deploy and monitor**:
|
||||
|
||||
```bash
|
||||
# Check status
|
||||
sudo systemctl status panoptikon-example
|
||||
|
||||
# View logs
|
||||
sudo journalctl -u panoptikon-example -f
|
||||
|
||||
# Trigger manually
|
||||
sudo systemctl start panoptikon-example
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
```
|
||||
┌─────────────┐
|
||||
│ systemd │
|
||||
│ timer │─── triggers ───┐
|
||||
└─────────────┘ │
|
||||
▼
|
||||
┌─────────────────────┐
|
||||
│ Panoptikon Service │
|
||||
│ (oneshot) │
|
||||
└─────────────────────┘
|
||||
│
|
||||
┌─────────┴─────────┐
|
||||
▼ │
|
||||
┌──────────────────┐ │
|
||||
│ Run watcher │ │
|
||||
│ script → current │ │
|
||||
└──────────────────┘ │
|
||||
│ │
|
||||
▼ │
|
||||
┌──────────────────┐ │
|
||||
│ Compare with │ │
|
||||
│ .old state │ │
|
||||
└──────────────────┘ │
|
||||
│ │
|
||||
diff? ─┼─── Yes ────────┤
|
||||
│ No │
|
||||
▼ ▼
|
||||
┌──────────────┐ ┌──────────────┐
|
||||
│ Exit quietly │ │ Pipe diff to │
|
||||
└──────────────┘ │ reporters │
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌────────────┴────────────┐
|
||||
▼ ▼
|
||||
[reporter 1] [reporter 2]
|
||||
│ │
|
||||
└──────────┬───────────────┘
|
||||
▼
|
||||
[Notifications sent]
|
||||
│
|
||||
▼
|
||||
┌──────────────────┐
|
||||
│ Rotate state: │
|
||||
│ current → old │
|
||||
└──────────────────┘
|
||||
```
|
||||
|
||||
**Key Points:**
|
||||
- Each watcher runs as its own systemd service with a dedicated timer
|
||||
- Output is saved to `/var/lib/panoptikon/<watcher-name>` and `<watcher-name>.old`
|
||||
- Only diffs are sent to reporters (full output never transmitted)
|
||||
- Reporters run in a clean, empty `/run` directory for isolation
|
||||
- State persists across reboots
|
||||
|
||||
## Configuration Reference
|
||||
|
||||
### Watcher Options
|
||||
|
||||
| Option | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `script` | path | **Required.** Executable whose stdout will be monitored. |
|
||||
| `frequency` | string | systemd.time(7) timer expression. Default: `"daily"` |
|
||||
| `reporters` | list of paths | **Required.** Scripts that receive the diff via stdin. |
|
||||
| `loadCredential` | list of strings | Credentials to pass from systemd (`LoadCredential=`). |
|
||||
|
||||
### Timer Syntax
|
||||
|
||||
Common patterns:
|
||||
|
||||
| Pattern | Meaning |
|
||||
|---------|---------|
|
||||
| `*:0/5` | Every 5 minutes |
|
||||
| `*:0/15` | Every 15 minutes |
|
||||
| `hourly` | At minute 0 of every hour |
|
||||
| `daily` | Once per day at midnight |
|
||||
| `*-*-1 0:0:0` | First day of month |
|
||||
| `Mon *-*-* 0:0:0` | Every Monday |
|
||||
| `Sat,Sun *-*-* 0:0:0` | Weekends |
|
||||
|
||||
See: `man systemd.time`
|
||||
|
||||
### Environment Variables
|
||||
|
||||
- `PANOPTIKON_WATCHER` - Name of the watcher (available to both watchers and reporters)
|
||||
|
||||
## Built-in Helpers
|
||||
|
||||
The overlay provides convenient watcher and reporter constructors:
|
||||
|
||||
### Watcher Helpers
|
||||
|
||||
```nix
|
||||
# Fetch raw content
|
||||
pkgs.panoptikonWatchers.plain "https://example.com"
|
||||
|
||||
# Convert HTML to plain text
|
||||
pkgs.panoptikonWatchers.html "https://example.com"
|
||||
|
||||
# Extract specific HTML elements using CSS selector
|
||||
pkgs.panoptikonWatchers.htmlSelector "#news h2" "https://example.com"
|
||||
|
||||
# Process JSON with jq
|
||||
pkgs.panoptikonWatchers.json { jqScript = ".data[] | .value" } "https://api.example.com"
|
||||
```
|
||||
|
||||
### Reporter Helpers
|
||||
|
||||
```nix
|
||||
# Send to wall (broadcast to all logged-in users)
|
||||
pkgs.panoptikonReporters.wall { }
|
||||
|
||||
# Send email
|
||||
pkgs.panoptikonReporters.mail {
|
||||
recipient = "admin@example.org";
|
||||
subjectPrefix = "[Alert]";
|
||||
}
|
||||
|
||||
# Telegram Bot API
|
||||
pkgs.panoptikonReporters.telegram {
|
||||
chatId = "123456";
|
||||
tokenPath = "/run/keys/telegram-token"; # Use LoadCredential=
|
||||
messagePrefix = "Change detected: ";
|
||||
}
|
||||
|
||||
# Matrix (via REST API)
|
||||
pkgs.panoptikonReporters.matrix {
|
||||
homeserver = "https://matrix.org";
|
||||
roomId = "!roomid:matrix.org";
|
||||
tokenPath = "/run/keys/matrix-token";
|
||||
}
|
||||
|
||||
# IRC (simple netcat-based)
|
||||
pkgs.panoptikonReporters.irc {
|
||||
target = "#channel";
|
||||
server = "irc.libera.chat";
|
||||
messagePrefix = "New NixOS blog post: ";
|
||||
})
|
||||
];
|
||||
};
|
||||
port = "6667";
|
||||
nick = "panoptikon-bot";
|
||||
}
|
||||
|
||||
# Monitor a local command
|
||||
disk-space = {
|
||||
script = pkgs.writers.writeDash "disk-space" ''
|
||||
df -h / | tail -1 | awk '{print $5 " used
|
||||
}'';
|
||||
frequency = "*:0/30"; # Every 30 minutes
|
||||
reporters = [
|
||||
# Log to systemd journal
|
||||
(pkgs.writers.writeDash "journal-log" ''
|
||||
journalctl -t panoptikon-disk-space --since "1 hour ago" | tail -5
|
||||
'')
|
||||
];
|
||||
};
|
||||
};
|
||||
# kpaste + ircsink (for retiolum)
|
||||
pkgs.panoptikonReporters.kpaste-irc {
|
||||
target = "#nixos";
|
||||
server = "irc.r";
|
||||
retiolumLink = true; # Generate retiolum link
|
||||
}
|
||||
```
|
||||
|
||||
## Service Management
|
||||
**Important:** For security, always use `LoadCredential=` for tokens instead of embedding them in your Nix store.
|
||||
|
||||
### systemd Integration
|
||||
## Advanced Examples
|
||||
|
||||
Each watcher gets its own systemd service and timer:
|
||||
### Monitor Bitcoin Price with Telegram Alert
|
||||
|
||||
See [examples/bitcoin.nix](./examples/bitcoin.nix)
|
||||
|
||||
### Website Content Change with Email Notification
|
||||
|
||||
See [examples/nixos.nix](./examples/nixos.nix)
|
||||
|
||||
### System Metrics with Console Notifications
|
||||
|
||||
See [examples/system.nix](./examples/system.nix)
|
||||
|
||||
### Custom Script with Multiple Reporters
|
||||
|
||||
See [examples/simple.nix](./examples/simple.nix) for more.
|
||||
|
||||
## Testing
|
||||
|
||||
### Run a VM with Example Configurations
|
||||
|
||||
```bash
|
||||
# List all Panoptikon services
|
||||
systemctl list-units "panoptikon-*"
|
||||
|
||||
# Check a specific watcher
|
||||
systemctl status panoptikon-github-meta
|
||||
|
||||
# View logs
|
||||
journalctl -u panoptikon-github-meta -f
|
||||
|
||||
# Trigger a manual run
|
||||
systemctl start panoptikon-github-meta
|
||||
nix run .#panoptikon-vm
|
||||
```
|
||||
|
||||
### Timer Configuration
|
||||
This builds and boots a NixOS VM with all example watchers pre-configured.
|
||||
|
||||
Timers use systemd timer syntax. Common examples:
|
||||
### Inside the VM
|
||||
|
||||
- `*:0/5` - Every 5 minutes
|
||||
- `daily` - Once per day
|
||||
- `*:0/15` - Every 15 minutes
|
||||
- `weekly` - Once per week
|
||||
```bash
|
||||
# List all panoptikon services
|
||||
systemctl list-units "panoptikon-*"
|
||||
|
||||
See [systemd.time(7)](https://www.freedesktop.org/software/systemd/man/systemd.time.html) for full syntax.
|
||||
# Check Bitcoin watcher status
|
||||
systemctl status panoptikon-bitcoin-price
|
||||
|
||||
## Security Considerations
|
||||
# Follow its logs
|
||||
journalctl -u panoptikon-bitcoin-price -f
|
||||
|
||||
- Watchers run as the `panoptikon` system user
|
||||
- Scripts are executed in `/var/lib/panoptikon`
|
||||
- Use `LoadCredential=` to securely pass secrets
|
||||
- Scripts should be written defensively (use `set -euo pipefail`)
|
||||
# Force a run (useful for testing)
|
||||
systemctl start panoptikon-bitcoin-price
|
||||
|
||||
## Troubleshooting
|
||||
# Check state directory
|
||||
ls -la /var/lib/panoptikon/
|
||||
```
|
||||
|
||||
## Examples
|
||||
### Manual Script Testing
|
||||
|
||||
See the [examples directory](./examples/) for complete configurations.
|
||||
Run the watcher script as the panoptikon user to see its output:
|
||||
|
||||
Run `nix run .#panoptikon-vm` to start a VM with Panoptikon and example watchers pre-configured.
|
||||
```bash
|
||||
sudo -u panoptikon /nix/store/<hash>-watch-bitcoin
|
||||
```
|
||||
|
||||
Check that it produces clean output without errors.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can Panoptikon monitor FTP or SSH?**
|
||||
|
||||
A: Yes! Write a custom watcher script that uses `curl` (for SFTP), `ssh`, or any CLI tool. Example:
|
||||
|
||||
```nix
|
||||
script = pkgs.writers.writeDash "ssh-check" ''
|
||||
${pkgs.openssh}/bin/ssh user@host "uptime"
|
||||
'';
|
||||
```
|
||||
|
||||
**Q: What happens if the watcher script fails?**
|
||||
|
||||
A: The service will be marked as failed and will restart on next timer activation (unless configured otherwise). The error is logged to the systemd journal. Reporters are skipped.
|
||||
|
||||
**Q: Can I run multiple reporters for the same event?**
|
||||
|
||||
A: Yes! Reporters are executed sequentially. If one fails, others still run (errors are suppressed with `|| :`).
|
||||
|
||||
**Q: How do I handle JSON pretty-printing?**
|
||||
|
||||
A: Use jq:
|
||||
|
||||
```nix
|
||||
script = pkgs.panoptikonWatchers.json { jqScript = "." } "https://api.example.com/data";
|
||||
```
|
||||
|
||||
Or in a custom script:
|
||||
|
||||
```bash
|
||||
curl -s ... | jq -S . # -S sorts keys
|
||||
```
|
||||
|
||||
**Q: Can I send rich formatting (HTML, Markdown) to reporters?**
|
||||
|
||||
A: Yes! Reporters receive the raw diff. For IRC, use colors sparingly. For Telegram/Matrix, you can send Markdown or HTML by constructing appropriate payloads in custom reporters.
|
||||
|
||||
Reference in New Issue
Block a user