panoptikon/AGENTS.md

AGENTS.md

This file provides guidance for AI coding assistants working on the Panoptikon project.

Project Overview

Panoptikon is a NixOS module that monitors website content and command output changes. It periodically runs scripts, compares outputs, and reports changes to various destinations.

Key Components:

• flake.nix - Flake definition with modules, overlays, and VM app
• nix/module.nix - Main NixOS module implementation
• nix/overlay.nix - Overlay providing panoptikonWatchers and panoptikonReporters helpers
• examples/ - Example configurations
• README.md - User-facing documentation

Coding Conventions

Nix Code Style

• Use Nix expression language (not NixOS modules have lib available)
• Prefer lib.attrsets.mapAttrs' with lib.nameValuePair for transforming attrsets
• Use lib.strings.concatMapStringsSep for joining strings with separators
• Maintain consistent formatting (2-space indentation)

Systemd Configuration

• Services run as dedicated panoptikon system user
• Use RuntimeDirectory for temporary per-run isolation
• Use ReadWritePaths to restrict filesystem access
• Enable hardening flags: ProtectSystem, ProtectHome, PrivateTmp, NoNewPrivileges

State Management

• State files stored in /var/lib/panoptikon/
• Each watcher uses ${watcherName} and ${watcherName}.old
• Scripts ensure .old exists before diff
• Rotate state after comparing

Security Practices

• Never log sensitive data (tokens, credentials)
• Use set -efu in shell scripts (avoid -x)
• Use LoadCredential= for passing secrets
• Validate file existence before reading tokens
• Escape shell arguments with lib.escapeShellArg

Testing

Manual Testing

1. Start the VM with all examples:

   nix run .#panoptikon-vm
   

2. Inside the VM, check service status:

   systemctl status panoptikon-*
   journalctl -u panoptikon-bitcoin-price -f
   

3. Trigger a manual run:

   systemctl start panoptikon-bitcoin-price
   

Linting and Formatting

# Check Nix syntax
nix flake check

# Format Nix code
nix fmt

Debugging

Service Issues

1. Check if service is enabled and running:

   systemctl status panoptikon-<watcher-name>
   

2. View logs:

   journalctl -u panoptikon-<watcher-name> -f
   

3. Test scripts manually (as panoptikon user):

   sudo -u panoptikon /nix/store/...-script
   

State File Issues

• Check state directory: /var/lib/panoptikon/
• State files persist across reboots
• Delete state file to force change detection on next run

Timer Issues

• Timer uses RandomizedDelaySec of 1 hour to prevent thundering herd
• This means services may be delayed up to 1 hour from the scheduled time
• Check active timers: systemctl list-timers "panoptikon-*"

Architecture Notes

Data Flow

1. systemd timer triggers service
2. Script runs, stdout saved to ${watcherName}
3. Diff against ${watcherName}.old
4. If changes detected: pipe diff to each reporter
5. Rotate: current → old

Reporter Contract

• Reporters receive the diff via stdin
• Can access watcher name via $PANOPTIKON_WATCHER
• Should handle errors gracefully (exit code ignored in main script)
• Run in isolated empty /run/panoptikon/<watcher-name> directory

Watcher Contract

• Must be an executable path (script or binary)
• Output is stored verbatim for diffing
• Should be idempotent (running twice produces same output)
• Avoid side effects; output only relevant data

Extension Points

Adding New Watcher Helpers

Add to panoptikonWatchers in overlay.nix:

panoptikonWatchers = (prev.panoptikonWatchers or { }) // {
  myWatcher =
    args:
    prev.writers.writeDash "watch-my" ''
      ${prev.somePackage}/bin/tool --option ${lib.escapeShellArg args}
    '';
};

Adding New Reporter Helpers

Similar pattern in panoptikonReporters:

panoptikonReporters = (prev.panoptikonReporters or { }) // {
  myReporter =
    { requiredArg, optionalArg ? "default" }:
    prev.writers.writeDash "report-my" ''
      cat | ${prev.tool}/bin/send --to ''${requiredArg}
    '';
};

Common Dependencies

• writers.writeDash - Write shell scripts
• curl - HTTP requests
• jq - JSON processing
• htmlq - HTML selectors
• diffutils - Compare outputs

Known Limitations

• Large outputs may cause memory pressure in diff
• No built-in rate limiting for external APIs
• Reporters cannot access the original script output, only the diff
• State stored in plain files (no compression/rotation)

Future Considerations

• Adding watchdog timeouts
• Metrics collection (execution time, success rate)
• Backoff strategies for failing watchers
• Support for structured output formats (JSON events)