panoptikon/README.md

# Panoptikon - Website and Command Output Monitoring

A NixOS module for monitoring website content and command output changes.

## Overview

Panoptikon is a generic command output and website watcher that periodically runs scripts and reports changes. It's designed to be flexible and can monitor anything from API endpoints to system metrics.

## Features

- **Flexible Watchers**: Monitor any command output or website content
- **Custom Frequencies**: Run scripts at any interval using systemd.timer syntax
- **Multiple Reporters**: Report changes to various destinations (IRC, Telegram, Prometheus, etc.)
- **Secret Support**: Securely pass credentials to scripts without exposing them in the Nix store
- **Stateful Tracking**: Automatically tracks previous output and reports only changes
- **Modular Design**: Easy to extend with custom watchers and reporters

## Installation

Add Panoptikon to your NixOS configuration:

```nix
{ config, pkgs, ... }:

{
  imports = [ 
    # Add the Panoptikon module
    (pkgs.callPackage (builtins.fetchTarball https://github.com/kfm/panoptikon-library/archive/main.tar.gz) { }).nixosModules.default
  ];

  # Enable Panoptikon service
  services.panoptikon.enable = true;

  # Configure your watchers
  services.panoptikon.watchers = {
    # Your watcher configurations go here
  };
}
```

## Configuration

### Basic Watcher Configuration

```nix
{
  services.panoptikon.enable = true;
  
  services.panoptikon.watchers = {
    # Monitor GitHub metadata
    github-meta = {
      script = pkgs.writers.writeDash "github-meta" '''
        ${pkgs.curl}/bin/curl -sSL https://api.github.com/meta | ${pkgs.jq}/bin/jq
      ''';
      frequency = "*:0/5";  # Every 5 minutes
      reporters = [
        # Report changes to Telegram
        (pkgs.writers.writeDash "telegram-reporter" '''
          ${pkgs.curl}/bin/curl -X POST https://api.telegram.org/bot''${TOKEN}/sendMessage \
            -d chat_id=123456 \
            -d text="$(cat)"
        ''')
        # Also show desktop notifications
        (pkgs.writers.writeDash "notify" '''
          ${pkgs.libnotify}/bin/notify-send "$PANOPTIKON_WATCHER has changed."
        ''')
      ];
    };

    # Monitor a website for specific content
    nixos-updates = {
      script = pkgs.panoptikon.urlSelector "#news h2" "https://nixos.org/blog/";
      frequency = "daily";
      reporters = [
        # Report to IRC
        (pkgs.panoptikon.kpaste-irc {
          target = "#nixos";
          server = "irc.libera.chat";
          messagePrefix = "New NixOS blog post: ";
        })
      ];
    };

    # Monitor a local command
    disk-space = {
      script = pkgs.writers.writeDash "disk-space" '''
        df -h / | tail -1 | awk '{print $5 " used"}'
      ''';
      frequency = "*:0/30";  # Every 30 minutes
      reporters = [
        # Log to systemd journal
        (pkgs.writers.writeDash "journal-log" '''
          journalctl -t panoptikon-disk-space --since "1 hour ago" | tail -5
        ''')
      ];
    };
  };
}
```

### Advanced Configuration

#### Using Secrets

```nix
{
  services.panoptikon.watchers = {
    private-api = {
      script = pkgs.writers.writeDash "private-api" '''
        ${pkgs.curl}/bin/curl -sSL \
          -H "Authorization: Bearer $API_TOKEN" \
          https://api.example.com/data
      ''';
      frequency = "hourly";
      loadCredential = [ "API_TOKEN" ];
      reporters = [
        (pkgs.writers.writeDash "secure-reporter" '''
          ${pkgs.curl}/bin/curl -X POST https://secure.example.com/report \
            -d "$(cat)"
        ''')
      ];
    };
  };
}
```

#### JSON API Monitoring

```nix
{
  services.panoptikon.watchers = {
    crypto-prices = {
      script = pkgs.panoptikon.urlJSON {
        jqScript = ".[0] | { name: .name, price: .quote.USD.price }";
      } "https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&ids=bitcoin";
      frequency = "*:0/15";  # Every 15 minutes
      reporters = [
        (pkgs.writers.writeDash "price-alert" '''
          price=$(echo "$(cat)" | ${pkgs.jq}/bin/jq -r '.price')
          if (( $(echo "$price > 60000" | bc -l) )); then
            ${pkgs.libnotify}/bin/notify-send "Bitcoin price: $$price"
          fi
        ''')
      ];
    };
  };
}
```

## Available Helper Scripts

Panoptikon provides several helper scripts in the overlay:

### URL Monitoring

- `pkgs.panoptikon.url`: Fetch and convert HTML to text
- `pkgs.panoptikon.urlSelector`: Extract specific HTML elements using CSS selectors
- `pkgs.panoptikon.urlJSON`: Fetch JSON and apply jq transformations

### Reporters

- `pkgs.panoptikon.kpaste-irc`: Report changes to IRC via kpaste
  - `target`: IRC target (channel or user)
  - `server`: IRC server (default: irc.r)
  - `messagePrefix`: Prefix for messages (default: "change detected: ")
  - `nick`: Nick to use (default: watcher name + "-watcher")
  - `retiolumLink`: Use retiolum link (default: false)

## Service Management

### Systemd Integration

Each watcher gets its own systemd service and timer:

```bash
# List all Panoptikon services
systemctl list-units "panoptikon-*"

# Check a specific watcher
systemctl status panoptikon-github-meta

# View logs
journalctl -u panoptikon-github-meta -f

# Trigger a manual run
systemctl start panoptikon-github-meta
```

### Timer Configuration

Timers use systemd.timer syntax. Common examples:

- `*:0/5` - Every 5 minutes
- `daily` - Once per day
- `Mon..Fri 9:00-17:00` - Weekdays during business hours
- `*:0/15` - Every 15 minutes
- `weekly` - Once per week

See [systemd.time(7)](https://www.freedesktop.org/software/systemd/man/systemd.time.html) for full syntax.

## Security Considerations

- Watchers run as the `panoptikon` system user
- Scripts are executed in `/var/lib/panoptikon`
- Use `loadCredential` to securely pass secrets
- Scripts should be written defensively (use `set -euo pipefail`)

## Troubleshooting

### Common Issues

1. **No output**: Check if the script runs correctly manually
2. **Permission denied**: Ensure the panoptikon user can access required resources
3. **Network issues**: Add `network-online.target` as a dependency
4. **Rate limiting**: Add randomized delays using `RandomizedDelaySec`

### Debug Mode

Enable debug logging:

```nix
services.panoptikon.watchers.github-meta = {
  # ...
  script = ''
    set -x  # Enable debug output
    ${pkgs.curl}/bin/curl -sSL https://api.github.com/meta | ${pkgs.jq}/bin/jq
  '';
};
```

## Examples

### Monitor Cryptocurrency Prices

```nix
{
  services.panoptikon.enable = true;
  
  services.panoptikon.watchers = {
    bitcoin-price = {
      script = pkgs.panoptikon.urlJSON {
        jqScript = ".[0] | { name: .name, price: .quote.USD.price, change: .quote.USD.percent_change_24h }";
      } "https://api.coingecko.com/api/v3/coins/markets?vs_currency=usd&ids=bitcoin";
      frequency = "*:0/30";
      reporters = [
        (pkgs.writers.writeDash "btc-alert" '''
          price=$(echo "$(cat)" | ${pkgs.jq}/bin/jq -r '.price')
          change=$(echo "$(cat)" | ${pkgs.jq}/bin/jq -r '.change')
          
          ${pkgs.libnotify}/bin/notify-send \
            "Bitcoin: $$price ($$change%)"
        ''')
      ];
    };
  };
}
```

### Monitor System Metrics

```nix
{
  services.panoptikon.enable = true;
  
  services.panoptikon.watchers = {
    load-average = {
      script = pkgs.writers.writeDash "load-average" '''
        uptime | awk -F'load average:' '{print $2}' | awk '{print $1 " (1m), " $2 " (5m), " $3 " (15m)"}'
      ''';
      frequency = "*:0/2";  # Every 2 minutes
      reporters = [
        (pkgs.writers.writeDash "load-alert" '''
          load=$(echo "$(cat)" | awk '{print $1}' | tr -d ',')
          if (( $(echo "$load > 2.0" | bc -l) )); then
            ${pkgs.libnotify}/bin/notify-send "High load: $$load"
          fi
        ''')
      ];
    };
  };
}
```

### Monitor NixOS Updates

```nix
{
  services.panoptikon.enable = true;
  
  services.panoptikon.watchers = {
    nixos-updates = {
      script = pkgs.panoptikon.url "https://nixos.org/blog/";
      frequency = "daily";
      reporters = [
        (pkgs.writers.writeDash "update-notify" '''
          ${pkgs.libnotify}/bin/notify-send "NixOS blog updated: $(cat | head -5)"
        ''')
      ];
    };
  };
}
```

## Development

### Adding Custom Reporters

Create a new reporter in the overlay:

```nix
final: prev: {
  panoptikon = prev.panoptikon // {
    my-custom-reporter = {
      endpoint,
      authToken ? "",
      ...
    }:
      prev.writers.writeDash "my-reporter" ''
        ${prev.curl}/bin/curl -X POST ''${endpoint} \
          -H "Authorization: Bearer ''${authToken} \
          -d "$(cat)"
      '';
  };
};
```

### Contributing

1. Fork the repository
2. Create a feature branch
3. Add tests if applicable
4. Submit a pull request

## License

MIT License