Quentin Mayo

Blog

  • Setting up Safe-Settings on Coolify

    A step-by-step guide to deploying GitHub Safe-Settings — policy-as-code for a GitHub
    organization — on a self-hosted Coolify instance. This guide reflects
    the live securelyprogramming deployment (app served at
    https://safe-settings2.programmingsecurely.com).

    Safe-Settings is a Probot GitHub App. It reads settings from a
    central admin repo and enforces them across every repository in the org — repository
    settings, branch protection, teams, collaborators, labels, environments, and rulesets.

    1. How it fits together

    diagram

    The GitHub App forwards webhooks to the container; the container reads the desired state from
    the admin repo and reconciles every target repository to match.

    2. Prerequisites

    Requirement This deployment
    A GitHub org securelyprogramming
    A Coolify instance coolify4 (ingress 23.122.195.61)
    A public hostname + DNS you control safe-settings2.programmingsecurely.com (Route53)
    A GitHub App (created below) safe-settings.programmingsecurely, App ID 2413188
    An admin repo in the org holds all the settings YAML

    3. Create the GitHub App

    Create the App at https://github.com/organizations/securelyprogramming/settings/apps/new
    (or use Probot’s manifest flow). It needs these permissions and events:

    diagram

    Then, on the App page:

    1. Set the Webhook URL to https://safe-settings2.programmingsecurely.com/api/github/webhooks.
    2. Set a Webhook secret (openssl rand -base64 32) — you’ll reuse this exact value in the container.
    3. Generate and download a private key (.pem).
    4. Install the App on All repositories in the org.

    4. Deploy on Coolify

    Point a Coolify Application at the admin repo (branch main) and use the repository’s
    Dockerfile build pack. The image is node:20-alpine and starts Probot with
    npm start on port 3000.

    Environment variables

    Set these on the application (Coolify → the app → Environment Variables):

    Key Value
    APP_ID 2413188
    GH_ORG securelyprogramming
    PRIVATE_KEY base64 of the downloaded .pem — base64 -w0 key.pem
    WEBHOOK_SECRET the exact secret set on the App webhook
    PORT 3000

    PRIVATE_KEY must be the base64-encoded contents of the .pem, on a single line.

    5. Point DNS at the Coolify ingress

    The App’s webhook URL must resolve to the Coolify server that runs the container.

    diagram

    Create/point the A record for safe-settings2.programmingsecurely.com to 23.122.195.61
    (the same IP coolify4.programmingsecurely.com resolves to). Set the app’s FQDN to
    https://safe-settings2.programmingsecurely.com so Coolify requests a Let’s Encrypt cert —
    GitHub requires valid TLS on the webhook endpoint.

    6. Add a health check

    Coolify runs the health check inside the container with curl. Probot exposes /ping
    (returns 200 OK), so use that:

    Field Value
    Path /ping
    Port 3000
    Method / scheme GET / http

    Gotcha: node:20-alpine ships without curl, so the health check fails with
    curl: not found. Add RUN apk add --no-cache curl to the Dockerfile.

    7. Configuration repo layout

    All policy lives in the admin repo — never in individual repos. Settings merge from broad
    to specific, with the most specific winning.

    diagram

    Precedence: repos/ > suborgs/ > settings.yml.

    8. What happens on a change

    diagram

    • PRs run in dry-run and report via check runs / PR comments.
    • Pushes to the default branch apply the settings for real.

    9. Verify it works

    1. Container healthy — Coolify shows running:healthy.
    2. Endpoint reachable — curl -I https://safe-settings2.programmingsecurely.com/ping → 200.
    3. Webhook delivers — GitHub App → Advanced → Recent Deliveries → Redeliver a
      ping; expect a green 200. A red 400 with
      signature does not match event payload and secret means the WEBHOOK_SECRET in Coolify
      doesn’t match the App’s webhook secret.
    4. Settings apply — push a change to admin‘s default branch and watch the check run.

    10. Troubleshooting

    Symptom Cause Fix
    Build fails at npm ci, “package.json and package-lock.json … in sync” Lockfile out of sync for the build’s npm version Regenerate the lockfile with the build’s npm (node:20-alpine, npm install --package-lock-only) and commit
    Health check fails: curl: not found alpine image lacks curl RUN apk add --no-cache curl in the Dockerfile
    Webhook returns 400, signature does not match … secret WEBHOOK_SECRET mismatch Make the App webhook secret and the container WEBHOOK_SECRET identical
    Webhook returns 404 / TLS warning DNS points at the wrong server, or no LE cert Point the A record at the Coolify ingress IP; set the app FQDN so a cert is issued
    App starts but does nothing admin repo has no config, or App not installed on repos Add .github/settings.yml; install the App on All repos

    Generated for the securelyprogramming/admin deployment.