diff --git a/contrib/systemd/idx-ownership-snapshot-publish.service b/contrib/systemd/idx-ownership-snapshot-publish.service new file mode 100644 index 0000000..0e7351f --- /dev/null +++ b/contrib/systemd/idx-ownership-snapshot-publish.service @@ -0,0 +1,11 @@ +[Unit] +Description=Publish idx ownership snapshot to GitHub Releases +Wants=network-online.target +After=network-online.target + +[Service] +Type=oneshot +WorkingDirectory=/srv/idx-cli +Environment=HOME=/var/lib/idx-ownership-snapshot +EnvironmentFile=/etc/idx-ownership-snapshot.env +ExecStart=/bin/sh -lc 'nix develop --command scripts/publish-ownership-snapshot.sh --build --idx-bin ./target/debug/idx --output-dir /var/lib/idx-ownership-snapshot/current --repo 0xrsydn/idx-cli --release-tag ownership-snapshot-current' diff --git a/contrib/systemd/idx-ownership-snapshot-publish.timer b/contrib/systemd/idx-ownership-snapshot-publish.timer new file mode 100644 index 0000000..17b127e --- /dev/null +++ b/contrib/systemd/idx-ownership-snapshot-publish.timer @@ -0,0 +1,11 @@ +[Unit] +Description=Run monthly idx ownership snapshot publish job + +[Timer] +OnCalendar=*-*-02 09:00:00 +Persistent=true +RandomizedDelaySec=30m +Unit=idx-ownership-snapshot-publish.service + +[Install] +WantedBy=timers.target diff --git a/docs/OWNERSHIP_PUBLISH.md b/docs/OWNERSHIP_PUBLISH.md index d5b92ee..278a16e 100644 --- a/docs/OWNERSHIP_PUBLISH.md +++ b/docs/OWNERSHIP_PUBLISH.md @@ -93,3 +93,18 @@ Current behavior: This is intentionally manual-first. Add a schedule only after a few successful publish runs confirm the live source remains stable enough. + +## Self-Hosted Automation + +GitHub-hosted Actions are still not sufficient for this job on their own. Live +verification has shown IDX returning `403` to GitHub-hosted runners during +discovery/import. + +The recommended unattended path is therefore: + +- keep the public artifacts on GitHub Releases +- run the publish job from a self-hosted machine that IDX accepts +- use a `systemd` oneshot service plus `systemd.timer` on that machine + +See `docs/OWNERSHIP_SELF_HOSTED.md` for the reusable helper script, sample +systemd units, and the recommended split between this repo and `clan-private`. diff --git a/docs/OWNERSHIP_SELF_HOSTED.md b/docs/OWNERSHIP_SELF_HOSTED.md new file mode 100644 index 0000000..7457fbb --- /dev/null +++ b/docs/OWNERSHIP_SELF_HOSTED.md @@ -0,0 +1,95 @@ +# Self-Hosted Ownership Snapshot Publish + +This document covers the recommended self-hosted automation path for publishing +the monthly ownership snapshot consumed by `idx ownership sync`. + +Use this path when: + +- the repo's public release assets should stay on GitHub Releases +- IDX blocks GitHub-hosted Actions with `403` +- you have a VPS or other trusted machine that can reach the IDX source + +## Why Self-Hosted + +The current GitHub-hosted workflow remains useful as a manual reference, but it +is not reliable enough for unattended publishing because IDX has returned `403` +to GitHub-hosted runners. The self-hosted path keeps the end-user experience +simple: + +- maintainers publish from a trusted machine +- end users still run plain `idx ownership sync` + +No Nix setup is required for end users. Nix is only an implementation detail for +the maintainer host if that host already uses NixOS. + +## Manual Command + +From a checked-out repo with GitHub auth already configured: + +```bash +nix develop --command scripts/publish-ownership-snapshot.sh \ + --build \ + --idx-bin ./target/debug/idx \ + --output-dir /var/lib/idx/ownership-snapshot/current \ + --repo 0xrsydn/idx-cli \ + --release-tag ownership-snapshot-current +``` + +That helper: + +1. optionally builds `idx` +2. discovers the latest supported IDX/KSEI PDF +3. imports it into an isolated temp DB +4. emits the SQLite snapshot and manifest +5. ensures the stable GitHub release exists +6. uploads the manifest and SQLite asset with `--clobber` + +## systemd Service + +Example service: [contrib/systemd/idx-ownership-snapshot-publish.service](/Users/rasyidanakbar/Development/myApp/idx-cli/contrib/systemd/idx-ownership-snapshot-publish.service) + +Important assumptions: + +- the repo checkout lives at `/srv/idx-cli` +- a writable publish directory exists at `/var/lib/idx-ownership-snapshot/current` +- `GH_TOKEN` is provided via an env file such as `/etc/idx-ownership-snapshot.env` +- the host can run `nix develop` + +## systemd Timer + +Example timer: [contrib/systemd/idx-ownership-snapshot-publish.timer](/Users/rasyidanakbar/Development/myApp/idx-cli/contrib/systemd/idx-ownership-snapshot-publish.timer) + +The sample timer uses: + +- `OnCalendar=*-*-02 09:00:00` +- `Persistent=true` +- `RandomizedDelaySec=30m` + +That is intentionally conservative. The ownership source is monthly, but the +exact publish day can drift. Start with an early-month schedule and adjust after +observing a few real runs. + +## Clan Integration Later + +The clean split for `clan-private` is: + +1. keep the workflow logic in this repo +2. keep host-specific secrets and enablement in `clan-private` + +Recommended integration shape on `greencloud-vps`: + +- store a checked-out copy of `idx-cli` on the host, for example at `/srv/idx-cli` +- provide `GH_TOKEN` through Clan vars or another secret mechanism +- add a small NixOS module that installs/enables the service and timer +- set the module's `WorkingDirectory`, env file path, and publish directory + +The existing pattern in `clan-private/modules/workspace-backup.nix` is a good +fit: a oneshot `systemd` service plus a `systemd.timer`. + +## Operational Notes + +- Keep `ownership-snapshot-current` separate from versioned app releases. +- Prefer manual runs first, then enable the timer after a few successful + publishes. +- If the monthly source has not appeared yet, the service should fail loudly and + be retried later; do not silently publish stale assumptions. diff --git a/scripts/publish-ownership-snapshot.sh b/scripts/publish-ownership-snapshot.sh new file mode 100755 index 0000000..dd88f31 --- /dev/null +++ b/scripts/publish-ownership-snapshot.sh @@ -0,0 +1,152 @@ +#!/usr/bin/env bash + +set -euo pipefail + +usage() { + cat <<'EOF' +Usage: scripts/publish-ownership-snapshot.sh --output-dir [options] + +Build and publish the latest supported IDX/KSEI ownership snapshot to the stable +GitHub release used by `idx ownership sync`. + +Options: + --idx-bin idx binary to use (default: ./target/debug/idx) + --output-dir Directory to write the copied snapshot and manifest + --repo GitHub repo used for release upload and public URLs + (default: 0xrsydn/idx-cli) + --release-tag Stable GitHub release tag used for snapshot assets + (default: ownership-snapshot-current) + --build Run `cargo build` before publishing + --keep-workdir Keep the temp workdir created by the builder helper + --help Show this help +EOF +} + +IDX_BIN="./target/debug/idx" +OUTPUT_DIR="" +REPO_FULL_NAME="0xrsydn/idx-cli" +RELEASE_TAG="ownership-snapshot-current" +BUILD_FIRST="0" +KEEP_WORKDIR="0" + +while [[ $# -gt 0 ]]; do + case "$1" in + --idx-bin) + IDX_BIN="${2:-}" + shift 2 + ;; + --output-dir) + OUTPUT_DIR="${2:-}" + shift 2 + ;; + --repo) + REPO_FULL_NAME="${2:-}" + shift 2 + ;; + --release-tag) + RELEASE_TAG="${2:-}" + shift 2 + ;; + --build) + BUILD_FIRST="1" + shift + ;; + --keep-workdir) + KEEP_WORKDIR="1" + shift + ;; + --help|-h) + usage + exit 0 + ;; + *) + echo "unknown argument: $1" >&2 + usage >&2 + exit 2 + ;; + esac +done + +if [[ -z "$OUTPUT_DIR" ]]; then + echo "--output-dir is required" >&2 + usage >&2 + exit 2 +fi + +if ! command -v gh >/dev/null 2>&1; then + echo "gh is required for GitHub release upload" >&2 + exit 1 +fi + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +BUILDER="$SCRIPT_DIR/build-latest-ownership-snapshot.sh" + +if [[ ! -x "$BUILDER" ]]; then + echo "required helper script is missing or not executable: $BUILDER" >&2 + exit 1 +fi + +if [[ "$BUILD_FIRST" == "1" ]]; then + printf 'Building idx...\n' + cargo build +fi + +if ! "$IDX_BIN" version >/dev/null 2>&1; then + echo "failed to run idx binary: $IDX_BIN" >&2 + echo "build the CLI first or pass --idx-bin " >&2 + exit 1 +fi + +build_args=( + --idx-bin "$IDX_BIN" + --output-dir "$OUTPUT_DIR" + --repo "$REPO_FULL_NAME" + --release-tag "$RELEASE_TAG" +) + +if [[ "$KEEP_WORKDIR" == "1" ]]; then + build_args+=(--keep-workdir) +fi + +printf 'Preparing latest ownership snapshot artifacts...\n' +"$BUILDER" "${build_args[@]}" + +MANIFEST_PATH="$OUTPUT_DIR/ownership-snapshot-manifest.json" +if [[ ! -f "$MANIFEST_PATH" ]]; then + echo "manifest was not generated: $MANIFEST_PATH" >&2 + exit 1 +fi + +shopt -s nullglob +sqlite_matches=("$OUTPUT_DIR"/ownership-snapshot-*.sqlite) +shopt -u nullglob + +if [[ "${#sqlite_matches[@]}" -ne 1 ]]; then + echo "expected exactly one SQLite artifact in $OUTPUT_DIR" >&2 + exit 1 +fi + +SQLITE_PATH="${sqlite_matches[0]}" + +if gh release view "$RELEASE_TAG" --repo "$REPO_FULL_NAME" >/dev/null 2>&1; then + printf 'Release %s already exists.\n' "$RELEASE_TAG" +else + printf 'Creating stable snapshot release %s...\n' "$RELEASE_TAG" + gh release create "$RELEASE_TAG" \ + --repo "$REPO_FULL_NAME" \ + --target "$(git rev-parse HEAD)" \ + --title "Ownership Snapshot Current" \ + --notes "Stable release for idx ownership snapshot artifacts consumed by \`idx ownership sync\`." \ + --latest=false +fi + +printf 'Uploading snapshot assets to %s...\n' "$RELEASE_TAG" +gh release upload "$RELEASE_TAG" \ + "$MANIFEST_PATH" \ + "$SQLITE_PATH" \ + --repo "$REPO_FULL_NAME" \ + --clobber + +printf 'Published manifest: https://github.com/%s/releases/download/%s/ownership-snapshot-manifest.json\n' \ + "$REPO_FULL_NAME" "$RELEASE_TAG" +printf 'Published SQLite: %s\n' "$SQLITE_PATH"