devsam/image

Fork 3

Files

T

Hide_D a935b23571 feat: manage uploaded image retention

2026-08-07 07:14:14 +00:00

12 KiB

Raw Blame History

Shared image repository

icons/ is served as /image/icons/. General portraits follow these rules:

icons/장수/<장수명>.<확장자> is the canonical shared path.
Scenario-specific collections keep a separate directory such as icons/걸그룹/ or icons/롤시나리오/.
Root-level numeric files such as icons/1047.jpg are compatibility aliases for deployed legacy versions. Do not delete or replace them while those versions are in service.
A general name must map to one canonical file. If two records intentionally need different portraits, disambiguate the scenario name or use an explicit scenario-specific path.

The core2026 repository owns resources/general-icons.json and tools/manage-general-icons.mjs. Use that tool with this repository as --image-root to synchronize aliases, scenario paths, and verify that every catalog source exists and has identical bytes.

Public URLs

The same tracked files are exposed through both URL contracts:

https://sam.hided.net/image/game/... and /image/icons/...
https://sam-image.hided.net/game/... and /icons/...
https://sam-image.hided.net/image/... is a compatibility alias.

Do not redirect the old /image/* URLs to the dedicated domain. Existing PHP and core2026 clients use same-origin relative paths and can move independently. The image server never exposes the repository root, .git, PHP sources, or directory listings.

Node webhook deployment service

compose.yaml runs a read-only Nginx static edge and an internal Node service. Only the edge publishes port 8191. The Node service verifies the raw Gitea X-Gitea-Signature, accepts push events for devsam/image, fetches the exact remote branch tip, rejects dirty or non-fast-forward updates, and serializes all Git changes. The initial and only allowed branch is master unless IMAGE_ALLOWED_BRANCHES is explicitly expanded.

The service is reverse-proxy agnostic. Caddy, Nginx, HAProxy, or another proxy can use the same port and path contract.

For a native reverse proxy running on the same host as Docker, publish port 8191 on loopback only and proxy to 127.0.0.1:8191:

IMAGE_BIND_ADDRESS=127.0.0.1
IMAGE_PORT=8191
TRUSTED_PROXY_CIDRS=172.24.0.1/32

These two addresses intentionally describe different hops. The native proxy connects to the host loopback address, while the Nginx process inside image-web normally observes the connection as coming from that Compose network's Docker bridge gateway (172.24.0.1 in the example). Use the source shown in the image-web access/error log, then recreate image-web whenever TRUSTED_PROXY_CIDRS changes. Do not replace it with a Cloudflare CIDR: the direct peer of image-web remains the host-native reverse proxy path.

If the reverse proxy is on another host, bind an explicit private interface instead of loopback and apply the host DOCKER-USER (or equivalent) firewall rule. Keep every source CIDR observed by image-web in the untracked TRUSTED_PROXY_CIDRS value, separated by commas or spaces. Avoid 0.0.0.0 unless the firewall has already been verified.

Prepare

cp .env.example .env
./deploy/scripts/init-secrets.sh
docker compose config --quiet
docker compose build

Only for a proxy on another host, apply and inspect the dedicated Docker ingress chain with root privileges:

sudo env TRUSTED_PROXY_CIDRS=192.0.2.10/32 IMAGE_PORT=8191 \
  ./deploy/scripts/firewall-8191.sh apply
sudo ./deploy/scripts/firewall-8191.sh check

The script only owns the SAM_IMAGE_INGRESS chain and its port-8191 jump from DOCKER-USER; it does not flush shared firewall chains.

For the same-host native-proxy case, point the proxy upstream at loopback. A minimal Caddy site is:

sam-image.hided.net {
    reverse_proxy 127.0.0.1:8191
}

The equivalent Nginx/HAProxy configuration should use the same upstream. The application does not depend on Caddy-specific request behavior.

Secret values are generated under ignored secrets/ files with mode 0600 and are never printed. Put the contents of secrets/gitea_webhook_secret into the Gitea webhook configuration. Configure a JSON push webhook targeting:

https://sam-image.hided.net/v1/hooks/gitea

Use branch filter master. Disable the old PHP webhook before enabling the new writer. The legacy PHP files remain in hook/ for an explicit rollback, but PHP and Node must never mutate the checkout concurrently.

Fallback sync callers

If Gitea webhook delivery is missed, Core and Core2026 can request a restricted reconciliation through POST /v1/sync. This endpoint cannot select or change a branch: it only fetches the current active branch and applies the same clean worktree and fast-forward checks as a webhook deployment.

Each caller has a separate secret:

secrets/image_sync_core_secret for legacy Core
secrets/image_sync_core2026_secret for Core2026

The caller sends its name, a timestamp, a unique request ID, and an HMAC-SHA256 signature over timestamp.request-id.<exact JSON body>. Requests expire after five minutes and successful request IDs are persisted for replay protection. The body is either {} or { "commit": "<full-image-commit-sha>" }; all other fields are rejected. The optional commit asserts the expected remote tip and does not grant checkout selection.

Distribute only the matching caller secret through an ignored secret file. Never give either caller image_admin_secret, which also authorizes explicit branch changes. This fallback handles webhook delivery outages; if the image service itself is stopped, restore it and run the caller command again.

User uploads

Core and Core2026 can store validated account icons and editor attachments through this service with PUT /v1/uploads/<user-icons|content>/<client>/<random-32-hex>.<extension>. Each game server validates the authenticated user and image first, then sends the raw image body with X-Image-Client, X-Image-Expires, X-Image-Request-Id, and X-Image-Signature headers.

The signature is HMAC-SHA256 over expires.requestId.pathname.contentType.sha256(body). Expiry may be at most five minutes in the future, so a grant cannot be reused for another path, content type, body, or later upload. The service also checks the image magic, caller scope, and request replay before it writes one immutable file below the host bind directory runtime-data/uploads. User uploads are deliberately not added to Git; Nginx exposes that bind read-only at /icons/users/ for account icons and /uploads/ for editor content. User icons retain the 50KB limit; editor content retains the existing 1MB limit.

Create separate upload secrets with deploy/scripts/init-secrets.sh. Mount only the matching image_upload_core_secret or image_upload_core2026_secret on the game server. The shared secrets stay server-side in Docker secrets; they are not returned to browsers or forwarded to Cloudflare.

Run deploy/scripts/init-secrets.sh before the first Compose start so the upload directory exists with permissions that allow the hook container to write and the Nginx container to read. IMAGE_UPLOAD_PATH can point only the uploaded images at a separate ZFS dataset. Keep IMAGE_RUNTIME_PATH on SSD storage for the small SQLite database and service state. Back up both paths independently of the Git repository when moving servers.

Retention and image administration

Account icons below /icons/users/ are permanent. They are inventoried in the metadata database but never become deletion candidates. Tiptap images below /uploads/ become candidates after 730 days without a request reaching the origin Nginx. Nginx mirrors a small internal notification to Node for those requests; Node deduplicates paths in memory and writes each path to SQLite at most once per flush interval (60 seconds by default).

Cloudflare and browser cache hits do not necessarily reach the origin. The two-year interval is therefore an operational expiry policy, not proof that an old HTML document no longer contains the URL. Confirm that the Cloudflare edge TTL does not exceed one year before enabling this policy.

Candidates remain publicly available until an administrator quarantines them. Quarantine moves the file into a hidden directory on the same upload bind and is reversible. Permanent deletion is disabled until the 30-day quarantine period has elapsed. A request received while an image is only a candidate returns it to active state; requests do not restore quarantined files.

Open the minimal administration panel at:

https://sam-image.hided.net/admin/

Log in with the value in the ignored secrets/image_admin_panel_password file. The panel can filter and preview assets, show storage and retention timestamps, quarantine candidates, restore quarantined files, and permanently delete files whose grace period has elapsed. It uses a separate short-lived, HttpOnly, SameSite session; the stronger image_admin_secret used for branch deployment is never sent to the browser. Rotate the panel password by replacing its secret file and recreating image-hook.

Retention settings are available in .env:

IMAGE_CONTENT_RETENTION_DAYS=730
IMAGE_CONTENT_QUARANTINE_DAYS=30
IMAGE_ASSET_TOUCH_FLUSH_SECONDS=60
IMAGE_ASSET_MAINTENANCE_SECONDS=21600

The SQLite database uses WAL and synchronous writes. Back it up with the service stopped or with a SQLite-aware backup; do not copy only the main .sqlite3 file while the service is running because committed data can still be in -wal.

Legacy HTTP mutation is disabled by default. An emergency PHP rollback must first stop image-hook, then create the ignored hook/legacy-enabled sentinel in the legacy checkout before restoring its Caddy/Gitea route. Remove the sentinel before Node is started again. CLI execution of hook/git_pull.php from the hook/ directory remains available for local recovery without exposing the HTTP endpoint.

Start and verify

docker compose up -d --build
docker compose ps
curl -fsS https://sam-image.hided.net/healthz
curl -fsS https://sam-image.hided.net/v1/status
curl -fsS https://sam-image.hided.net/game/back.jpg -o /dev/null
curl -fsS https://sam-image.hided.net/image/icons/default.jpg -o /dev/null

Then log in to /admin/ and verify that an existing account icon is shown as permanent (user-icons, active) and a Tiptap upload is shown as content.

Nginx serves the static game/, icons/, and uploads/ paths, the two public inventory files, and the authenticated /admin/ panel. The API inventory additionally reports the deployed branch, full commit, generation time, asset list, and both public base URLs.

Explicit branch deployment

Automatic pushes only update the active branch. Add a branch to IMAGE_ALLOWED_BRANCHES, recreate image-hook, and switch it with the internal signed administration command:

./deploy/scripts/admin-deploy.sh <branch> [expected-commit]

The signed branch-deployment route under /v1/admin/ is not exposed through the public reverse proxy. It is separate from the password-authenticated image management panel at /admin/.

Tests and rollback

docker build -t sam-image-hook:test node-hook
docker run --rm sam-image-hook:test npm test
docker compose exec -T image-web nginx -t -c /tmp/nginx.conf

Stopping image-hook does not remove the last checked-out files from the static service. For a full rollback, stop Node mutation first, restore the old Caddy/PHP webhook route, and only then enable the legacy writer. Preserve both checkouts until public file hashes and representative browser requests have been verified.

12 KiB Raw Blame History