test: 300접속 capacity calibration을 재현 가능하게 구성
비밀값 자동 준비, 4 CPU/8GiB API launcher, fixture 검증 기반 calibration config를 추가한다. tRPC v11 GET input encoding 오류를 고치고 300 SSE/HTTP 실제 실행 절차를 문서화한다.
This commit is contained in:
+43
-24
@@ -30,24 +30,23 @@ unchanged/snapshot/patch 경로를 구분하며 raw JSON에는 종류별 count
|
|||||||
|
|
||||||
### 1. 격리 PostgreSQL/Redis와 1,200장수 fixture
|
### 1. 격리 PostgreSQL/Redis와 1,200장수 fixture
|
||||||
|
|
||||||
아래 Compose는 loopback에만 포트를 열고 PostgreSQL 18.4, Redis 8.2.7을 고정한다. 실제 password와 URL은
|
아래 Compose는 loopback에만 포트를 열고 PostgreSQL 18.4, Redis 8.2.7을 고정한다. PostgreSQL 18의
|
||||||
Git ignored `secrets/`에 두며 명령행이나 결과 JSON에는 기록하지 않는다. `capacity.env`에는 최소
|
versioned data-directory 계약에 맞춰 volume은 `/var/lib/postgresql`에 붙인다. host port는 기본
|
||||||
`LOAD_TEST_DATABASE_URL`(query의 `schema=load_capacity_300_900_5m` 포함),
|
`15442/16379`이며 `CAPACITY_POSTGRES_PORT`/`CAPACITY_REDIS_PORT`로 충돌 없이 바꿀 수 있다. `prepare`는
|
||||||
`LOAD_TEST_REDIS_URL`(`/15` 포함), API 기동에 필요한 `GAME_TOKEN_SECRET`,
|
PostgreSQL password, API token/image secret과 정확한 URL을 무작위 생성해 Git ignored `secrets/`의 새
|
||||||
`GAME_IMAGE_UPLOAD_SECRET_FILE`을 넣는다. URL의 password는 percent-encoding한다.
|
파일 세 개에 `0600`으로 저장한다. 기존 파일을 덮어쓰거나 비밀값을 stdout에 쓰지 않는다.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
install -m 600 /dev/null tools/load-tests/secrets/postgres-password.txt
|
pnpm --filter @sammo-ts/load-tests prepare:capacity \
|
||||||
install -m 600 /dev/null tools/load-tests/secrets/capacity.env
|
--config tools/load-tests/config/300-users-900-npcs-5m.json
|
||||||
# 두 파일은 로컬 편집기로 채우고 내용을 stdout에 출력하지 않는다.
|
|
||||||
|
|
||||||
docker compose -f tools/load-tests/compose.capacity.yml config --quiet
|
|
||||||
docker compose -f tools/load-tests/compose.capacity.yml up -d --wait
|
|
||||||
|
|
||||||
set -a
|
set -a
|
||||||
source tools/load-tests/secrets/capacity.env
|
source tools/load-tests/secrets/capacity.env
|
||||||
set +a
|
set +a
|
||||||
|
|
||||||
|
docker compose -f tools/load-tests/compose.capacity.yml config --quiet
|
||||||
|
docker compose -f tools/load-tests/compose.capacity.yml up -d --wait
|
||||||
|
|
||||||
pnpm --filter @sammo-ts/load-tests seed \
|
pnpm --filter @sammo-ts/load-tests seed \
|
||||||
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
||||||
--tokens tools/load-tests/secrets/game-tokens.json
|
--tokens tools/load-tests/secrets/game-tokens.json
|
||||||
@@ -64,20 +63,21 @@ fixture와 같은 환경으로 API를 띄울 때 핵심 namespace는 다음과
|
|||||||
풀어 쓰지 않는다.
|
풀어 쓰지 않는다.
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
export DATABASE_URL="$LOAD_TEST_DATABASE_URL"
|
pnpm --filter @sammo-ts/common build
|
||||||
export REDIS_URL="$LOAD_TEST_REDIS_URL"
|
pnpm --filter @sammo-ts/logic build
|
||||||
export PROFILE=load_capacity_300_900_5m
|
pnpm --filter @sammo-ts/infra build
|
||||||
export SCENARIO=2601
|
pnpm --filter @sammo-ts/game-engine build
|
||||||
export GAME_PROFILE_NAME=load-tests:capacity-300-900-5m
|
pnpm --filter @sammo-ts/game-api build
|
||||||
export GAME_API_HOST=127.0.0.1
|
|
||||||
export GAME_API_PORT=15001
|
systemd-run --user --unit=sammo-capacity-api --collect \
|
||||||
export GAME_TRPC_PATH=/api/trpc
|
--property=MemoryMax=8G \
|
||||||
export GAME_API_EVENTS_PATH=/events
|
--working-directory="$(pwd)" \
|
||||||
pnpm --filter @sammo-ts/game-api start
|
"$(pwd)/tools/load-tests/scripts/run-capacity-api.sh"
|
||||||
```
|
```
|
||||||
|
|
||||||
API와 driver는 별도 terminal/process로 실행한다. dev-sam2026와 같은 판정이 필요하면 API/engine container에
|
runner script는 `capacity.env`의 Node binary를 사용하고 `taskset 0-3`으로 API를 4 logical CPU에 제한한다.
|
||||||
4 CPU/8 GiB 제한을 주고 driver는 그 cgroup 밖에서 실행하며, image digest를 아래 run config에 기록한다.
|
systemd unit은 API에 8 GiB memory limit을 적용한다. `systemctl --user show`로 얻은 main PID의 affinity와
|
||||||
|
unit `MemoryMax`를 각각 `taskset -pc`와 `systemctl --user show`로 확인한다. API와 driver는 별도 process로 실행한다.
|
||||||
공개 `dev-sam2026.hided.net` profile에는 이 fixture나 driver를 연결하지 않는다.
|
공개 `dev-sam2026.hided.net` profile에는 이 fixture나 driver를 연결하지 않는다.
|
||||||
|
|
||||||
### 2. 인증 HTTP/SSE driver
|
### 2. 인증 HTTP/SSE driver
|
||||||
@@ -90,7 +90,7 @@ pnpm --filter @sammo-ts/load-tests validate --config tools/load-tests/config/300
|
|||||||
pnpm --filter @sammo-ts/load-tests dry-run \
|
pnpm --filter @sammo-ts/load-tests dry-run \
|
||||||
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
||||||
--tokens tools/load-tests/secrets/game-tokens.json
|
--tokens tools/load-tests/secrets/game-tokens.json
|
||||||
pnpm --filter @sammo-ts/load-tests run \
|
pnpm --filter @sammo-ts/load-tests run run \
|
||||||
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
||||||
--tokens tools/load-tests/secrets/game-tokens.json \
|
--tokens tools/load-tests/secrets/game-tokens.json \
|
||||||
--output tools/load-tests/results/300-users-900-npcs-5m.json
|
--output tools/load-tests/results/300-users-900-npcs-5m.json
|
||||||
@@ -101,6 +101,25 @@ pnpm --filter @sammo-ts/load-tests run \
|
|||||||
host/cgroup에서 실행하고 두 host의 CPU quota와 competing load를 별도로 기록한다.
|
host/cgroup에서 실행하고 두 host의 CPU quota와 competing load를 별도로 기록한다.
|
||||||
`run`은 sample의 runtime metadata placeholder가 하나라도 남아 있으면 시작하지 않는다.
|
`run`은 sample의 runtime metadata placeholder가 하나라도 남아 있으면 시작하지 않는다.
|
||||||
|
|
||||||
|
300 SSE/HTTP의 짧은 연결·query calibration은 검증된 fixture에서 ignored runtime config를 먼저 만든다.
|
||||||
|
기본 calibration은 idle 5초와 own/global/mixed 각 10초이며 capacity 합격 판정용 soak test가 아니다.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pnpm --filter @sammo-ts/load-tests materialize-calibration \
|
||||||
|
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
||||||
|
--output tools/load-tests/results/calibration-config.json
|
||||||
|
|
||||||
|
pnpm --filter @sammo-ts/load-tests run run \
|
||||||
|
--config tools/load-tests/results/calibration-config.json \
|
||||||
|
--tokens tools/load-tests/secrets/game-tokens.json \
|
||||||
|
--output tools/load-tests/results/calibration-result.json
|
||||||
|
```
|
||||||
|
|
||||||
|
`materialize-calibration`은 DB/Redis count와 manifest/hash를 다시 확인하고 실제 PostgreSQL/Redis version을
|
||||||
|
기록한다. `LOAD_TEST_IMAGE_DIGEST`가 없으면 image 결과라고 부르지 않고 현재 dirty source-tree commit을
|
||||||
|
명시한다. driver JSON의 process CPU/RSS는 driver 자체 값이다. API target 값은 systemd unit의
|
||||||
|
`CPUUsageNSec`, `MemoryCurrent`, `MemoryPeak`를 run 직전/직후 별도로 수집한다.
|
||||||
|
|
||||||
## 결과와 해석
|
## 결과와 해석
|
||||||
|
|
||||||
raw JSON은 Git commit/tree/dirty 상태, config/runtime/host hash, Node/V8, host CPU/memory와 cgroup limit,
|
raw JSON은 Git commit/tree/dirty 상태, config/runtime/host hash, Node/V8, host CPU/memory와 cgroup limit,
|
||||||
|
|||||||
@@ -7,8 +7,10 @@
|
|||||||
"run": "pnpm -w exec tsx tools/load-tests/src/cli.ts run",
|
"run": "pnpm -w exec tsx tools/load-tests/src/cli.ts run",
|
||||||
"dry-run": "pnpm -w exec tsx tools/load-tests/src/cli.ts dry-run",
|
"dry-run": "pnpm -w exec tsx tools/load-tests/src/cli.ts dry-run",
|
||||||
"validate": "pnpm -w exec tsx tools/load-tests/src/cli.ts validate",
|
"validate": "pnpm -w exec tsx tools/load-tests/src/cli.ts validate",
|
||||||
|
"prepare:capacity": "pnpm -w exec tsx tools/load-tests/src/cli.ts prepare",
|
||||||
"seed": "pnpm -w exec tsx tools/load-tests/src/cli.ts seed",
|
"seed": "pnpm -w exec tsx tools/load-tests/src/cli.ts seed",
|
||||||
"verify-fixture": "pnpm -w exec tsx tools/load-tests/src/cli.ts verify-fixture",
|
"verify-fixture": "pnpm -w exec tsx tools/load-tests/src/cli.ts verify-fixture",
|
||||||
|
"materialize-calibration": "pnpm -w exec tsx tools/load-tests/src/cli.ts materialize-calibration",
|
||||||
"cleanup": "pnpm -w exec tsx tools/load-tests/src/cli.ts cleanup",
|
"cleanup": "pnpm -w exec tsx tools/load-tests/src/cli.ts cleanup",
|
||||||
"test": "pnpm -w exec tsx --test tools/load-tests/test/*.test.ts",
|
"test": "pnpm -w exec tsx --test tools/load-tests/test/*.test.ts",
|
||||||
"typecheck": "pnpm -w tsc7 -p tools/load-tests/tsconfig.json --noEmit"
|
"typecheck": "pnpm -w tsc7 -p tools/load-tests/tsconfig.json --noEmit"
|
||||||
|
|||||||
Executable
+25
@@ -0,0 +1,25 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
workspace_root=$(CDPATH= cd -- "$(dirname -- "$0")/../../.." && pwd)
|
||||||
|
secret_env="$workspace_root/tools/load-tests/secrets/capacity.env"
|
||||||
|
|
||||||
|
if [ ! -f "$secret_env" ]; then
|
||||||
|
echo "capacity.env is missing; run the load-test prepare command first" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
secret_mode=$(stat -c '%a' "$secret_env")
|
||||||
|
if [ "$secret_mode" != "600" ]; then
|
||||||
|
echo "capacity.env must have mode 0600" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
. "$secret_env"
|
||||||
|
set +a
|
||||||
|
|
||||||
|
export GAME_API_ROLE=server
|
||||||
|
cd "$workspace_root"
|
||||||
|
exec taskset -c "${CAPACITY_CPUSET:-0-3}" "${CAPACITY_NODE_BINARY:-node}" app/game-api/dist/index.js
|
||||||
@@ -2,19 +2,25 @@ import { mkdir, writeFile } from 'node:fs/promises';
|
|||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
|
|
||||||
import { assertRuntimeMetadataFinalized, loadConfig, loadTokens } from './config.js';
|
import { assertRuntimeMetadataFinalized, loadConfig, loadTokens } from './config.js';
|
||||||
import { cleanupCapacityFixture, seedCapacityFixture, verifyCapacityFixture } from './fixture.js';
|
import {
|
||||||
|
cleanupCapacityFixture,
|
||||||
|
materializeCalibrationConfig,
|
||||||
|
prepareCapacitySecrets,
|
||||||
|
seedCapacityFixture,
|
||||||
|
verifyCapacityFixture,
|
||||||
|
} from './fixture.js';
|
||||||
import { describeDryRun, runLoadTest } from './runner.js';
|
import { describeDryRun, runLoadTest } from './runner.js';
|
||||||
|
|
||||||
type Command = 'run' | 'dry-run' | 'validate' | 'seed' | 'verify-fixture' | 'cleanup';
|
type Command = 'run' | 'dry-run' | 'validate' | 'prepare' | 'seed' | 'verify-fixture' | 'materialize-calibration' | 'cleanup';
|
||||||
|
|
||||||
const usage = (): never => {
|
const usage = (): never => {
|
||||||
process.stderr.write('usage: cli.ts <validate|dry-run|run|seed|verify-fixture|cleanup> --config <file> [--tokens <0600-gitignored-file>] [--output <new-json-file>] [--confirm <load_schema>]\n');
|
process.stderr.write('usage: cli.ts <validate|dry-run|run|prepare|seed|verify-fixture|materialize-calibration|cleanup> --config <file> [--tokens <0600-gitignored-file>] [--output <new-json-file>] [--confirm <load_schema>]\n');
|
||||||
process.exit(64);
|
process.exit(64);
|
||||||
};
|
};
|
||||||
|
|
||||||
const parseArguments = (argv: readonly string[]): { command: Command; config: string; tokens?: string; output?: string; confirm?: string } => {
|
const parseArguments = (argv: readonly string[]): { command: Command; config: string; tokens?: string; output?: string; confirm?: string } => {
|
||||||
const command = argv[0];
|
const command = argv[0];
|
||||||
if (!['run', 'dry-run', 'validate', 'seed', 'verify-fixture', 'cleanup'].includes(command ?? '')) usage();
|
if (!['run', 'dry-run', 'validate', 'prepare', 'seed', 'verify-fixture', 'materialize-calibration', 'cleanup'].includes(command ?? '')) usage();
|
||||||
const values = new Map<string, string>();
|
const values = new Map<string, string>();
|
||||||
for (let index = 1; index < argv.length; index += 2) {
|
for (let index = 1; index < argv.length; index += 2) {
|
||||||
const flag = argv[index];
|
const flag = argv[index];
|
||||||
@@ -26,8 +32,10 @@ const parseArguments = (argv: readonly string[]): { command: Command; config: st
|
|||||||
if (!config) usage();
|
if (!config) usage();
|
||||||
if (command === 'run' && (!values.get('--tokens') || !values.get('--output'))) usage();
|
if (command === 'run' && (!values.get('--tokens') || !values.get('--output'))) usage();
|
||||||
if (command === 'seed' && (!values.get('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
if (command === 'seed' && (!values.get('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
||||||
|
if (command === 'prepare' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
||||||
if (command === 'cleanup' && (!values.get('--confirm') || values.has('--tokens') || values.has('--output'))) usage();
|
if (command === 'cleanup' && (!values.get('--confirm') || values.has('--tokens') || values.has('--output'))) usage();
|
||||||
if (command === 'verify-fixture' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
if (command === 'verify-fixture' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
||||||
|
if (command === 'materialize-calibration' && (!values.get('--output') || values.has('--tokens') || values.has('--confirm'))) usage();
|
||||||
if (command === 'validate' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
if (command === 'validate' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
|
||||||
if (command === 'dry-run' && (values.has('--output') || values.has('--confirm'))) usage();
|
if (command === 'dry-run' && (values.has('--output') || values.has('--confirm'))) usage();
|
||||||
if (command === 'run' && values.has('--confirm')) usage();
|
if (command === 'run' && values.has('--confirm')) usage();
|
||||||
@@ -57,10 +65,26 @@ const main = async (): Promise<void> => {
|
|||||||
process.stdout.write(`${JSON.stringify(result)}\n`);
|
process.stdout.write(`${JSON.stringify(result)}\n`);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if (args.command === 'prepare') {
|
||||||
|
process.stdout.write(`${JSON.stringify(await prepareCapacitySecrets({ config, workspaceRoot }))}\n`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (args.command === 'verify-fixture') {
|
if (args.command === 'verify-fixture') {
|
||||||
process.stdout.write(`${JSON.stringify(await verifyCapacityFixture(config))}\n`);
|
process.stdout.write(`${JSON.stringify(await verifyCapacityFixture(config))}\n`);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
if (args.command === 'materialize-calibration') {
|
||||||
|
process.stdout.write(
|
||||||
|
`${JSON.stringify(
|
||||||
|
await materializeCalibrationConfig({
|
||||||
|
config,
|
||||||
|
outputPath: args.output!,
|
||||||
|
workspaceRoot,
|
||||||
|
})
|
||||||
|
)}\n`
|
||||||
|
);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if (args.command === 'cleanup') {
|
if (args.command === 'cleanup') {
|
||||||
process.stdout.write(`${JSON.stringify(await cleanupCapacityFixture(config, args.confirm!))}\n`);
|
process.stdout.write(`${JSON.stringify(await cleanupCapacityFixture(config, args.confirm!))}\n`);
|
||||||
return;
|
return;
|
||||||
|
|||||||
+160
-17
@@ -1,6 +1,6 @@
|
|||||||
import { execFile } from 'node:child_process';
|
import { execFile } from 'node:child_process';
|
||||||
import { randomUUID } from 'node:crypto';
|
import { randomBytes, randomUUID } from 'node:crypto';
|
||||||
import { lstat, mkdir, open, realpath } from 'node:fs/promises';
|
import { lstat, mkdir, open, realpath, unlink } from 'node:fs/promises';
|
||||||
import path from 'node:path';
|
import path from 'node:path';
|
||||||
import { promisify } from 'node:util';
|
import { promisify } from 'node:util';
|
||||||
|
|
||||||
@@ -8,7 +8,7 @@ import { seedScenarioToDatabase } from '@sammo-ts/game-engine';
|
|||||||
import {
|
import {
|
||||||
createGamePostgresConnector,
|
createGamePostgresConnector,
|
||||||
createRedisConnector,
|
createRedisConnector,
|
||||||
type GamePrisma,
|
GamePrisma,
|
||||||
type GamePrismaClient,
|
type GamePrismaClient,
|
||||||
} from '@sammo-ts/infra';
|
} from '@sammo-ts/infra';
|
||||||
|
|
||||||
@@ -20,6 +20,105 @@ const SCENARIO_ID = 2601;
|
|||||||
|
|
||||||
type FixtureEnvironment = { databaseUrl: string; redisUrl: string };
|
type FixtureEnvironment = { databaseUrl: string; redisUrl: string };
|
||||||
|
|
||||||
|
const assertNewSecretPath = async (secretPath: string, workspaceRoot: string): Promise<string> => {
|
||||||
|
const secretRoot = await realpath(path.join(workspaceRoot, 'tools/load-tests/secrets'));
|
||||||
|
const output = path.resolve(secretPath);
|
||||||
|
const relative = path.relative(secretRoot, output);
|
||||||
|
if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) {
|
||||||
|
throw new Error('generated secret files must stay inside tools/load-tests/secrets');
|
||||||
|
}
|
||||||
|
await mkdir(path.dirname(output), { recursive: true, mode: 0o700 });
|
||||||
|
if ((await realpath(path.dirname(output))) !== secretRoot) {
|
||||||
|
throw new Error('generated secret files must not traverse a symbolic-link directory');
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await lstat(output);
|
||||||
|
throw new Error('generated secret output already exists');
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
|
||||||
|
}
|
||||||
|
return output;
|
||||||
|
};
|
||||||
|
|
||||||
|
const writeNewSecret = async (secretPath: string, content: string): Promise<void> => {
|
||||||
|
const handle = await open(secretPath, 'wx', 0o600);
|
||||||
|
try {
|
||||||
|
await handle.writeFile(content, { encoding: 'utf8' });
|
||||||
|
} finally {
|
||||||
|
await handle.close();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
export const prepareCapacitySecrets = async (options: {
|
||||||
|
config: LoadConfig;
|
||||||
|
workspaceRoot: string;
|
||||||
|
env?: NodeJS.ProcessEnv;
|
||||||
|
}) => {
|
||||||
|
const env = options.env ?? process.env;
|
||||||
|
const postgresPort = Number(env.CAPACITY_POSTGRES_PORT ?? '15442');
|
||||||
|
const redisPort = Number(env.CAPACITY_REDIS_PORT ?? '16379');
|
||||||
|
if (!Number.isSafeInteger(postgresPort) || postgresPort < 1024 || postgresPort > 65_535) {
|
||||||
|
throw new Error('CAPACITY_POSTGRES_PORT must be an unprivileged TCP port');
|
||||||
|
}
|
||||||
|
if (!Number.isSafeInteger(redisPort) || redisPort < 1024 || redisPort > 65_535) {
|
||||||
|
throw new Error('CAPACITY_REDIS_PORT must be an unprivileged TCP port');
|
||||||
|
}
|
||||||
|
if (postgresPort === redisPort) throw new Error('capacity PostgreSQL and Redis ports must differ');
|
||||||
|
|
||||||
|
const secretRoot = path.join(options.workspaceRoot, 'tools/load-tests/secrets');
|
||||||
|
const postgresPasswordPath = await assertNewSecretPath(
|
||||||
|
path.join(secretRoot, 'postgres-password.txt'),
|
||||||
|
options.workspaceRoot
|
||||||
|
);
|
||||||
|
const imageSecretPath = await assertNewSecretPath(
|
||||||
|
path.join(secretRoot, 'image-upload-secret.txt'),
|
||||||
|
options.workspaceRoot
|
||||||
|
);
|
||||||
|
const capacityEnvPath = await assertNewSecretPath(path.join(secretRoot, 'capacity.env'), options.workspaceRoot);
|
||||||
|
|
||||||
|
const postgresPassword = randomBytes(32).toString('hex');
|
||||||
|
const gameTokenSecret = randomBytes(32).toString('hex');
|
||||||
|
const imageUploadSecret = randomBytes(32).toString('hex');
|
||||||
|
const databaseUrl =
|
||||||
|
`postgresql://sammo_capacity:${postgresPassword}@127.0.0.1:${postgresPort}/sammo_capacity` +
|
||||||
|
`?schema=${options.config.isolation.postgresSchema}`;
|
||||||
|
const redisUrl = `redis://127.0.0.1:${redisPort}/${options.config.isolation.redisDatabase}`;
|
||||||
|
const envLines = [
|
||||||
|
`CAPACITY_POSTGRES_PORT='${postgresPort}'`,
|
||||||
|
`CAPACITY_REDIS_PORT='${redisPort}'`,
|
||||||
|
`LOAD_TEST_DATABASE_URL='${databaseUrl}'`,
|
||||||
|
`LOAD_TEST_REDIS_URL='${redisUrl}'`,
|
||||||
|
`DATABASE_URL='${databaseUrl}'`,
|
||||||
|
`REDIS_URL='${redisUrl}'`,
|
||||||
|
`GAME_TOKEN_SECRET='${gameTokenSecret}'`,
|
||||||
|
`GAME_IMAGE_UPLOAD_SECRET_FILE='${imageSecretPath}'`,
|
||||||
|
`PROFILE='${options.config.isolation.postgresSchema}'`,
|
||||||
|
`SCENARIO='2601'`,
|
||||||
|
`GAME_PROFILE_NAME='${options.config.isolation.profileName}'`,
|
||||||
|
`GAME_API_HOST='127.0.0.1'`,
|
||||||
|
`GAME_API_PORT='${new URL(options.config.target.baseUrl).port || '80'}'`,
|
||||||
|
`GAME_TRPC_PATH='${options.config.target.trpcPath}'`,
|
||||||
|
`GAME_API_EVENTS_PATH='${options.config.target.ssePath}'`,
|
||||||
|
`CAPACITY_NODE_BINARY='${process.execPath}'`,
|
||||||
|
"CAPACITY_CPUSET='0-3'",
|
||||||
|
'',
|
||||||
|
].join('\n');
|
||||||
|
|
||||||
|
const created: string[] = [];
|
||||||
|
try {
|
||||||
|
await writeNewSecret(postgresPasswordPath, `${postgresPassword}\n`);
|
||||||
|
created.push(postgresPasswordPath);
|
||||||
|
await writeNewSecret(imageSecretPath, `${imageUploadSecret}\n`);
|
||||||
|
created.push(imageSecretPath);
|
||||||
|
await writeNewSecret(capacityEnvPath, envLines);
|
||||||
|
created.push(capacityEnvPath);
|
||||||
|
} catch (error) {
|
||||||
|
await Promise.all(created.map((file) => unlink(file).catch(() => undefined)));
|
||||||
|
throw error;
|
||||||
|
}
|
||||||
|
return { prepared: true, secretFilesWritten: created.length, mode: '0600' };
|
||||||
|
};
|
||||||
|
|
||||||
const requireEnvironment = (env: NodeJS.ProcessEnv): FixtureEnvironment => {
|
const requireEnvironment = (env: NodeJS.ProcessEnv): FixtureEnvironment => {
|
||||||
const databaseUrl = env.LOAD_TEST_DATABASE_URL;
|
const databaseUrl = env.LOAD_TEST_DATABASE_URL;
|
||||||
const redisUrl = env.LOAD_TEST_REDIS_URL;
|
const redisUrl = env.LOAD_TEST_REDIS_URL;
|
||||||
@@ -177,21 +276,10 @@ const resizeSeededGenerals = async (db: GamePrismaClient, config: LoadConfig): P
|
|||||||
};
|
};
|
||||||
|
|
||||||
const assertNewTokenPath = async (tokenPath: string, workspaceRoot: string): Promise<string> => {
|
const assertNewTokenPath = async (tokenPath: string, workspaceRoot: string): Promise<string> => {
|
||||||
const secretRoot = await realpath(path.join(workspaceRoot, 'tools/load-tests/secrets'));
|
const output = await assertNewSecretPath(tokenPath, workspaceRoot);
|
||||||
const output = path.resolve(tokenPath);
|
if (path.extname(output) !== '.json') {
|
||||||
const relative = path.relative(secretRoot, output);
|
|
||||||
if (!relative || relative.startsWith('..') || path.isAbsolute(relative) || path.extname(output) !== '.json') {
|
|
||||||
throw new Error('seed token output must be a new JSON file inside tools/load-tests/secrets');
|
throw new Error('seed token output must be a new JSON file inside tools/load-tests/secrets');
|
||||||
}
|
}
|
||||||
await mkdir(path.dirname(output), { recursive: true, mode: 0o700 });
|
|
||||||
const parent = await realpath(path.dirname(output));
|
|
||||||
if (parent !== secretRoot) throw new Error('seed token output must not traverse a symbolic-link directory');
|
|
||||||
try {
|
|
||||||
await lstat(output);
|
|
||||||
throw new Error('seed token output already exists');
|
|
||||||
} catch (error) {
|
|
||||||
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
|
|
||||||
}
|
|
||||||
return output;
|
return output;
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -313,7 +401,10 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
|
|||||||
const redis = createRedisConnector({ url: environment.redisUrl });
|
const redis = createRedisConnector({ url: environment.redisUrl });
|
||||||
await postgres.connect();
|
await postgres.connect();
|
||||||
try {
|
try {
|
||||||
const state = await projectFixtureState(postgres.prisma);
|
const [state, postgresRows] = await Promise.all([
|
||||||
|
projectFixtureState(postgres.prisma),
|
||||||
|
postgres.prisma.$queryRaw<Array<{ version: string }>>(GamePrisma.sql`SELECT version()`),
|
||||||
|
]);
|
||||||
const fixtureSha256 = `sha256:${sha256(canonicalJson(state))}`;
|
const fixtureSha256 = `sha256:${sha256(canonicalJson(state))}`;
|
||||||
await redis.connect();
|
await redis.connect();
|
||||||
try {
|
try {
|
||||||
@@ -329,6 +420,8 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
const accessTokens = await countMatchingRedisKeys(redis.client, `${accessKeyPrefix(config)}ga_*`);
|
const accessTokens = await countMatchingRedisKeys(redis.client, `${accessKeyPrefix(config)}ga_*`);
|
||||||
|
const redisInfo = await redis.client.info('server');
|
||||||
|
const redisVersion = /^redis_version:(.+)$/mu.exec(redisInfo)?.[1]?.trim() ?? 'unknown';
|
||||||
const npcGenerals = state.generals.filter((general) => general.npcState >= 2).length;
|
const npcGenerals = state.generals.filter((general) => general.npcState >= 2).length;
|
||||||
const humanGenerals = state.generals.filter(
|
const humanGenerals = state.generals.filter(
|
||||||
(general) => general.npcState === 0 && general.userId
|
(general) => general.npcState === 0 && general.userId
|
||||||
@@ -348,6 +441,8 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
|
|||||||
accessTokens,
|
accessTokens,
|
||||||
redisManifestPresent: rawManifest !== null,
|
redisManifestPresent: rawManifest !== null,
|
||||||
redisManifestMatches: manifestFixtureSha256 === fixtureSha256,
|
redisManifestMatches: manifestFixtureSha256 === fixtureSha256,
|
||||||
|
postgresVersion: postgresRows[0]?.version ?? 'unknown',
|
||||||
|
redisVersion,
|
||||||
};
|
};
|
||||||
} finally {
|
} finally {
|
||||||
await redis.disconnect();
|
await redis.disconnect();
|
||||||
@@ -357,6 +452,54 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
|
export const materializeCalibrationConfig = async (options: {
|
||||||
|
config: LoadConfig;
|
||||||
|
outputPath: string;
|
||||||
|
workspaceRoot: string;
|
||||||
|
env?: NodeJS.ProcessEnv;
|
||||||
|
}) => {
|
||||||
|
const env = options.env ?? process.env;
|
||||||
|
const output = path.resolve(options.outputPath);
|
||||||
|
const resultsRoot = await realpath(path.join(options.workspaceRoot, 'tools/load-tests/results'));
|
||||||
|
const relative = path.relative(resultsRoot, output);
|
||||||
|
if (!relative || relative.startsWith('..') || path.isAbsolute(relative) || path.extname(output) !== '.json') {
|
||||||
|
throw new Error('calibration config output must be a new JSON file inside tools/load-tests/results');
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
await lstat(output);
|
||||||
|
throw new Error('calibration config output already exists');
|
||||||
|
} catch (error) {
|
||||||
|
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
|
||||||
|
}
|
||||||
|
const verified = await verifyCapacityFixture(options.config, env);
|
||||||
|
if (!verified.valid) throw new Error('fixture verification failed; refusing to materialize calibration config');
|
||||||
|
const gitCommit = (
|
||||||
|
await execFileAsync('git', ['rev-parse', 'HEAD'], { cwd: options.workspaceRoot })
|
||||||
|
).stdout.trim();
|
||||||
|
const runtimeConfig: LoadConfig = {
|
||||||
|
...options.config,
|
||||||
|
name: `${options.config.name}-calibration`,
|
||||||
|
runtimeMetadata: {
|
||||||
|
fixtureSha256: verified.fixtureSha256,
|
||||||
|
imageDigest: env.LOAD_TEST_IMAGE_DIGEST ?? `source-tree:${gitCommit}:dirty`,
|
||||||
|
postgresVersion: verified.postgresVersion,
|
||||||
|
redisVersion: verified.redisVersion,
|
||||||
|
},
|
||||||
|
phases: options.config.phases.map((phase) => ({
|
||||||
|
...phase,
|
||||||
|
name: phase.name.replace(/-(?:5|10|30)m$/u, '-calibration'),
|
||||||
|
durationMs: phase.kind === 'idle' ? 5_000 : 10_000,
|
||||||
|
})),
|
||||||
|
};
|
||||||
|
await writeNewSecret(output, `${JSON.stringify(runtimeConfig, null, 2)}\n`);
|
||||||
|
return {
|
||||||
|
materialized: true,
|
||||||
|
fixtureSha256: verified.fixtureSha256,
|
||||||
|
phaseDurationMs: runtimeConfig.phases.map((phase) => phase.durationMs),
|
||||||
|
runtimeKind: env.LOAD_TEST_IMAGE_DIGEST ? 'image' : 'dirty-source-tree',
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
export const cleanupCapacityFixture = async (
|
export const cleanupCapacityFixture = async (
|
||||||
config: LoadConfig,
|
config: LoadConfig,
|
||||||
confirmation: string,
|
confirmation: string,
|
||||||
|
|||||||
@@ -9,7 +9,7 @@ export interface TrpcRequest {
|
|||||||
export const buildTrpcQuery = (baseUrl: string, trpcPath: string, operation: LoadOperation, token: string): TrpcRequest => {
|
export const buildTrpcQuery = (baseUrl: string, trpcPath: string, operation: LoadOperation, token: string): TrpcRequest => {
|
||||||
const normalizedPath = trpcPath.endsWith('/') ? trpcPath.slice(0, -1) : trpcPath;
|
const normalizedPath = trpcPath.endsWith('/') ? trpcPath.slice(0, -1) : trpcPath;
|
||||||
const url = new URL(`${normalizedPath}/${operation.procedure}`, baseUrl);
|
const url = new URL(`${normalizedPath}/${operation.procedure}`, baseUrl);
|
||||||
if (operation.input !== undefined) url.searchParams.set('input', JSON.stringify({ json: operation.input }));
|
if (operation.input !== undefined) url.searchParams.set('input', JSON.stringify(operation.input));
|
||||||
return {
|
return {
|
||||||
url: url.toString(),
|
url: url.toString(),
|
||||||
init: {
|
init: {
|
||||||
|
|||||||
@@ -1,9 +1,11 @@
|
|||||||
import assert from 'node:assert/strict';
|
import assert from 'node:assert/strict';
|
||||||
import { readFile } from 'node:fs/promises';
|
import { mkdir, mkdtemp, readFile, rm, stat } from 'node:fs/promises';
|
||||||
|
import { tmpdir } from 'node:os';
|
||||||
|
import path from 'node:path';
|
||||||
import test from 'node:test';
|
import test from 'node:test';
|
||||||
|
|
||||||
import { validateLoadConfig } from '../src/config.js';
|
import { validateLoadConfig } from '../src/config.js';
|
||||||
import { assertFixtureIsolation } from '../src/fixture.js';
|
import { assertFixtureIsolation, prepareCapacitySecrets } from '../src/fixture.js';
|
||||||
|
|
||||||
const samplePath = new URL('../config/300-users-900-npcs-5m.json', import.meta.url);
|
const samplePath = new URL('../config/300-users-900-npcs-5m.json', import.meta.url);
|
||||||
|
|
||||||
@@ -44,3 +46,31 @@ void test('fixture refuses a shared schema, shared Redis database, and public ho
|
|||||||
/loopback or private/u
|
/loopback or private/u
|
||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
void test('prepare creates only three 0600 ignored-secret inputs without returning their values', async () => {
|
||||||
|
const workspaceRoot = await mkdtemp(path.join(tmpdir(), 'sammo-capacity-prepare-'));
|
||||||
|
try {
|
||||||
|
await mkdir(path.join(workspaceRoot, 'tools/load-tests/secrets'), { recursive: true });
|
||||||
|
const config = validateLoadConfig(JSON.parse(await readFile(samplePath, 'utf8')));
|
||||||
|
const result = await prepareCapacitySecrets({
|
||||||
|
config,
|
||||||
|
workspaceRoot,
|
||||||
|
env: { CAPACITY_POSTGRES_PORT: '25442', CAPACITY_REDIS_PORT: '26379' },
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(result, { prepared: true, secretFilesWritten: 3, mode: '0600' });
|
||||||
|
for (const name of ['postgres-password.txt', 'image-upload-secret.txt', 'capacity.env']) {
|
||||||
|
const info = await stat(path.join(workspaceRoot, 'tools/load-tests/secrets', name));
|
||||||
|
assert.equal(info.mode & 0o777, 0o600);
|
||||||
|
}
|
||||||
|
const envText = await readFile(
|
||||||
|
path.join(workspaceRoot, 'tools/load-tests/secrets/capacity.env'),
|
||||||
|
'utf8'
|
||||||
|
);
|
||||||
|
assert.match(envText, /127\.0\.0\.1:25442/u);
|
||||||
|
assert.match(envText, /127\.0\.0\.1:26379\/15/u);
|
||||||
|
assert.equal(JSON.stringify(result).includes('postgresql://'), false);
|
||||||
|
} finally {
|
||||||
|
await rm(workspaceRoot, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ void test('tRPC query uses bearer auth without putting the token in the URL', ()
|
|||||||
assert.equal(new Headers(request.init.headers).get('authorization'), `Bearer ${token}`);
|
assert.equal(new Headers(request.init.headers).get('authorization'), `Bearer ${token}`);
|
||||||
assert.equal(request.url.includes(token), false);
|
assert.equal(request.url.includes(token), false);
|
||||||
assert.equal(new URL(request.url).pathname, '/api/trpc/dashboard.getContextBundleDelta');
|
assert.equal(new URL(request.url).pathname, '/api/trpc/dashboard.getContextBundleDelta');
|
||||||
assert.deepEqual(JSON.parse(new URL(request.url).searchParams.get('input')!), { json: { include: { context: true } } });
|
assert.deepEqual(JSON.parse(new URL(request.url).searchParams.get('input')!), { include: { context: true } });
|
||||||
});
|
});
|
||||||
|
|
||||||
void test('dashboard observations retain only opaque revisions and aggregate-safe result kinds', () => {
|
void test('dashboard observations retain only opaque revisions and aggregate-safe result kinds', () => {
|
||||||
|
|||||||
Reference in New Issue
Block a user