test: 300접속 capacity calibration을 재현 가능하게 구성

비밀값 자동 준비, 4 CPU/8GiB API launcher, fixture 검증 기반 calibration config를 추가한다. tRPC v11 GET input encoding 오류를 고치고 300 SSE/HTTP 실제 실행 절차를 문서화한다.
This commit is contained in:
2026-08-16 18:54:34 +00:00
parent 1a77bb3df1
commit fb3ce3da70
8 changed files with 292 additions and 49 deletions
+43 -24
View File
@@ -30,24 +30,23 @@ unchanged/snapshot/patch 경로를 구분하며 raw JSON에는 종류별 count
### 1. 격리 PostgreSQL/Redis와 1,200장수 fixture ### 1. 격리 PostgreSQL/Redis와 1,200장수 fixture
아래 Compose는 loopback에만 포트를 열고 PostgreSQL 18.4, Redis 8.2.7을 고정한다. 실제 password와 URL은 아래 Compose는 loopback에만 포트를 열고 PostgreSQL 18.4, Redis 8.2.7을 고정한다. PostgreSQL 18의
Git ignored `secrets/`에 두며 명령행이나 결과 JSON에는 기록하지 않는다. `capacity.env`에는 최소 versioned data-directory 계약에 맞춰 volume은 `/var/lib/postgresql`에 붙인다. host port는 기본
`LOAD_TEST_DATABASE_URL`(query의 `schema=load_capacity_300_900_5m` 포함), `15442/16379`이며 `CAPACITY_POSTGRES_PORT`/`CAPACITY_REDIS_PORT`로 충돌 없이 바꿀 수 있다. `prepare`
`LOAD_TEST_REDIS_URL`(`/15` 포함), API 기동에 필요한 `GAME_TOKEN_SECRET`, PostgreSQL password, API token/image secret과 정확한 URL을 무작위 생성해 Git ignored `secrets/`의 새
`GAME_IMAGE_UPLOAD_SECRET_FILE`을 넣는다. URL의 password는 percent-encoding한다. 파일 세 개에 `0600`으로 저장한다. 기존 파일을 덮어쓰거나 비밀값을 stdout에 쓰지 않는다.
```sh ```sh
install -m 600 /dev/null tools/load-tests/secrets/postgres-password.txt pnpm --filter @sammo-ts/load-tests prepare:capacity \
install -m 600 /dev/null tools/load-tests/secrets/capacity.env --config tools/load-tests/config/300-users-900-npcs-5m.json
# 두 파일은 로컬 편집기로 채우고 내용을 stdout에 출력하지 않는다.
docker compose -f tools/load-tests/compose.capacity.yml config --quiet
docker compose -f tools/load-tests/compose.capacity.yml up -d --wait
set -a set -a
source tools/load-tests/secrets/capacity.env source tools/load-tests/secrets/capacity.env
set +a set +a
docker compose -f tools/load-tests/compose.capacity.yml config --quiet
docker compose -f tools/load-tests/compose.capacity.yml up -d --wait
pnpm --filter @sammo-ts/load-tests seed \ pnpm --filter @sammo-ts/load-tests seed \
--config tools/load-tests/config/300-users-900-npcs-5m.json \ --config tools/load-tests/config/300-users-900-npcs-5m.json \
--tokens tools/load-tests/secrets/game-tokens.json --tokens tools/load-tests/secrets/game-tokens.json
@@ -64,20 +63,21 @@ fixture와 같은 환경으로 API를 띄울 때 핵심 namespace는 다음과
풀어 쓰지 않는다. 풀어 쓰지 않는다.
```sh ```sh
export DATABASE_URL="$LOAD_TEST_DATABASE_URL" pnpm --filter @sammo-ts/common build
export REDIS_URL="$LOAD_TEST_REDIS_URL" pnpm --filter @sammo-ts/logic build
export PROFILE=load_capacity_300_900_5m pnpm --filter @sammo-ts/infra build
export SCENARIO=2601 pnpm --filter @sammo-ts/game-engine build
export GAME_PROFILE_NAME=load-tests:capacity-300-900-5m pnpm --filter @sammo-ts/game-api build
export GAME_API_HOST=127.0.0.1
export GAME_API_PORT=15001 systemd-run --user --unit=sammo-capacity-api --collect \
export GAME_TRPC_PATH=/api/trpc --property=MemoryMax=8G \
export GAME_API_EVENTS_PATH=/events --working-directory="$(pwd)" \
pnpm --filter @sammo-ts/game-api start "$(pwd)/tools/load-tests/scripts/run-capacity-api.sh"
``` ```
API와 driver는 별도 terminal/process로 실행한다. dev-sam2026와 같은 판정이 필요하면 API/engine container에 runner script는 `capacity.env`의 Node binary를 사용하고 `taskset 0-3`으로 API를 4 logical CPU에 제한한다.
4 CPU/8 GiB 제한을 주고 driver는 그 cgroup 밖에서 실행하며, image digest를 아래 run config에 기록한다. systemd unit은 API에 8 GiB memory limit을 적용한다. `systemctl --user show`로 얻은 main PID의 affinity와
unit `MemoryMax`를 각각 `taskset -pc``systemctl --user show`로 확인한다. API와 driver는 별도 process로 실행한다.
공개 `dev-sam2026.hided.net` profile에는 이 fixture나 driver를 연결하지 않는다. 공개 `dev-sam2026.hided.net` profile에는 이 fixture나 driver를 연결하지 않는다.
### 2. 인증 HTTP/SSE driver ### 2. 인증 HTTP/SSE driver
@@ -90,7 +90,7 @@ pnpm --filter @sammo-ts/load-tests validate --config tools/load-tests/config/300
pnpm --filter @sammo-ts/load-tests dry-run \ pnpm --filter @sammo-ts/load-tests dry-run \
--config tools/load-tests/config/300-users-900-npcs-5m.json \ --config tools/load-tests/config/300-users-900-npcs-5m.json \
--tokens tools/load-tests/secrets/game-tokens.json --tokens tools/load-tests/secrets/game-tokens.json
pnpm --filter @sammo-ts/load-tests run \ pnpm --filter @sammo-ts/load-tests run run \
--config tools/load-tests/config/300-users-900-npcs-5m.json \ --config tools/load-tests/config/300-users-900-npcs-5m.json \
--tokens tools/load-tests/secrets/game-tokens.json \ --tokens tools/load-tests/secrets/game-tokens.json \
--output tools/load-tests/results/300-users-900-npcs-5m.json --output tools/load-tests/results/300-users-900-npcs-5m.json
@@ -101,6 +101,25 @@ pnpm --filter @sammo-ts/load-tests run \
host/cgroup에서 실행하고 두 host의 CPU quota와 competing load를 별도로 기록한다. host/cgroup에서 실행하고 두 host의 CPU quota와 competing load를 별도로 기록한다.
`run`은 sample의 runtime metadata placeholder가 하나라도 남아 있으면 시작하지 않는다. `run`은 sample의 runtime metadata placeholder가 하나라도 남아 있으면 시작하지 않는다.
300 SSE/HTTP의 짧은 연결·query calibration은 검증된 fixture에서 ignored runtime config를 먼저 만든다.
기본 calibration은 idle 5초와 own/global/mixed 각 10초이며 capacity 합격 판정용 soak test가 아니다.
```sh
pnpm --filter @sammo-ts/load-tests materialize-calibration \
--config tools/load-tests/config/300-users-900-npcs-5m.json \
--output tools/load-tests/results/calibration-config.json
pnpm --filter @sammo-ts/load-tests run run \
--config tools/load-tests/results/calibration-config.json \
--tokens tools/load-tests/secrets/game-tokens.json \
--output tools/load-tests/results/calibration-result.json
```
`materialize-calibration`은 DB/Redis count와 manifest/hash를 다시 확인하고 실제 PostgreSQL/Redis version을
기록한다. `LOAD_TEST_IMAGE_DIGEST`가 없으면 image 결과라고 부르지 않고 현재 dirty source-tree commit을
명시한다. driver JSON의 process CPU/RSS는 driver 자체 값이다. API target 값은 systemd unit의
`CPUUsageNSec`, `MemoryCurrent`, `MemoryPeak`를 run 직전/직후 별도로 수집한다.
## 결과와 해석 ## 결과와 해석
raw JSON은 Git commit/tree/dirty 상태, config/runtime/host hash, Node/V8, host CPU/memory와 cgroup limit, raw JSON은 Git commit/tree/dirty 상태, config/runtime/host hash, Node/V8, host CPU/memory와 cgroup limit,
+2
View File
@@ -7,8 +7,10 @@
"run": "pnpm -w exec tsx tools/load-tests/src/cli.ts run", "run": "pnpm -w exec tsx tools/load-tests/src/cli.ts run",
"dry-run": "pnpm -w exec tsx tools/load-tests/src/cli.ts dry-run", "dry-run": "pnpm -w exec tsx tools/load-tests/src/cli.ts dry-run",
"validate": "pnpm -w exec tsx tools/load-tests/src/cli.ts validate", "validate": "pnpm -w exec tsx tools/load-tests/src/cli.ts validate",
"prepare:capacity": "pnpm -w exec tsx tools/load-tests/src/cli.ts prepare",
"seed": "pnpm -w exec tsx tools/load-tests/src/cli.ts seed", "seed": "pnpm -w exec tsx tools/load-tests/src/cli.ts seed",
"verify-fixture": "pnpm -w exec tsx tools/load-tests/src/cli.ts verify-fixture", "verify-fixture": "pnpm -w exec tsx tools/load-tests/src/cli.ts verify-fixture",
"materialize-calibration": "pnpm -w exec tsx tools/load-tests/src/cli.ts materialize-calibration",
"cleanup": "pnpm -w exec tsx tools/load-tests/src/cli.ts cleanup", "cleanup": "pnpm -w exec tsx tools/load-tests/src/cli.ts cleanup",
"test": "pnpm -w exec tsx --test tools/load-tests/test/*.test.ts", "test": "pnpm -w exec tsx --test tools/load-tests/test/*.test.ts",
"typecheck": "pnpm -w tsc7 -p tools/load-tests/tsconfig.json --noEmit" "typecheck": "pnpm -w tsc7 -p tools/load-tests/tsconfig.json --noEmit"
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
set -eu
workspace_root=$(CDPATH= cd -- "$(dirname -- "$0")/../../.." && pwd)
secret_env="$workspace_root/tools/load-tests/secrets/capacity.env"
if [ ! -f "$secret_env" ]; then
echo "capacity.env is missing; run the load-test prepare command first" >&2
exit 1
fi
secret_mode=$(stat -c '%a' "$secret_env")
if [ "$secret_mode" != "600" ]; then
echo "capacity.env must have mode 0600" >&2
exit 1
fi
set -a
# shellcheck disable=SC1090
. "$secret_env"
set +a
export GAME_API_ROLE=server
cd "$workspace_root"
exec taskset -c "${CAPACITY_CPUSET:-0-3}" "${CAPACITY_NODE_BINARY:-node}" app/game-api/dist/index.js
+28 -4
View File
@@ -2,19 +2,25 @@ import { mkdir, writeFile } from 'node:fs/promises';
import path from 'node:path'; import path from 'node:path';
import { assertRuntimeMetadataFinalized, loadConfig, loadTokens } from './config.js'; import { assertRuntimeMetadataFinalized, loadConfig, loadTokens } from './config.js';
import { cleanupCapacityFixture, seedCapacityFixture, verifyCapacityFixture } from './fixture.js'; import {
cleanupCapacityFixture,
materializeCalibrationConfig,
prepareCapacitySecrets,
seedCapacityFixture,
verifyCapacityFixture,
} from './fixture.js';
import { describeDryRun, runLoadTest } from './runner.js'; import { describeDryRun, runLoadTest } from './runner.js';
type Command = 'run' | 'dry-run' | 'validate' | 'seed' | 'verify-fixture' | 'cleanup'; type Command = 'run' | 'dry-run' | 'validate' | 'prepare' | 'seed' | 'verify-fixture' | 'materialize-calibration' | 'cleanup';
const usage = (): never => { const usage = (): never => {
process.stderr.write('usage: cli.ts <validate|dry-run|run|seed|verify-fixture|cleanup> --config <file> [--tokens <0600-gitignored-file>] [--output <new-json-file>] [--confirm <load_schema>]\n'); process.stderr.write('usage: cli.ts <validate|dry-run|run|prepare|seed|verify-fixture|materialize-calibration|cleanup> --config <file> [--tokens <0600-gitignored-file>] [--output <new-json-file>] [--confirm <load_schema>]\n');
process.exit(64); process.exit(64);
}; };
const parseArguments = (argv: readonly string[]): { command: Command; config: string; tokens?: string; output?: string; confirm?: string } => { const parseArguments = (argv: readonly string[]): { command: Command; config: string; tokens?: string; output?: string; confirm?: string } => {
const command = argv[0]; const command = argv[0];
if (!['run', 'dry-run', 'validate', 'seed', 'verify-fixture', 'cleanup'].includes(command ?? '')) usage(); if (!['run', 'dry-run', 'validate', 'prepare', 'seed', 'verify-fixture', 'materialize-calibration', 'cleanup'].includes(command ?? '')) usage();
const values = new Map<string, string>(); const values = new Map<string, string>();
for (let index = 1; index < argv.length; index += 2) { for (let index = 1; index < argv.length; index += 2) {
const flag = argv[index]; const flag = argv[index];
@@ -26,8 +32,10 @@ const parseArguments = (argv: readonly string[]): { command: Command; config: st
if (!config) usage(); if (!config) usage();
if (command === 'run' && (!values.get('--tokens') || !values.get('--output'))) usage(); if (command === 'run' && (!values.get('--tokens') || !values.get('--output'))) usage();
if (command === 'seed' && (!values.get('--tokens') || values.has('--output') || values.has('--confirm'))) usage(); if (command === 'seed' && (!values.get('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
if (command === 'prepare' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
if (command === 'cleanup' && (!values.get('--confirm') || values.has('--tokens') || values.has('--output'))) usage(); if (command === 'cleanup' && (!values.get('--confirm') || values.has('--tokens') || values.has('--output'))) usage();
if (command === 'verify-fixture' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage(); if (command === 'verify-fixture' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
if (command === 'materialize-calibration' && (!values.get('--output') || values.has('--tokens') || values.has('--confirm'))) usage();
if (command === 'validate' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage(); if (command === 'validate' && (values.has('--tokens') || values.has('--output') || values.has('--confirm'))) usage();
if (command === 'dry-run' && (values.has('--output') || values.has('--confirm'))) usage(); if (command === 'dry-run' && (values.has('--output') || values.has('--confirm'))) usage();
if (command === 'run' && values.has('--confirm')) usage(); if (command === 'run' && values.has('--confirm')) usage();
@@ -57,10 +65,26 @@ const main = async (): Promise<void> => {
process.stdout.write(`${JSON.stringify(result)}\n`); process.stdout.write(`${JSON.stringify(result)}\n`);
return; return;
} }
if (args.command === 'prepare') {
process.stdout.write(`${JSON.stringify(await prepareCapacitySecrets({ config, workspaceRoot }))}\n`);
return;
}
if (args.command === 'verify-fixture') { if (args.command === 'verify-fixture') {
process.stdout.write(`${JSON.stringify(await verifyCapacityFixture(config))}\n`); process.stdout.write(`${JSON.stringify(await verifyCapacityFixture(config))}\n`);
return; return;
} }
if (args.command === 'materialize-calibration') {
process.stdout.write(
`${JSON.stringify(
await materializeCalibrationConfig({
config,
outputPath: args.output!,
workspaceRoot,
})
)}\n`
);
return;
}
if (args.command === 'cleanup') { if (args.command === 'cleanup') {
process.stdout.write(`${JSON.stringify(await cleanupCapacityFixture(config, args.confirm!))}\n`); process.stdout.write(`${JSON.stringify(await cleanupCapacityFixture(config, args.confirm!))}\n`);
return; return;
+160 -17
View File
@@ -1,6 +1,6 @@
import { execFile } from 'node:child_process'; import { execFile } from 'node:child_process';
import { randomUUID } from 'node:crypto'; import { randomBytes, randomUUID } from 'node:crypto';
import { lstat, mkdir, open, realpath } from 'node:fs/promises'; import { lstat, mkdir, open, realpath, unlink } from 'node:fs/promises';
import path from 'node:path'; import path from 'node:path';
import { promisify } from 'node:util'; import { promisify } from 'node:util';
@@ -8,7 +8,7 @@ import { seedScenarioToDatabase } from '@sammo-ts/game-engine';
import { import {
createGamePostgresConnector, createGamePostgresConnector,
createRedisConnector, createRedisConnector,
type GamePrisma, GamePrisma,
type GamePrismaClient, type GamePrismaClient,
} from '@sammo-ts/infra'; } from '@sammo-ts/infra';
@@ -20,6 +20,105 @@ const SCENARIO_ID = 2601;
type FixtureEnvironment = { databaseUrl: string; redisUrl: string }; type FixtureEnvironment = { databaseUrl: string; redisUrl: string };
const assertNewSecretPath = async (secretPath: string, workspaceRoot: string): Promise<string> => {
const secretRoot = await realpath(path.join(workspaceRoot, 'tools/load-tests/secrets'));
const output = path.resolve(secretPath);
const relative = path.relative(secretRoot, output);
if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) {
throw new Error('generated secret files must stay inside tools/load-tests/secrets');
}
await mkdir(path.dirname(output), { recursive: true, mode: 0o700 });
if ((await realpath(path.dirname(output))) !== secretRoot) {
throw new Error('generated secret files must not traverse a symbolic-link directory');
}
try {
await lstat(output);
throw new Error('generated secret output already exists');
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
}
return output;
};
const writeNewSecret = async (secretPath: string, content: string): Promise<void> => {
const handle = await open(secretPath, 'wx', 0o600);
try {
await handle.writeFile(content, { encoding: 'utf8' });
} finally {
await handle.close();
}
};
export const prepareCapacitySecrets = async (options: {
config: LoadConfig;
workspaceRoot: string;
env?: NodeJS.ProcessEnv;
}) => {
const env = options.env ?? process.env;
const postgresPort = Number(env.CAPACITY_POSTGRES_PORT ?? '15442');
const redisPort = Number(env.CAPACITY_REDIS_PORT ?? '16379');
if (!Number.isSafeInteger(postgresPort) || postgresPort < 1024 || postgresPort > 65_535) {
throw new Error('CAPACITY_POSTGRES_PORT must be an unprivileged TCP port');
}
if (!Number.isSafeInteger(redisPort) || redisPort < 1024 || redisPort > 65_535) {
throw new Error('CAPACITY_REDIS_PORT must be an unprivileged TCP port');
}
if (postgresPort === redisPort) throw new Error('capacity PostgreSQL and Redis ports must differ');
const secretRoot = path.join(options.workspaceRoot, 'tools/load-tests/secrets');
const postgresPasswordPath = await assertNewSecretPath(
path.join(secretRoot, 'postgres-password.txt'),
options.workspaceRoot
);
const imageSecretPath = await assertNewSecretPath(
path.join(secretRoot, 'image-upload-secret.txt'),
options.workspaceRoot
);
const capacityEnvPath = await assertNewSecretPath(path.join(secretRoot, 'capacity.env'), options.workspaceRoot);
const postgresPassword = randomBytes(32).toString('hex');
const gameTokenSecret = randomBytes(32).toString('hex');
const imageUploadSecret = randomBytes(32).toString('hex');
const databaseUrl =
`postgresql://sammo_capacity:${postgresPassword}@127.0.0.1:${postgresPort}/sammo_capacity` +
`?schema=${options.config.isolation.postgresSchema}`;
const redisUrl = `redis://127.0.0.1:${redisPort}/${options.config.isolation.redisDatabase}`;
const envLines = [
`CAPACITY_POSTGRES_PORT='${postgresPort}'`,
`CAPACITY_REDIS_PORT='${redisPort}'`,
`LOAD_TEST_DATABASE_URL='${databaseUrl}'`,
`LOAD_TEST_REDIS_URL='${redisUrl}'`,
`DATABASE_URL='${databaseUrl}'`,
`REDIS_URL='${redisUrl}'`,
`GAME_TOKEN_SECRET='${gameTokenSecret}'`,
`GAME_IMAGE_UPLOAD_SECRET_FILE='${imageSecretPath}'`,
`PROFILE='${options.config.isolation.postgresSchema}'`,
`SCENARIO='2601'`,
`GAME_PROFILE_NAME='${options.config.isolation.profileName}'`,
`GAME_API_HOST='127.0.0.1'`,
`GAME_API_PORT='${new URL(options.config.target.baseUrl).port || '80'}'`,
`GAME_TRPC_PATH='${options.config.target.trpcPath}'`,
`GAME_API_EVENTS_PATH='${options.config.target.ssePath}'`,
`CAPACITY_NODE_BINARY='${process.execPath}'`,
"CAPACITY_CPUSET='0-3'",
'',
].join('\n');
const created: string[] = [];
try {
await writeNewSecret(postgresPasswordPath, `${postgresPassword}\n`);
created.push(postgresPasswordPath);
await writeNewSecret(imageSecretPath, `${imageUploadSecret}\n`);
created.push(imageSecretPath);
await writeNewSecret(capacityEnvPath, envLines);
created.push(capacityEnvPath);
} catch (error) {
await Promise.all(created.map((file) => unlink(file).catch(() => undefined)));
throw error;
}
return { prepared: true, secretFilesWritten: created.length, mode: '0600' };
};
const requireEnvironment = (env: NodeJS.ProcessEnv): FixtureEnvironment => { const requireEnvironment = (env: NodeJS.ProcessEnv): FixtureEnvironment => {
const databaseUrl = env.LOAD_TEST_DATABASE_URL; const databaseUrl = env.LOAD_TEST_DATABASE_URL;
const redisUrl = env.LOAD_TEST_REDIS_URL; const redisUrl = env.LOAD_TEST_REDIS_URL;
@@ -177,21 +276,10 @@ const resizeSeededGenerals = async (db: GamePrismaClient, config: LoadConfig): P
}; };
const assertNewTokenPath = async (tokenPath: string, workspaceRoot: string): Promise<string> => { const assertNewTokenPath = async (tokenPath: string, workspaceRoot: string): Promise<string> => {
const secretRoot = await realpath(path.join(workspaceRoot, 'tools/load-tests/secrets')); const output = await assertNewSecretPath(tokenPath, workspaceRoot);
const output = path.resolve(tokenPath); if (path.extname(output) !== '.json') {
const relative = path.relative(secretRoot, output);
if (!relative || relative.startsWith('..') || path.isAbsolute(relative) || path.extname(output) !== '.json') {
throw new Error('seed token output must be a new JSON file inside tools/load-tests/secrets'); throw new Error('seed token output must be a new JSON file inside tools/load-tests/secrets');
} }
await mkdir(path.dirname(output), { recursive: true, mode: 0o700 });
const parent = await realpath(path.dirname(output));
if (parent !== secretRoot) throw new Error('seed token output must not traverse a symbolic-link directory');
try {
await lstat(output);
throw new Error('seed token output already exists');
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
}
return output; return output;
}; };
@@ -313,7 +401,10 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
const redis = createRedisConnector({ url: environment.redisUrl }); const redis = createRedisConnector({ url: environment.redisUrl });
await postgres.connect(); await postgres.connect();
try { try {
const state = await projectFixtureState(postgres.prisma); const [state, postgresRows] = await Promise.all([
projectFixtureState(postgres.prisma),
postgres.prisma.$queryRaw<Array<{ version: string }>>(GamePrisma.sql`SELECT version()`),
]);
const fixtureSha256 = `sha256:${sha256(canonicalJson(state))}`; const fixtureSha256 = `sha256:${sha256(canonicalJson(state))}`;
await redis.connect(); await redis.connect();
try { try {
@@ -329,6 +420,8 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
} }
} }
const accessTokens = await countMatchingRedisKeys(redis.client, `${accessKeyPrefix(config)}ga_*`); const accessTokens = await countMatchingRedisKeys(redis.client, `${accessKeyPrefix(config)}ga_*`);
const redisInfo = await redis.client.info('server');
const redisVersion = /^redis_version:(.+)$/mu.exec(redisInfo)?.[1]?.trim() ?? 'unknown';
const npcGenerals = state.generals.filter((general) => general.npcState >= 2).length; const npcGenerals = state.generals.filter((general) => general.npcState >= 2).length;
const humanGenerals = state.generals.filter( const humanGenerals = state.generals.filter(
(general) => general.npcState === 0 && general.userId (general) => general.npcState === 0 && general.userId
@@ -348,6 +441,8 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
accessTokens, accessTokens,
redisManifestPresent: rawManifest !== null, redisManifestPresent: rawManifest !== null,
redisManifestMatches: manifestFixtureSha256 === fixtureSha256, redisManifestMatches: manifestFixtureSha256 === fixtureSha256,
postgresVersion: postgresRows[0]?.version ?? 'unknown',
redisVersion,
}; };
} finally { } finally {
await redis.disconnect(); await redis.disconnect();
@@ -357,6 +452,54 @@ export const verifyCapacityFixture = async (config: LoadConfig, env: NodeJS.Proc
} }
}; };
export const materializeCalibrationConfig = async (options: {
config: LoadConfig;
outputPath: string;
workspaceRoot: string;
env?: NodeJS.ProcessEnv;
}) => {
const env = options.env ?? process.env;
const output = path.resolve(options.outputPath);
const resultsRoot = await realpath(path.join(options.workspaceRoot, 'tools/load-tests/results'));
const relative = path.relative(resultsRoot, output);
if (!relative || relative.startsWith('..') || path.isAbsolute(relative) || path.extname(output) !== '.json') {
throw new Error('calibration config output must be a new JSON file inside tools/load-tests/results');
}
try {
await lstat(output);
throw new Error('calibration config output already exists');
} catch (error) {
if ((error as NodeJS.ErrnoException).code !== 'ENOENT') throw error;
}
const verified = await verifyCapacityFixture(options.config, env);
if (!verified.valid) throw new Error('fixture verification failed; refusing to materialize calibration config');
const gitCommit = (
await execFileAsync('git', ['rev-parse', 'HEAD'], { cwd: options.workspaceRoot })
).stdout.trim();
const runtimeConfig: LoadConfig = {
...options.config,
name: `${options.config.name}-calibration`,
runtimeMetadata: {
fixtureSha256: verified.fixtureSha256,
imageDigest: env.LOAD_TEST_IMAGE_DIGEST ?? `source-tree:${gitCommit}:dirty`,
postgresVersion: verified.postgresVersion,
redisVersion: verified.redisVersion,
},
phases: options.config.phases.map((phase) => ({
...phase,
name: phase.name.replace(/-(?:5|10|30)m$/u, '-calibration'),
durationMs: phase.kind === 'idle' ? 5_000 : 10_000,
})),
};
await writeNewSecret(output, `${JSON.stringify(runtimeConfig, null, 2)}\n`);
return {
materialized: true,
fixtureSha256: verified.fixtureSha256,
phaseDurationMs: runtimeConfig.phases.map((phase) => phase.durationMs),
runtimeKind: env.LOAD_TEST_IMAGE_DIGEST ? 'image' : 'dirty-source-tree',
};
};
export const cleanupCapacityFixture = async ( export const cleanupCapacityFixture = async (
config: LoadConfig, config: LoadConfig,
confirmation: string, confirmation: string,
+1 -1
View File
@@ -9,7 +9,7 @@ export interface TrpcRequest {
export const buildTrpcQuery = (baseUrl: string, trpcPath: string, operation: LoadOperation, token: string): TrpcRequest => { export const buildTrpcQuery = (baseUrl: string, trpcPath: string, operation: LoadOperation, token: string): TrpcRequest => {
const normalizedPath = trpcPath.endsWith('/') ? trpcPath.slice(0, -1) : trpcPath; const normalizedPath = trpcPath.endsWith('/') ? trpcPath.slice(0, -1) : trpcPath;
const url = new URL(`${normalizedPath}/${operation.procedure}`, baseUrl); const url = new URL(`${normalizedPath}/${operation.procedure}`, baseUrl);
if (operation.input !== undefined) url.searchParams.set('input', JSON.stringify({ json: operation.input })); if (operation.input !== undefined) url.searchParams.set('input', JSON.stringify(operation.input));
return { return {
url: url.toString(), url: url.toString(),
init: { init: {
+32 -2
View File
@@ -1,9 +1,11 @@
import assert from 'node:assert/strict'; import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises'; import { mkdir, mkdtemp, readFile, rm, stat } from 'node:fs/promises';
import { tmpdir } from 'node:os';
import path from 'node:path';
import test from 'node:test'; import test from 'node:test';
import { validateLoadConfig } from '../src/config.js'; import { validateLoadConfig } from '../src/config.js';
import { assertFixtureIsolation } from '../src/fixture.js'; import { assertFixtureIsolation, prepareCapacitySecrets } from '../src/fixture.js';
const samplePath = new URL('../config/300-users-900-npcs-5m.json', import.meta.url); const samplePath = new URL('../config/300-users-900-npcs-5m.json', import.meta.url);
@@ -44,3 +46,31 @@ void test('fixture refuses a shared schema, shared Redis database, and public ho
/loopback or private/u /loopback or private/u
); );
}); });
void test('prepare creates only three 0600 ignored-secret inputs without returning their values', async () => {
const workspaceRoot = await mkdtemp(path.join(tmpdir(), 'sammo-capacity-prepare-'));
try {
await mkdir(path.join(workspaceRoot, 'tools/load-tests/secrets'), { recursive: true });
const config = validateLoadConfig(JSON.parse(await readFile(samplePath, 'utf8')));
const result = await prepareCapacitySecrets({
config,
workspaceRoot,
env: { CAPACITY_POSTGRES_PORT: '25442', CAPACITY_REDIS_PORT: '26379' },
});
assert.deepEqual(result, { prepared: true, secretFilesWritten: 3, mode: '0600' });
for (const name of ['postgres-password.txt', 'image-upload-secret.txt', 'capacity.env']) {
const info = await stat(path.join(workspaceRoot, 'tools/load-tests/secrets', name));
assert.equal(info.mode & 0o777, 0o600);
}
const envText = await readFile(
path.join(workspaceRoot, 'tools/load-tests/secrets/capacity.env'),
'utf8'
);
assert.match(envText, /127\.0\.0\.1:25442/u);
assert.match(envText, /127\.0\.0\.1:26379\/15/u);
assert.equal(JSON.stringify(result).includes('postgresql://'), false);
} finally {
await rm(workspaceRoot, { recursive: true, force: true });
}
});
+1 -1
View File
@@ -14,7 +14,7 @@ void test('tRPC query uses bearer auth without putting the token in the URL', ()
assert.equal(new Headers(request.init.headers).get('authorization'), `Bearer ${token}`); assert.equal(new Headers(request.init.headers).get('authorization'), `Bearer ${token}`);
assert.equal(request.url.includes(token), false); assert.equal(request.url.includes(token), false);
assert.equal(new URL(request.url).pathname, '/api/trpc/dashboard.getContextBundleDelta'); assert.equal(new URL(request.url).pathname, '/api/trpc/dashboard.getContextBundleDelta');
assert.deepEqual(JSON.parse(new URL(request.url).searchParams.get('input')!), { json: { include: { context: true } } }); assert.deepEqual(JSON.parse(new URL(request.url).searchParams.get('input')!), { include: { context: true } });
}); });
void test('dashboard observations retain only opaque revisions and aggregate-safe result kinds', () => { void test('dashboard observations retain only opaque revisions and aggregate-safe result kinds', () => {