merge: 최신 main을 레거시 DB 재이관에 통합
# Conflicts: # app/gateway-api/test/releaseManifest.test.ts # release-manifest.json
This commit is contained in:
@@ -36,6 +36,8 @@ versioned data-directory 계약에 맞춰 volume은 `/var/lib/postgresql`에 붙
|
||||
`15442/16379`이며 `CAPACITY_POSTGRES_PORT`/`CAPACITY_REDIS_PORT`로 충돌 없이 바꿀 수 있다. `prepare`는
|
||||
PostgreSQL password, API token/image secret과 정확한 URL을 무작위 생성해 Git ignored `secrets/`의 새
|
||||
파일 세 개에 `0600`으로 저장한다. 기존 파일을 덮어쓰거나 비밀값을 stdout에 쓰지 않는다.
|
||||
기존 fixture volume을 보존하면서 별도 실행이 필요하면
|
||||
`CAPACITY_COMPOSE_PROJECT_NAME`과 `CAPACITY_POSTGRES_VOLUME_NAME`을 함께 고유하게 지정한다.
|
||||
|
||||
```sh
|
||||
pnpm --filter @sammo-ts/load-tests prepare:capacity \
|
||||
@@ -152,7 +154,34 @@ pnpm --filter @sammo-ts/game-engine profile:npc-capacity-1200
|
||||
이 프로필은 자연 통일 소요시간 시험이 아니라 고정 1개월 engine 처리량 시험이다. 기존
|
||||
`profile:npc-unification-timing`의 무보정 자연 진행 의미는 바꾸지 않는다.
|
||||
|
||||
### 4. 명시적 cleanup
|
||||
### 4. E2 PostgreSQL flush와 profile 간 경합
|
||||
|
||||
`measure-turn-flush`는 검증된 fixture를 production loader로 읽고 daemon lease/fencing, 장수 턴,
|
||||
dirty-state transaction, journal/outbox와 commit 이후 Redis 발행을 거쳐 정확히 한 월 경계를 실행한다.
|
||||
정상 realtime daemon처럼 장수는 한 transaction에 하나씩 commit한다. 권위 순서는 `turn_tick`이며,
|
||||
JavaScript `Date`의 밀리초보다 세밀한 tick을 포함하도록 cutoff를 보정한다.
|
||||
|
||||
```sh
|
||||
pnpm --filter @sammo-ts/load-tests measure-turn-flush \
|
||||
--config tools/load-tests/config/300-users-900-npcs-5m.json \
|
||||
--confirm load_capacity_300_900_5m \
|
||||
--output tools/load-tests/results/turn-flush.json
|
||||
```
|
||||
|
||||
결과에는 장수 transaction과 월 transaction/Redis 발행 latency, 처리량, 시작·종료 장수 수,
|
||||
process CPU/RSS/event-loop lag, database-wide `pg_stat_database` delta와 connection/active/lock-wait 최대값이
|
||||
들어간다. PostgreSQL delta에는 별도 observer sampler의 read transaction도 포함되므로 transaction 수를
|
||||
daemon commit 수와 동일하다고 해석하지 않는다.
|
||||
|
||||
`nya`와 `pya` 동시 1분 경합용 config는 각각 Redis DB 14/13과 별도 `load_` schema를 사용한다.
|
||||
같은 PostgreSQL/CPU에서 두 fixture를 seed한 뒤 두 `measure-turn-flush` process를 동시에 시작하여
|
||||
schema 간 row-lock 격리와 공유 CPU/I/O/connection 경합을 확인한다. 이 실행은 실제 운영 profile이나
|
||||
공개 URL을 대상으로 하지 않는다.
|
||||
|
||||
- `tools/load-tests/config/nya-10-users-800-npcs-1m.json`
|
||||
- `tools/load-tests/config/pya-10-users-800-npcs-1m.json`
|
||||
|
||||
### 5. 명시적 cleanup
|
||||
|
||||
token 파일은 별도로 안전하게 삭제하고, fixture schema/Redis token은 schema명을 그대로 확인 인자로 주어
|
||||
정리한다. named volume은 보존한다. 데이터 폐기가 필요하지 않으면 이 명령을 실행하지 않는다.
|
||||
@@ -166,9 +195,11 @@ docker compose -f tools/load-tests/compose.capacity.yml down
|
||||
|
||||
## 아직 남은 측정 경계
|
||||
|
||||
- `seed`/`verify-fixture`는 실제 PostgreSQL schema와 Redis access-token 상태를 만든다. 그러나 E2의 daemon
|
||||
fast-forward, 한 달치 PostgreSQL flush/outbox publish, schedule lag와 DB statement count를 하나로
|
||||
계측하는 실행기는 아직 없다. 따라서 E1이나 API/SSE driver 결과를 E2 합격으로 대체하지 않는다.
|
||||
- `measure-turn-flush`는 한 달을 wall-clock보다 빠르게 replay하는 처리량 시험이다. 실제 schedule lag,
|
||||
장시간 pool wait와 autovacuum/checkpoint 영향을 보려면 profile별 속도로 pacing한 soak가 별도로 필요하다.
|
||||
- 월 경계 latency는 실행당 표본이 하나다. scenario 진행 시점과 월별 event 차이를 포괄하지 않는다.
|
||||
- 두 1분 profile 동시 실행은 최악 turn-rate 조합의 국소 증거이며, 여섯 profile의 전체 PM2 RSS,
|
||||
Gateway·worker connection과 운영 container cgroup을 재현하지 않는다.
|
||||
- own/global phase의 mutation stimulus는 driver가 만들지 않는다. 격리 runtime의 실제 engine/API mutation과
|
||||
함께 실행하지 않았다면 A2/A3/M1 전체 합격으로 보고하지 않는다.
|
||||
- 이 repository에서 실행한 로컬 E1 수치는 source-tree 회귀 근거다. dev-sam2026 동급 4 CPU/8 GiB container
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
name: sammo-capacity-fixture
|
||||
name: ${CAPACITY_COMPOSE_PROJECT_NAME:-sammo-capacity-fixture}
|
||||
|
||||
services:
|
||||
postgres:
|
||||
@@ -39,4 +39,4 @@ secrets:
|
||||
|
||||
volumes:
|
||||
capacity_postgres_data:
|
||||
name: sammo_capacity_fixture_postgres_data
|
||||
name: ${CAPACITY_POSTGRES_VOLUME_NAME:-sammo_capacity_fixture_postgres_data}
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"$schema": "./load-test.schema.json",
|
||||
"version": 1,
|
||||
"name": "nya-10-users-800-npcs-1m",
|
||||
"target": {
|
||||
"baseUrl": "http://127.0.0.1:15001",
|
||||
"trpcPath": "/api/trpc",
|
||||
"ssePath": "/events",
|
||||
"publicProfile": false,
|
||||
"allowedHosts": ["127.0.0.1", "localhost"]
|
||||
},
|
||||
"isolation": {
|
||||
"postgresSchema": "load_capacity_nya_10_800_1m",
|
||||
"redisPrefix": "load-tests:capacity-nya-10-800-1m:",
|
||||
"redisDatabase": 14,
|
||||
"profileName": "load-tests:capacity-nya-10-800-1m"
|
||||
},
|
||||
"capacity": {
|
||||
"authenticatedViewers": 10,
|
||||
"npcGenerals": 800,
|
||||
"humanGenerals": 10,
|
||||
"turnIntervalMs": 60000
|
||||
},
|
||||
"runtimeMetadata": {
|
||||
"fixtureSha256": "sha256:0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"imageDigest": "replace-before-measurement",
|
||||
"postgresVersion": "replace-before-measurement",
|
||||
"redisVersion": "replace-before-measurement"
|
||||
},
|
||||
"phases": [
|
||||
{
|
||||
"name": "idle-calibration",
|
||||
"kind": "idle",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": null,
|
||||
"operations": []
|
||||
},
|
||||
{
|
||||
"name": "own-calibration",
|
||||
"kind": "own",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": 1000,
|
||||
"operations": [
|
||||
{
|
||||
"name": "own-context",
|
||||
"procedure": "dashboard.getContextBundleDelta",
|
||||
"type": "query",
|
||||
"weight": 1,
|
||||
"input": {
|
||||
"include": { "context": true, "commandTable": true, "boardAccess": true },
|
||||
"forceSnapshot": false
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "global-calibration",
|
||||
"kind": "global",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": 1000,
|
||||
"operations": [
|
||||
{ "name": "global-front", "procedure": "general.getFrontStatus", "type": "query", "weight": 1 }
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "mixed-calibration",
|
||||
"kind": "mixed",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": 1000,
|
||||
"operations": [
|
||||
{
|
||||
"name": "own-context",
|
||||
"procedure": "dashboard.getContextBundleDelta",
|
||||
"type": "query",
|
||||
"weight": 1,
|
||||
"input": {
|
||||
"include": { "context": true, "commandTable": true, "boardAccess": true },
|
||||
"forceSnapshot": false
|
||||
}
|
||||
},
|
||||
{ "name": "global-front", "procedure": "general.getFrontStatus", "type": "query", "weight": 1 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
{
|
||||
"$schema": "./load-test.schema.json",
|
||||
"version": 1,
|
||||
"name": "pya-10-users-800-npcs-1m",
|
||||
"target": {
|
||||
"baseUrl": "http://127.0.0.1:15001",
|
||||
"trpcPath": "/api/trpc",
|
||||
"ssePath": "/events",
|
||||
"publicProfile": false,
|
||||
"allowedHosts": ["127.0.0.1", "localhost"]
|
||||
},
|
||||
"isolation": {
|
||||
"postgresSchema": "load_capacity_pya_10_800_1m",
|
||||
"redisPrefix": "load-tests:capacity-pya-10-800-1m:",
|
||||
"redisDatabase": 13,
|
||||
"profileName": "load-tests:capacity-pya-10-800-1m"
|
||||
},
|
||||
"capacity": {
|
||||
"authenticatedViewers": 10,
|
||||
"npcGenerals": 800,
|
||||
"humanGenerals": 10,
|
||||
"turnIntervalMs": 60000
|
||||
},
|
||||
"runtimeMetadata": {
|
||||
"fixtureSha256": "sha256:0000000000000000000000000000000000000000000000000000000000000000",
|
||||
"imageDigest": "replace-before-measurement",
|
||||
"postgresVersion": "replace-before-measurement",
|
||||
"redisVersion": "replace-before-measurement"
|
||||
},
|
||||
"phases": [
|
||||
{
|
||||
"name": "idle-calibration",
|
||||
"kind": "idle",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": null,
|
||||
"operations": []
|
||||
},
|
||||
{
|
||||
"name": "own-calibration",
|
||||
"kind": "own",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": 1000,
|
||||
"operations": [
|
||||
{
|
||||
"name": "own-context",
|
||||
"procedure": "dashboard.getContextBundleDelta",
|
||||
"type": "query",
|
||||
"weight": 1,
|
||||
"input": {
|
||||
"include": { "context": true, "commandTable": true, "boardAccess": true },
|
||||
"forceSnapshot": false
|
||||
}
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "global-calibration",
|
||||
"kind": "global",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": 1000,
|
||||
"operations": [
|
||||
{ "name": "global-front", "procedure": "general.getFrontStatus", "type": "query", "weight": 1 }
|
||||
]
|
||||
},
|
||||
{
|
||||
"name": "mixed-calibration",
|
||||
"kind": "mixed",
|
||||
"durationMs": 1000,
|
||||
"sseConnections": 10,
|
||||
"requestIntervalMs": 1000,
|
||||
"operations": [
|
||||
{
|
||||
"name": "own-context",
|
||||
"procedure": "dashboard.getContextBundleDelta",
|
||||
"type": "query",
|
||||
"weight": 1,
|
||||
"input": {
|
||||
"include": { "context": true, "commandTable": true, "boardAccess": true },
|
||||
"forceSnapshot": false
|
||||
}
|
||||
},
|
||||
{ "name": "global-front", "procedure": "general.getFrontStatus", "type": "query", "weight": 1 }
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -11,6 +11,7 @@
|
||||
"seed": "pnpm -w exec tsx tools/load-tests/src/cli.ts seed",
|
||||
"verify-fixture": "pnpm -w exec tsx tools/load-tests/src/cli.ts verify-fixture",
|
||||
"activate-coverage": "pnpm -w exec tsx tools/load-tests/src/cli.ts activate-coverage",
|
||||
"measure-turn-flush": "pnpm -w exec tsx tools/load-tests/src/cli.ts measure-turn-flush",
|
||||
"materialize-calibration": "pnpm -w exec tsx tools/load-tests/src/cli.ts materialize-calibration",
|
||||
"cleanup": "pnpm -w exec tsx tools/load-tests/src/cli.ts cleanup",
|
||||
"test": "pnpm -w exec tsx --test tools/load-tests/test/*.test.ts",
|
||||
|
||||
@@ -11,6 +11,7 @@ import {
|
||||
verifyCapacityFixture,
|
||||
} from './fixture.js';
|
||||
import { describeDryRun, runLoadTest } from './runner.js';
|
||||
import { measureTurnFlush } from './turnFlush.js';
|
||||
|
||||
type Command =
|
||||
| 'run'
|
||||
@@ -20,12 +21,13 @@ type Command =
|
||||
| 'seed'
|
||||
| 'verify-fixture'
|
||||
| 'activate-coverage'
|
||||
| 'measure-turn-flush'
|
||||
| 'materialize-calibration'
|
||||
| 'cleanup';
|
||||
|
||||
const usage = (): never => {
|
||||
process.stderr.write(
|
||||
'usage: cli.ts <validate|dry-run|run|prepare|seed|verify-fixture|activate-coverage|materialize-calibration|cleanup> --config <file> [--tokens <0600-gitignored-file>] [--output <new-json-file>] [--confirm <load_schema>]\n'
|
||||
'usage: cli.ts <validate|dry-run|run|prepare|seed|verify-fixture|activate-coverage|measure-turn-flush|materialize-calibration|cleanup> --config <file> [--tokens <0600-gitignored-file>] [--output <new-json-file>] [--confirm <load_schema>]\n'
|
||||
);
|
||||
process.exit(64);
|
||||
};
|
||||
@@ -43,6 +45,7 @@ const parseArguments = (
|
||||
'seed',
|
||||
'verify-fixture',
|
||||
'activate-coverage',
|
||||
'measure-turn-flush',
|
||||
'materialize-calibration',
|
||||
'cleanup',
|
||||
].includes(command ?? '')
|
||||
@@ -69,6 +72,11 @@ const parseArguments = (
|
||||
(!values.get('--confirm') || values.has('--tokens') || values.has('--output'))
|
||||
)
|
||||
usage();
|
||||
if (
|
||||
command === 'measure-turn-flush' &&
|
||||
(!values.get('--confirm') || !values.get('--output') || values.has('--tokens'))
|
||||
)
|
||||
usage();
|
||||
if (
|
||||
command === 'materialize-calibration' &&
|
||||
(!values.get('--output') || values.has('--tokens') || values.has('--confirm'))
|
||||
@@ -116,6 +124,20 @@ const main = async (): Promise<void> => {
|
||||
process.stdout.write(`${JSON.stringify(await activateCapacityCoverage(config, args.confirm!))}\n`);
|
||||
return;
|
||||
}
|
||||
if (args.command === 'measure-turn-flush') {
|
||||
const output = path.resolve(args.output!);
|
||||
await mkdir(path.dirname(output), { recursive: true });
|
||||
const result = await measureTurnFlush({ config, confirmation: args.confirm! });
|
||||
await writeFile(output, `${JSON.stringify(result, null, 2)}\n`, {
|
||||
encoding: 'utf8',
|
||||
flag: 'wx',
|
||||
mode: 0o600,
|
||||
});
|
||||
process.stdout.write(
|
||||
`${JSON.stringify({ completed: true, processedGenerals: result.throughput.processedGenerals, outputWritten: true })}\n`
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (args.command === 'materialize-calibration') {
|
||||
process.stdout.write(
|
||||
`${JSON.stringify(
|
||||
|
||||
@@ -0,0 +1,328 @@
|
||||
import { monitorEventLoopDelay } from 'node:perf_hooks';
|
||||
|
||||
import {
|
||||
createTurnDaemonRuntime,
|
||||
getNextTickTime,
|
||||
type TurnCheckpoint,
|
||||
type TurnRunResult,
|
||||
} from '@sammo-ts/game-engine';
|
||||
import { createGamePostgresConnector, GamePrisma } from '@sammo-ts/infra';
|
||||
|
||||
import type { LoadConfig } from './config.js';
|
||||
import { verifyCapacityFixture } from './fixture.js';
|
||||
import { summarizeDistribution } from './metrics.js';
|
||||
|
||||
type DatabaseStatsRow = {
|
||||
xactCommit: bigint;
|
||||
xactRollback: bigint;
|
||||
blocksRead: bigint;
|
||||
blocksHit: bigint;
|
||||
tuplesReturned: bigint;
|
||||
tuplesFetched: bigint;
|
||||
tuplesInserted: bigint;
|
||||
tuplesUpdated: bigint;
|
||||
tuplesDeleted: bigint;
|
||||
tempFiles: bigint;
|
||||
tempBytes: bigint;
|
||||
deadlocks: bigint;
|
||||
};
|
||||
|
||||
type ActivityRow = {
|
||||
connections: bigint;
|
||||
active: bigint;
|
||||
waitingLocks: bigint;
|
||||
};
|
||||
|
||||
const readDatabaseStats = async (
|
||||
database: ReturnType<typeof createGamePostgresConnector>['prisma']
|
||||
): Promise<DatabaseStatsRow> => {
|
||||
const rows = await database.$queryRaw<DatabaseStatsRow[]>(GamePrisma.sql`
|
||||
SELECT
|
||||
xact_commit AS "xactCommit",
|
||||
xact_rollback AS "xactRollback",
|
||||
blks_read AS "blocksRead",
|
||||
blks_hit AS "blocksHit",
|
||||
tup_returned AS "tuplesReturned",
|
||||
tup_fetched AS "tuplesFetched",
|
||||
tup_inserted AS "tuplesInserted",
|
||||
tup_updated AS "tuplesUpdated",
|
||||
tup_deleted AS "tuplesDeleted",
|
||||
temp_files AS "tempFiles",
|
||||
temp_bytes AS "tempBytes",
|
||||
deadlocks
|
||||
FROM pg_stat_database
|
||||
WHERE datname = current_database()
|
||||
`);
|
||||
const row = rows[0];
|
||||
if (!row) throw new Error('pg_stat_database did not return the current database');
|
||||
return row;
|
||||
};
|
||||
|
||||
const subtractDatabaseStats = (before: DatabaseStatsRow, after: DatabaseStatsRow): Record<string, string> =>
|
||||
Object.fromEntries(
|
||||
Object.keys(before).map((key) => {
|
||||
const name = key as keyof DatabaseStatsRow;
|
||||
return [key, (after[name] - before[name]).toString()];
|
||||
})
|
||||
);
|
||||
|
||||
const readActivity = async (
|
||||
database: ReturnType<typeof createGamePostgresConnector>['prisma']
|
||||
): Promise<ActivityRow> => {
|
||||
const rows = await database.$queryRaw<ActivityRow[]>(GamePrisma.sql`
|
||||
SELECT
|
||||
(SELECT count(*) FROM pg_stat_activity WHERE datname = current_database()) AS connections,
|
||||
(
|
||||
SELECT count(*)
|
||||
FROM pg_stat_activity
|
||||
WHERE datname = current_database() AND state = 'active' AND pid <> pg_backend_pid()
|
||||
) AS active,
|
||||
(
|
||||
SELECT count(*)
|
||||
FROM pg_locks
|
||||
WHERE NOT granted AND database = (SELECT oid FROM pg_database WHERE datname = current_database())
|
||||
) AS "waitingLocks"
|
||||
`);
|
||||
const row = rows[0];
|
||||
if (!row) throw new Error('PostgreSQL activity sampler returned no row');
|
||||
return row;
|
||||
};
|
||||
|
||||
const round = (value: number): number => Math.round(value * 1000) / 1000;
|
||||
|
||||
const includeSubMillisecondGameTick = (turnTime: Date): Date =>
|
||||
// Game ticks are finer than JavaScript Date's millisecond precision. The
|
||||
// loader projects authoritative turn_tick to a floored Date, so replaying
|
||||
// that exact Date can map to a tick just before the general is due.
|
||||
new Date(turnTime.getTime() + 1);
|
||||
|
||||
/**
|
||||
* Runs exactly one logical month through the production loader, lease/fencing,
|
||||
* dirty-state flush, journal/outbox and Redis publication boundaries. General
|
||||
* turns are committed one at a time in chronological order, matching a healthy
|
||||
* realtime daemon rather than a catch-up chunk.
|
||||
*/
|
||||
export const measureTurnFlush = async (options: {
|
||||
config: LoadConfig;
|
||||
confirmation: string;
|
||||
env?: NodeJS.ProcessEnv;
|
||||
}) => {
|
||||
const env = options.env ?? process.env;
|
||||
if (options.confirmation !== options.config.isolation.postgresSchema) {
|
||||
throw new Error('turn-flush confirmation must exactly equal isolation.postgresSchema');
|
||||
}
|
||||
const databaseUrl = env.LOAD_TEST_DATABASE_URL;
|
||||
const redisUrl = env.LOAD_TEST_REDIS_URL;
|
||||
if (!databaseUrl || !redisUrl) {
|
||||
throw new Error('LOAD_TEST_DATABASE_URL and LOAD_TEST_REDIS_URL are required');
|
||||
}
|
||||
const fixture = await verifyCapacityFixture(options.config, env);
|
||||
if (!fixture.valid) throw new Error('fixture verification failed; refusing turn-flush measurement');
|
||||
|
||||
const observer = createGamePostgresConnector({ url: databaseUrl });
|
||||
await observer.connect();
|
||||
let beforeStats: DatabaseStatsRow;
|
||||
try {
|
||||
beforeStats = await readDatabaseStats(observer.prisma);
|
||||
} catch (error) {
|
||||
await observer.disconnect().catch(() => undefined);
|
||||
throw error;
|
||||
}
|
||||
const activity = {
|
||||
samples: 0,
|
||||
failures: 0,
|
||||
maxConnections: 0,
|
||||
maxActive: 0,
|
||||
maxWaitingLocks: 0,
|
||||
};
|
||||
let sampling = true;
|
||||
const sampleActivity = async (): Promise<void> => {
|
||||
while (sampling) {
|
||||
try {
|
||||
const sample = await readActivity(observer.prisma);
|
||||
activity.samples += 1;
|
||||
activity.maxConnections = Math.max(activity.maxConnections, Number(sample.connections));
|
||||
activity.maxActive = Math.max(activity.maxActive, Number(sample.active));
|
||||
activity.maxWaitingLocks = Math.max(activity.maxWaitingLocks, Number(sample.waitingLocks));
|
||||
} catch {
|
||||
activity.failures += 1;
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 10));
|
||||
}
|
||||
};
|
||||
const activityPromise = sampleActivity();
|
||||
|
||||
const histogram = monitorEventLoopDelay({ resolution: 20 });
|
||||
histogram.enable();
|
||||
const cpuStart = process.cpuUsage();
|
||||
const wallStartNs = process.hrtime.bigint();
|
||||
let maxRssBytes = process.memoryUsage().rss;
|
||||
const generalTransactionMs: number[] = [];
|
||||
const monthlyTransactionMs: number[] = [];
|
||||
const publicationMs: number[] = [];
|
||||
let processedGenerals = 0;
|
||||
let processedMonths = 0;
|
||||
let runtime: Awaited<ReturnType<typeof createTurnDaemonRuntime>> | null = null;
|
||||
let startYearMonth: string | null = null;
|
||||
let endYearMonth: string | null = null;
|
||||
let initialGeneralCount: number | null = null;
|
||||
let finalGeneralCount: number | null = null;
|
||||
let runError: unknown;
|
||||
|
||||
try {
|
||||
runtime = await createTurnDaemonRuntime({
|
||||
// Omitting profileName deliberately disables the Gateway admin-action
|
||||
// consumer. The load schema contains game tables only; the scoped
|
||||
// profile value still isolates the lease and Redis channel.
|
||||
profile: options.config.isolation.profileName,
|
||||
databaseUrl,
|
||||
redisUrl,
|
||||
gameClockMode: 'manual',
|
||||
enableDatabaseFlush: true,
|
||||
enableLeaseHeartbeat: true,
|
||||
databaseTransactionTimeoutMs: 30_000,
|
||||
});
|
||||
const initialState = runtime.world.getState();
|
||||
initialGeneralCount = runtime.world.listGenerals().length;
|
||||
startYearMonth = `${initialState.currentYear}-${String(initialState.currentMonth).padStart(2, '0')}`;
|
||||
const tickMinutes = Math.max(1, Math.round(initialState.tickSeconds / 60));
|
||||
const boundary = getNextTickTime(initialState.lastTurnTime, tickMinutes);
|
||||
let checkpoint: TurnCheckpoint | undefined = await runtime.stateStore.loadCheckpoint();
|
||||
|
||||
const execute = async (target: Date, maxGenerals: number): Promise<TurnRunResult> => {
|
||||
const started = performance.now();
|
||||
const result = await runtime!.stateManager.transaction(async () => {
|
||||
await runtime!.stateStore.advanceGameClockTo(target, new Date());
|
||||
const next = await runtime!.processor.run(
|
||||
target,
|
||||
{ budgetMs: 30_000, maxGenerals, catchUpCap: 1 },
|
||||
checkpoint
|
||||
);
|
||||
await runtime!.stateStore.saveLastTurnTime(new Date(next.lastTurnTime));
|
||||
await runtime!.stateStore.saveCheckpoint(next.checkpoint);
|
||||
await runtime!.hooks?.flushChanges?.(next);
|
||||
return next;
|
||||
});
|
||||
const transactionMs = performance.now() - started;
|
||||
checkpoint = result.checkpoint;
|
||||
maxRssBytes = Math.max(maxRssBytes, process.memoryUsage().rss);
|
||||
const publishStarted = performance.now();
|
||||
await runtime!.hooks?.publishEvents?.(result);
|
||||
publicationMs.push(performance.now() - publishStarted);
|
||||
if (result.processedTurns > 0) monthlyTransactionMs.push(transactionMs);
|
||||
else generalTransactionMs.push(transactionMs);
|
||||
processedGenerals += result.processedGenerals;
|
||||
processedMonths += result.processedTurns;
|
||||
return result;
|
||||
};
|
||||
|
||||
while (true) {
|
||||
const nextGeneral = await runtime.stateStore.loadNextGeneralTurnTime();
|
||||
if (!nextGeneral || nextGeneral.getTime() >= boundary.getTime()) break;
|
||||
const result = await execute(includeSubMillisecondGameTick(nextGeneral), 1);
|
||||
if (result.processedGenerals !== 1 || result.processedTurns !== 0) {
|
||||
throw new Error(
|
||||
`chronological turn-flush run expected one general and zero months; got ${result.processedGenerals} generals and ${result.processedTurns} months`
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const monthly = await execute(boundary, 200);
|
||||
if (monthly.processedTurns !== 1) {
|
||||
throw new Error('turn-flush measurement did not cross exactly one monthly boundary');
|
||||
}
|
||||
const finalState = runtime.world.getState();
|
||||
finalGeneralCount = runtime.world.listGenerals().length;
|
||||
endYearMonth = `${finalState.currentYear}-${String(finalState.currentMonth).padStart(2, '0')}`;
|
||||
} catch (error) {
|
||||
runError = error;
|
||||
} finally {
|
||||
try {
|
||||
await runtime?.close();
|
||||
} catch (error) {
|
||||
runError ??= error;
|
||||
}
|
||||
sampling = false;
|
||||
try {
|
||||
await activityPromise;
|
||||
} catch (error) {
|
||||
runError ??= error;
|
||||
}
|
||||
}
|
||||
if (runError !== undefined) {
|
||||
histogram.disable();
|
||||
await observer.disconnect().catch(() => undefined);
|
||||
throw runError;
|
||||
}
|
||||
if (
|
||||
startYearMonth === null ||
|
||||
endYearMonth === null ||
|
||||
initialGeneralCount === null ||
|
||||
finalGeneralCount === null
|
||||
) {
|
||||
histogram.disable();
|
||||
await observer.disconnect().catch(() => undefined);
|
||||
throw new Error('turn-flush measurement completed without a full result');
|
||||
}
|
||||
|
||||
let afterStats: DatabaseStatsRow;
|
||||
try {
|
||||
afterStats = await readDatabaseStats(observer.prisma);
|
||||
} catch (error) {
|
||||
histogram.disable();
|
||||
await observer.disconnect().catch(() => undefined);
|
||||
throw error;
|
||||
}
|
||||
await observer.disconnect();
|
||||
histogram.disable();
|
||||
const elapsedMs = Number(process.hrtime.bigint() - wallStartNs) / 1_000_000;
|
||||
const cpu = process.cpuUsage(cpuStart);
|
||||
const cpuMs = (cpu.user + cpu.system) / 1_000;
|
||||
const fromNs = (value: number): number => (Number.isFinite(value) ? round(value / 1_000_000) : 0);
|
||||
|
||||
return {
|
||||
version: 1,
|
||||
fixture: {
|
||||
name: options.config.name,
|
||||
fixtureSha256: fixture.fixtureSha256,
|
||||
capacity: options.config.capacity,
|
||||
},
|
||||
mode: 'chronological-one-general-per-transaction-plus-month-boundary',
|
||||
startYearMonth,
|
||||
endYearMonth,
|
||||
elapsedMs: round(elapsedMs),
|
||||
throughput: {
|
||||
generalTurnsPerSecond: round(processedGenerals / Math.max(elapsedMs / 1_000, 0.001)),
|
||||
processedGenerals,
|
||||
processedMonths,
|
||||
},
|
||||
population: {
|
||||
initialGenerals: initialGeneralCount,
|
||||
finalGenerals: finalGeneralCount,
|
||||
generalDelta: finalGeneralCount - initialGeneralCount,
|
||||
},
|
||||
latencyMs: {
|
||||
generalTransaction: summarizeDistribution(generalTransactionMs),
|
||||
monthlyTransaction: summarizeDistribution(monthlyTransactionMs),
|
||||
redisPublication: summarizeDistribution(publicationMs),
|
||||
},
|
||||
postgres: {
|
||||
statsScope: 'database-wide-including-observer-sampler',
|
||||
statsDelta: subtractDatabaseStats(beforeStats, afterStats),
|
||||
activity,
|
||||
},
|
||||
process: {
|
||||
cpuPercentOfOneCore: round((cpuMs / Math.max(elapsedMs, 1)) * 100),
|
||||
maxRssBytes,
|
||||
eventLoopLagMs: {
|
||||
min: fromNs(histogram.min),
|
||||
max: fromNs(histogram.max),
|
||||
mean: fromNs(histogram.mean),
|
||||
p50: fromNs(histogram.percentile(50)),
|
||||
p95: fromNs(histogram.percentile(95)),
|
||||
p99: fromNs(histogram.percentile(99)),
|
||||
},
|
||||
},
|
||||
measuredAt: new Date().toISOString(),
|
||||
};
|
||||
};
|
||||
@@ -6,6 +6,10 @@ import test from 'node:test';
|
||||
import { assertRuntimeMetadataFinalized, canonicalJson, expandWeightedOperations, loadTokens, validateLoadConfig } from '../src/config.js';
|
||||
|
||||
const samplePath = new URL('../config/300-users-900-npcs-5m.json', import.meta.url);
|
||||
const oneMinuteProfilePaths = [
|
||||
new URL('../config/nya-10-users-800-npcs-1m.json', import.meta.url),
|
||||
new URL('../config/pya-10-users-800-npcs-1m.json', import.meta.url),
|
||||
];
|
||||
|
||||
void test('the 300 viewer, 900 NPC, five-minute sample validates', async () => {
|
||||
const config = validateLoadConfig(JSON.parse(await readFile(samplePath, 'utf8')));
|
||||
@@ -18,6 +22,24 @@ void test('the 300 viewer, 900 NPC, five-minute sample validates', async () => {
|
||||
assert.deepEqual(new Set(config.phases.map((phase) => phase.kind)), new Set(['idle', 'own', 'global', 'mixed']));
|
||||
});
|
||||
|
||||
void test('nya and pya one-minute profiles use distinct database and Redis isolation', async () => {
|
||||
const configs = await Promise.all(
|
||||
oneMinuteProfilePaths.map(async (configPath) =>
|
||||
validateLoadConfig(JSON.parse(await readFile(configPath, 'utf8')))
|
||||
)
|
||||
);
|
||||
assert.deepEqual(
|
||||
configs.map((config) => config.capacity),
|
||||
[
|
||||
{ authenticatedViewers: 10, npcGenerals: 800, humanGenerals: 10, turnIntervalMs: 60_000 },
|
||||
{ authenticatedViewers: 10, npcGenerals: 800, humanGenerals: 10, turnIntervalMs: 60_000 },
|
||||
]
|
||||
);
|
||||
assert.equal(new Set(configs.map((config) => config.isolation.postgresSchema)).size, 2);
|
||||
assert.equal(new Set(configs.map((config) => config.isolation.redisDatabase)).size, 2);
|
||||
assert.equal(new Set(configs.map((config) => config.isolation.profileName)).size, 2);
|
||||
});
|
||||
|
||||
void test('validation rejects public, non-allowlisted, and mutating targets', async () => {
|
||||
const raw = JSON.parse(await readFile(samplePath, 'utf8')) as Record<string, any>;
|
||||
raw.target.publicProfile = true;
|
||||
|
||||
Reference in New Issue
Block a user