Section 22Experimentation and Evidence
Performance and scalability requirements
Defines measurement, profiling, budgets, parallelism, throughput, memory, and representative workload reporting.
Section relationships
Quantitative targets remain open until the reference hardware and mission are fixed. A TBD-* is a required product decision, not permission to omit measurement.
| ID | Name | Priority | Requirement | Verification |
|---|---|---|---|---|
| PERF-001 | Reference benchmark definition | MVP | BeforeMetal SHALL publish the exact reference scene, mission, machine configuration, software configuration, and measurement procedure used for performance claims. | Inspection |
| PERF-002 | Interactive real-time factor | MVP | The portable product SHALL meet TBD-PERF-RTF for the reference rover with the MVP sensor set. | Analysis |
| PERF-003 | Headless throughput | MVP | The portable product SHALL meet TBD-PERF-HEADLESS simulated seconds or control steps per wall second for the reference workload. | Analysis |
| PERF-004 | Startup time | MVP | The portable product SHALL meet TBD-PERF-START from process launch to compiled reference scenario readiness on reference hardware. | Test |
| PERF-005 | Reference memory | MVP | The portable product SHALL remain within TBD-PERF-MEM peak resident memory for the reference scenario. | Test |
| PERF-006 | Telemetry overhead | MVP | Default metrics and tracing SHALL remain within TBD-PERF-TRACE overhead relative to an equivalent unrecorded run. | Analysis |
| PERF-007 | CPU-only headless | MVP | The reference headless workload SHALL run without initializing a graphics device. | Test |
| PERF-008 | Rendering decoupling | MVP | Reducing, disabling, or increasing viewer frame rate SHALL not change committed non-render sensor and mechanics results under deterministic mode. | Test |
| PERF-009 | Independent-world scaling | Beta | Batched independent worlds SHALL meet TBD-PERF-SCALE efficiency across the approved core-count range. | Analysis |
| PERF-010 | Memory per environment | Beta | A batched workload SHALL report and meet TBD-PERF-ENV-MEM incremental memory per equivalent environment. | Analysis |
| PERF-011 | Bulk sensor bandwidth | Beta | The supported image and point-cloud workload SHALL meet TBD-PERF-SENSOR throughput without unbounded queue growth. | Analysis |
| PERF-012 | Profiling visibility | MVP | BeforeMetal SHALL report wall time by kernel, mechanics, environment, devices, sensors, controller, rendering, and data output. | Test |
| PERF-013 | Performance regression gate | MVP | A committed benchmark regression beyond TBD-PERF-REGRESSION SHALL require explicit recorded approval. | Analysis |
| PERF-014 | Determinism-performance separation | MVP | Performance documentation SHALL not imply that strict reproducibility and maximum-throughput modes have identical cost or guarantees. | Inspection |
Generated from the canonical specification. Edit section metadata or prose in docs/requirements.md; the website rebuilds this page and its relationships automatically.