ADR-0008 — Bounded sampling for large and backed matrices¶
- Status: Accepted
- Date: 2026-07 (as embodied in the 0.2.0b1 codebase)
Context¶
Full densification or exhaustive scans of large matrices are unsafe for memory and runtime. Backed AnnData must fail gracefully.
Decision¶
- Never densify a full sparse matrix merely to validate it.
- Numeric / AI-readiness checks use bounded row sampling (
sample_rows, default 5000) for large or backed inputs. - Sparse in-memory paths may inspect
.datawithout densifying. - Unsupported modes skip with an explicit check execution record rather than crashing the run.
Consequences¶
- Rare defects outside the sample may be missed (documented limitation).
- Large public profiles remain practical to validate.
Alternatives considered¶
- Always full-matrix scan — rejected for scalability.
- Skip numeric checks on backed data silently — rejected; skips are recorded.
References¶
src/cp_anndata_validator/sampling.py,loading.py- Matrix / AI-readiness checks and their tests
- Large or backed AnnData