Generic-delimited ingestion

Reading long- and wide-format CSV/TSV via a manifest. See Generic delimited and Manifest version 1.

The manifest

GenericManifest

Bases: _Base

The top-level manifest model.

load_manifest

load_manifest(path: Path) -> GenericManifest

Load and validate a manifest from a JSON or YAML file.

Raises:

Type	Description
ManifestError	if the file is missing, unparseable, or fails schema validation.

Source code in src/openauc/formats/manifest.py

def load_manifest(path: Path) -> GenericManifest:
    """Load and validate a manifest from a JSON or YAML file.

    Raises:
        ManifestError: if the file is missing, unparseable, or fails schema
            validation.
    """
    if not path.is_file():
        raise ManifestError(f"manifest file not found: {path}")
    text = path.read_text(encoding="utf-8")
    suffix = path.suffix.lower()
    try:
        if suffix in {".yaml", ".yml"}:
            raw = yaml.safe_load(text)
        elif suffix == ".json":
            raw = json.loads(text)
        else:
            raise ManifestError(
                f"unsupported manifest suffix {suffix!r} (expected .json/.yaml/.yml)"
            )
    except (json.JSONDecodeError, yaml.YAMLError) as exc:
        raise ManifestError(f"could not parse manifest {path.name}: {exc}") from exc

    if not isinstance(raw, dict):
        raise ManifestError(
            f"manifest {path.name} must contain a mapping at the top level"
        )
    try:
        return GenericManifest.model_validate(raw)
    except ValidationError as exc:
        raise ManifestError(f"manifest {path.name} failed validation: {exc}") from exc

Parser registry

get_parser

get_parser(format_id: str) -> Parser

Return the parser registered under format_id.

Raises:

Type	Description
UnsupportedFormatError	if no parser is registered under that id.

Source code in src/openauc/formats/registry.py

def get_parser(format_id: str) -> Parser:
    """Return the parser registered under ``format_id``.

    Raises:
        UnsupportedFormatError: if no parser is registered under that id.
    """
    try:
        return _REGISTRY[format_id]
    except KeyError:
        available = ", ".join(sorted(_REGISTRY)) or "(none)"
        raise UnsupportedFormatError(
            f"unknown format {format_id!r}; registered formats: {available}"
        ) from None

registered_ids

registered_ids() -> tuple[str, ...]

All registered format identifiers, sorted (parsers and archives).

Source code in src/openauc/formats/registry.py

def registered_ids() -> tuple[str, ...]:
    """All registered format identifiers, sorted (parsers and archives)."""
    return tuple(sorted({*_REGISTRY, *_ARCHIVE_FORMATS}))

FormatInfo dataclass

FormatInfo(
    format_id: str,
    name: str,
    suffixes: tuple[str, ...],
    layouts: tuple[str, ...],
    limitations: tuple[str, ...],
    doc_reference: str,
)

Public description of a registered parser (for available_formats).

Extending

A first-party parser subclasses Parser and registers itself. See Parser detection and ADR-0004.

Parser

Bases: ABC

Abstract base class for a format parser plugin.

Concrete subclasses set the class attributes below and implement :meth:detect and :meth:parse.

detect abstractmethod

detect(
    table: Table, manifest: GenericManifest | None
) -> DetectionResult

Report how confidently this parser can handle table.

Source code in src/openauc/formats/base.py

@abstractmethod
def detect(self, table: Table, manifest: GenericManifest | None) -> DetectionResult:
    """Report how confidently this parser can handle ``table``."""

parse abstractmethod

parse(
    table: Table,
    source: ResolvedSource,
    manifest: GenericManifest | None,
) -> ParseResult

Parse table into an :class:AUCExperiment (without provenance).

Source code in src/openauc/formats/base.py

@abstractmethod
def parse(
    self,
    table: Table,
    source: ResolvedSource,
    manifest: GenericManifest | None,
) -> ParseResult:
    """Parse ``table`` into an :class:`AUCExperiment` (without provenance)."""

info

info() -> FormatInfo

Return the public :class:FormatInfo description of this parser.

Source code in src/openauc/formats/base.py

def info(self) -> FormatInfo:
    """Return the public :class:`FormatInfo` description of this parser."""
    return FormatInfo(
        format_id=self.format_id,
        name=self.name,
        suffixes=self.suffixes,
        layouts=self.layouts,
        limitations=self.limitations,
        doc_reference=self.doc_reference,
    )