Metadata

Every remote entity in PySUS — clients, datasets, groups and files — exposes a unified metadata surface via the .metadata property:

from pysus.api.saude import SaudeClient

async with SaudeClient() as c:
    pkg = await c.fetch_dataset("arboviroses-dengue")
    # pkg.metadata is a MetadataBag

The MetadataBag

pysus.api.metadata.models.MetadataBag organizes metadata into eight typed facets:

identity — name, slug, aliases, and cross_origin_id (the shared CKAN UUID that links the same dataset across the dados.gov.br and dadosabertos.saude.gov.br portals);
description — title, long name, description, tags, themes;
temporal — created/modified timestamps, periodicity, year/month;
spatial — geographic scope, UF list, municipalities, state;
provenance — origin, organization, author, contact, license;
structure — columns, row count, file count, format;
access — URL, format, size, auth requirement, policy;
quality — content fingerprint, integrity, freshness.

Every bag serializes to JSON (bag.to_dict() / MetadataBag.from_dict(...)) so it can be persisted in the DuckLake catalogs or snapshot files.

Extractors

Each client ships its own extractors, one per entity type (MetadataExtractor):

Client	Extractors
FTP	`FtpDatasetExtractor`, `FtpGroupExtractor`, `FtpFileExtractor`
DadosGov	`DadosGovDatasetExtractor`, `DadosGovGroupExtractor`, `DadosGovFileExtractor`
DuckLake	`DuckLakeDatasetExtractor`, `DuckLakeGroupExtractor`, `DuckLakeFileExtractor`
Saude	`SaudeDatasetExtractor`, `SaudeGroupExtractor`, `SaudeFileExtractor`

Concrete model classes declare their extractors via the extractor_types class attribute; the base classes only know that some extractor exists:

file = await some_dataset.search(name="DENGBR25.csv.zip")[0]
bag = file.metadata          # merged bag, cached
print(bag.temporal.year)     # 2025 (from the filename formatter)
print(bag.access.size_bytes)

Merging across origins

merge_bags() combines bags from different origins with a documented per-facet precedence: Saude wins for descriptive fields, DuckLake for structure and content fingerprints, and so on:

from pysus.api.metadata.models import merge_bags

merged = merge_bags([ftp_file.metadata, saude_file.metadata])

# Descriptive metadata comes from Saude, structure from DuckLake:
print(merged.description.title)
print(merged.structure.row_count)

Local files

Tabular local files (Parquet, CSV, DBF, …) also expose .metadata computed from their content — columns, row count and size — without any network access:

local = await file.download()
print(local.metadata.structure.columns)

Curated column descriptions

PySUS ships curated YAML schemas with field names and Portuguese descriptions transcribed from official data dictionaries. These cover SIM Declaração de Óbito, SINAN disease notifications (dengue, peste, chikungunya, etc.), and other datasets.

The SIM schema is covered by pysus/api/saude/schemas/vigilanciameioambiente.yaml. Its field names and Portuguese descriptions are transcribed from the official SIM data dictionary (portal resource updated 9 June 2025; the file is marked 06/2025); no records are bundled with PySUS. The source is the official SIM data dictionary.

The schema is keyed by the downloaded resource basename, so it can be applied without relying on a particular data file or downloading a sample:

from pysus.api.saude.schemas import load_endpoint_columns

columns = load_endpoint_columns(
    "vigilanciameioambiente", "DO24OPEN_csv"
)
assert any(column["name"] == "TIPOBITO" for column in columns)

SINAN column metadata includes categories and characteristics extracted from the official tarball dictionaries:

from pysus import search_columns, available_databases, load_column_metadata

# List available curated schemas
print(available_databases())  # ["sim", "sinan"]

# Load column metadata for a specific SINAN disease
columns = load_column_metadata("sinan", "Dengue")

# Search for a column across all SINAN diseases
results = search_columns("CON_CLASSI")
for r in results:
    print(r.dataset, r.name, r.categories)