Persistence data

This page is the canonical reference for NAHPU persistence boundaries, schema changes, migrations, storage locations, and Darwin Core field mappings. Other architecture and code-contribution pages should link here instead of duplicating these details.

For the complete import/export feature inventory and service-to-crate data flows, see Data import and export. Keep this page focused on storage ownership, schema compatibility, and field mappings.

NAHPU is an offline-first application. Persistence changes must preserve existing user data because field teams may update the app or restore backups far from reliable internet access.

Persistence boundaries

NAHPU uses separate stores because project data, reproducible configuration, installation-local state, and files have different ownership and backup needs.

Store	Owner	Contents	Reproducibility
Drift over SQLite	Flutter app	Projects, specimens, sites, collecting events, personnel, taxonomy, media metadata, biological attributes, narratives, custom fields, and relationships	Canonical project dataset, exported as project-scoped JSON and CSV resources in NAHPU Data Packages
redb	nahpu_configs through Rust bridge	Custom option lists, record-export presets, document-template presets, and document layouts	Exportable when reproducing outputs on another installation
SharedPreferences	Flutter app	Installation-local UI and device state such as theme settings and migration markers	Not project data, do not store export-relevant configuration here
Documents directory	Dart services	SQLite databases, backups, project media, personnel media, custom fonts, and custom maps	Managed by app workflows and package and backup exporters
Temporary directory	Dart and platform services	Staging snapshots, intermediate files, and temporary archives	Disposable, never the source of truth

The normal application storage layout is:

Documents/nahpu/
├── nahpu.db
├── nahpu_configs.db
├── backup/
├── appMedia/personnel/
├── UserConfigs/
│   ├── fonts/
│   └── maps/
└── <project_uuid>/media/
    ├── site/
    ├── specimen/
    └── narrative/

The Flutter Drift database remains the canonical project store. nahpu_db generates Rust-side models and tabular helpers from the Drift schema, but it does not own the app's SQLite connection. nahpu_dwc and nahpu_dp receive caller-prepared data and do not replace the project database.

Version responsibilities

Keep these version concepts separate when reviewing a persistence change.

Version	Current value	Owner	Meaning
Drift schema version	22	kSchemaVersion in lib/services/database/database.dart	SQLite structure expected by the Flutter application
SQLite user version	Set by Drift	Drift	On-disk marker used to select upgrade steps
redb internal schema version	Not implemented	nahpu_configs	A future marker for redb table and value migrations
Configuration export schema	4	USER_CONFIG_SCHEMA_VERSION	Compatibility version for exported configuration payloads
Rust crate version	0.7.1	crates/nahpu_configs/Cargo.toml	Package release version, not a data migration marker
Schema mirror crate version	0.6.0	crates/nahpu_db/Cargo.toml	Release carrying the mirrored tables.drift; a schema change bumps it
redb engine version	4.1.0 dependency	Cargo dependency resolution	Storage engine behavior, not NAHPU application schema
Record-exchange format	6	Record-exchange model	Version of individual record and QR payloads
Project transfer format	9	Transfer model and archive code	Version of project import and export payloads

redb tables

The nahpu_configs crate stores reproducible configuration and export presets in a redb database. The tables use string keys and serialized JSON values.

Table	Key	Value	Purpose
user_configs	Configuration name	Configuration JSON	Installation configuration values such as custom option lists
record_export_presets	Preset name	Preset JSON	Named record-export field mappings and custom-field expressions
template_presets	Preset name	Preset JSON	Named reusable document-template definitions
document_layouts	Layout name	Layout JSON	Named document layout settings, including page and template-block configuration

ConfigDb::init opens all four tables and commits an initialization transaction. CRUD operations serialize model values directly to JSON bytes. The current implementation has no internal redb database migration runner. Existing databases without metadata are therefore treated as the only known layout and must be preserved by any future migration.

redb versioning and migration guidance

Use a dedicated application schema version for redb. Do not use the crate version, the redb engine version, or USER_CONFIG_SCHEMA_VERSION as a replacement. The recommended implementation is:

1. Add CONFIG_DB_SCHEMA_VERSION and table definitions in crates/nahpu_configs/src/schema.rs.
2. Add a metadata table with a reserved key such as schema_version. Treat a database that has the four current tables but no metadata row as version 1.
3. Add ordered migration functions in crates/nahpu_configs/src/migrations.rs, such as migrate_v1_to_v2.
4. Make ConfigDb::init run migrations before publishing the global database handle. Reject a database with a future version instead of guessing.
5. Run every step in order inside one redb write transaction. Update the metadata row only in that transaction so a failed migration can roll back.
6. Keep export schema migration separate. Validate an incoming UserConfigsExport.schema_version, transform old payloads in memory, reject future versions, then replace all tables atomically.
7. Add a JSON Lines metadata record so parsing preserves the source schema version instead of silently replacing it with the current version.

Required tests cover a fresh database, a legacy database without metadata, each migration step, future-version rejection, rollback after a failed step, idempotent reopen, JSON export and import compatibility, JSON Lines version preservation, and corrupted input that must not partially clear stored data.

Drift table schema

The canonical project schema is defined in lib/services/database/tables.drift and is currently at schema version 22. Drift manages these tables over SQLite. The summary below describes the current table set and the important ownership relationships.

Table	Role and important relationships
project	Project identity and lifecycle dates. Parent of project-scoped rows.
geography	Shared, project-independent locality fields deduplicated by matchKey. Added in v21.
site	Collection-site identity. Belongs to project and references shared geography.
siteAttribute	Habitat type, condition, description, and canopy cover for a site. Added in v19 when these columns moved out of site.
fossilSite	Fossil-specific extension of site through siteID. Renamed from paleontologySite in v13.
coordinate	Coordinates and georeferencing details associated with a site.
collEvent	Collecting events, dates, times, methods, and site associations.
environment	Environmental and astronomical observations for collecting events. Renamed from weather in v19.
collPersonnel	Personnel and collecting roles associated with events.
collEffort	Collecting methods, equipment, counts, and effort notes for events.
narrative	Project or site field narratives and their metadata.
media	Project media metadata. v13 introduced nullable uri.
narrativeMedia	Links narratives to media files.
siteMedia	Links sites to media files.
eventMedia	Links collecting events to media files.
specimenMedia	Links specimens to media files.
associatedData	Project-scoped external data with name, type, date, description, and the v13 uri field.
specimenAssociatedData	Composite link between specimens and associated data. Replaces associatedData.specimenUuid.
siteAssociatedData	Composite link between sites and associated data.
eventAssociatedData	Composite link between collecting events and associated data.
personnelList	Links personnel to projects.
personnel	People, contact details, roles, and field-number settings.
taxonomy	Taxonomic names, classifications, status, and notes.
specimen	Core specimen records and project, taxon, event, coordinate, and personnel relationships.
mammalAttribute	Mammal measurements, reproductive observations, and bat or echolocation attributes.
birdAttribute	Bird measurements, colors, molt, reproductive observations, and habitat attributes.
herpAttribute	Herpetofauna sex, life stage, weight, snout-vent length, and remark attributes.
invertebrateAttribute	Invertebrate sex, life stage, caste, host interactions, morphometrics, and remarks.
fossilAttribute	Fossil-specific specimen attributes.
parasiteDetection	Detection records connecting specimens and parasite observations.
parasite	Parasite taxonomy and observation details.
specimenPart	Specimen parts, tissues, barcodes, treatments, loans, and preparation details.
customFieldDefinition	Global and project custom-field definitions, types, placement, ordering, export mapping, and archive state.
customFieldValue	Owner-specific custom values. A non-legacy row belongs to exactly one event, site, specimen, specimen part, or parasite. v20 added nullable eventId ownership.

The custom-field service validates definition scope, project ownership, placement, catalog applicability for specimen-related placements, and one value per definition and owner. Environmental custom fields use FieldUISection.environmentalData, belong to collEvent, and deliberately do not apply specimen catalog-format filtering. Archived definitions disappear from forms but remain available to exports so historical values are preserved.

Foreign keys, triggers, and indexes

Project-owned tables use cascading foreign keys where deleting a project must remove dependent data. Composite link tables use cascading foreign keys and composite primary keys. specimenAssociatedData, siteAssociatedData, and eventAssociatedData link their respective owner records to associatedData.

The specimen_associated_data_same_project, site_associated_data_same_project, and event_associated_data_same_project triggers run before link inserts. They abort a link when the associated data row belongs to a different project. There is no trigger on associatedData itself. The v13 migration first moves legacy specimen links, then drops the old direct column, so an additional associatedData trigger is not needed.

The custom_field_value_validate_insert and custom_field_value_validate_update triggers verify that each custom value's owner matches its definition placement and project. Partial unique indexes enforce one value per definition for each owner type, including (fieldDefinitionId, eventId) for environmental data.

Current indexes cover project lookups and common joins, including project and species queries on specimen, project and site queries on collEvent, site and coordinate lookups, specimen part lookups, and associated-data link scans. Before adding a new index, measure the query and write path because every index increases migration and insert cost.

Schema workflow

1. Open or discuss a GitHub issue before making a schema change. Record the data preservation rule and the expected upgrade path.

2. Update the Drift schema in lib/services/database/tables.drift.

3. Add a dated comment at the top of tables.drift explaining the change.

4. Bump kSchemaVersion in lib/services/database/database.dart by exactly one version.

5. Add the release step to releaseSteps in lib/services/database/migration_coordinator.dart. Keep historical migrations in place because an old database may skip directly to the new release.

6. Put reusable SQL and schema helpers in lib/services/database/migration_utilities.dart. Keep the legacy v1 to v11 path in database.dart unchanged unless the change explicitly repairs legacy behavior.

7. Regenerate Drift code, then dump the new schema. Code generation must run first: the dump reads the generated database, so dumping before it writes the previous schema under the new file name. Point scripts/dump_schema.sh at the new version before running it.

   bash scripts/codegen.sh
   bash scripts/dump_schema.sh
   cp lib/services/database/tables.drift db_schemas/drift_tables/tables_v22.drift

8. Regenerate the migration-test schemas. drift_dev schema generate lists its input directory non-recursively and rewrites the GeneratedHelper switch in test/database/generated_migrations/schema.dart from only the dumps it finds. Flatten db_schemas/legacy_schema/ into db_schemas/ for the run, or the helper loses every version before v19 and the fixtures that start at v6 stop compiling. Remove the copies afterwards.

   cp db_schemas/legacy_schema/*.json db_schemas/
   dart run drift_dev schema generate db_schemas test/database/generated_migrations

9. Never call migrator.createTable(db.<table>) from a historical migration step for a table the change renames or reshapes. That getter always builds the current shape, so an old database would gain the new table mid-chain and the later step would fail. Spell the historical CREATE TABLE out instead, as _Version12Migration and _Version19Migration do for arthropodAttribute, and leave those steps' validation lists on the old names.

10. Mirror lib/services/database/tables.drift into crates/nahpu_db/schema/tables.drift in nahpu_api. The two files must be byte-identical; nahpu_db generates its Rust models from the mirror, so a renamed table renames the generated struct. Bump nahpu_db, update the exact pins in nahpu_dwc and nahpu_export, and review the Darwin Core audit below. See API for building the app against the unpublished crates.

11. Add migration fixtures and tests in test/database/migration_test.dart. The suite currently upgrades fixtures from v6 through v21 to v22. Point every migrateAndValidate call at the new version: a step that recreates a trigger builds the current shape, so a call left on the previous version fails schema comparison.

12. Run flutter test, flutter analyze, cargo check, and cargo clippy when generated Rust or bridge data is affected.

Worked example: v21 to v22

The v22 migration in lib/services/database/migration_coordinator.dart:

1. Drops both custom-field validation triggers first. They compare customFieldDefinition.catalogFormat against specimen.taxonGroup, and the backfills below rewrite both sides.
2. Renames arthropodAttribute to invertebrateAttribute. The columns are unchanged, so _renameTableIfPresent is enough.
3. Backfills specimen.taxonGroup from Arthropods to Invertebrates, and rewrites customFieldDefinition.catalogFormat from the taxon-based names to the discipline names mammalogy, ornithology, herpetology, and invertebrateZoology.
4. Recreates both triggers from the generated definitions, then verifies the row count survived, that no stale taxon group or catalog format remains, and that no custom-field value is left whose definition no longer matches its specimen. That last check matters because the recreated trigger would otherwise reject the user's next edit rather than the migration.

Catalog formats and taxon groups are now separate concepts: CatalogFmt names the discipline a collection is curated under, specimen.taxonGroup names the taxon, and the trigger CASE translates one to the other. The CASE appears six times in tables.drift and must stay in step with matchTaxonGroupToCatFmt in lib/services/types/specimens.dart. If the two disagree, the app offers fields that SQLite then refuses to store.

Historical example: v20 to v21

The v21 migration in lib/services/database/migration_coordinator.dart:

1. Creates the shared geography table.
2. Normalizes the six former site-locality fields into a stable matchKey and inserts one geography row per distinct non-empty locality.
3. Rebuilds site, replacing those six columns with nullable geographyId, then restores every migrated relationship.
4. Adds site_geography_idx, verifies that no legacy geography columns remain, rejects duplicate match keys, and runs foreign-key and integrity checks.
5. Leaves sites with no locality values unlinked instead of creating blank geography rows.

The v12-to-v13 example below remains historical context. Keep these examples aligned with the migration code when future schema versions are added.

Historical example: v12 to v13

The v13 migration in lib/services/database/migration_coordinator.dart performs these operations in order.

1. Find legacy associatedData rows with a non-null specimenUuid.
2. Insert missing rows into specimenAssociatedData only when the specimen and associated-data rows have the same projectUuid.
3. Abort if any remaining legacy relationship cannot be represented safely.
4. Rebuild associatedData with Drift TableMigration, copying old url values into new uri values and dropping specimenUuid.
5. Rename paleontologySite to fossilSite when the old table exists.
6. Add media.uri and customFieldValue.unit.
7. Validate that fossilSite exists, paleontologySite is absent, associatedData has uri and no legacy columns, foreign keys pass, and SQLite integrity checks succeed.

The v12 migration creates historical paleontologySite and unit-less customFieldValue shapes with raw SQL because generated v13 accessors no longer expose those old definitions. This is why migration fixtures must use the historical schema rather than only current Drift table helpers.

Migration rules and test expectations

• Never rely on users deleting their database to recover from a migration.
• Preserve rows and values unless the issue explicitly documents a lossy conversion.
• Keep each version step deterministic and safe when an older development build may have partially applied the change.
• Disable foreign keys only for the controlled migration transaction, then restore them and run foreign-key and integrity checks.
• Abort when project ownership or relationship data cannot be represented in the new schema. Do not silently attach data to another project.
• Test fresh creation, direct upgrades from every supported historical version, populated rows, null values, duplicate links, invalid ownership, rollback, and reopening the migrated database.
• Include migration risk, backup expectations, generated artifacts, and test coverage in the pull request description.

Data integrity

Import, export, backup, and migration changes can permanently affect user data. Favor small migrations, explicit tests, and clear pull request notes.

Persistence data flows

Database-backed workflows run from Dart services and providers. Screens render state and dispatch actions, they do not own database queries or migrations.

Taxonomy import and record export reads or assembles rows from Drift. Dart performs application-specific validation and database updates, while nahpu_db may provide reusable tabular readers and writers.

Bundle export builds a transport-safe request from database records, configuration, vocabularies, and files. nahpu_dwc writes Darwin Core Archive and Darwin Core Data Package outputs. nahpu_dp writes the complete NAHPU Data Package, including the project-transfer JSON, matching CSV resources, and reproducibility metadata.

Project transfer remains coordinated by Dart because it owns database rows, media manifests, conflict handling, and import decisions. nahpu_archive provides the archive container operations.

Configuration persistence belongs in nahpu_configs when values must be exported or reproduced. Installation-local preferences remain in SharedPreferences.

Darwin Core field mapping

This is the audited schema-v21 mapping surface for nahpu_dwc 0.6.0. It uses the 2026-05-26 Darwin Core List of Terms and the Darwin Core Data Package 1.0 profile. Source fields use table::field notation.

Every persisted column has exactly one status:

• Mapped: one exact Darwin Core or Dublin Core term.
• Composite/measurement: combined with other values or emitted as an assertion or MeasurementOrFact row.
• Relationship: represented by a DwC-DP relationship or agent-role row.
• Dynamic: custom-field configuration decides its exported representation. A value with no exact term is withheld from Darwin Core bundles.
• Unmapped: no exact current standard representation. Tabular output keeps the value under the shown nahpu:<table>.<column> header; Darwin Core bundles omit it, and the NAHPU Data Package carries it.

When a generic JSON or XML conversion finds multiple populated source fields that target the same standard term, it preserves each value under its NAHPU header instead of overwriting data. Structured bundle builders may combine those fields where the combination rule is explicit.

Bundle export contract

A Darwin Core bundle column exists only when it is registered in crates/nahpu_dwc/src/dwc/terms.rs. Each registered entry names the CSV header, the standard term the header stands for, and the bundle profile the column is legal in. A value that resolves to no registered term is withheld from the bundle, reported in the manifest at plan time, and remains available in a NAHPU Data Package. No column is ever published under an invented term IRI.

Only three namespaces may be written:

Namespace	Base IRI
dwc	http://rs.tdwg.org/dwc/terms/
dcterms	http://purl.org/dc/terms/
ac	http://rs.tdwg.org/ac/terms/

The Darwin Core Data Package profile defines no term namespace of its own. Every column in its table schemas versions a Darwin Core, Dublin Core, or Audubon Core term, so dwc-dp never appears as a field IRI.

The two writers publish different shapes, and the registry is keyed by table and profile so a header is only legal where the standard puts it. The Archive is one flat occurrence core with extensions. The Data Package is relational and its Occurrence class is thin: determination ranks belong to identification, location and collecting values belong to event, and catalog and preparation values belong to material. Values with no class column, such as the host association, are written as assertions. _pk and _fk columns are Data Package structural keys that version the identifier term they stand for, and never appear in an Archive.

Filtering happens at two different levels, and at neither is it row-level. A Darwin Core bundle is filtered by term; a NAHPU Data Package is filtered by populated table, so a mammalogy-only project carries no bird, herpetofauna, invertebrate, or fossil resources and no enum mappings for them. A persisted row is exported by both even when only its defaults are set.

The table audit is grouped by data domain:

Category	Tables
Projects and locations	project, geography, site, siteAttribute, fossilSite, coordinate
Collecting events	collEvent, environment, collPersonnel, collEffort
Narratives and linked resources	narrative, media, media link tables, associatedData, associated-data link tables
People and taxonomy	personnelList, personnel, taxonomy
Specimens and attributes	specimen, mammalAttribute, birdAttribute, herpAttribute, invertebrateAttribute, fossilAttribute
Parasites and specimen material	parasiteDetection, parasite, specimenPart
Custom fields	customFieldDefinition, customFieldValue

Projects and locations

project

Fields	Status	Darwin Core or export handling
uuid; name; created; lastAccessed	Mapped	dwc:projectID; dwc:projectTitle; dcterms:created; dcterms:modified
description; principalInvestigator; accession; catalogNumberPrefix; currentCatalogNumber; catalogNumberSuffix; location; timeZone; startDate; endDate	Unmapped	nahpu:project.description; nahpu:project.principalInvestigator; nahpu:project.accession; nahpu:project.catalogNumberPrefix; nahpu:project.currentCatalogNumber; nahpu:project.catalogNumberSuffix; nahpu:project.location; nahpu:project.timeZone; nahpu:project.startDate; nahpu:project.endDate. These are project configuration or lifecycle values without one exact row-level term.

geography

Fields	Status	Darwin Core or export handling
country; islandGroup; stateProvince; county; municipality; locality	Mapped	dwc:country; dwc:islandGroup; dwc:stateProvince; dwc:county; dwc:municipality; dwc:verbatimLocality
id; matchKey	Unmapped	nahpu:geography.id; nahpu:geography.matchKey. Internal identity and normalized deduplication key.

site

Fields	Status	Darwin Core or export handling
siteID; projectUuid; remark	Mapped	dwc:locationID; dwc:datasetID; dwc:locationRemarks
geographyId	Relationship	Resolved to the shared location fields in structured bundles.
id; leadStaffId; siteType; mediaID	Unmapped	nahpu:site.id; nahpu:site.leadStaffId; nahpu:site.siteType; nahpu:site.mediaID. Internal identity and site workflow links are not currently represented by an exact bundle relationship.

siteAttribute

Fields	Status	Darwin Core or export handling
siteID; habitatType; habitatCondition; habitatDescription	Mapped	dwc:locationID; the three habitat fields target dwc:habitat and are joined with `
canopyCover	Composite/measurement	Assertion with type canopy cover; no unit is imposed.

fossilSite

Fields	Status	Darwin Core or export handling
siteID; formation; narrowerGeologicStage; broaderGeologicStage	Mapped	dwc:locationID; dwc:formation; dwc:latestAgeOrHighestStage; dwc:earliestAgeOrLowestStage
geologicEra; geologicPeriod; geologicSeries; geologicEpoch; biozone; rockType; depositionalEnvironmentType; depositionalContinent; depositionalMarine; standardPreservationType; stratigraphyRemark; stratigraphicSource; sedimentologyRemark	Unmapped	nahpu:fossilSite.geologicEra; nahpu:fossilSite.geologicPeriod; nahpu:fossilSite.geologicSeries; nahpu:fossilSite.geologicEpoch; nahpu:fossilSite.biozone; nahpu:fossilSite.rockType; nahpu:fossilSite.depositionalEnvironmentType; nahpu:fossilSite.depositionalContinent; nahpu:fossilSite.depositionalMarine; nahpu:fossilSite.standardPreservationType; nahpu:fossilSite.stratigraphyRemark; nahpu:fossilSite.stratigraphicSource; nahpu:fossilSite.sedimentologyRemark. Enum indices, free-text geology, and sedimentology are not forced into approximate terms.

coordinate

Fields	Status	Darwin Core or export handling
nameId; siteID; decimalLatitude; decimalLongitude; verbatimLatitude; verbatimLongitude; verbatimCoordinates; verbatimCoordinateSystem; datum; uncertaintyInMeters; notes	Mapped	dwc:locationID; dwc:locationID; dwc:decimalLatitude; dwc:decimalLongitude; dwc:verbatimLatitude; dwc:verbatimLongitude; dwc:verbatimCoordinates; dwc:verbatimCoordinateSystem; dwc:geodeticDatum; dwc:coordinateUncertaintyInMeters; dwc:georeferenceRemarks
elevationInMeter	Composite/measurement	Written to both dwc:minimumElevationInMeters and dwc:maximumElevationInMeters.
id; gpsUnit	Unmapped	nahpu:coordinate.id; nahpu:coordinate.gpsUnit. Internal key and device label.

Collecting events

collEvent

Fields	Status	Darwin Core or export handling
id; projectUuid; siteID; startDate; endDate; startTime; endTime; primaryCollMethod; collMethodNotes	Mapped	dwc:eventID; dwc:datasetID; dwc:locationID; date values target dwc:eventDate; time values target dwc:eventTime; dwc:samplingProtocol; dwc:samplingEffort
idSuffix	Unmapped	nahpu:collEvent.idSuffix. NAHPU display suffix rather than a complete event identifier.

environment

Fields	Status	Darwin Core or export handling
eventID; notes	Mapped	dwc:eventID; dwc:eventRemarks
lowestDayTempC; highestDayTempC; lowestNightTempC; highestNightTempC; averageHumidity; dewPointTemp; sunriseTime; sunsetTime; moonPhase; cloudCover; rainfallInMm; ambientTemperature; ambientHumidity; waterTemperature; pH; dissolvedOxygen; flowVelocity	Composite/measurement	Event assertions with explicit measurement types and defined units where available.

collPersonnel

Fields	Status	Darwin Core or export handling
eventID; personnelId; name	Mapped	dwc:eventID; dwc:recordedByID; dwc:recordedBy
role	Relationship	Event agent-role value in DwC-DP.
id	Unmapped	nahpu:collPersonnel.id. Internal relationship key.

collEffort

Fields	Status	Darwin Core or export handling
eventID; method; notes	Mapped	dwc:eventID; dwc:samplingProtocol; dwc:samplingEffort
id; brand; count; size	Unmapped	nahpu:collEffort.id; nahpu:collEffort.brand; nahpu:collEffort.count; nahpu:collEffort.size. Internal identity and equipment details lack exact standalone terms.

Narratives and linked resources

narrative

Fields	Status	Darwin Core or export handling
projectUuid; date; siteID	Mapped	dwc:datasetID; dcterms:date; dwc:locationID
id; time; writerId; narrative; mediaID	Unmapped	nahpu:narrative.id; nahpu:narrative.time; nahpu:narrative.writerId; nahpu:narrative.narrative; nahpu:narrative.mediaID. Narratives are not a DwC-DP resource, and a narrative can belong to a project or site, so its text is not forced into event remarks.

media

Fields	Status	Darwin Core or export handling
primaryId; secondaryId; projectUuid; category; tag; taken; camera; lenses; additionalExif; personnelId; fileName; uri; caption	Mapped	Identifiers use dcterms:identifier; project uses dwc:datasetID; category dcterms:type; tag dcterms:subject; taken dcterms:created; camera/lenses/EXIF/caption dcterms:description; personnel dcterms:creator; file name dcterms:title. Structured bundles preserve media relationships separately.

narrativeMedia

Fields	Status	Darwin Core or export handling
narrativeId; mediaId	Relationship	Narrative-to-media relationship; identifiers use dcterms:identifier in flat output.

siteMedia

Fields	Status	Darwin Core or export handling
siteId; mediaId	Relationship	Location-to-media relationship; dwc:locationID and dcterms:identifier in flat output.

eventMedia

Fields	Status	Darwin Core or export handling
eventID; mediaId	Relationship	Event-to-media relationship; dwc:eventID and dcterms:identifier in flat output.

specimenMedia

Fields	Status	Darwin Core or export handling
specimenUuid; mediaId	Relationship	Occurrence-to-media relationship; dwc:occurrenceID and dcterms:identifier in flat output.

associatedData

Fields	Status	Darwin Core or export handling
primaryId; projectUuid; name; type; date; description; uri	Mapped	dcterms:identifier; dwc:datasetID; dcterms:title; dcterms:type; dcterms:created; dcterms:description; dcterms:identifier

specimenAssociatedData

Fields	Status	Darwin Core or export handling
specimenUuid; associatedDataId	Relationship	Occurrence-to-associated-data relationship; dwc:occurrenceID and dcterms:identifier in flat output.

siteAssociatedData

Fields	Status	Darwin Core or export handling
siteId; associatedDataId	Relationship	Location-to-associated-data relationship; dwc:locationID and dcterms:identifier in flat output.

eventAssociatedData

Fields	Status	Darwin Core or export handling
eventID; associatedDataId	Relationship	Event-to-associated-data relationship; dwc:eventID and dcterms:identifier in flat output.

People and taxonomy

personnelList

Fields	Status	Darwin Core or export handling
projectUuid; personnelUuid	Relationship	Dataset-to-agent relationship; dwc:datasetID and dwc:agentID in flat output.

personnel

Fields	Status	Darwin Core or export handling
uuid; orcid; name; notes	Mapped	dwc:agentID; canonical ORCID also uses dwc:agentID; dwc:preferredAgentName; dwc:agentRemarks
initial; email; phone; affiliation; role; currentFieldNumber; photoPath; isRegisterField	Unmapped	nahpu:personnel.initial; nahpu:personnel.email; nahpu:personnel.phone; nahpu:personnel.affiliation; nahpu:personnel.role; nahpu:personnel.currentFieldNumber; nahpu:personnel.photoPath; nahpu:personnel.isRegisterField. Contact data, project roles, counters, paths, and UI state are not agent type fields.

taxonomy

Fields	Status	Darwin Core or export handling
id; taxonRank; kingdom; phylum; taxonClass; taxonOrder; taxonFamily; genus; specificEpithet; subspecificEpithet; authors; commonName; notes	Mapped	dwc:taxonID; dwc:taxonRank; dwc:kingdom; dwc:phylum; dwc:class; dwc:order; dwc:family; dwc:genus; dwc:specificEpithet; dwc:infraspecificEpithet; dwc:scientificNameAuthorship; dwc:vernacularName; dwc:taxonRemarks
citesStatus; redListCategory; countryStatus; sortingOrder; mediaId	Unmapped	nahpu:taxonomy.citesStatus; nahpu:taxonomy.redListCategory; nahpu:taxonomy.countryStatus; nahpu:taxonomy.sortingOrder; nahpu:taxonomy.mediaId. Conservation classifications require vocabularies and provenance; sorting and the local media link are not currently represented in the bundle.

Specimens and attributes

specimen

Fields	Status	Darwin Core or export handling
uuid; projectUuid; speciesID; iDConfidence; iDMethod; taxonGroup; collectionDate; captureDate; collectionTime; captureTime; trapType; methodID; coordinateID; fieldNumber; collEventID; collPersonnelID; collMethodID; determinerID	Mapped	dwc:occurrenceID; dwc:datasetID; dwc:taxonID; dwc:identificationVerificationStatus; dwc:identificationType; dwc:higherClassification; dates use dwc:eventDate; times use dwc:eventTime; collection methods use dwc:samplingProtocol; coordinate dwc:locationID; field number dwc:recordNumber; event dwc:eventID; collector dwc:recordedByID; determiner dwc:identifiedByID.
condition; coordinateExtentMeters; projectFieldNumber	Composite/measurement	Condition derives dwc:basisOfRecord; positive coordinate extent contributes to dwc:coordinateUncertaintyInMeters; project field number participates in the bundle catalog number.
catalogerID; preparatorID	Relationship	Occurrence agent-role relationships.
prepDate; prepTime; isRelativeTime; relativeCaptureTime; isMultipleCollector; museumID	Unmapped	nahpu:specimen.prepDate; nahpu:specimen.prepTime; nahpu:specimen.isRelativeTime; nahpu:specimen.relativeCaptureTime; nahpu:specimen.isMultipleCollector; nahpu:specimen.museumID. Preparation workflow, relative-time state, UI flags, and unqualified museum identifiers remain NAHPU fields.

mammalAttribute

Fields	Status	Darwin Core or export handling
specimenUuid; sex; lifeStage; reproductiveStage; remark	Mapped	dwc:occurrenceID; dwc:sex; dwc:lifeStage; dwc:reproductiveCondition; dwc:occurrenceRemarks
totalLength; tailLength; hindFootLength; earLength; forearm; tibia; echolocation; frequencyMax; frequencyMin; frequencyAtMaxEnergy; duration; weight; weightUnit; testisPosition; testisLength; testisWidth; epididymisAppearance; leftPlacentalScars; rightPlacentalScars; mammaeCondition; mammaeInguinalCount; mammaeAxillaryCount; mammaeAbdominalCount; vaginaOpening; pubicSymphysis; embryoLeftCount; embryoRightCount; embryoCR	Composite/measurement	MeasurementOrFact rows with field-specific types and units; weightUnit supplies the row-level weight unit.
showBatFields; showEchoFields; accuracy; accuracySpecify	Unmapped	nahpu:mammalAttribute.showBatFields; nahpu:mammalAttribute.showEchoFields; nahpu:mammalAttribute.accuracy; nahpu:mammalAttribute.accuracySpecify. UI visibility and NAHPU measurement-quality state.

birdAttribute

Fields	Status	Darwin Core or export handling
specimenUuid; sex; lifeStage; specimenRemark; habitatRemark	Mapped	dwc:occurrenceID; dwc:sex; dwc:lifeStage; dwc:occurrenceRemarks; dwc:habitat
weight; weightUnit; wingspan; irisColor; irisHex; billColor; billHex; maxillaColor; maxillaHex; mandibleColor; mandibleHex; toeColor; toeHex; tarsusColor; tarsusHex; broodPatch; skullOssification; hasBursa; bursaWidth; bursaLength; fat; stomachContent; testisLength; testisWidth; testisRemark; ovaryLength; ovaryWidth; oviductWidth; ovaryAppearance; firstOvaSize; secondOvaSize; thirdOvaSize; oviductAppearance; ovaryRemark; wingIsMolt; wingMolt; tailIsMolt; tailMolt; bodyMolt; moltRemark	Composite/measurement	MeasurementOrFact rows with field-specific types and units; weightUnit supplies the row-level weight unit.

herpAttribute

Fields	Status	Darwin Core or export handling
specimenUuid; sex; lifeStage; remark	Mapped	dwc:occurrenceID; dwc:sex; dwc:lifeStage; dwc:occurrenceRemarks
weight; weightUnit; svl	Composite/measurement	Weight and snout-vent-length MeasurementOrFact rows; weightUnit supplies the row-level weight unit.

invertebrateAttribute

Fields	Status	Darwin Core or export handling
specimenUuid; sex; lifeStage; caste; hostOrganism; remark	Mapped	dwc:occurrenceID; dwc:sex; dwc:lifeStage; dwc:caste; dwc:associatedTaxa; dwc:occurrenceRemarks
headWidth; bodyLength; wingspanUpper; wingspanLower; hostPart	Composite/measurement	MeasurementOrFact rows with field-specific types and units.

fossilAttribute

Fields	Status	Darwin Core or export handling
specimenUuid; fossilType; sex; ontogeneticStage; specimenDescription; remark	Mapped	dwc:occurrenceID; dwc:materialEntityType; dwc:sex; dwc:lifeStage; descriptions and remarks use dwc:materialEntityRemarks.
weight; weightUnit	Composite/measurement	Weight MeasurementOrFact row with its selected unit.

Parasites and specimen material

parasiteDetection

Fields	Status	Darwin Core or export handling
specimenUuid; detectionRemark	Mapped	dwc:occurrenceID; dwc:occurrenceRemarks
parasiteExamined; parasiteDetected	Composite/measurement	MeasurementOrFact rows describing examination and detection.

parasite

Fields	Status	Darwin Core or export handling
speciesID; identifierID; parasiteID; parasiteUuid; count; preparationMethod; treatment; lifeStage; detectionMethod; dateCollected; timeCollected; remark	Mapped	dwc:taxonID; dwc:identifiedByID; dwc:catalogNumber; dwc:occurrenceID; dwc:individualCount; preparation values use dwc:preparations; dwc:lifeStage; dwc:samplingProtocol; dwc:eventDate; dwc:eventTime; dwc:occurrenceRemarks
specimenUuid; anatomicalLocation; category; associationStatus	Relationship	Host occurrence and organism-interaction fields; structured bundles emit subject/related occurrence links, interaction type, related organism part, and description.
id; storage; storageLocation; datePreserved; timePreserved; museumPermanent; museumLoan	Unmapped	nahpu:parasite.id; nahpu:parasite.storage; nahpu:parasite.storageLocation; nahpu:parasite.datePreserved; nahpu:parasite.timePreserved; nahpu:parasite.museumPermanent; nahpu:parasite.museumLoan. Internal identity and preservation, storage, and museum workflow values need a material-record model before standard mapping.

specimenPart

Fields	Status	Darwin Core or export handling
specimenUuid; personnelId	Relationship	Material-to-occurrence and preparator agent-role relationships. Flat output exposes the occurrence identifier for specimenUuid.
tissueID; barcodeID; count; treatment; additionalTreatment; dateTaken; timeTaken; remark	Mapped	dwc:materialSampleID; dwc:otherCatalogNumbers; dwc:objectQuantity; preparation values use dwc:preparations; dwc:eventDate; dwc:eventTime; dwc:materialEntityRemarks
type	Composite/measurement	dwc:materialEntityType and dwc:objectQuantityType
id; storage; storageLocation; pmi; museumPermanent; museumLoan	Unmapped	nahpu:specimenPart.id; nahpu:specimenPart.storage; nahpu:specimenPart.storageLocation; nahpu:specimenPart.pmi; nahpu:specimenPart.museumPermanent; nahpu:specimenPart.museumLoan. Internal, storage, PMI, and museum workflow values have no exact current representation.

Custom fields

customFieldDefinition

Fields	Status	Darwin Core or export handling
id; uuid; sourceTemplateUuid; name; type; uiSection; options; scope; projectUuid; catalogFormat; sortOrder; isArchived; dwcTarget; dwcField; dwcMode; allowDwcConflict; createdAt; updatedAt	Dynamic	Configuration determines whether a value targets an exact registered term or an assertion. A value with no Darwin Core term is withheld from Darwin Core bundles and remains in the NAHPU Data Package. Definition metadata is not itself statically mapped.

customFieldValue

Fields	Status	Darwin Core or export handling
id; fieldDefinitionId; projectUuid; value; unit; eventId; siteId; specimenUuid; specimenPartId; parasiteId; isLegacy	Dynamic	The referenced definition and owner select the target resource and representation.

Legacy source aliases

The mapper retains event:: for collEvent::, weather:: for environment::, and the historical mammalMeasurement::, avianMeasurement::, and herpMeasurement:: namespaces. Site geography keys remain aliases for records created before v21, while current exports use geography::. associatedData::url remains an alias for the current uri column.

Unmapped-field follow-ups

These fields remain Unmapped until their persisted semantics support an exact representation. They are absent from both Darwin Core outputs and are carried only by the NAHPU Data Package:

• Convert fossil era, period, series, epoch, and biozone values from stored enum indices into controlled labels before creating GeologicalContext rows.
• Define controlled vocabularies and provenance for rock, depositional, preservation, stratigraphy, sedimentology, and conservation-status fields.
• Give narratives an explicit resource owner before choosing dwc:fieldNotes or a resource-specific remarks field.
• Model specimen-part PMI with a defined measurement type and unit, and model preparation or preservation dates as explicit events when appropriate.
• Define whether museum and storage identifiers describe material entities, collections, loans, or installation-local workflow before mapping them.
• Resolve the product distinction between specimen fieldNumber, derived project field numbers, dwc:recordNumber, and dwc:catalogNumber before changing the established bundle contract.
• Decide whether NAHPU ever publishes a namespaced Data Package extension for its own terms, or stays strictly standard-only.
• Re-verify every registered namespace whenever TDWG republishes the term list. Ratification can move a term between namespaces.
• Migrate the tabular mapper's dwc:preferredAgentName prefix to dcterms:title. The term is published in Dublin Core, and the bundle writers already resolve it correctly; the flat exports keep the legacy prefix because renaming it changes columns in saved user presets.

The mapping is maintained with nahpu_dwc. A schema, mapper, or bundle change must update this audit, the term registry in dwc/terms.rs, and the schema-classification tests in the same change.

Missing improvements and follow-up work

• Add the redb metadata table and ordered migrations described above. The current crate has no internal redb migration path.
• Add uniqueness constraints where the data model requires one. Several child tables rely on application checks rather than database uniqueness.
• Add a documentation check that compares the table inventory and schema version here with generated Drift artifacts. This would catch stale table names such as paleontologySite.
• Reconcile export-bundle table counts with the actual v21 inventory and keep backup and restore documentation tested against a real archive.
• Add restore verification that opens a restored SQLite file, runs foreign-key checks, and records the schema version before the file is offered to users.