Lewati ke konten

Tutorial 2: Convert a NAHPU Data Package to the LSUMZ Specify sheet

Konten ini belum tersedia dalam bahasa Anda.

Tutorial 1 builds the LSUMZ Mammalogy WorkBench sheet inside NAHPU with an export preset. This tutorial produces the same sheet outside the app: you export a NAHPU Data Package and convert it with a ready-made script in Python or R.

Use this route when:

  • the data needs checking or cleaning in a script before it goes to Specify;
  • someone without NAHPU on their device prepares the upload;
  • you want a reproducible, versioned record of how the sheet was made.

Both scripts live in the lsumz-mammal-specify-frictionless repository. They read the package with the Frictionless library for each language and write identical CSV files.

You need:

  • a NAHPU project with mammal specimen records;
  • uv, Python 3.10 or newer with pip, or R with permission to install packages;
  • a copy of the repository, cloned with Git or downloaded as a ZIP from GitHub.

Set up your computer covers each of these.

The repository is organized like this:

  • Directorydata/
    • country-continent.csv
    • Directoryexample/
      • ambang-shrews.nahpu-dp.zip
      • expected-specify.csv
  • Directorypython/
    • nahpu_to_specify.py
    • Directorytests/
      • …
  • DirectoryR/
    • nahpu_to_specify.R
    • Directorytests/
      • …
  • Directoryresults/
    • …

Do this once per computer. The commands run in a terminal: Terminal on macOS, PowerShell on Windows, or your distribution's terminal on Linux.

With Git:

Terminal window
git clone https://github.com/nahpu/lsumz-mammal-specify-frictionless.git
cd lsumz-mammal-specify-frictionless

Without Git, open the repository on GitHub, select Code, then Download ZIP, extract the archive, and open a terminal in the extracted folder.

Choose uv or pip for Python, or install R. uv is the simpler route: it installs a suitable Python version and the script's packages for you.

  1. Install uv.

    On macOS or Linux:

    Terminal window
    curl -LsSf https://astral.sh/uv/install.sh | sh

    On Windows, in PowerShell:

    Terminal window
    powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

    With a package manager, use brew install uv on macOS or winget install --id=astral-sh.uv -e on Windows.

  2. Close and reopen the terminal, then check the installation:

    Terminal window
    uv --version
  3. From the repository folder, install the script's dependencies:

    Terminal window
    cd python
    uv sync

    uv downloads Python 3.10 or newer if your computer does not have it and creates a virtual environment in python/.venv. Prefix Python commands with uv run to use it; you do not need to activate it.

  1. Open the project menu and select Bundle records.
  2. Set Bundle format to NAHPU Data Package.
  3. Keep TAR.GZ or choose ZIP as the Archive format. The scripts read both.
  4. Name the file, choose a folder, and select Create bundle.
  5. Copy the file, such as ambang.nahpu-dp.tar.gz, into the repository's data/ folder.

Run a converter once without arguments. It converts data/example/ambang-shrews.nahpu-dp.zip and writes results/ambang-shrews-specify.csv, which should match data/example/expected-specify.csv, five rows of the real LSUMZ incoming sheet.

With uv, from the python/ folder:

Terminal window
uv run python nahpu_to_specify.py

With pip, from the repository folder with .venv activated:

Terminal window
python python/nahpu_to_specify.py

If the result matches the expected file, your setup works.

Pass your package as the first argument. The sheet is written to results/ under the package name.

With uv, from the python/ folder:

Terminal window
uv run python nahpu_to_specify.py ../data/ambang.nahpu-dp.tar.gz

With pip, from the repository folder with .venv activated:

Terminal window
python python/nahpu_to_specify.py data/ambang.nahpu-dp.tar.gz

Add -o PATH to choose the output file, and --all-taxa to keep specimens outside the mammal taxon groups.

The scripts also accept an extracted package folder or its datapackage.json.

Open the CSV and compare a few specimens with their NAHPU records:

  • Sex reads Male or Female. The scripts decode the stored code with the package's mappings/sqlite_enums.csv.
  • Continent is filled from Country. A country missing from data/country-continent.csv leaves it blank and prints a warning; add the country to that file and run again.
  • Collector1LastName reads Last, First. The last word of the cataloger's name is moved to the front, so check two-word family names.
  • Start Date is the specimen's collection date as dd/mm/yyyy.
  • Each part fills one PrepType group, with its storage location, such as a freezer box position, in PrepTypenLocation. Remarks follows the second group, as on the LSUMZ sheet.
  • A measurement in brackets, such as [150], means its accuracy note marks it as unreliable, the same rule the LSUMZ-mammalogy preset from Tutorial 1 uses.

Import the CSV as a new WorkBench data set, apply the LSUMZ Mammalogy upload plan, validate, and upload, as in Tutorial 1. The columns are the same, so the same upload plan works for sheets from either route.

Each script follows the package's relational tables:

  1. It keeps specimens whose taxonGroup is Mammals, General Mammals, or Bats.
  2. It joins taxonomy through speciesID, mammalAttribute through the specimen UUID, and the cataloger in personnel.
  3. It finds the locality through the collecting event's site and its geography row.
  4. It uses the specimen's own coordinate, or the site's coordinate when the site has exactly one, as NAHPU does.
  5. It spreads specimenPart rows into numbered PrepType columns in part order.

Empty NAHPU tables are left out of a Data Package, so the scripts treat a missing table as empty rather than failing. To adapt the scripts to another collection's sheet, change the column lists at the top of either file. For a different layout altogether, write your own script as in Tutorial 3.