Methods

Understanding data provenance is essential in conducting high quality research.

This page describes how data from DESE’s public-facing portal are collected, compiled, and validated.

Source

All data come from the Missouri Department of Elementary and Secondary Education’s public ASBR portal (apps.dese.mo.gov), which publishes each district’s Annual Secretary of the Board Report and related financial reports via a self-serve platform that loads one district-year at a time.

Collection

An automated pipeline (R + headless Chrome browser) retrieves reports through the same public interface a person would use:

  1. Discovery — enumerate the portal’s own district list and, for each district, the years it actually offers. Each discovery run is saved as a dated snapshot, since the set of districts changes (e.g., a charter opens or closes).
  2. Capture — for each district-year, select reports in the portal and export the official report rendering to CSV (the same export a person gets from the report viewer’s CSV button). Requests are paced to limit load on servers.
  3. Parsing — each report export contains multiple stacked tables; these are located by their exact header signatures and parsed into tidy tables. Dollar strings ("$1,234", parenthesized negatives) become numeric; identifiers keep their zero-padding.

Validation

  • Every captured ASBR’s total assessed valuation is cross-checked against DESE’s separately published assessed-valuation workbooks; any mismatch in year or amount invalidates the capture and forces a re-scrape and manual review to ensure the correct report was captured.
  • Completeness is audited against DESE’s district directory to identify district-years that should exist but were not captured.
  • Release manifests record row counts, MD5 checksums, and a changelog documenting any alterations to the scraping process.

Versioning

The working database is rebuilt as audited ASBR reports are made available. What researchers receive is different: releases are dated snapshots in parquet format with a manifest (MD5 checksums, row counts, source-code version, report version, and build provenance). Data in releases are never modified after they are published, so any result computed from a release can be reproduced from the exact same data.

Using the data

For what these data support well — and the things to keep in mind while analyzing them, like how DESE’s chart of accounts evolves over time — see Use Cases.