Add LegiScan legislation ingestion and analysis queries

Adds ingest_legiscan.py to pull all US state + federal bills (2016-2026)
from the LegiScan API into legiscan_sessions and legiscan_bills tables.
Bills are keyword-tagged across 8 research categories (data_center,
ratepayer_protection, large_load, grid_impact, tax_incentive, etc.).
Loads ~1.3M bills; ~60K tagged relevant. Adds query_legiscan_bills.sql
with pre-built analysis queries including state/DC joins. Updates
database-tables.md, README.md, and research-ideas.md accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-05-27 21:30:31 -07:00
parent 46c8c58545
commit 4525ea3f97
5 changed files with 1046 additions and 1 deletions

View File

@@ -87,6 +87,7 @@ Facilities in DBSCAN clusters differ significantly from isolated sites:
- **NOAA HMS Smoke Data**: Wildfire smoke exposure (2005-2025)
- **USDM Drought Data**: Drought severity
- **Utility Rate Tracker**: State-level electricity rate increases
- **LegiScan Legislative Data**: All US state + federal bills 20162026 (1.3M bills, 646 sessions), tagged for data center, ratepayer, grid, tax, and siting topics
## Repository Structure
@@ -99,6 +100,7 @@ Facilities in DBSCAN clusters differ significantly from isolated sites:
- `load_postgis_internet_cables.py` - Load submarine cables and landing points
- `ingest_eia_energy_layers.py` - Ingest EIA energy data via API
- `build_watershed_huc8_tables.py` - Load USGS HUC8 watersheds
- `ingest_legiscan.py` - Download all US state/federal bills 20162026 via LegiScan API, tag for data center research topics
**Enrichment**
- `create_data_center_census_tract_table.py` - Join data centers to Census tracts with ACS demographics
@@ -151,6 +153,7 @@ Credentials stored in `~/.zsh_secrets`, loaded via environment variables:
- `FCC_USERNAME`, `FCC_API_KEY`: FCC broadband data
- `RDH_USERNAME`, `RDH_PASSWORD`: Redistricting Data Hub
- `CENSUS_API_KEY`: Census ACS API
- `LEGISCAN_API_KEY`: LegiScan legislative data
## Quick Start
@@ -174,6 +177,11 @@ python3 analyze_cables_concentration.py > output/cables_analysis.txt
# 4. Execute notebooks
jupyter notebook cluster_analysis.ipynb
# 5. Load legislation (all states, 2016-2026)
python3 ingest_legiscan.py --all
# Weekly refresh (skips unchanged sessions):
python3 ingest_legiscan.py --fetch --load
```
### Generate Maps