Add LegiScan legislation ingestion and analysis queries
Adds ingest_legiscan.py to pull all US state + federal bills (2016-2026) from the LegiScan API into legiscan_sessions and legiscan_bills tables. Bills are keyword-tagged across 8 research categories (data_center, ratepayer_protection, large_load, grid_impact, tax_incentive, etc.). Loads ~1.3M bills; ~60K tagged relevant. Adds query_legiscan_bills.sql with pre-built analysis queries including state/DC joins. Updates database-tables.md, README.md, and research-ideas.md accordingly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -87,6 +87,7 @@ Facilities in DBSCAN clusters differ significantly from isolated sites:
|
||||
- **NOAA HMS Smoke Data**: Wildfire smoke exposure (2005-2025)
|
||||
- **USDM Drought Data**: Drought severity
|
||||
- **Utility Rate Tracker**: State-level electricity rate increases
|
||||
- **LegiScan Legislative Data**: All US state + federal bills 2016–2026 (1.3M bills, 646 sessions), tagged for data center, ratepayer, grid, tax, and siting topics
|
||||
|
||||
## Repository Structure
|
||||
|
||||
@@ -99,6 +100,7 @@ Facilities in DBSCAN clusters differ significantly from isolated sites:
|
||||
- `load_postgis_internet_cables.py` - Load submarine cables and landing points
|
||||
- `ingest_eia_energy_layers.py` - Ingest EIA energy data via API
|
||||
- `build_watershed_huc8_tables.py` - Load USGS HUC8 watersheds
|
||||
- `ingest_legiscan.py` - Download all US state/federal bills 2016–2026 via LegiScan API, tag for data center research topics
|
||||
|
||||
**Enrichment**
|
||||
- `create_data_center_census_tract_table.py` - Join data centers to Census tracts with ACS demographics
|
||||
@@ -151,6 +153,7 @@ Credentials stored in `~/.zsh_secrets`, loaded via environment variables:
|
||||
- `FCC_USERNAME`, `FCC_API_KEY`: FCC broadband data
|
||||
- `RDH_USERNAME`, `RDH_PASSWORD`: Redistricting Data Hub
|
||||
- `CENSUS_API_KEY`: Census ACS API
|
||||
- `LEGISCAN_API_KEY`: LegiScan legislative data
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -174,6 +177,11 @@ python3 analyze_cables_concentration.py > output/cables_analysis.txt
|
||||
|
||||
# 4. Execute notebooks
|
||||
jupyter notebook cluster_analysis.ipynb
|
||||
|
||||
# 5. Load legislation (all states, 2016-2026)
|
||||
python3 ingest_legiscan.py --all
|
||||
# Weekly refresh (skips unchanged sessions):
|
||||
python3 ingest_legiscan.py --fetch --load
|
||||
```
|
||||
|
||||
### Generate Maps
|
||||
|
||||
Reference in New Issue
Block a user