j40-cejst-2/data/data-pipeline/data_pipeline/ipython
Matt Bowen d5fbb802e8
Add FUDS ETL (#1817)
* Add spatial join method (#1871)

Since we'll need to figure out the tracts for a large number of points
in future tickets, add a utility to handle grabbing the tract geometries
and adding tract data to a point dataset.

* Add FUDS, also jupyter lab (#1871)

* Add YAML configs for FUDS (#1871)

* Allow input geoid to be optional (#1871)

* Add FUDS ETL, tests, test-datae noteobook (#1871)

This adds the ETL class for Formerly Used Defense Sites (FUDS). This is
different from most other ETLs since these FUDS are not provided by
tract, but instead by geographic point, so we need to assign FUDS to
tracts and then do calculations from there.

* Floats -> Ints, as I intended (#1871)

* Floats -> Ints, as I intended (#1871)

* Formatting fixes (#1871)

* Add test false positive GEOIDs (#1871)

* Add gdal binaries (#1871)

* Refactor pandas code to be more idiomatic (#1871)

Per Emma, the more pandas-y way of doing my counts is using np.where to
add the values i need, then groupby and size. It is definitely more
compact, and also I think more correct!

* Update configs per Emma suggestions (#1871)

* Type fixed! (#1871)

* Remove spurious import from vscode (#1871)

* Snapshot update after changing col name (#1871)

* Move up GDAL (#1871)

* Adjust geojson strategy (#1871)

* Try running census separately first (#1871)

* Fix import order (#1871)

* Cleanup cache strategy (#1871)

* Download census data from S3 instead of re-calculating (#1871)

* Clarify pandas code per Emma (#1871)
2022-08-16 13:28:39 -04:00
..
experiment_4_weighting CDC SVI Index: Additions to data-pipeline and comparison tool (#1096) 2022-01-14 14:52:37 -05:00
home_foreclosures_analysis_nhpd Issue 1075: Add refactored ETL tests to NRI (#1088) 2022-02-08 19:05:32 -05:00
ACS Validate.ipynb Adding persistent poverty tracts (#738) 2021-09-22 17:57:08 -04:00
agricultural_loss_indicator.ipynb FEMA data check (#1270) 2022-02-17 16:53:04 -05:00
census_explore.ipynb Issue 844: Add island areas to Definition L (#957) 2021-12-03 15:46:10 -05:00
compare_two_score_files_for_differences.ipynb Issue 1044: Add low HS education fields to tiles and download (#1046) 2021-12-14 15:41:06 -05:00
county_lookup.ipynb Adding persistent poverty tracts (#738) 2021-09-22 17:57:08 -04:00
EJScreen Validate.ipynb Adding persistent poverty tracts (#738) 2021-09-22 17:57:08 -04:00
ejscreen_load.ipynb Add EJSCREEN Areas of Concern (#843) 2021-11-02 15:38:42 -04:00
generate_fuds_test_data.ipynb Add FUDS ETL (#1817) 2022-08-16 13:28:39 -04:00
Score Validate.ipynb Adding persistent poverty tracts (#738) 2021-09-22 17:57:08 -04:00
Score_Dissolve_Script.ipynb Adding persistent poverty tracts (#738) 2021-09-22 17:57:08 -04:00
score_explore.ipynb Adding persistent poverty tracts (#738) 2021-09-22 17:57:08 -04:00
scoring_comparison.ipynb FEMA data check (#1270) 2022-02-17 16:53:04 -05:00
TractArea.ipynb Fix on large AK tracts that are off screen (#1740) 2022-07-13 19:01:43 -07:00
urban_vs_rural.ipynb [ISS-751] Updating comments for geocorr ETL (#913) 2021-12-03 10:10:05 -05:00