publicdata/j40-cejst-2
1
0
Fork 0
mirror of https://github.com/DOI-DO/j40-cejst-2.git synced 2025-02-22 17:44:20 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions
j40-cejst-2/data/data-pipeline/pyproject.toml

115 lines
3.2 KiB
TOML
Raw Normal View History

Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
[tool.poetry]
Adding notebook to create score dissolve (#333)
2021-07-21 16:10:32 -04:00
authors = ["Your Name <you@example.com>"]
description = "ETL and Generation of Justice 40 Score"
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
name = "data-pipeline"
Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
version = "0.1.0"
[tool.poetry.dependencies]
Data folder restructuring in preparation for 361 (#376) * initial checkin * gitignore and docker-compose update * readme update and error on hud * encoding issue * one more small README change * data roadmap re-strcuture * pyproject sort * small update to score output folders * checkpoint * couple of last fixes
2021-07-20 14:55:39 -04:00
CensusData = "^1.13"
click = "^8.0.1"
dynaconf = "^3.1.4"
Adding notebook to create score dissolve (#333)
2021-07-21 16:10:32 -04:00
geopandas = "^0.9.0"
Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
ipython = "^7.24.1"
jupyter = "^1.0.0"
jupyter-contrib-nbextensions = "^0.5.1"
numpy = "^1.21.0"
pandas = "^1.2.5"
Adding notebook to create score dissolve (#333)
2021-07-21 16:10:32 -04:00
python = "^3.7.1"
wip
2021-08-09 22:24:14 -05:00
pypandoc = "^1.6.3"
Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
requests = "^2.25.1"
wip
2021-08-09 22:24:14 -05:00
tqdm = "4.62.0"
Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
types-requests = "^2.25.0"
wip
2021-08-09 22:24:14 -05:00
us = "^2.0.2"
xlsxwriter = "^2.0.0"
Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
[tool.poetry.dev-dependencies]
black = {version = "^21.6b0", allow-prereleases = true}
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
flake8 = "^3.9.2"
liccheck = "^0.6.2"
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
mypy = "^0.910"
Backend change for Zipfile pt. 2 (#469) * Fixes #303 : adding downloadable zip archive logic * linter recommendations * Pushes data directory to AWS. We'll want to move to use AWS for this ASAP, but this works for now * updating pattern
2021-08-09 10:39:59 -04:00
openpyxl = "^3.0.7"
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
pylint = "^2.9.6"
pytest = "^6.2.4"
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
safety = "^1.10.3"
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
tox = "^3.24.0"
Modularization + Poetry + Docker (#213) * reorg * added configuration management; initial click cmds * reset dirs completed * major modularization effort * prepping mbtiles * first round of PR review updates * round 2 of feedback review * checkpoint * habemus dockerfile 🎉 * updated dock-er-compose with long running container * census generation works * logging working * updated README * updated README * last small update to README * added instructions for log visualization * census etl update for reusable fips module * ejscreem etl updated * further modularization * score modularization * tmp cleanup
2021-06-28 16:16:14 -04:00
[build-system]
build-backend = "poetry.core.masonry.api"
Adding notebook to create score dissolve (#333)
2021-07-21 16:10:32 -04:00
requires = ["poetry-core>=1.0.0"]
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
[tool.pylint]
[tool.pylint."MESSAGE CONTROL"]
disable = [
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
"C0114", # Disables module docstrings
"R0201", # Disables method could have been a function
"R0903", # Disables too few public methods
"C0103", # Disables name case styling
"W0511", # Disables FIXME warning
"W1203", # Disables f-string interpolation for logging warning # Errors temporarily ignored for further discussion
"W0107", # Disables unnecessary pass
"W0221", # Disables arguments differ
"R0902", # Disables too many instance attributes
"R0914", # Disables too many local variables
"W0621", # Disables redefined outer name
"C0302", # Disables too many lines in module
"R1732", # Disables consider using "with"
"R1720", # Disables unnecessary "else" after "raise"
"C0206", # Disables consider iteratig with ".items()"
"C0200", # Disables consider using "enumerate" instead of "range" + "len"
"W0612", # Disables unused variable
"W0613", # Disables unused argument
"C0116", # Disables missing function or method docstring
"C0115", # Disables missing class docstring
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
]
[tool.pylint.FORMAT]
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
max-line-length = 150
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
[tool.pylint.SIMILARITIES]
# Configures how pylint detects repetitive code
ignore-comments = "yes"
ignore-docstrings = "yes"
ignore-imports = "yes"
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
min-similarity-lines = 4
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
[tool.liccheck]
# Authorized and unauthorized licenses in LOWER CASE
authorized_licenses = [
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
"bsd",
"new bsd",
"bsd license",
"bsd 3-clause",
"new bsd license",
"simplified bsd",
"apache",
"apache 2.0",
"apache license 2.0",
"apache software license",
"apache software",
"gnu lgpl",
"gnu lesser general public license v2 (lgplv2)",
"gnu general public license v2 (gplv2)",
"gnu library or lesser general public license (lgpl)",
"lgpl with exceptions or zpl",
"isc license",
"isc license (iscl)",
"mit",
"mit license",
"mozilla public license 2.0 (mpl 2.0)",
"public domain",
"python software foundation license",
"python software foundation",
"zpl 2.1",
"gpl v3",
Issue 308 python linting (#443) * Adds flake8, pylint, liccheck, flake8 to dependencies for data-pipeline * Sets up and runs black autoformatting * Adds flake8 to tox linting * Fixes flake8 error F541 f string missing placeholders * Fixes flake8 E501 line too long * Fixes flake8 F401 imported but not used * Adds pylint to tox and disables the following pylint errors: - C0114: module docstrings - R0201: method could have been a function - R0903: too few public methods - C0103: name case styling - W0511: fix me - W1203: f-string interpolation in logging * Adds utils.py to tox.ini linting, runs black on utils.py * Fixes import related pylint errors: C0411 and C0412 * Fixes or ignores remaining pylint errors (for discussion later) * Adds safety and liccheck to tox.ini
2021-08-02 12:16:38 -04:00
]
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
[tool.poetry.scripts]
Backend change for Zipfile pt. 2 (#469) * Fixes #303 : adding downloadable zip archive logic * linter recommendations * Pushes data directory to AWS. We'll want to move to use AWS for this ASAP, but this works for now * updating pattern
2021-08-09 10:39:59 -04:00
cleanup_census = 'data_pipeline.application:census_cleanup'
Data directory should adopt standard Poetry-suggested python package structure (#457) * Fixes #456 - Our data directory should adopt standard python package structure * a few missed references * updating readme * updating requirements * Running Black * Fixes for flake8 * updating pylint
2021-08-05 15:35:54 -04:00
cleanup_data = 'data_pipeline.application:data_cleanup'
download_census = 'data_pipeline.application:census_data_download'
etl = 'data_pipeline.application:etl_run'
generate_tiles = 'data_pipeline.application:generate_map_tiles'
score = 'data_pipeline.application:score_run'
Poetry updates for application (#483)
2021-08-06 16:24:30 -04:00
etl_and_score = 'data_pipeline.application:score_full_run'
Reference in a new issue Copy permalink