diff --git a/DATASETS.md b/DATASETS.md new file mode 100644 index 00000000..9486ee7e --- /dev/null +++ b/DATASETS.md @@ -0,0 +1,8 @@ +# Justice40 Datasets + +This page contains web links to the datasets that are uploaded as part of our data pipeline, if you want to access them directly. + +For now, we have two datasets: + +- Census data, generated by the [Generate Census Github Action](https://github.com/usds/justice40-tool/blob/main/.github/workflows/generate-census.yml): https://justice40-data.s3.us-east-1.amazonaws.com/data-sources/census.zip +- Score data, generated by the [Generate Score Github Action](https://github.com/usds/justice40-tool/blob/main/.github/workflows/generate-score.yml): https://justice40-data.s3.us-east-1.amazonaws.com/data-pipeline/data/score/csv/full/usa.csv diff --git a/INSTALLATION.md b/INSTALLATION.md index 9b19fea5..791a7aac 100644 --- a/INSTALLATION.md +++ b/INSTALLATION.md @@ -1,11 +1,47 @@ -# Installing the Justice40 Tool +# Installing Required Software for the Justice40 Tool *[¡Lea esto en español!](INSTALLATION-es.md)* -## Required Software -You'll need the following set up in your environment before continuing to run the Justice40 Tool code. +This page documents the installation steps for some of the software used to work with this project, but you may not need all of it depending on what you are trying to do. -- git, node, and an IDE +If you want to quickly run everything locally, see [`QUICKSTART.md`](QUICKSTART.md). + +If you are working with the frontend, see [`client/README.md`](client/README.md) for installation steps specific to that part of the project. + +If you are working with the data pipeline, see [`data/data-pipeline/README.md`](data/data-pipeline/README.md) for installation steps specific to that part of the project. + +## Install Homebrew (MacOS only) + +Homebrew is an easy way to manage software downloads on MacOS. You don't *have* to use it, but we recommend it. + +1. First, open your terminal and run `brew -v` to determine whether you have Homebrew installed. If you get a resopnse that looks something like `Homebrew 3.1.9`, you've already got it! If you get nothing back, or an error, continue. +2. Open the terminal and copy / paste this command and hit RETURN. Go through the prompts (you will need to grant `sudo` access). + +`/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"` + +2. Validate installation by typing `brew -v` in the terminal and ensure a version number is shown. + +You should regularly run `brew update` and `brew doctor` to make sure your packages are up to date and in good condition. + +## IDE set up +While any IDE can be used, we're outlining how to set up VS Code + +1. Open the terminal and type `brew install --cask visual-studio-code` and hit RETURN. + 1. If this doesn't work or for Win10, you can download VS Code from the [website](https://code.visualstudio.com/). +2. After [forking this repo](https://github.com/usds/justice40-tool/blob/main/CONTRIBUTING.md#code-contributions), you can clone your forked repo into VS Code +3. VS Code with prompt you to "clone a repo" +4. Open the terminal and navigate to `client` directory +5. Type `npm install` to load dependencies +6. Type `gatsby develop` to spin up the app +7. Navigate to `localhost:8000` to view the app + +### Recommended VS Code Extensions + +1. [Browser Preview](https://github.com/auchenberg/vscode-browser-preview) +2. [Live Server](https://github.com/ritwickdey/vscode-live-server) +3. [Live Share](https://github.com/MicrosoftDocs/live-share) +4. [Live Share Audio](https://github.com/MicrosoftDocs/live-share) +5. [Live Share Extention Pack](https://github.com/MicrosoftDocs/live-share) ### Install Git @@ -18,67 +54,7 @@ You'll need the following set up in your environment before continuing to run th #### Win10 Download from [website](https://git-scm.com/download/win) +## Install Docker -### Install Homebrew (MacOS only) -Homebrew is an easy way to manage software downloads on MacOS. You don't *have* to use it, but we recommend it. - -1. First, open your terminal and run `brew -v` to determine whether you have Homebrew installed. If you get a resopnse that looks something like `Homebrew 3.1.9`, you've already got it! If you get nothing back, or an error, continue. -2. Open the terminal and copy / paste this command and hit RETURN. Go through the prompts (you will need to grant `sudo` access). - -`/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"` - -2. Validate installation by typing `brew -v` in the terminal and ensure a version number is shown. - -### Install Node using NVM - -This will work for both MacOS and Win10. Follow instructions on this [link](https://medium.com/@nodesource/installing-node-js-tutorial-using-nvm-5c6ff5925dd8). Be sure to read through the whole doc to find the sections within each step relevant to you (e.g. if you're using Homebrew, when you get to Step 2 look for the section, "Install NVM with Homebrew"). - -If you install NVM using Homebrew, make sure to read the output in terminal after you run `brew install nvm`. You will need to add a few lines to your ~/.bash_profile and perhaps complete a couple other tasks. - -Once you install NVM, don't forget to install Node! This is included in the linked tutorial above. - -### IDE set up -While any IDE can be used, we're outlining how to set up VS Code - -1. Open the terminal and type `brew install --cask visual-studio-code` and hit RETURN. - 1. If this doesn't work or for Win10, you can download VS Code from the [website](https://code.visualstudio.com/). -2. After [forking this repo](https://github.com/usds/justice40-tool/blob/main/CONTRIBUTING.md#code-contributions), you can clone your forked repo into VS Code -3. VS Code with prompt you to "clone a repo" -4. Open the terminal and navigate to `client` directory -5. Type `npm install` to load dependencies -6. Type `gatsby develop` to spin up the app -7. Navigate to `localhost:8000` to view the app - -#### Recommended VS Code Extensions - -1. [Browser Preview](https://github.com/auchenberg/vscode-browser-preview) -2. [Live Server](https://github.com/ritwickdey/vscode-live-server) -3. [Live Share](https://github.com/MicrosoftDocs/live-share) -4. [Live Share Audio](https://github.com/MicrosoftDocs/live-share) -5. [Live Share Extention Pack](https://github.com/MicrosoftDocs/live-share) - - -## Running the code - -Once you have the required software installed and ready on your computer, you're ready to begin installing and running the Justice40 Tool code. - -1. Fork this repository and create a clone locally. In your terminal/CLI, in the directory you wish to put this project, run `git clone` plus the URL of your fork. -1. If you're using a Mac and Homebrew, go ahead and run `brew update` and `brew doctor`. It's always good to run these regularly to make sure your packages are up to date and in good condition. - -Now, check out the next section on the organization of this repository to determine where to go next. - -### Repo organization -Our repo is organized into multiple directories (folders) to make it easier to work on and run different parts of the system independently. - -The top level of the repo contains these directories, as well as project-wide documentation such as this INSTALLATION file, the README, LICENSE, CODE OF CONDUCT, etc. - -There is also a docker-compose file that will eventually include everything you need to run every part of the application in one docker command; however, at the moment, this file only spins up the score creation part of our data pipeline. - -**Running the client** -If you wish to check out our client-side code (i.e. the GatsbyJS statically generated website with the map), check out the `client` directory and its [README](client/README.md). - -**Running the backend** -If you want to run the data pipeline with ETL and score generation, check out the `data/data-pipeline` directory and its [README](data/data-pipeline/README.md). - -**Deployment** -For core team contributors working on deployment, check out the `infrastructure` directory and its [README](infrastructure/README.md) for information on deploying the backend to AWS. +Follow the [Docker installation +instructions](https://docs.docker.com/get-docker/) for your platform. diff --git a/QUICKSTART.md b/QUICKSTART.md new file mode 100644 index 00000000..cd6c7d3a --- /dev/null +++ b/QUICKSTART.md @@ -0,0 +1,23 @@ +# Local Quickstart + +Install [`git`](https://git-scm.com/). See [Install Git](INSTALLATION.md#install-git). + +Open a terminal, clone this repository, and change directory to the repository root: + +```sh +$ git clone https://github.com/usds/justice40-tool +Cloning into 'justice40-tool'... +$ cd justice40-tool +``` + +Install [`docker`](https://www.docker.com/). See [Install Docker](INSTALLATION.md#install-docker). + +> *Important*: To be able to run the entire application, you may need to increase the memory allocated for docker to at least 8096 MB. See [this post](https://stackoverflow.com/a/44533437) for more details. + +Use `docker-compose` to run the application: + +```sh +$ docker-compose up +``` + +After some setup, you should be able to open the application in your browser at [http://localhost:8000](http://localhost:8000). diff --git a/README.md b/README.md index f60ad927..f5de511b 100644 --- a/README.md +++ b/README.md @@ -44,9 +44,17 @@ Contributions are always welcome! We encourage contributions in the form of disc See [CONTRIBUTING.md](CONTRIBUTING.md) for ways to get started. -## Installation +## Using The Datasets -The installation is a typical gatsby install and the details can be found in [INSTALLATION.md](https://github.com/usds/justice40-tool/blob/main/INSTALLATION.md) +The intermediate steps of the data pipeline and the final output that is consumed by the frontend are all public and can be accessed directly. See [DATASETS.md](DATASETS.md) for these direct download links. + +## Local Quickstart + +If you want to run the entire application locally, see [QUICKSTART.md](QUICKSTART.md). + +## Power User Guides + +If you have software experience or more specific use cases, the [project wiki](https://github.com/usds/justice40-tool/wiki) has more in depth documentation of how to work with this project. ## Glossary diff --git a/client/README.md b/client/README.md index 638f8fc5..7e1bdca5 100644 --- a/client/README.md +++ b/client/README.md @@ -16,8 +16,21 @@ This README contains the following content: ## Installing and running the client site ### Via npm -1. Confirm you have the base required software installed. See [INSTALLATION](INSTALLATION.md) for more details. -1. Install yarn if you do not have it yet. Open your terminal and run `sudo npm install -global yarn`. This works on MacOS and Win10. To confirm it is installed, run `yarn -v`. A version number should be returned. + +#### Install Node using NVM + +This will work for both MacOS and Win10. Follow instructions on this [link](https://medium.com/@nodesource/installing-node-js-tutorial-using-nvm-5c6ff5925dd8). Be sure to read through the whole doc to find the sections within each step relevant to you (e.g. if you're using Homebrew, when you get to Step 2 look for the section, "Install NVM with Homebrew"). + +If you install NVM using Homebrew, make sure to read the output in terminal after you run `brew install nvm`. You will need to add a few lines to your ~/.bash_profile and perhaps complete a couple other tasks. + +Once you install NVM, don't forget to install Node! This is included in the linked tutorial above. This will also install `npm` which you need for the steps below. + +#### Install Yarn + +Install yarn if you do not have it yet. Open your terminal and run `sudo npm install -global yarn`. This works on MacOS and Win10. To confirm it is installed, run `yarn -v`. A version number should be returned. + +#### Run The Application + 1. Navigate to the base directory of this repository, and go to the `client` directory (`cd client`). 1. Run the command `npm install` to install the dependencies. 1. Run `npm start` to start up the client app.