publicdata/UW-CALMA-datarescue
1
0
Fork 0
mirror of https://github.com/UW-CALMA/datarescue.git synced 2025-02-21 09:11:31 -08:00
Code Issues Projects Releases Packages Wiki Activity Actions

GITBOOK-12: No subject

Browse source
This commit is contained in:
bksstudio 2025-01-06 05:41:33 +00:00 • committed by gitbook-bot
parent ac506d3d81
commit 84e0d713b5
No known key found for this signature in database
GPG key ID: 07D2180C7B12D0FF
5 changed files with 97 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
SUMMARY.md
View file

@ -8,5 +8,8 @@
* [🎙️ Track 1 (Communications)](how-to-start/track-1-communications.md)
* [🔍 Track 2 (Data Assessment)](how-to-start/track-2-data-assessment.md)
* [🕵️ Track 3 (Technical)](how-to-start/track-3-technical.md)
* [🛠️ Resources & Tools](resources-and-tools.md)
* [🛠️ Resources & Tools](resources-and-tools/README.md)
* [Tools](resources-and-tools/tools.md)
* [Readings](resources-and-tools/readings.md)
* [Relevant Projects](resources-and-tools/relevant-projects.md)
* [🙋 Stay in Touch](stay-in-touch.md)

6
resources-and-tools.md → resources-and-tools/README.md
View file

@ -6,11 +6,7 @@ description: Readings and tools available online
### Tools
* Making signed BagIt files: [https://github.com/harvard-lil/bag-nabit](https://github.com/harvard-lil/bag-nabit)
* https://github.com/climate-mirror/climate-mirror-tools
* [https://www.datalumos.org/](https://www.datalumos.org/) - has simple drag and drop add tags and basic metadata
* https://www.sucho.org/ This is another initiative which was focused on Ukrainian Digital Cultural Heritage. It was kind of modelled off of Data rescue v01 but a little more broad in terms of what to “save” and there were different threats, because the physical infrastructure was also in danger
* Harvard Library Innovation Lab seeking Government Datasets for Preservation form [https://docs.google.com/forms/d/11qyuKUEkbh0OPNyAyMVCXviiSDTYYA4RLdIlTnWNiTE/edit](https://docs.google.com/forms/d/11qyuKUEkbh0OPNyAyMVCXviiSDTYYA4RLdIlTnWNiTE/edit)
Here you will find tools to assist with basic to advanced digital preservation tasks.
### References

41
resources-and-tools/readings.md Normal file
View file

@ -0,0 +1,41 @@
---
description: >-
References included in the Gitbook as well as other notable research findings
and reflections
---
# Readings
ClimateWire, S. W. (n.d.). Climate Web Pages Erased and Obscured under Trump. Scientific American. Retrieved January 3, 2025, from[ https://www.scientificamerican.com/article/climate-web-pages-erased-and-obscured-under-trump/](https://www.scientificamerican.com/article/climate-web-pages-erased-and-obscured-under-trump/)
Dillon, L., Walker, D., Shapiro, N., Underhill, V., Martenyi, M., Wylie, S., Lave, R., Murphy, M., Brown, P., & Environmental Data and Governance Initiative. (2017). Environmental Data Justice and the Trump Administration: Reflections from the Environmental Data and Governance Initiative. Environmental Justice, 10(6), 186–192.[ https://doi.org/10.1089/env.2017.0020](https://doi.org/10.1089/env.2017.0020)
Earthjustice. (2024, November 12). What Project 2025 Would Do to the Environment – and How We Will Respond. Earthjustice.[ https://earthjustice.org/article/what-project-2025-would-do-to-the-environment-and-how-we-will-respond](https://earthjustice.org/article/what-project-2025-would-do-to-the-environment-and-how-we-will-respond)
Environmental Data and Governance Initiative. (n.d.-a). Changing the Digital Climate: How Climate Change Web Content is Being Censored Under the Trump Administration,. Retrieved January 3, 2025, from[ https://envirodatagov.org/publication/changing-digital-climate/](https://envirodatagov.org/publication/changing-digital-climate/)
Environmental Data and Governance Initiative. (n.d.-b). Federal Environmental Web Tracker. Environmental Data and Governance Initiative. Retrieved January 3, 2025, from[ https://envirodatagov.org/federal-environmental-web-tracker-about-page/](https://envirodatagov.org/federal-environmental-web-tracker-about-page/)
Harmon, A. (2017, March 6). Activists Rush to Save Government Science Data—If They Can Find It. The New York Times.[ https://www.nytimes.com/2017/03/06/science/donald-trump-data-rescue-science.html](https://www.nytimes.com/2017/03/06/science/donald-trump-data-rescue-science.html)
Johnson, E., & Kubas, A. (2018, February 7). Spotlight on Digital Government Information Preservation: Examining the Context, Outcomes, Limitations, and Successes of the DataRefuge Movement. In the Library with the Lead Pipe.[ https://www.inthelibrarywiththeleadpipe.org/2018/information-preservation/](https://www.inthelibrarywiththeleadpipe.org/2018/information-preservation/)
Kosoff, M. (2017, January 25). Trump White House Orders E.P.A. to Delete Climate-Change Web Page. Vanity Fair.[ https://www.vanityfair.com/news/2017/01/trump-white-house-orders-epa-to-delete-climate-change-web-pages](https://www.vanityfair.com/news/2017/01/trump-white-house-orders-epa-to-delete-climate-change-web-pages)
Lamdan, S. (2018). Lessons from Datarescue: The Limits of Grassroots Climate Change Data Preservation and the Need for Federal Records Law Reform. University of Pennsylvania Law Review, 231.[ https://papers.ssrn.com/abstract=3163616](https://papers.ssrn.com/abstract=3163616)
Nost, E., Gehrke, G., Poudrier, G., Lemelin, A., Beck, M., Wylie, S., & Initiative, on behalf of the E. D. & G. (2021). Visualizing changes to US federal environmental agency websites, 2016–2020. PLOS ONE, 16(2), e0246450.[ https://doi.org/10.1371/journal.pone.0246450](https://doi.org/10.1371/journal.pone.0246450)
Sens. Markey, Hirono and Rep. Adams Introduce Legislation to Promote Conservation and Preservation of Government and Historic Records. Retrieved January 3, 2025, from[ https://www.markey.senate.gov/news/press-releases/sens-markey-hirono-and-rep-adams-introduce-legislation-to-promote-conservation-and-preservation-of-government-and-historic-records](https://www.markey.senate.gov/news/press-releases/sens-markey-hirono-and-rep-adams-introduce-legislation-to-promote-conservation-and-preservation-of-government-and-historic-records)
Sisak, M. R., Colvin, J., & Whitehurst, L. (2023, June 10). A timeline of events leading to Donald Trump’s indictment in the classified documents case. AP News.[ https://apnews.com/article/trump-documents-investigation-timeline-087f0c9a8368bb983a16b67dd31dcd4c](https://apnews.com/article/trump-documents-investigation-timeline-087f0c9a8368bb983a16b67dd31dcd4c)
Stein, R. (2024, November 12). With Trump coming into power, the NIH is in the crosshairs. NPR.[ https://www.npr.org/2024/11/12/nx-s1-5183014/trump-election-2024-nih-rfk](https://www.npr.org/2024/11/12/nx-s1-5183014/trump-election-2024-nih-rfk)
Sunlight Foundation. (n.d.). How federal agencies are quietly removing government Web resources, and why it matters. Retrieved January 3, 2025, from[ https://sunlightfoundation.com/2017/11/15/how-federal-agencies-are-quietly-removing-web-resources-and-why-it-matters/](https://sunlightfoundation.com/2017/11/15/how-federal-agencies-are-quietly-removing-web-resources-and-why-it-matters/)
Tirrell, C., Senier, L., Wylie, S. A., Alder, C., Poudrier, G., DiValli, J., Beck, M., Nost, E., Brackett, R., & Gehrke, G. (2020). Learning in Crisis: Training students to monitor and address irresponsible knowledge construction by U.S. federal agencies under Trump. Engaging Science, Technology, and Society, 6, 81–93.[ https://doi.org/10.17351/ests2020.313](https://doi.org/10.17351/ests2020.313)
Vinik, D. (2017, July 25). What happened to Trump’s war on data? The Agenda.[ https://www.politico.com/agenda/story/2017/07/25/what-happened-trump-war-data-000481](https://www.politico.com/agenda/story/2017/07/25/what-happened-trump-war-data-000481)
Williams, R. (2017, January 29). Michigan web developers and archivists join race to back up federal agency data. Michigan Public.[ https://www.michiganpublic.org/environment-science/2017-01-29/michigan-web-developers-and-archivists-join-race-to-back-up-federal-agency-data](https://www.michiganpublic.org/environment-science/2017-01-29/michigan-web-developers-and-archivists-join-race-to-back-up-federal-agency-data)

10
resources-and-tools/relevant-projects.md Normal file
View file

@ -0,0 +1,10 @@
---
description: >-
Information about relevant digital preservation work of at-risk data (both
inactive & active)
---
# Relevant Projects
Saving Ukrainian Cultural Heritage Online (SUCHO) https://www.sucho.org/ \
Initiative focused on Ukrainian Digital Cultural Heritage. It was kind of modeled off of Data rescue v01 but a little more broad in terms of what to “save” and there were different threats, because the physical infrastructure was also in danger.

41
resources-and-tools/tools.md Normal file
View file

@ -0,0 +1,41 @@
---
description: >-
Information about tools needed for tasks or other important digital
preservation work
---
# Tools
For Authenticity and Verification
* Making signed BagIt files: [https://github.com/harvard-lil/bag-nabit](https://github.com/harvard-lil/bag-nabit)
* Make Bags: [https://github.com/WeAreAVP/fixity](https://github.com/WeAreAVP/fixity) or [https://github.com/LibraryOfCongress/bagger](https://github.com/LibraryOfCongress/bagger)
* Create checksums: [https://corz.org/windows/software/checksum/](https://corz.org/windows/software/checksum/)
Metadata Creation and Description
* Analyze file & produce basic metadata: [https://coptr.digipres.org/index.php/NARA\_File\_Analyzer\_and\_Metadata\_Harvester](https://coptr.digipres.org/index.php/NARA_File_Analyzer_and_Metadata_Harvester)
* Index web archive files: 
For Data an Web Archive Capturing and Harvesting
* Conifer tool (website interactions): [https://conifer.rhizome.org/\_faq](https://conifer.rhizome.org/_faq)
* Browser extension web crawl (single page): [https://warcreate.com/](https://warcreate.com/)
* Browser extension to add webpage to Internet Archive: [https://web.archive.org/](https://web.archive.org/)
* Copy websites (HTTrack): [http://www.httrack.com/](http://www.httrack.com/)
* Crawl website (Heritrix): [https://sourceforge.net/projects/heritrix.mirror/](https://sourceforge.net/projects/heritrix.mirror/)
* Capture backend of websites: [https://deeparc.sourceforge.net/](https://deeparc.sourceforge.net/)
**For Website Monitoring and Assessments**
* Estimate website size: [https://github.com/izkreny/website-size](https://github.com/izkreny/website-size)
* Monitor websites in bulk (thousands): [https://github.com/edgi-govdata-archiving/web-monitoring](https://github.com/edgi-govdata-archiving/web-monitoring)
* Monitor websites (single or small batch): [https://distill.io/](https://distill.io/)
* Detect website changes: [https://github.com/openpreserve/pagelyzer](https://github.com/openpreserve/pagelyzer)
* Assess websites (note differences in stories): [https://github.com/DocNow/diffengine](https://github.com/DocNow/diffengine)
* Assess websites (compare two pages): [http://pagelyzer.openpreservation.org/](http://pagelyzer.openpreservation.org/)
General Lists of Digital Preservation Tools
* Community Owned digital Preservation Tool Registry (COPTR) [https://www.digipres.org/tools/by-function/#createorreceive(acquire):webcrawl](https://www.digipres.org/tools/by-function/#createorreceive\(acquire\):webcrawl)