irma.nps.gov-datastore (14 files)
 zipfiles.txt.zst |
10.08kB |
| steps.txt |
0.38kB |
| README |
1.14kB |
| profiles.tar.zst |
45.95MB |
| pdfs.tar.zst |
473.98GB |
| pages.tar.zst |
11.62MB |
| others.tar.zst |
252.37GB |
| html.tar.zst |
48.05MB |
| get-profile.sh |
0.25kB |
| holdings.tar.zst |
7.28MB |
| extracted-zip.tar.zst |
2.18TB |
| get-holdings.sh |
0.80kB |
| download-page.sh |
1.00kB |
| download-file-types.txt.zst |
631.36kB |
Type: Dataset
Bibtex:
Tags:
Bibtex:
@article{,
title= {irma.nps.gov-datastore},
journal= {},
author= {},
year= {},
url= {},
abstract= {# IRMA NPS DataStore
A mirror of https://irma.nps.gov/DataStore/Search/Quick -- sent a search for the empty string, and crawled through all results and file downloads.
Data captured on 2025-03-04
This archive contains 3317 pages of search results, amounting to 165806 records
("references" in DataStore lingo)
pages.tar.zst contains all pages from search results, in JSON format.
profiles.tar.zst contains JSON metadata (description, date published, author)
for each reference.
holdings.tar.zst contains JSON file listings per reference, i.e. file
MIME-types and sizes.
html.tar.zst pdfs.tar.zst extracted-zip.tar.zst and others.tar.zst are the
actual downloaded files segmented by filetypes for compressability.
extracted-zip.tar.zst does not contain the original zipfiles but rather
extracted folders so that they can be more effectively recompressed by
ZStandard. The original total size of all zipfiles was 2.2 TiB, all data fully
extracted was 3.1 TiB.
Detailed code for how the data was scraped is available in steps.txt. Data is
packed in ZStandard-compressed tarballs with -9 --long to reduce torrent
metadata and disk usage.},
keywords= {national park service,irma,nps,gov,united states},
terms= {},
license= {},
superseded= {}
}