Webarchiving Collection

◆

◆

◆

◆

📦Normal

★★

“An Awesome List for getting started with web archiving”

◆

◆

◆

◆

🔮Psychic

★★

“Wayback Machine API interface & a command-line tool”

◆

◆

◆

◆

🔮Psychic

★

“WARC + AI - Experimental Retrieval Augmented Generation Pipeline for Web Archive Collections.”

◆

◆

◆

◆

⚡Electric

★

“Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head”

◆

◆

◆

◆

⚔️Fighting

★

“Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.”

◆

◆

◆

◆

📦Normal

★

“A list of things related to software, literature, and other content for 🕣 Memento”

◆

◆

◆

◆

⚡Electric

★

“Parse And Create Web ARChive (WARC) files with node.js”

◆

◆

◆

◆

📦Normal

★

“Various Jupyter notebooks about Common Crawl data”

◆

◆

◆

◆

🔮Psychic

★

“A dockerized, queued high fidelity web archiver based on Squidwarc”