Recently a good friend of mine reached out to me asking for help. There was an issue with payment for his website and it was not caught so the provider deleted the website. First thing I did was check the Wayback Machine at https://archive.org and he was lucky to have some snapshots of the site! Great now what do we do? I started to Google around for tools to download a website from the Wayback Machine.
The first tool I came across was https://github.com/bitdruid/python-wayback-machine-downloader great! I started poking around with it and found it worked but would only do one web link at a time, so it would take a while to download the whole site. I started going down this path for about 15 minutes and decided to check if there are any other tools out there as this was gonna take a while.
I then came across https://github.com/StrawberryMaster/wayback-machine-downloader . Hmmm a ruby app have not used one of those in a while. Just need to run the ` ./wayback_machine_downloader https://www.mycoolwebsite.com -a ` press enter and the script hangs for a moment. I’m like great this script no longer works, just as I was about to press control+c the script starts coming back [SAVED] https://www.mycoolwebsite.com and starts chugging through all the snapshots. Wooooo!!!!!
I tarball up the files and throw them on one of my Podman servers that runs some other websites as well. Spin up a webserver container and point it to the folder with the website’s files and create a new sub-domain for testing. It worked perfectly! No issues at all other than a few pictures that did not get archived. Not a big deal I was able to recover almost everything else. I did some more poking around with script and found that you can restore from a particular Wayback archive by passing -f %snapshot-number%(This can be found in the url string when searching for the sites archives) I decide to give it a shot on another site that was an AngelFire site. (remember those?) ./wayback_machine_downloader -f 20160307203310 http://angelfire.com/mi/sitename -a Script chugs through that website like a champ and now have a backup of another website that I’m gonna link to another site. So amazing that this worked so well! I am really impressed with this ruby script.
If you ever need a lifeline for data recovery on a website, I highly recommend StrawberryMaster’s wayback-machine-downloader. Better yet: make sure you have backups of your data, and remember to test those restores.
I’ve seen it firsthand — a restore attempt happens, and the archive or tape turns out to be faulty. Now you’re screwed. That kind of failure can have real business continuity consequences, including lost revenue. So save, backup, test your backups, and have a disaster recovery plan.
This was just a hobby project for a friend, but they were scrambling to get a backup of their site. They got a little lucky, too — since it was just flat HTML files, the restore was straightforward. Had it been a WordPress site, things wouldn’t have gone nearly as smoothly.