Chicken Scratches on Crawler
(More notes to myself) Inspired by CommonCrawl, but their data isn’t all that I want. After building out my redback system, which I’ve now let sit and die for the last two years (it worked, but had some small scaling issued). Revised idea based on some of the CommonCrawl thinking… – Fetcher Built in Tornado, with [...]