One could say I’ve got too many projects with too much free time, but another way to look at things is that constant exploration can put a smile on your face. I’ve been working on a few projects:
This is my big project, which I really need some product marketing help on. I’ve gotten most of the infrastructure in place, but need somebody to come in an help me put the finishing 20% on top to make the first time experience right and probably catch where I’ve just been wrong headed about the thinking.
Note – Really, could use help…
One of the things that I keep on needing is a “midscale” web crawler. Would like to be able to crawl 100M pages without much thought along with obey all of robots.txt and be nice to sites, etc.etc. What’s interesting is the crawler is the easy part, it’s being nice to sites that is difficult.
This is where synergies come from! Originally I started Spiro with the idea of having a nice Bootstrap and Backbone UI, but it was klunky. In one of those moments of frustration I started looking at EmberJS, which is way overkill… Ok, not really overkill, but not what I wanted.
So after looking at EmberJS I was inspired to produce a view framework that really tried to mimic the best parts, but keep the Backbone spirit alive. Thus Distal was born.
What’s great is that I’ve had a chance to use it in GearTracker – which is where I’ve started to understand the whole client side UI development model, then take it back to Spiro to use again and break some of the ideas that I had in GearTracker.
It’s all about Synergies!
Ideas build on ideas, tools build on tools, those moments where you want a big old toggle switch to turn off your crawler can end up being 15 lines of JS (+15 of boilerplate) and 10 lines of Python. And you’ve got a working control on a page, which does a real time refresh of the server to enable and disable things.
You did one of those “big deal” – but it is really a big deal – It’s not like I had to build a bunch of templates, do a full server reload of the page. It’s a button it operates like a button.