Skis / Toys / Fun

I admit it, I’ve been working on my middleware python server.   Here’s the basic design of the app at this point:

PHP <— JSON-RPC —-> Python <–> MySql

The assumption is that as things grow, you’ve already defined a good layer to scale at and have a reasonable Object abstraction from the front end.  At one level it’s turning out to be duplication of work, like duh!  I almost need three versions of the same object instance.

1. the ORM version (read/write to the DB)
2. the Python version — manipulation and management
3. the PHP version — slightly different mix of methods, but same data

That’s the object trials and tribulations.  Now we’re off to RPC land.  First you have to pick a framework, which if you want to use Python and JSON-RPC [lighter weight than XML-RPC] you end up in a world of hurt since the jsonrpc support for the long laundry list of frameworks is pretty piss poor. 

Currently using Twisted as the framework de jour, but it’s json-rpc library (via a code.google.com project) doesn’t quite match the spec!  For instance by default it always returns a list.  Second problem, which turns out to be a fun rant…  This is where one really discovers that Python is not Java, is that if you want to serialize anything out via the Twisted mess of frameworks you’re waiting for pain.

1. simplejson (as used by Twisted JSON-RPC) is slow
2. cjson is fast, but suffers even more than simplejson

Biggest problem, I want to have this nice simple return statement in my json handler:

return { ‘data’ : object }

Looks nice, but wait!  simplejson doesn’t know what to do with an object.  The only “fix” is to subclass simplejson to extend the handlers.  Of course since I’m sitting eight layers down in the stack from where simplejson is instantiated you can’t really do that.  So, what’s the fix?

The Fix: tear apart the Twisted JSON-RPC handler make it use both simplejson and cjson (why not) and add 4 lines of code to cjson encode_object:

        if ((result = PyObject_CallMethod(object, "__getstate__", NULL)) != NULL) {
            PyObject *r = encode_object(result);
            Py_XDECREF(result);
            return r;
        }

Ah, now we’ve duck typed the pickle __getstate__ method to give us a back a dictionary…  The problem of course is that you can’t register “unknown” type handlers for any of the json serializes out there.

Had an interesting conversation the other day with another rider while heading up to the top of Page Mill — of course the whole point was to pass the time while riding the last mile and a half.  The basic premise of the conversation went something like this:

California has a problem with a budget that varies quite radically, the core of the problem is that in a good economy the tax revenue from capital gains is large, so the state then sets up a whole bunch of projects that need that revenue to exist.  Then, as typical, a few years later in a weaker economy you end up with massive shortfalls, thus cutting teaching or school funding (ok, you cut a lot, but some groups yell louder).

So the proposal as we approached the top of the hill, was to insure that schools and other basic programs could be paid for out of the general tax revenue.  While capital project — typically financed via bonds — would be funded via the capital gains revenue.  Thus you could start more projects in good years and not start projects in down years, you could also insure that you budgeted some amount of $$ for future bond payments based on your current projections.

I’m getting really frustrated at looking at site after site after site that really doesn’t solve your problem.  I want products that make me smarter or reduce the amount of work I have to do!

My favorite whipping boy is groups, specifically Y!Groups or Google Groups, they’re products that are designed for “On the Internet nobody knows if you’re a dog.” mentality.  Fundamentally 90% of the groups that I would actively participate in are “people I know.”

I have very simple requirements, it would probably only take a month to build out a whole service to do it all…  Easily replace Y!Groups with it, but can I find a market, can I plug it into FB or some other thing that really makes people smarter?

Hmm…

Ok, it the last month I’ve used every one of these languages and most of them for the same project.  Some highlights:

• Java — decompilers are very useful in reverse engineering
• C++ — std::string in naive use sucks for performance (wrote a pooled string ala Java String — performance increased by 10x)
• Python – is easy to extend in C or C++
• PHP – has the best web frameworks
• Python – the spaces thing is prone to error, it really is hard to add addtional conditionals and insure that you got all of a if/for/… loop
• PHP & Python – Lists are good as a basic structure
• Python – As far as I can tell, it’s not a true lambda
• PHP – Lax typing, both good and bad…
• C++ – when it’s right, it rocks for performance
• Java – a good way to rapidly build app and algs that work
• Java – it’s easy to over object an API
• C++ – boost is your friend, but it comes with a cost
• Python – documentation is almost one step below C++ in quality — long topic around virtuous circles and how Java/PHP rock for docs.
• Python – has the libraries that PHP only wishes it had, written by people who know what a library should look like.

What’s the upshot…

• Use PHP to build web front ends — the ability to find a PHP programmer is pretty easy, odds are it’s not going to make/break your application.
• Use simple RPC mechanisms (e.g. JSON RPC) to Python back ends for the heavy lifting — lots of choices here, but odds are it’s not going to make much difference.
• Extend your Python with C++ for the computational parts of your heavy lifting — think peep hole optimization.

I used to think that using Java as a back end was the “only” way, but it’s more like the “cult of Java” vs. the “cult of …”, I’m not into religion just getting a job done.   When it needs to be fast both languages will get it there, but you have to remember to spend your time optimizing only where it counts…

I recently got yet another notification that somebody was following a feed of mine… every now and then you go, who is this person, why are they interested in my stream of ramblings.   So, I looked and found this wonderful statement on their linked in page:

Track record in establishing a vision, identifying opportunities, building organizations and delivering strong revenues and profits despite intense market conditions. Repeated success guiding sizeable, cross-functional teams in the design, re-design, and launch of leading-edge business solutions driving record-setting sales revenues. Combines strategic planning with strong technical qualifications to establish financial operations responsive for long-range objectives for growth and profitability. Verifiable track record of increasing revenue, profits and market share through savvy product acquisition, design, management and marketing. Practiced in analyzing competitive landscapes and designing product vision and global market strategies. Versed in defining cost-effective branding and marketing programs to quickly drive market awareness. Natural communicator and team leader with strong motivational skills and ability to build, produce and succeed.

You half to ask yourself…  Are they building a linkedin profile for buzzword bingo purposes, trying to impress their first date?  What in the world does this say..  Or decomposed:

• My org made $$ when a new competitor entered the market
• We figured out how to market our stuff to make $$
• I hung out in planning meetings
• I helped acquirer a company or two
• I built a marketing program, that had no analytics associated with it…
• I know a lot of words to build sentences to confuse

Now if we were to write this in English, we could say:

I’m a kick butt marketing professional who’s lead teams to grow our products in many markets against our competitors.  Driving teams to build growth product both internally and through acquisition.

Of course, I don’t know if that’s really true for this person… but heck at least it’s back into a language that we can read.