Playing with the Real-Time Web

Since I left Facebook, I’ve been working on a little project to take its place as an aggregator, or a central hub of all me-related activity on the internet: Planetoid.

Right now, if you’ve got the patience to get it up and running without documentation, Planetoid is just a run of the mill, particularly ugly feed aggregator. It’s built on Django, and it has a cron job that well pull in updates from all your feeds. (You edit the list of feeds in the Django admin.)

But that’s boring.

This is just the platform I needed. Now, I want to make it real-time.

The first step is implementing Pubsubhubbub subscriber support. Then the cron will only be necessary for feeds that don’t push update notifications.

The next step is where things get interesting: both the cron and pubsubhubbub subscriber will push notifications using Redispubsub feature. Node.js, running in parallel, will subscribe to the channel in Redis and will provide the server half of a Comet/long-polling setup, the rest of which will be implemented on the client side.

Then I just need to enable pubsubhubbub in a few places, and anyone sitting on jamessocol.com should see things like blog posts in real-time, and everything else automatically with a small lag.

Real-time and the tools to do it are very, very fun.