Playing with the Real-Time Web

Since I left Facebook, I’ve been working on a little project to take its place as an aggregator, or a central hub of all me-related activity on the internet: Planetoid.

Right now, if you’ve got the patience to get it up and running without documentation, Planetoid is just a run of the mill, particularly ugly feed aggregator. It’s built on Django, and it has a cron job that well pull in updates from all your feeds. (You edit the list of feeds in the Django admin.)

But that’s boring.

This is just the platform I needed. Now, I want to make it real-time.

The first step is implementing Pubsubhubbub subscriber support. Then the cron will only be necessary for feeds that don’t push update notifications.

The next step is where things get interesting: both the cron and pubsubhubbub subscriber will push notifications using Redis‘ pubsub feature. Node.js, running in parallel, will subscribe to the channel in Redis and will provide the server half of a Comet/long-polling setup, the rest of which will be implemented on the client side.

Then I just need to enable pubsubhubbub in a few places, and anyone sitting on jamessocol.com should see things like blog posts in real-time, and everything else automatically with a small lag.

Real-time and the tools to do it are very, very fun.