This is the second half of a two-part post. Start with part 1.
TodaysMeet is an interesting challenge because it has components that are absolutely real-time and should be built like a messaging system, not a CMS, and parts that aren’t real-time at all, and can totally be built like a CMS.
TodaysMeet has one loosely-coupled component, its Twitter integration. The site knows nothing about the daemon that executes Twitter searches and populates the database. If anything, I think it’s currently too loosely coupled, but it’s given me some insight into how to build the new system.
The guiding principles of this design are:
- DRY. Only one part of the app should know about the canonical data store (MySQL for now).
- Loosely coupled. A failure in one part of the service shouldn’t affect other parts.
- Real-time, event-driven. Periodic polling is bad. Periodic Twitter searches are particularly so.
- Use the right tool for the right job.
Obviously I’m a fan of Django. I could port TodaysMeet directly, using something like Playdoh to bootstrap the process, and get done relatively quickly. Django gets me a lot of stuff for free: a solid data store, users (something I want to add in a limited capacity, eventually), even a framework for building the crons and daemons necessary for running the site.
But Django is a lousy way to serve real-time content via long polling or socket connections.
I can easily serve real-time content via Node.js. I can do long polling or sockets. It’s also a great way to handle Twitter integration because I can just leave a connection to their streaming API, and handle new tweets as soon as they happen instead of checking periodically.
But even with things like Sequelize, MySQL connectivity through Node still feels awkward. (I think it will get better but I’m impatient.) And serving fairly static content is just weird.
So, Towelie, do you want to go real-time with Node, or do you want to use Django?
I’ll use… both!
Read on for a more technical overview. And a picture!