RSS
 

Weekly Update 04/03/2011

04 Mar

OK, in line with my 2011 goals and because I’m sick of not remembering what I did last week, I’m restarting the weekly update posts. I hope you like hearing about the minutia of my job! (Just kidding. I write these for me.) I’m going to try a “sometime Friday” schedule instead of weekends or at night.

Also, this one will cover the last four-and-a-half-ish weeks, since it’s been a while.

 
Comments Off

Posted in Articles

 

O Hai Django AdminPlus

04 Mar

Last night, as happens sometimes, I was wishing it was possible to add some of our custom admin views to the Django admin’s index page. It’s kind of a pain to have to type the URL every time, especially when talking to other people: “It’s in the admin, but no, you can’t… just go to this URL.”

So I was reading the Django admin docs and, frankly, they aren’t great. They tell you about the benefits of subclassing AdminSite, or running multiple AdminSites, but they never really mention how to do it.

So, with a little trial and error, I wrote my AdminSite subclass and figured out how to use it. It simplified a bunch of code in our custom admin app, so I thought I would share it.

Here’s Django AdminPlus (and on PyPI).

AdminPlus adds a method, admin.site.register_view, that connects an arbitrary view function to your admin, and puts a link to it on the admin index page. If you put calls to register_view in someapp/admin.py, they’re picked up during the normal admin.autodiscover() process.

  1. # myapp/admin.py
  2. from django.contrib import admin
  3. from django.shortcuts import render_to_response
  4.  
  5. def my_admin_view(request):
  6.     return render_to_response(‘myapp/admin/view.html’)
  7. admin.site.register_view(‘mypath’, my_admin_view)
  8.  
  9. # With an optional ‘name’ parameter:
  10. def my_other_admin_view(request):
  11.     return render_to_response(‘myapp/admin/other.html’)
  12. admin.site.register_view(‘otherpath’, my_other_admin_view, ‘Fancy Stuff!’)

Assuming your admin URLs start with admin/, this will make my_admin_view available at admin/mypath and my_other_admin_view available at admin/otherpath, but it also puts them in a new section on the admin index, so you don’t even need to worry about the URL.

If no name is given we try to guess from the name of the view function.

That’s about it! Read the README and the other docs and enjoy!

 
2 Comments

Posted in Articles

 

SUMO in Q2

02 Mar

At the end of 2010, I issued a challenge to my team: deploy support.mozilla.com continuously by the end of 2011. So, as we move into the last part of Q1, how are we doing, and what’s next?

So Far

This quarter we’ve managed to completely break free from our old svn repository. All our code is in git now. This has simplified our deployments significantly. We’ve also moved a bunch of code from difficult-to-maintain RewriteRules into the product and made it easier to manage.

We’ve still got some work to do on JavaScript unit tests and moving our crontab into version control, but these are both on track for this quarter, we just have to make sure we make the time.

Overall, I think we’ve done pretty well so far this quarter. I would give us a B, and we can still get to an A if we can tackle JS tests and crontabs.

Coming Up

The two things I’d like to see next are faster cycles and moving toward a more CD friendly way of building things. There are two challenges these help address.

Faster release cycles will get us thinking about managing time and planning work when releases aren’t centered around a big piece of functionality. Instead of starting with a big thing and picking a set of little things to lump in with it, we’ll need to balance fixing little things with work on bigger features.

By the beginning of June, I want us to release every week, whether there’s something big or not.

CD also presents a new challenge for big features: how do you deal with code that isn’t completely ready yet?

There are a couple of ways of handling this. One is to use longer-lived feature branches. That works pretty well for desktop software like Firefox where you can hand someone a binary and say “test this.” But it’s a challenge with a web app because, unless we move our staging environments around all the time, it blocks QA from testing something until it’s not only ready but, in fact, already on production.

A better solution is to use feature flags to hide new functionality until it’s ready. In this model, everything lives on the same branch (you still do feature branches for bug fix/review and for those times something does need to live a little longer before merging) and all staging servers—and eventually production—run the same code, but with a different set of flags turned on. You can turn a feature on in production after it’s been verified on stage.

This isn’t easier. It means structuring code differently, thinking about a new set of constraints. It’s what we should do.

We will screw up, probably a couple of times, before we get this down. That’s why we should start now, when our mistakes will be confined to staging servers.

In Q2 I want us to stop using the ‘next’ branch and start using Waffle to control features until they’re ready.

Not everything needs to hide behind a flag: small bug fixes obviously don’t. This may be giving us a window into what QA is like under CD: big things are manually tested before we turn them on, but little fixes, already verified by the developer and reviewer, make it through to production without manual intervention from QA.

We also need to start planning time to remove flags and dead code once a feature has shipped. That is part of the development cost of a feature, and will hopefully be offset by reducing the time it takes to get something to production once it is ready.

Big Changes

While Q1 has been about a larger set of smaller, more concrete and isolated changes, Q2 is going to be about a smaller set of much bigger changes.

Q1 was about changing our code. Q2 is about changing our thinking.

I’m excited about diving into these challenges. I can’t wait for Q2.

 
4 Comments

Posted in Articles

 

The Future of TodaysMeet

30 Jan

This is the second half of a two-part post. Start with part 1.

TodaysMeet is an interesting challenge because it has components that are absolutely real-time and should be built like a messaging system, not a CMS, and parts that aren’t real-time at all, and can totally be built like a CMS.

TodaysMeet has one loosely-coupled component, its Twitter integration. The site knows nothing about the daemon that executes Twitter searches and populates the database. If anything, I think it’s currently too loosely coupled, but it’s given me some insight into how to build the new system.

The guiding principles of this design are:

  • DRY. Only one part of the app should know about the canonical data store (MySQL for now).
  • Loosely coupled. A failure in one part of the service shouldn’t affect other parts.
  • Real-time, event-driven. Periodic polling is bad. Periodic Twitter searches are particularly so.
  • Use the right tool for the right job.

Obviously I’m a fan of Django. I could port TodaysMeet directly, using something like Playdoh to bootstrap the process, and get done relatively quickly. Django gets me a lot of stuff for free: a solid data store, users (something I want to add in a limited capacity, eventually), even a framework for building the crons and daemons necessary for running the site.

But Django is a lousy way to serve real-time content via long polling or socket connections.

I can easily serve real-time content via Node.js. I can do long polling or sockets. It’s also a great way to handle Twitter integration because I can just leave a connection to their streaming API, and handle new tweets as soon as they happen instead of checking periodically.

But even with things like Sequelize, MySQL connectivity through Node still feels awkward. (I think it will get better but I’m impatient.) And serving fairly static content is just weird.

So, Towelie, do you want to go real-time with Node, or do you want to use Django?

I’ll use… both!

Read on for a more technical overview. And a picture!

Read the rest of this entry »

 
Comments Off

Posted in Articles

 

The Problem with TodaysMeet

29 Jan

TodaysMeet is a project I started in 2008 to help my father solve a problem in one of his classes. The fact that it’s as popular as it is—mostly in education—never ceases to amaze me.

Unfortunately, I don’t give TodaysMeet the attention it, and more importantly its users, deserve. This is because TodaysMeet has two fatal flaws that, if they haven’t crippled it yet, will someday.

  • The UI is based on proof-of-concept JavaScript.
  • The back-end is based on my own framework.

What follows is the sad history of TodaysMeet development.

Origin Story

TodaysMeet came out of a conversation between my father and I, but it’s origins are slightly older. In some downtime in late 2007 I was trying to familiarize myself with various JavaScript frameworks by writing a UI for the same back-end in each of them. It was a pretty basic Ajax comment system. I believe it polled the server every minute. If I remember correctly, I got busy and abandoned it after creating Prototype and jQuery versions.

Around the same time I was enamored of Rails, and trying to round out Maveric into a decent Rails-inspired PHP framework.

So when my father said he wanted something like Twitter for a single classroom, that he could project on a wall, and wouldn’t require signing up, I put these things together in my head. TodaysMeet is basically the proof-of-concept Prototype JS running on top of an old version of Maveric.

The Situation Now

Every developer should write a framework, I think it’s a fantastic learning experience. But they should never build a production website out of it.

Even though Maveric got a little better after I created TodaysMeet, it’s still based on an untested, unsupported framework with no support for basic things like storage back-ends or caching.

The UI is still based on Prototype, which I haven’t used in years, and the fundamental client-server interactions are still that original “learning the library” code.

Essentially, TodaysMeet is a prototype masquerading as a production-ready product.

The result is that working on it is slow, difficult, and frankly unpleasant. Adding features—like the long-promised password protected rooms—is painful and, with no test suite, dangerous. The one real feature I added, Twitter integration, barely works when it works at all.

But users don’t care about any of that. They see that it works, mostly. They might see that it doesn’t get much attention and the UI feels three years old (because it is, of course).

Where Do We Go From Here?

TodaysMeet could be awesome, but it needs to go all the way down to the basic stack and get rebuilt. TodaysMeet is an absolutely perfect candidate for all sorts of new, exciting tools and techniques. To use any of them means starting over.

This is the first of a two-part post. In the next part, I’m going to outline the architecture I want, instead of the architecture I have.

Hopefully, some social aspect of talking about this will lead to me actually doing something about it.

 
3 Comments

Posted in Articles

 
« Older Entries
Newer Entries »