RSS
 

Posts Tagged ‘python’

Acronyms you should know: MTTD and MTTR

10 May

If you’re a SUMO contributor, there are two acronyms you will start to hear more often from us developers: MTTD and MTTR.

They mean “mean time to detect” and “mean time to resolve,” respectively, and they refer to how long it takes to detect an issue in production, and how long it takes to resolve that issue once it’s detected.

As we move toward continuous deployment, these are two of the metrics we’ll be using to gauge the effectiveness of our tools and processes.

For major production issues, our MTTR is actually fairly good right now—if it’s something that cannot wait until the next scheduled release, it takes us 60-90 minutes from becoming aware of an issue to pushing a fix. I think we can do better with better release processes, but we’re starting off pretty good and going to get better, which is great.

Our MTTD, on the other hand, needs work. SUMO 2.8.1 upgraded Django and included a sweeping change to our CSRF protection—this necessarily affected every form on the site. We discovered three related issues that warranted immediate hotfixes, but we didn’t discover two of them for almost two days when our contributors brought them to our attention.

It’s great that our contributors pointed out these issues to us. Our community is a critical part of “detection” and I want to encourage everyone to point out issues in the forums or IRC. It’s extremely helpful!

But there are things we can do, too, to notice things faster. One thing we are working to add is business metric graphs. We have useful data in Ganglia right now, but we will be using Graphite and Etsy‘s StatsD to peer into what our users are doing. If we deploy a change and notice that no one is previewing articles, for example, we know immediately that we have an issue and can start diagnosing and fixing it.

If you follow SUMO development, you’ll hear us start using terms like MTTD, MTTR, “detection,” more, and talking about how to reduce them. We welcome your input and ideas as we start working on these challenges. And of course, keep telling us when things are broken!

 
1 Comment

Posted in Articles

 

O Hai Django AdminPlus

04 Mar

Last night, as happens sometimes, I was wishing it was possible to add some of our custom admin views to the Django admin’s index page. It’s kind of a pain to have to type the URL every time, especially when talking to other people: “It’s in the admin, but no, you can’t… just go to this URL.”

So I was reading the Django admin docs and, frankly, they aren’t great. They tell you about the benefits of subclassing AdminSite, or running multiple AdminSites, but they never really mention how to do it.

So, with a little trial and error, I wrote my AdminSite subclass and figured out how to use it. It simplified a bunch of code in our custom admin app, so I thought I would share it.

Here’s Django AdminPlus (and on PyPI).

AdminPlus adds a method, admin.site.register_view, that connects an arbitrary view function to your admin, and puts a link to it on the admin index page. If you put calls to register_view in someapp/admin.py, they’re picked up during the normal admin.autodiscover() process.

  1. # myapp/admin.py
  2. from django.contrib import admin
  3. from django.shortcuts import render_to_response
  4.  
  5. def my_admin_view(request):
  6.     return render_to_response(‘myapp/admin/view.html’)
  7. admin.site.register_view(‘mypath’, my_admin_view)
  8.  
  9. # With an optional ‘name’ parameter:
  10. def my_other_admin_view(request):
  11.     return render_to_response(‘myapp/admin/other.html’)
  12. admin.site.register_view(‘otherpath’, my_other_admin_view, ‘Fancy Stuff!’)

Assuming your admin URLs start with admin/, this will make my_admin_view available at admin/mypath and my_other_admin_view available at admin/otherpath, but it also puts them in a new section on the admin index, so you don’t even need to worry about the URL.

If no name is given we try to guess from the name of the view function.

That’s about it! Read the README and the other docs and enjoy!

 
2 Comments

Posted in Articles

 

The Future of TodaysMeet

30 Jan

This is the second half of a two-part post. Start with part 1.

TodaysMeet is an interesting challenge because it has components that are absolutely real-time and should be built like a messaging system, not a CMS, and parts that aren’t real-time at all, and can totally be built like a CMS.

TodaysMeet has one loosely-coupled component, its Twitter integration. The site knows nothing about the daemon that executes Twitter searches and populates the database. If anything, I think it’s currently too loosely coupled, but it’s given me some insight into how to build the new system.

The guiding principles of this design are:

  • DRY. Only one part of the app should know about the canonical data store (MySQL for now).
  • Loosely coupled. A failure in one part of the service shouldn’t affect other parts.
  • Real-time, event-driven. Periodic polling is bad. Periodic Twitter searches are particularly so.
  • Use the right tool for the right job.

Obviously I’m a fan of Django. I could port TodaysMeet directly, using something like Playdoh to bootstrap the process, and get done relatively quickly. Django gets me a lot of stuff for free: a solid data store, users (something I want to add in a limited capacity, eventually), even a framework for building the crons and daemons necessary for running the site.

But Django is a lousy way to serve real-time content via long polling or socket connections.

I can easily serve real-time content via Node.js. I can do long polling or sockets. It’s also a great way to handle Twitter integration because I can just leave a connection to their streaming API, and handle new tweets as soon as they happen instead of checking periodically.

But even with things like Sequelize, MySQL connectivity through Node still feels awkward. (I think it will get better but I’m impatient.) And serving fairly static content is just weird.

So, Towelie, do you want to go real-time with Node, or do you want to use Django?

I’ll use… both!

Read on for a more technical overview. And a picture!

Read the rest of this entry »

 
Comments Off

Posted in Articles

 

Bleach 1.0rc2

25 Jan

Bleach is very nearly ready for a 1.0!

After I announced Bleach 1.0rc1, a couple of important issues were found. Those have now been fixed. (Thanks, guys!) One of these was backported to 0.5, and I uploaded 0.5.1 to PyPI yesterday.

If you use Bleach, please try out the new version. You can get it from Github and install with pip.

Upgrade Path

Bleach 1.0 drops the Bleach class. 0.5.x is intended as a stepping stone that will let you start using the new API.

  1. # Bleach <= 0.5
  2. from bleach import Bleach
  3. b = Bleach()
  4. b.clean(‘<script>evil()</script>’)
  5.  
  6. # Bleach >= 0.5
  7. import bleach
  8. bleach.clean(‘<script>evil()</script>’)

Hopefully this change will be minor unless you were using URL filters on linkify(). In that case, the change is:

  1. # Bleach <= 0.5
  2. from bleach import Bleach
  3. class MyBleach(Bleach):
  4.     def filter_url(self, url):
  5.         return ‘http://google.com’
  6. b = MyBleach()
  7. b.linkify(‘a link to example.com’)
  8.  
  9. # Bleach >= 0.5
  10. import bleach
  11. def filter_url(url):
  12.     return ‘http://google.com’
  13. bleach.linkify(‘a link to example.com’, filter_url=filter_url)

This is a bigger change, but I hope it won’t be too bad for users, and I think it makes the whole API much more consistent. Lots of kwargs, yes, but more consistent and logical.

Features

Bleach 1.0 doesn’t have much in the way of new features, instead focusing on clean up, improving test coverage and security, and being ready to call it a 1.0. It does have the following changes over 0.5:

  • A skip_pre option for linkify() to ignore URLs in <pre> sections.
  • Drops support for nofollow_relative. I still think that’s a good option but it needs work.

So please, check out Bleach. I’d love feedback on the new direction. Feel free to comment or open issues.

 
Comments Off

Posted in Articles

 

Introducing Waffle for Django

23 Jan

Waffle is a feature-flipping library for Django that strives to be easy and intuitive, and work with both the Django/Jingo/Jinja2 stack we use at Mozilla, and Django templates out of the box.

Waffle lets you define various reasons a flag can be active for a given request. You can make flags active for all superusers, or authenticated users, for example. You can also define a percentage of “everyone else” who will see the flag as active.

If Waffle uses a dice roll to determine if the current request will have the flag turned on, it sets a cookie so the flag will continue to be active or inactive for subsequent requests. For all subsequent requests, the cookie value is respected. Waffle only sets cookies if a given flag was actually used during the request.

Flags are global: once a given flag is set for a user, that flag is set on every request or page view. You can test a flag on /foo and trust that a given user will see the same flag value on /bar.

Flags can be used in templates, in views, or even to hide whole views.

Optionally, Waffle can activate flags based on the query string, so you can guarantee a flag will be on or off for any request for testing.

Check out Waffle on Github and let me know what you think!

 
6 Comments

Posted in Articles

 
« Older Entries