RSS
 

Posts Tagged ‘webdev’

The Future of TodaysMeet

30 Jan

This is the second half of a two-part post. Start with part 1.

TodaysMeet is an interesting challenge because it has components that are absolutely real-time and should be built like a messaging system, not a CMS, and parts that aren’t real-time at all, and can totally be built like a CMS.

TodaysMeet has one loosely-coupled component, its Twitter integration. The site knows nothing about the daemon that executes Twitter searches and populates the database. If anything, I think it’s currently too loosely coupled, but it’s given me some insight into how to build the new system.

The guiding principles of this design are:

  • DRY. Only one part of the app should know about the canonical data store (MySQL for now).
  • Loosely coupled. A failure in one part of the service shouldn’t affect other parts.
  • Real-time, event-driven. Periodic polling is bad. Periodic Twitter searches are particularly so.
  • Use the right tool for the right job.

Obviously I’m a fan of Django. I could port TodaysMeet directly, using something like Playdoh to bootstrap the process, and get done relatively quickly. Django gets me a lot of stuff for free: a solid data store, users (something I want to add in a limited capacity, eventually), even a framework for building the crons and daemons necessary for running the site.

But Django is a lousy way to serve real-time content via long polling or socket connections.

I can easily serve real-time content via Node.js. I can do long polling or sockets. It’s also a great way to handle Twitter integration because I can just leave a connection to their streaming API, and handle new tweets as soon as they happen instead of checking periodically.

But even with things like Sequelize, MySQL connectivity through Node still feels awkward. (I think it will get better but I’m impatient.) And serving fairly static content is just weird.

So, Towelie, do you want to go real-time with Node, or do you want to use Django?

I’ll use… both!

Read on for a more technical overview. And a picture!

Read the rest of this entry »

 
Comments Off

Posted in Articles

 

The Problem with TodaysMeet

29 Jan

TodaysMeet is a project I started in 2008 to help my father solve a problem in one of his classes. The fact that it’s as popular as it is—mostly in education—never ceases to amaze me.

Unfortunately, I don’t give TodaysMeet the attention it, and more importantly its users, deserve. This is because TodaysMeet has two fatal flaws that, if they haven’t crippled it yet, will someday.

  • The UI is based on proof-of-concept JavaScript.
  • The back-end is based on my own framework.

What follows is the sad history of TodaysMeet development.

Origin Story

TodaysMeet came out of a conversation between my father and I, but it’s origins are slightly older. In some downtime in late 2007 I was trying to familiarize myself with various JavaScript frameworks by writing a UI for the same back-end in each of them. It was a pretty basic Ajax comment system. I believe it polled the server every minute. If I remember correctly, I got busy and abandoned it after creating Prototype and jQuery versions.

Around the same time I was enamored of Rails, and trying to round out Maveric into a decent Rails-inspired PHP framework.

So when my father said he wanted something like Twitter for a single classroom, that he could project on a wall, and wouldn’t require signing up, I put these things together in my head. TodaysMeet is basically the proof-of-concept Prototype JS running on top of an old version of Maveric.

The Situation Now

Every developer should write a framework, I think it’s a fantastic learning experience. But they should never build a production website out of it.

Even though Maveric got a little better after I created TodaysMeet, it’s still based on an untested, unsupported framework with no support for basic things like storage back-ends or caching.

The UI is still based on Prototype, which I haven’t used in years, and the fundamental client-server interactions are still that original “learning the library” code.

Essentially, TodaysMeet is a prototype masquerading as a production-ready product.

The result is that working on it is slow, difficult, and frankly unpleasant. Adding features—like the long-promised password protected rooms—is painful and, with no test suite, dangerous. The one real feature I added, Twitter integration, barely works when it works at all.

But users don’t care about any of that. They see that it works, mostly. They might see that it doesn’t get much attention and the UI feels three years old (because it is, of course).

Where Do We Go From Here?

TodaysMeet could be awesome, but it needs to go all the way down to the basic stack and get rebuilt. TodaysMeet is an absolutely perfect candidate for all sorts of new, exciting tools and techniques. To use any of them means starting over.

This is the first of a two-part post. In the next part, I’m going to outline the architecture I want, instead of the architecture I have.

Hopefully, some social aspect of talking about this will lead to me actually doing something about it.

 
3 Comments

Posted in Articles

 

Bleach 1.0rc2

25 Jan

Bleach is very nearly ready for a 1.0!

After I announced Bleach 1.0rc1, a couple of important issues were found. Those have now been fixed. (Thanks, guys!) One of these was backported to 0.5, and I uploaded 0.5.1 to PyPI yesterday.

If you use Bleach, please try out the new version. You can get it from Github and install with pip.

Upgrade Path

Bleach 1.0 drops the Bleach class. 0.5.x is intended as a stepping stone that will let you start using the new API.

  1. # Bleach <= 0.5
  2. from bleach import Bleach
  3. b = Bleach()
  4. b.clean(‘<script>evil()</script>’)
  5.  
  6. # Bleach >= 0.5
  7. import bleach
  8. bleach.clean(‘<script>evil()</script>’)

Hopefully this change will be minor unless you were using URL filters on linkify(). In that case, the change is:

  1. # Bleach <= 0.5
  2. from bleach import Bleach
  3. class MyBleach(Bleach):
  4.     def filter_url(self, url):
  5.         return ‘http://google.com’
  6. b = MyBleach()
  7. b.linkify(‘a link to example.com’)
  8.  
  9. # Bleach >= 0.5
  10. import bleach
  11. def filter_url(url):
  12.     return ‘http://google.com’
  13. bleach.linkify(‘a link to example.com’, filter_url=filter_url)

This is a bigger change, but I hope it won’t be too bad for users, and I think it makes the whole API much more consistent. Lots of kwargs, yes, but more consistent and logical.

Features

Bleach 1.0 doesn’t have much in the way of new features, instead focusing on clean up, improving test coverage and security, and being ready to call it a 1.0. It does have the following changes over 0.5:

  • A skip_pre option for linkify() to ignore URLs in <pre> sections.
  • Drops support for nofollow_relative. I still think that’s a good option but it needs work.

So please, check out Bleach. I’d love feedback on the new direction. Feel free to comment or open issues.

 
Comments Off

Posted in Articles

 

Introducing Waffle for Django

23 Jan

Waffle is a feature-flipping library for Django that strives to be easy and intuitive, and work with both the Django/Jingo/Jinja2 stack we use at Mozilla, and Django templates out of the box.

Waffle lets you define various reasons a flag can be active for a given request. You can make flags active for all superusers, or authenticated users, for example. You can also define a percentage of “everyone else” who will see the flag as active.

If Waffle uses a dice roll to determine if the current request will have the flag turned on, it sets a cookie so the flag will continue to be active or inactive for subsequent requests. For all subsequent requests, the cookie value is respected. Waffle only sets cookies if a given flag was actually used during the request.

Flags are global: once a given flag is set for a user, that flag is set on every request or page view. You can test a flag on /foo and trust that a given user will see the same flag value on /bar.

Flags can be used in templates, in views, or even to hide whole views.

Optionally, Waffle can activate flags based on the query string, so you can guarantee a flag will be on or off for any request for testing.

Check out Waffle on Github and let me know what you think!

 
6 Comments

Posted in Articles

 

Bleach 1.0rc

16 Jan

After nearly a year, I’ve got something I’d like to call Bleach 1.0. But first, I want your feedback!

I incorporated some patches from the community this afternoon, and closed an issue that had been bugging me. These are all available in backwards-compatible changes between versions 0.3.5 and 0.4.0.

Then there’s 0.5.0, which is the current version on PyPI and is only backwards-incompatible if you’re using the linkify filters.

In 0.5.0 I added a new/old API that doesn’t require a Bleach object. I say “new/old” because this is actually how it worked in the first place. As of 0.5.0, much like 0.1.0, you can do:

>>> import bleach
>>> bleach.clean('a <script>bad()</script> string')
'a &lt;script&gt;bad()&lt;/script&gt; string'

At some point, there’s a commit that says “Move to a more maintainable architecture.” I really wish I knew what I meant by that. My suspicion is that I was trying to avoid passing callables into linkify().

So why did I change my mind and alter the API completely?

As I was considering a set of patches, I started thinking about whether this was going to make the API more or less logical. I didn’t want to add a huge collection of kwargs, but I didn’t want to make it necessary to instantiate multiple Bleach objects just because you wanted slightly different options in different places. I also wasn’t thrilled with needing to subclass Bleach at all.

Thinking this through, I came to the conclusion that lots of kwargs, with sane defaults, is better than a kind-of-stateful, unnecessarily-class-based API.

Please look at the changes between 0.3.5 and 1.0-branch and let me know what you think.

At the very least, before creating an official 1.0, I’m going to take the time to fix all the PEP-8 violations in sanitizer.py.

Here are the recent changes, by version:

0.3.5
  • Add a strip kwarg to clean that strips blacklisted HTML instead of escaping it. (Default: False.)
0.4.0
  • Add a strip_comments kwarg to clean that strips HTML comments. (Note that this always happened before.) (Default: True.)
  • Add a styles kwarg to clean that takes a list of whitelisted CSS properties. (Note that before, allowing style attributes essentially allowed all CSS properties.) (Default: [].)
0.5.0
  • Add a nofollow_relative kwarg to linkify that controls setting rel="nofollow" on relative links within the text (links starting with /). (Default: True.)
  • Add the optional, class-less API. bleach.clean() works exactly like bleach.Bleach().clean(), and similarly with bleach.linkify().
  • Drop Bleach.filter_url and Bleach.filter_text. These are now kwargs passed into either version of linkify.
1.0.0rc1
  • Drop the Bleach class completely. All access is through the new API.
latest
  • Clean up PEP8 violations and some general coding style.
  • Add a skip_pre option to linkify that can skip creating links inside <pre> sections. (Default: False.)
  • Drop nofollow_relative. Fred’s concerns below are 100% valid and, even though it’s not a security issue per se, I don’t want to give false impressions.
 
3 Comments

Posted in Articles

 
« Older Entries
Newer Entries »