Recently though, I have started to pay more attention to usability – in software and in other aspects of life. The image to the right is of my television and associated devices (apologies for the blur). I was looking at this tonight and I started wondering why those of us who work with software are so concerned with usability when it seems like no one else is. Let’s take a look at the various indicator lights that are visible in this picture. First off, there’s my Wii. It is in standby mode and is displaying an amber light. That makes sense, right? It’s not fully on and not fully off. So, amber indicates standby. Below that is my stereo receiver. It is on and has a green light on it. Again, this makes logical sense. Then there is my TV. It is on but it sports a red light to indicate the on state. Finally, in the lower right is my satellite receiver which has both a green and an amber light on it. This would seem to indicate a state somewhere between standby and on, but the device is fully on.

So, has a single sale of one of these devices ever been lost due to the lack of usability the indicator lights represent? I highly doubt it. I also don’t know anyone who has ever test-driven a television’s menus before purchase. Sure, we probably see the menus when the sales guy is talking to us about resolution and contrast ratio, but we’re not paying attention to that. We completely ignore the usability aspects of these common items because we are more concerned with how well the device does its primary function. In the case of a television, we only care about how good the picture looks.

Why then do we as software developers hold ourselves, in theory anyway, to such a higher standard? Why is usability so much more important with software than other products? Could it be a hold over from the days when there was resistance to computers from older generations? Perhaps when there was widespread resistance to anything to do with computers there was a much greater need to make the applications on the computers so amazingly easy to use in order for their benefit to be readily apparent. If so, do we still need to try so hard?

Perhaps it has nothing to do with the user and everything to do with the software. With non-software items it is usually very easy to describe what they do in a short sentence. What does your television do? It allows me to watch shows and movies. What does your razor do? It shaves my facial hair. Now, think about the software you use. Can you easily describe the function of the software the same? Firefox lets me view web pages. It also lets me search Google. It shows me videos. It plays Flash games. The list goes on. Could usability be a problem for software because we are simply trying to jam too much functionality into our applications?

There is an example from the non-software world which makes me think this might be the case. Think of a Swiss Army Knife. You can’t easily describe all of its functions in a single sentence. You also cannot say that the damn thing is in any way a usability success. It is a pain in the ass to use, but it saves you from carrying more stuff. Maybe the answer to software usability is not massive amounts of research and testing to make a Swiss Army Knife application a little more usable. Maybe the answer is to make our applications simpler and make more of them. The popularity of gadgets/widgets and the thousands of wee little Facebook applications could be an indication that this is true.

I don’t know if any of this is really relevant to a discussion on usability, but it is the way things look to me. I have many powerful applications on my computer, but I do tend to prefer using a dashboard widget if available. I tend to frequent websites that focus on one thing and do it really well. It might very well be that what we perceive as usability issues are just a case of Jack of all trades, master of none. If anyone more enlightened on the subject that me has some additional insights, please leave me some comments.

I first started using Django when I started at VendAsta and this issue didn’t creep up there because we are use Google App Engine and its DataStore on the backend instead of a traditional database. With no database there was no issue with creating/dropping anything. It was only when I started using Django on my own that I came across that problem. There was one more issue with Django that I came across only when I started doing things outside of work: Django expects all of your models to be in a single file called models.py. I hate that. A lot.

I did some googling about the problem and found that there were a bunch of other people who didn’t like it much either. There were a few solutions but they all involved manually mucking about with the __all__ attribute in the __init__.py file of your models directory. Sure, it works, but I don’t like having to define a model in two places to use it. Oddly enough it was in some test code at work that I found the answer.

For our project at VendAsta we are using the Django App Engine Helper to provide a nice easy interface to the DataStore on the backend. A part of this is a small __init__.py file which allows you to spread your tests out in multiple files. Django likes its tests to be in a single tests.py file just like it likes its models to be in models.py. The wee bit of Google code is shown in Listing 1.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59

#!/usr/bin/python2.4
#
# Copyright 2008 Google Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
 
 
 
 
"""Loads all the _test.py files into the top level of the package.
 
This file is a hack around the fact that Django expects the tests "module" to
be a single tests.py file and cannot handle a tests package inside an
application.
 
All _test.py files inside this package are imported and any classes derived
from unittest.TestCase are then referenced from this file itself so that they
appear at the top level of the tests "module" that Django will import.
"""
 
import os
import re
import types
import unittest
 
PACKAGE = 'appname.tests'
TEST_RE = r"^.*_test.py$"
 
# Search through every file inside this package.
test_names = []
test_dir = os.path.dirname( __file__)
for filename in os.listdir(test_dir):
  if not re.match(TEST_RE, filename):
    continue
  # Import the test file and find all TestClass clases inside it.
  test_module = __import__('%s.%s' % (PACKAGE, filename[:-3]), 
                           {}, {},
                           filename[:-3])
  for name in dir(test_module):
    item = getattr(test_module, name)
    if not (isinstance(item, (type, types.ClassType)) and
            issubclass(item, unittest.TestCase)):
      continue
    # Found a test, bring into the module namespace.
    exec "%s = item" % name
    test_names.append(name)
 
# Hide everything other than the test cases from other modules.
__all__ = test_names

Listing 1: __init__.py file from Google for a tests module

Essentially what this file does is loop through all files in the same directory and for each on that matches the pattern listed on line 40 (in this case, files that end in ‘_test.py’) and import all of the Classes from those files that are subclasses of the TestCase class. Finally, the code on line 62 does the magic with __all__ that I wanted to avoid doing manually for my models. It was this line that caught my attention and started me wondering if this code could be used for spreading models across multiple files.

In order for this to work for models there are a few things that need to be changed. First of all, on line 40, the pattern should be changed to match on *.py to grab all of the python files in the directory. Then, on line 46, add

or filename == “__init__.py”

to ensure that we don’t try to import the file we are in. Finally, remove the subclass requirement from line 55, as we are just concerned with importing classes here not what they inherit from. Putting it all together results in Listing 2. Place that in the __init__.py of a directory called models and you’ll not have to manually manage the __all__ list for your models in separate files.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

#!/usr/bin/python2.4
 
import os
import re
import types
import unittest
 
PACKAGE = 'my_app.models'
MODEL_RE = r"^.*.py$"
 
# Search through every file inside this package.
model_names = []
model_dir = os.path.dirname( __file__)
for filename in os.listdir(model_dir):
  if not re.match(MODEL_RE, filename) or filename == "__init__.py":
    continue
  # Import the model file and find all clases inside it.
  model_module = __import__('%s.%s' % (PACKAGE, filename[:-3]),
                           {}, {},
                           filename[:-3])
  for name in dir(model_module):
    item = getattr(model_module, name)
    if not isinstance(item, (type, types.ClassType)):
      continue
    # Found a model, bring into the module namespace.
    exec "%s = item" % name
    model_names.append(name)
 
# Hide everything other than the classes from other modules.
__all__ = model_names

Listing 2: __init__.py file modified for a models module

Now, there is one last thing that you are going to need to do in order for everything to run properly in Django. It is a very simple addition to the end of each of your classes. I have seem some people complaining about the need to do this in various forums, but honestly, if you are using any form of modern text editing application, it should only take you about 2 minutes to create a new template for a model that has the little piece of code shown in Listing 3 at the bottom of the class (replace your_app_name with your actual app name).

1
2
3

class Meta:
        """Meta Class for your model."""
        app_label = 'your_app_name'

Listing 3: Meta class required at the end of each model for proper functioning.

First and foremost, I absolutely love the amount of planning and forethought that goes into each 30-day development sprint. My past two jobs had nothing that even remotely compares to this. One was an online grocery store (which just recently discontinued operations) which meant that most development was done within the demands of an operating e-business and as a result needed to be very seat-of-the-pants. The other was a manufacturing company which meant that development processes were heavily driven by the demands of sales and manufacturing departments and those demands could and would change in the blink of an eye. Needless to say, I have found the change to be very liberating.

That is not to say that we couldn’t do more planning. One of the things that we found working through our first sprint was that while we felt we had done a great deal of planning up front, there was actually a lot of granularity that we were lacking. This is something we are endeavoring to overcome in the planning for Sprint 2. As much as I love the planning sessions we are having and as exciting as I find the whole process, it is a very tiring and draining experience. Well worth it, but wearying nonetheless.

A second major thing that struck me was how well we worked as a team. Considering the fact that five people were placed on a development team, given a couple of weeks to do some research and get to know each other and then thrust into the middle of a very ambitious project, we somehow managed to work very well together. Coupling this with the fact that we are working with a large number of cutting edge technologies, I would have expected at least a little friction to develop somewhere in the team over the course of 30 days, but none did. I believe that this is largely due to the environment at VendAsta and their hiring process. Everyone who gets hired there has to go through a lunch with other developers whom they will be working with. If those devs don’t think that you are a good fit for the team and the company, you don’t get hired. I don’t know why more companies don’t operate this way. People from HR departments should use VendAsta as a case study in how to build a whole company that just “gels”.

Finally, in the past two days I have been very impressed with the manner in which some of the trouble spots from Sprint 1 have been discussed. Previously, I have experienced project post mortems that are nothing more than a gathering of people around a table to point fingers at each other. In our post mortem yesterday we were encouraged to be completely honest and frank about what we did and didn’t like about how Sprint 1 went. At no point was there any finger pointing and the blame game just didn’t happen. The maturity of my team mates and our “chickens” impressed me to no end.

So, that is about it for my impressions on Sprint 1. We’re neck deep in planning Sprint 2 right now so I am afraid that Episode 4 of Roger’s Wars will still be a couple of days away. I’ve had a few people contact me asking when the rest of the saga was going to unfold but with the holidays and now our sprint planning, I’m having trouble finding the time to do it when my head is not either a) jell-o or b) utterly focused on work. Tomorrow or the next day I hope to sit myself down with a pot of Earl Grey and hammer out the fourth installment. Thanks for your patience

The first hurdle is actually getting a Django site running on Dreamhost. Luckily, there is this blog post which details exactly how to get your site up and running on Dreamhost. I have used this method a couple of times (once modified to get a Satchmo install running) and have met with success each time.

Once the site is running, setting up tests poses another problem. The default test behaviour for a Django site creates and then destroys a whole test database on each test run. Normally this is not a problem, but the way Dreamhost has their MySQL setup running, the commands to create and drop databases are not available outside of their administration panel. What follows is the solution I have come up with and am currently using to run Django unittests with my Dreamhost MySQL databases.

First of all, it is important to understand a little about the default testing setup that Django uses. When you run the command python manage.py test, Django looks at the TEST_RUNNER setting to determine what to do. By default this setting points to django.test.simple.run_tests. A quick look and we can see that this is the functionality we want to alter. Specifically, lines 2 and 4 shown in listing #1 are what we need to replace.

1
2
3
4

from django.db import connection
connection.creation.create_test_db(verbosity, autoclobber=not interactive)
result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
connection.creation.destroy_test_db(old_name, verbosity)

Listing 1: Excerpt from django.test.simple.run_tests

It is these lines which make the calls to create and then drop the test database. The functionality needed, however, in order to work with Dreamhost’s setup, is to “flush” and existing test database both before and after the test run is complete. I have accomplished this through the creation of a method called flush_test_db, shown in Listing #2.

1
2
3
4
5
6
7
8
9
10
11

def flush_test_db(test_db):
    """Method to flush out the test database, removing tables, before
    and after test runs"""
 
    from django.db import connection 
    cursor = connection.cursor() 
 
    current_tables =  connection.introspection.table_names()
    for table in current_tables:
        cursor.execute("use %s" % test_db)
        cursor.execute("drop table %s" % table)

Listing 2: New method to remove all tables from the test db

With this method, we can drop all of the tables in the test db both before and after a test run. It is necessary to do both just in case something goes horribly awry during a test run and we end up with leftover data from a previous test run. With this new method, we can then create a new version of the run_tests method and place both methods into a new file. I called my file test_runner.py and placed it in a tests directory within my app. The complete test_runner.py is shown in Listing 3.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

"""Override of the simple tests module from django in order to skip
database creation and use an existing db"""
import unittest
from django.conf import settings
from django.db.models import get_app, get_apps
from django.db.backends.creation import TEST_DATABASE_PREFIX
from django.test import simple
from django.test.utils import setup_test_environment, teardown_test_environment
from django.core import management 
 
def flush_test_db(test_db):
    """Method to flush out the test database, removing tables, before
    and after test runs"""
 
    from django.db import connection 
    cursor = connection.cursor() 
 
    current_tables =  connection.introspection.table_names()
    for table in current_tables:
        cursor.execute("use %s" % test_db)
        cursor.execute("drop table %s" % table)
 
def run_tests(test_labels, verbosity=1, interactive=True, extra_tests=[]):
    """
    Replacement for the run_tests in Django.
    """
    setup_test_environment()
 
    settings.DEBUG = False    
    suite = unittest.TestSuite()
 
    if test_labels:
        for label in test_labels:
            if '.' in label:
                suite.addTest(simple.build_test(label))
            else:
                app = get_app(label)
                suite.addTest(simple.build_suite(app))
    else:
        for app in get_apps():
            suite.addTest(simple.build_suite(app))
 
    for test in extra_tests:
        suite.addTest(test)
 
    old_name = settings.DATABASE_NAME
 
    if settings.TEST_DATABASE_NAME:
        test_database_name = settings.TEST_DATABASE_NAME
    else:
        test_database_name = TEST_DATABASE_PREFIX + settings.DATABASE_NAME
 
    #replace the normal DATABASE_NAME with the name of the test db
    settings.DATABASE_NAME = test_database_name
 
    #flush the test db
    flush_test_db(test_database_name)
    #load the test db with models
    management.call_command('syncdb')
 
    result = unittest.TextTestRunner(verbosity=verbosity).run(suite)
 
    #flush the test db again
    flush_test_db(test_database_name)
 
    #reset the value of DATABASE_NAME in settings
    settings.DATABASE_NAME = old_name
    teardown_test_environment()
 
    return len(result.failures) + len(result.errors)

Listing 3: The complete test_runner.py

The final step is to point the test framework to your new test runner by placing a new line in your settings.py. The exact value will vary based on where in your project you have placed the new test runner. In this example I have placed the test_runner.py file, containing the run_tests method in the tests directory of an app called ‘your_app’. This is shown in listing 4.

1

TEST_RUNNER = 'your_app.tests.test_runner.run_tests'

Listing 4: Settings.py setting pointing the test framework towards your new test_runner

Finally a few words of caveat. I have omitted any comments I am using to tell pylint to ignore certain things and I have done little in this sample code to properly handle Exceptions that might arise. Nevertheless, this should get you a good start on running unit tests in Django if you host your site with Dreamhost.

For the past three and a half years, I have been working in a very heavy Microsoft shop. As a result, it was always difficult to get acceptance for open source projects at Cover-All. The more work I started doing on web projects the easier it became to bring some open source into the fold but there certainly wasn’t any interest in giving anything back to those projects we benefited from and, even if there was interest, there wouldn’t have been time to do so anyway.

At VendAsta, things are a lot different. Here is a company that not only embraces the use of open source projects but actually encourages everyone to give back to those projects if they make any improvements / changes to them. I absolutely love that! This has given me the opportunity to give back to my first open source project!

I have been doing some work on developing Facebook applications over the past few weeks. Specifically, I have been using the awesome pyFacebook library. This is a Python library which gives the same functionality as the Facebook-supplied PHP client library, except with the goodness of Python. A total bonus for us is the fact that the library includes functionality to work with Google App Engine, another new technology we are using. There was a small gap in this functionality and that is where I found my opportunity. While everything was setup to work with retrieving information from Facebook in GAE (Google has disabled socket access in the version of Python available through GAE, so you need to use their urlfetch method instead of calls to httplib), the Photos.upload() method was not setup in the same manner. I needed to upload files to Facebook, so I modified the library to work the way I needed.

Once my mods were complete I sent them to the maintainer of the pyFacebook project and, after a few helpful suggestions from him, I had a version that he was willing to commit to the project’s svn repository: http://code.google.com/p/pyfacebook/source/detail?r=166. After that I was able to track down and fix an actual bug involving some infinite redirects: http://code.google.com/p/pyfacebook/source/detail?r=168.

Two very small contributions to a single project, but I am hopeful that this is only the beginning!

I finally got a chance this week to take a crack at customizing things a little bit. I have been listening to Internet Radio .977 lately, a set of “radio” channels I found in my ShoutCast playlist in XBMC. One thing that is very cool about working on a Mac is that so many of the applications play really well together. For instance, if you are listening to music in iTunes, with a couple of clicks you can have the current song showing up in your status in iChat or Adium. Unfortunately for me, the .977 stream does not play in iTunes. I listen to it in VLC, which does not fit together with the chat apps the way iTunes does.

I did a little Googling and wasn’t able to find much on the subject. There are a few ways to get current media information out of VLC, however, most of that is geared towards playing mp3s and using these techniques I was only able to extract the name of the radio station I was listening to, not the track name that was appearing in the VLC window. Additionally, I was able to find a site that detailed writing some Applescript and BASH scripts to set iChat/Adium status to dynamically indicate your location, based on IP address. After a little tinkering I was able to put something together.

I won’t bother with all of the details here (anyone who wants to know how it was done, drop me a line). Basically my solution was this: setup VLC to write its log out to a file and set the verbosity of the logging to max. When the track playing changes a line is written to the log including “New Title=XXX” where XXX is the track listing as it appears in the main VLC window. This information was utilized with two BASH scripts (one to pull the last song entry from the log and parse the song from that line and another to control this whole process) and one Applescript to push the status into the chat apps. A little cron magic and it works like a charm!

At the end of my time at Cover-All I was in charge of three other programmers. I’ll refer to them as the good, the bad and the kid. The kid was the newest member of the team and the youngest. The good was one of the best programmers I have ever worked with. Finally, the bad was, in reality, not a programmer. He was trained as a CAD guy and evolved into a role as a programmer, but despite that he never really became a programmer. He just didn’t care about getting better at programming and didn’t want to actually learn the tech he was using. As you have probably guessed, the rest of this post is about the bad. The non-programmer.

It really is amazing how far someone can get with a very minimal skillset and a good library of code to draw from. The bad would typically follow the same methodology for each project he worked on. Step one would be to find a piece or pieces of code (either from elsewhere in our projects or from somewhere on the web) which had the same or similar functionality to that which he wanted to implement. Step two would would be to copy it into the project he was working on and make the modifications he thought were required to make meet his needs. That last bit is key and it is what separates the good from the bad.

A good programmer takes the time to understand the code that he is using – whether it is code he has written beforehand, code another dev wrote in the same or another project or code that he has found on the web. It is this understanding which makes everything come together. The bad usually did not understand the code he was trying to use and would end up with completely non-functional code. Code he would invariably send to the good with the subject line “what’s wrong with this?”. After a few weeks of the good doing all of the difficult parts of his work, I told the good it was okay for him to tell the bad to do his own work.

The full extent of just how bad the bad was only became apparent to me during my last week at Cover-All. He was working on adding some functionality to an aspx page that the good had created to display a listing of parts which had been added to an order. The good had done a very good job of creating the page – he had placed a datagrid on the form and was populating it dynamically using a datasource in the code-behind, and updated via AJAX. The bad, when faced with this page was completely unable to understand what was being done there and therefore could not add the functionality he needed. So, after being told to do his own work yet again, he did some googling and came up with something. A few hours later he was again asking the good why no text was showing up on his page at all – he had even broken that which had been working. The good took a look at his work and asked, “Where did all of my code go?”. The response was a chilling, “I found a better way.”. That better way was to remove the datagrid and remove the code-behind and fill the aspx page with a tonne of inline asp. It turns out that the bad doesn’t know that there is a difference between asp and asp.net.

The moral of this little story is twofold. First, if you are a programmer and you have an honest desire to become better at what you do, using code written by other developers can be a great resource for learning new techniques and new technologies. Just make sure that you take the necessary time to understand the code you are using. Secondly, if you are working with someone who seems to be asking for a lot of help with code that he didn’t write himself, you might want to think twice about solving his problems for him. He might be a little inexperienced and trying to use code a little beyond his abilities (in that case, you’ll want to encourage him to spend more time on understanding the code he uses) or he may be your own version of the bad and he might be using your help and other people’s code as a crutch for his lack of real skill.