The spirit of unit testing
At it’s core, unit testing is supposed to test a single atomic “unit” of functionality without dependencies on anything else. Dependencies are important to eliminate because a failure in a dependency can incorrectly show up as a failure in the unit you’re testing. For example, if a call to
JSON.stringify() returns the wrong value, that’s not the fault of your code. Your code always expects
JSON.stringify() to work correctly and return the correct value, and if it doesn’t, that’s an error outside of your control.
How to help yourself
- Don’t use global variables. Anytime you need to test something that uses global variables, you need to recreate all of them just so the code will run. Save yourself the trouble.
- Don’t modify objects that don’t belong to you. That goes for native object prototypes. Once again, this creates environmental dependencies that need to be recreated when you want to test the code.
- **Create small pieces of functionality. **The more atomic the pieces of your code are, the less environmental dependencies they will have. Try to group functionality together logically and in a way that allows you to pass in the necessary data instead of expecting it to be in a particular location.
- **Rely on a library for core functionality. **The library acts as an abstraction between your code and the environment, making it easier to stub or mock out functionality to eliminate dependencies.
Run tests in the browser
It’s the automation, stupid
The real reason that command line tools keep trying to appear is for the purposes of automation. When the developer is sitting in front of his computer and running tests in browsers, the unit testing process is pretty simple. But that’s terribly redundant and, of course, boring. It would be much easier if the tests were automatically run periodically and the results were recorded. Really, the command line appeal is integrate test running into a continuous integration (CI) system.
The two CI systems I hear the most about are CruiseControl and Hudson. Both work in a similar manner, periodically running a series of tasks related to your build. They are capable of checking out code, running scripts, and of course, executing command-line operations. Command-line utilities fit perfectly into these systems because the output can easily be monitored for completion and errors. This represents a major problem since most of the browsers people use are GUI-based (Lynx is still around, though).
Another interesting tool that recently popped up is TestSwarm. TestSwarm’s approach is different than that of Selenium. Instead of manually starting browsers and navigating them to a page, TestSwarm relies on browsers to already be set up and attached to the TestSwarm server. The browsers can then poll the server to see if there are any new jobs that must be processed. The advantage is that you can add new browsers simply by opening a browser and pointing it to the TestSwarm server. Since the browsers are very loosely coupled to the system, upgrading to include new browsers is ridiculously simple.
TestSwarm also enables the crowd sourcing of tests. Anyone who wants to help test a product can joined a swarm and volunteer to leave the browser open for testing.
As I stated earlier, this is an area that I’m currently researching heavily, both inside and outside of Yahoo!. I’m hoping to make significant progress over the next year and share my findings with everyone.