Designing software

So much of what I do revolves around design, I don't mean the beautiful works of visual art that painters, sculptors and architects bring to our world, I mean the hidden beauty of elegant software, that sweet-spot between achievable and maintainable, that magic place where you have just enough code and no more. The deceptively simple beauty of well articulated algorithms brought together with an overall design that makes the problems it addresses parseable by all who are lucky enough to work within it.

So what goes in to designing great software? I can't speak for everyone of course, but in my own experience of developing software since the mid 90's there are a number of recurring problems that need to be overcome if you want your project to succeed, and there are a number of good habits you can adopt to ensure that you avoid these problems by default.

First and foremost, solve the right problem

First and foremost, make sure you are solving the right problem. By this I mean, make sure that you understand the scope of what the software is supposed to change in the world rather than focusing over-much on how the software should work. There is a recurring tendency I come across in people who want software to do a particular job; they describe how they want the software to work without first describing why the software should exist. Get your customer to specify a list of problems the software is supposed to solve, do this first, and make sure you do it before you hop into designing how the software is to function. This list should be expressed in problem form, not in solution form. By this I mean your list should have statements like:

"I need to know where my vehicles are, I typically check on each of them at least once every fifteen minutes, I especially need to know immediately if the driver presses the panic button, or if some other alarm condition occurs, because I have to respond to alarms within a minute if I am to adhere to the security standards I'm contractually obliged to follow."

Rather than:

"The vehicle tracking device should report in once a minute"

When we look at these two examples, no software developer worth his salt should accept the second of these two with out asking a loud resounding "why??". The worst thing you can do with a "requirement" like this is to blindly implement it. While the problem illustrated is a fairly easy one to spot and ask the customer to elaborate on, it's this sort of thing when ignored that causes all sorts of knock-on effects. Imagine if the device being designed was battery powered, imagine if you have thousands of these devices all calling in every minute all at once twenty four hours a day, the bandwidth spikes! The first of these two examples focuses on what the problems are that the customer needs software to solve for them, the second example, is the customer (badly) designing a solution to their problems. You can make your customer much happier, and your software more useful if you understand the nature of the problem domain.

Stay on track

Use this list of problems throughout the life of your project and check to make sure you're actually solving them. It's OK to add to this list as the technical people learn more of the problem domain as they may see secondary issues that were missed in the initial draft. Be aware though, this list of problems is the primary way for you to tell if your software is complete, if the problems are solved, then the software is done. Yes of course their are bugs and wrinkles to work out along the way, but fundamentally, if these goals are not met, then the software is simply not complete.

There is another recurring risk here, scope creep. If you're writing software to make a living, being paid for your time and effort is the key to keeping the lights on and to running a business that works. The urge for both the developer and the customer to add items to the problem list is undeniable. As people think more deeply about what the software is trying to achieve, and as they try to explain their problem domain to a software developer, it is inevitable that they will gloss over details that they find easy or commonplace, much to the dismay of the software developer when they finally figure out the extent of the problem(s) they are trying to solve.

There are so many software design principles that try to manage this fundamental challenge of communicating ideas from those who know what the problems are to those who can do something about them. In a perfect world, the domain expert would simply and flawlessly brain-dump on to paper, then the developer would read, understand, and write amazing software with no gaps or architectural oddities because the software would be perfectly adapted to it's environment. All to often this is simply not the case, humans makes mistakes, humans make assumptions, humans forget stuff, and leave things out of their writings. And when problems are finally unearthed, late in the project, they can do fairly horrible things to a design that hasn't accounted for them. So how do you stay on track?

Given that we know we're all flawed creatures with an incomplete understanding of the universe, and given that we know, both ourselves and our customers will make mistakes. Then there is simply no excuse for not allowing for some contingency in a project. Beyond this however, if you have a problem/goal list and you share it with your customer, then you can point out that the scope of the project is growing when it does, and you can empower them to decide how they would like to proceed; either leave that problem unsolved, or extend the deadline and costs to add it to the list of things to do. Clear communication with your customer is so important, as is educating them, and making sure they understand the mystery and the muscle of the problem list.

Data, data, data

So often I see developers naturally start to code the user interface first, because that is the bit the customer sees and the bit everyone can look at and discuss. While this approach has some merits, especially when you don't know precisely what problems you're software is trying to solve, I think it's a mistake to ignore the nature and quantity of the data that will need to be gathered, stored, and queried by the system. Understand the data! Know what you are recording and why, understand the limits that should be placed on data, understand the constraints and relationships between the facts your system works with. 

Start at the destination

When designing a solution, it's a good idea to start at the destination you wish to reach and work backwards. Focus on your goals, those problems that our software will solve, and for each goal, figure out precisely what steps need to be taken to achieve it. Then for each of these new steps, figure out what steps need to be taken to achieve them. Keep iterating on this until you have a solution.

Working this way makes sure you stay on track, avoid developing code you don't need, and gets you to a working system faster. This method also has the side effect of deconstructing your goals into manageable steps.

Data, data, data 

If you hope to do a good job at selecting a database technology, then you need to understand the nature of the data you will be asking it to store. In order to understand what you are storing, you need to consider what output you expect to generate from the data. In order to understand output, you need to understand the problems you are solving. This can be something of a circular challenge to grasp. Choice of database technology does not escape this problem. Some systems promise a kind of developer nirvana by saying you don't need to structure your database. What they mean is, you don't need to structure and constraint your data NOW, but you will need to when you try to get sensible output from it. Remember, it's too late to validate the input once the user has logged off; you can't validate data after it has been written down to the database. So how do we validate data?