This morning, while reading one of Coding Horror's classic blog posts on software schedules, On Our Project, We're Always 90% Done, I came across an essay from Mary Poppendieck, which argues against rewarding team members by merit pay.
If you are a software developer or a manager, I highly recommend reading this thought-provoking article. It can be downloaded here as a PDF document. Here's my summary of highlights of the document. It lists the major disadvantages and unintended side-effects of incentive pay for your team members.
Destroying Intrinsic Motivation
Once employees get used to receiving financial rewards for meeting goals, they begin to work for the rewards, not the intrinsic motivation that comes from doing a good job and helping their company be successful. Many studies have shown that extrinsic rewards like grades and pay will, over time, destroy the intrinsic reward that comes from the work itself.
Ranking people for merit raises pits individual employees against each other and strongly discourages collaboration, a cornerstone of Agile practices. Even when the rankings are not made public, the fact that they happen does not remain a secret. Sometimes ranking systems are used as a basis for dismissing the lowest per-formers, making the practice even more threatening.
When team members are in competition with each other for their livelihood, teamwork quickly evaporates. Competition between teams, rather than individuals, may seem like a good idea, but it can be equally damaging.
The Perception of Unfairness
People perceive unfairness when they miss out on rewards they think they should have shared. There is no greater de-motivator than a reward system that is perceived to be un-fair. It doesn’t matter if the system is fair or not. If there is a perception of unfairness, then those who think that they have been treated unfairly will rapidly lose their motivation.
For example, if developers on a brand new project are rewarded, the operations people who worked day and night to deploy the software may feel cheated. The developers who took over the role of maintaining legacy software so others could work on the new project would also feel the pain of unfairness.
The Perception of Impossibility
When people find management exhorting them to do what is clearly impossible rather than helping to make the task possible, they are likely to be insulted by the offer of a reward and give up without even trying.
When we optimize a part of a chain, we invariably sub-optimize overall performance. For example, if the manager only rewards the developers for meeting the schedule, they will make the schedule at the cost of shipping low quality, not tested software. That will put the burden on the support and maintenance team. What to do?
If you are now convinced that the incentive-pay is considered harmful, the next obvious question is, how do you reward the team members so that they feel motivated to continue doing good work, and not start looking for greener pastures. Fortunately, the article lists a few solutions.
Find Better Motivators than Money
While monetary rewards can be a powerful driver of behavior, the motivation they provide is not sustainable. Once people have an adequate income, motivation comes from things such as achievement, growth, control over one’s work, recognition, advancement, and a friendly working environment.
Improving the Promotion System
Instead of tying salary gains to merit pay, have a good promotion system with different levels. One downside of typical promotion system in software companies is that it encourages solid technical people to move into management as they run out of titles for technical roles.
Instead, introduce multiple levels of promotions for both developers and managers. Establish clear promotion criteria for each level. When someone has met the criteria, promote them. To summarize, Make sure that people are fairly and adequately compensated, and then move on to more effective ways to improve performance.
In his book, Practice of System and Network Administration, Tom Limoncelli explains the importance of writing down the requirements for the system you are building. I think the list applies equally well to software projects. Having a list of clear requirements written down is probably one of the most important things you can do before starting a project.
Requirements are a list of what the service will be able to do. Requirements should list desired functionality, features, and capabilities. Focus on the end goal: what the system will enable people to achieve. The list of requirements guides all the other steps: design, engineering, implementation, testing, and so on.
Requirements are written down. They are not simply agreed to in a verbal discussion, tracked on a dry-erase board in your office, or kept in your head. Writing them down in a shared requirements document has many benefits.
Transparency: Unlike your brain or the dry-erase board in your office, everyone can read the requirements document.
Fewer gaps: Writing down requirements reduces errors caused by a request being misheard or forgotten. People can verify that their request was recorded properly.
Fewer misunderstandings: Two people often think they have verbal agreement when they do not. Seeing the decision in writing verifies that everyone is agreeing to the same thing.
Buy-in and approval: Written requirements provide the ability to get buy-in and management approval in an accountable way. People know what they are agreeing to. People can’t claim they didn’t sign off on the requirements if they’ve literally signed their name on a printout. They may claim they didn’t understand the document or that circumstances and budgets have changed, but at least you have baseline agreement.
Fixed scope: A formal requirements document provides a mechanism for handling additional feature requests that arrive after the agreement has been reached, preventing feature-creep. New requirements need to be formally agreed to, documented as a scope change, and subjected to buy-in and management approval before they can be added to the scope.
Accountability: Accountability is a two-way street. Customers can point to the document when a requirement you’ve agreed to turns out to be missing or incomplete. Documentation of requirements also helps delineate features versus bugs. A bug is a requirement that is incorrectly implemented. A feature request is a new requirement that wasn’t previously agreed to. While a bug should be fixed, a feature request requires approval, resource allocation, and possibly budget approval.
A requirements document does not need to be hundreds of pages long. A bullet list may be sufficient for small projects. The essence of business writing is brevity. A short, concise document is easier to read and understand.
Define terminology early in the document. Ontology is the system of terms and definitions that define the system and its parts. Often during a heated debate one realizes that everyone is using the same words but meaning different things. Getting agreement to the ontology that will be used is very important.
The requirements should focus on the list of features, stated from the perspective of what the customer should be able to accomplish using business terms, not technical terms. Record the “what,” not the “how.”
This also means not proscribing particular technology. For example, when an email system is being designed, users might request that it be compatible with the IMAP protocol (RFC3501). They should, instead, be stating their needs at a higher level of abstraction. Ask them why they need such a specific thing. For example, maybe they need to be able to read their email from their smartphone and they know that their phone supports IMAP.
Conversely, specifying IMAP support may under-specify the feature. Imagine the user’s surprise when the IMAP support is available but his or her particular smartphone is unable to access email. The feature is complete as specified—IMAP is supported—but the user is unable to read email. Requesting that this problem be fixed would be a feature request, not a bug. It would be rejected much to the surprise and dismay of the customer. Technically speaking, the product works as requested.
It is this kind of situation that makes perfect sense to a technical person but is frustrating and unreasonable to a typical user. This is one reason why users view IT departments as difficult to deal with. They’re being told they got what they asked for, but in their mind that’s not true: They can’t read email from their phone. Phrasing requirements at the right level of abstraction is one way that we can prevent this problem.
One thing I am learning as a developer working on enterprise software is the importance of a configuration management system. When you have a software system that's installed on-premises for each customer, that is configured and customized separately, managing configuration becomes a challenge. You can't just use environment variables or simple files to store config. You need to have a good strategy to deal with the configuration.
When code relies on values that may change after the application has gone live, keep those values external to the app. When your application will run in different environments, and potentially for different customers, keep the environment- and customer-specific values outside the app.
Common things you will probably want to put in configuration data include:
Credentials for external services (database, third party APIs, and so on)
Logging levels and destinations
Port, IP address, machine, and cluster names the app uses
Environment-specific validation parameters
Externally set parameters, such as tax rates
Site-specific formatting details
Basically, look for anything that you know will have to change that you can express outside your main body of code, and slap it into some configuration bucket. In this way, you’re parameterizing your application; the code adapts to the places it runs.
Parameterize Your App Using External Configuration.
If the configuration is simple, you can store it in JSON or XML files. For more complicated ones, you might need an application that stores the config in the database. When your application starts, it reads this configuration as a data structure in-memory and uses throughout the life of the application. If you change the configuration, the application needs to restart to fetch the new values. This is how we manage configuration at CityView.
Though this is a common approach to managing configuration, the pragmatic programmers recommend against it. Instead, their suggestion is to wrap the configuration information behind a thin API. This decouples your code from the details of the representation of configuration.
In this approach, the config is still external to the application, but instead of storing in a file or database, it's stored behind a service API. This has following benefits:
Changing configuration doesn't require the application to restart. Then next API call will fetch the latest configuration value.
Multiple applications can share configuration, with correct authentication and authorization.
The first benefit is important. Modern applications are expected to be up and running all the time. You shouldn't have to stop and start the whole app to change a single config value. Using config-as-a-service, the application always fetches the latest value from the configuration, dynamically. When config changes, there's no need to rebuild the code.
However, I am not totally sold on this benefit. If I expose my configuration as a service behind an API, my application becomes chatty, making a lot of HTTP calls with the API, which would have been available in-memory if they were loaded as a data structure when the application started. An HTTP call is always going to be slower than an in-memory call.
You have to use your best judgement and choose the strategy that suits the needs of the users and application and what's important to you. If the config is changing all the time and you don't want to restart app each and every time, use config-as-a-service. If the config is done initially, and changes rarely, use a static configuration which gets loaded when the application boots up.
In an IaC environment, we don’t make changes to systems directly. Instead, we update the code and data that are used to create our environments.
When we administer systems this way, the code we use to control our infrastructure is not just part of our infrastructure, it is our infrastructure. It describes the network connections between machines, the machines themselves, and the applications that run on the machines.
To create a new machine, we update our machine-processable description and let the automation create it. If we do not like that change, we can revert to a previous version of that machine-processable description. We fix our infrastructure as a developer fixes a bug: We make a change to code and test it in a test environment; if that is successful, we push the change to production.
When we manage our infrastructure as code, every change would be made by updating machine-processable definition files that, when processed, created machines, deleted machines, configured systems, created files, started processes, and so on. We would gain the ability to track changes, know the history of our infrastructure, and identify who made which change. Our infrastructure would be parameterized so that we could build the same environment with different quantities, names, and locations. System administration could reap the benefits that software engineers have enjoyed for decades.
Reduced Cost: Manual labor is reduced or eliminated. Automation is a workforce multiplier: It permits one person to do the work of many.
Improved Speed: Not only can tasks be done faster, but they can also be done in parallel. They can be done without waiting for a person to be available to do the work.
Reduced Risk: Security risk is reduced because we can prove compliance. We can reduce the risk of errors and bugs by applying software engineering techniques like unit tests and version control.
Make all changes via configuration files, such as a Dockerfile or PowerShell scripts.
Document systems and processes in code to make it the source of truth.
Use version control system to track changes in the configuration files.
It can be intimidating to get started with IaC, especially in a preexisting environment. The best strategy is to start small. Automate one thing on one machine to get comfortable. Then manage more aspects of the system over time and build toward an environment where all changes are made via the CM system.