One of the Strategic Goals and Objectives defined on the OSTP website is to "Energize and nurture the processes by which government programs in science and technology are resourced, evaluated, and coordinated." I have yet to see adequate evaluation of IT projects in the government. I suggest a more robust process for evaluating technology programs (particularly related to software development). Anyone with experience developing software for the government has heard of government programs that cost millions of dollars resulting in a software product that is not used. There are many reasons for such failures:
- the end product doesn't satisfy the user's needs (e.g., due to poor requirements or lack of communication with end users)
- the users needs changed during the development of the product (e.g., due to changes in command)
- the product was developed to spend money that was allocated without a need ever existing
- the end product is of poor quality
Regardless of the reason, millions of tax payer dollars are being spent on programs that result in failure. The government should have a more robust system for evaluating software development programs to identify the following:
- is the end product actually being used
- does the end product satisfy the user's needs
- if there was a failure, what was the cause
- was the money allocated wisely
- is the quality of the product satisfactory
These evaluations will help prevent failures from occurring in the future.
To expand on the software quality aspect of the proposed evaluation, I suggest the government impose software quality standards that all government IT initiatives must conform to. In particular, software testing is one of the only areas of IT where the industry is behind academia. Formal methods of evaluating the quality of software exist now, but are only being used for a small portion of software projects. The government should define acceptable testing approaches, guidance for testing software, and the minimum criteria required for software products to be accepted (e.g., level of detail for model-based approaches and specific coverage criteria). This will provide a basis for ensuring the quality of software and help prevent waste.
The government has a responsibility to ensure our tax dollars are spent responsibly. These suggestions will provide a first step to ensure the government is accountable for the money they spend.