How Important is Testing and Quality Control When Purchasing a Software System?

November 22, 2013, 3:41 PM

 

Healthcare.gov. Need I say more? Or, for those of you nostalgic for your law school days, res ipsa loquitur.

Actually, most business purchasers of software and related hardware systems will not be able to draw any direct lessons from the Obamacare website rollout fiasco. It is just too unique, complicated and fraught with too many problems, technical and otherwise. However, it should serve as a reminder to anyone planning to replace or acquire any important business software system and an Internet website is essentially a software system -- that there is a right way and a wrong way to do these things. So let me just offer some guidance we have shared with our clients before, in the following Software Acquisition Checklist:

Replacement of a business software system can constitute a major investment of both dollars and management time and entails the risk of disruption of ongoing processes facilitated by the software it if is not done well and on schedule. For these reasons advance planning is a "must" for any significant software upgrade. The following checklist is a summary of some of the more important advance planning points that should guide any software acquisition.
  1. If your internal IT resources/personnel are not sufficient, engage a software consultant to help guide you in the acquisition. The right consultant can be invaluable to help ensure both that the software system you are buying is what you need, and that it is installed, integrated and implemented properly.
  2. Determine what you need the software system to do and how well you need it to do it (speed, accuracy, guaranteed uptime, etc.) and from this develop specific requirements based on functionality and performance. These should be the basis for the "specifications" the software vendor will agree to meet.
  3. Assess the environment in which the software system will have to perform, including interfaces to existing equipment and other software and databases you use, and any new equipment needed to optimize performance of the new software. These things will end up being your responsibility rather than the software vendor's, unless you hire the vendor to take care of certain of them. So identifying environmental issues in advance will enable you to work with the software vendor to make everything work together properly.
  4. Identify your specific security and control requirements in order to assure that the software you are buying will meet them, or can be readily modified or augmented to meet them.
  5. Focus on scheduling as early as possible in the process, in order to be sure the software system vendor can meet your timing requirements. It is important to keep scheduling realistic to avoid surprises. There will always be hiccups and delays in any but the simplest software replacement. Realistic scheduling will help moderate their impact.
  6. Determine early on what types and stages of testing you will put the new system through before accepting it, and work out with the new software vendor the schedule and procedures for testing and acceptance.
  7. Plan for essential training by identifying what will be required and which of your personnel will be trained as operators of the software or to provide internal training to other users of the software.
  8. Be sure to focus on the ongoing maintenance commitment of the software system, to be sure it provides what you need. Solid maintenance terms will include such things as guaranteed uptime for the system and response time for handling of software problems.
  9. Spread your payment for the software and services provided by the vendor over the life of the project (delivery, installation, testing and acceptance), both to keep the vendor motivated to perform and to be sure you have leverage throughout the process to keep the project on schedule. If possible a significant portion of the software vendor's license and other fees should be deferred until and linked to successful testing and final acceptance of the installed system.
  10. You will want legal input and guidance not only for contracts and other documentation of your business deal with the software vendor, but also to help assure that the rights you get in the software are sufficient for your purposes and to help spot any lurking intellectual property issues or concerns with the vendor's rights in the software.

    In context of the ongoing healthcare.gov debacle, I want to emphasize and expand a bit on the sixth numbered item in the above list, relating to testing. In any major software acquisition and this includes websites the best way to be sure you get what you pay for is to work up and follow a rigorous testing regime that ends with a final test in which actual intended users of the new system put it through its paces to confirm that it not only works on a technical level, but also where the rubber meets the road, when the person who needs to use the system to do what it is intended to do tries to make it perform as advertised. We all know now that quality control and testing appear to have been given short shrift in the governments acquisition of the healthcare.gov website. Would that they had paid more attention to the common sense advice in the checklist above? - Robert E. Smartschan