Test Baby Test

Published: by

A short while ago, when reviewing a company's operations, I suggested their testing was deficient. Specifically, they had some decent unit testing, but very weak integration testing. In simplest terms:

  • Unit testing checks if small, individual components function as expected
  • Integration testing checks if larger systems, or the system as a whole, function as expected

Testing is the only way you can have confidence that what worked before works now, and that what you put in new works as expected. Of course, testing takes time and therefore money. It may seem expensive up front, but it more than pays for itself over time.

A longer while ago, while working with a company that stored consumer information in the cloud, I impressed upon the COO that he had 3 problems: performance, user interface (UI), and security. I also told him to fix security first.

The reason is simple: if you are slow, or the UI is poor, you can convince consumers to try you again in 6 months or a year.

But if you lose their data, you will never get those users (or anyone they speak to, or anyone who sees the news reports) again!

Put the two together, you begin to wonder about Apple's massive  iOS and OS X SSL security hole. If you want to read the Google Engineer's details (it isn't that gory), go here. If you prefer TheVerge's write-up on the implications of what they call an "Epic Security Flaw", read here.

You always need good testing, especially integration testing. But there is never any excuse for a lack of extensive automated testing of core security services. Let me repeat that:

There is never any excuse for a lack of extensive automated testing of core security services.

You can survive all other bugs, some of which you risk by shipping early for market reasons. Those are valid business considerations (I am a business technology consultant after all). Security risks are inexcusable.

I don't want to know how the bug got in; others do. I don't want to know why they were so quiet about repairing it. I don't want to know why iOS was repaired last week and OS X is still not repaired; I want the patch.

I just want to know how management and culture failed so miserably as to ship security code that sits at the heart of every secure connection to be without proper testing over an 18-month period.