Recently I developed several projects using Ruby on Rails.  One of the cool features of it is the built-in test tools. With this amazing tool, I can test my web page such as data validation in few seconds by myself. It reminds me the time how a software was tested in the company I worked for previously.

In the formal company I worked for, they had a very well Quality Assurance Team for testing the software functionality(The ratio of developer to QA is about 3:2). Usually we (developer) need to come up many test cases for them (e.g., testing the data field with positive, negative or invalid value) because the QA team does not have any technical background. Other than that, the QA team would only focus on the GUI level testing, such as whether this control is aligned with the label or not etc. One of the key components that has been ignored is the stress test.

I still remember that I developed a resource-consuming feature, and I was worrying about its impact of overall performance when many uses are using this feature at the same time. So I talked with my QA and see what he think. Initially I expected that he would run a stress test (e.g., simulating 100 thousands of users to use the feature at the same time), and guess what his answer?

I can schedule an appointment with the rest of the QA team, we can sit down together and try to use your feature at the same time.

No, this is not a joke. In fact, about a month later, someone from other application team really does something fancy about it. They sent out a mass email and ask people to sign up. They end up having about 900 uses to use the application at the same time. That’s about 1/3 of the whole population in the company!

I think it is kind of stupid to ask 1/3 of the whole company to do a simple test, which can be done using computer automatically! Think about how expensive is the test:

900 people x $40/hr x 2 hour = $72,000

With this amount of money, they can probably hire one to two QAs to work on the test for a year!