What is the part of testing in software development life-cycle?

I would like to present my assignment on that topic.

In our days, software plays a big role in every aspect of life and software produces for every single need of different structures such as banks, military, industrial systems, vendor specific etc. Because of growing of the importance of the software, the software development became more and more important as well. This lead to the evolution in software development process. In order to finish project on time and on the budget project managers are trying to apply different approaches to the projects and estimate the effort of them. As a part of software development life-cycle, software testing is not the exception in this case. It plays a big role not only in increase of product quality but also reduce costs in the future by preventing the changes in release. One of the responsibilities of the test manager is to give a correct estimation of the time effort. “Time is money” – an old cliche which is the best way to describes the purpose of the correct time-effort estimation.

There are a bunch of different approaches for testing time-effort estimation. One of them was taken from mathematics and shows the dependence between effort and delivery time – this is Putnam-Norden-Rayliegh Curve. (Pressman, R.) Other approach is to use own experience or experience from previous projects in order to estimate the time-effort. In this case percentage should be given as an estimation of the time. Different experts give different percentage expectations. For example, Rubin says that code test have to be approximately 34% and with 18% of System test it results in 52%.(Rubin, H.) Other point of view come from R. Grady who gives a slightly different estimation with 34% for code test and 29% for System test which overall results in 63%.(Grady, R.) For some other methods and approaches, specialists give a wide range of percentage of the time-effort in software testing from 30% to 70%.(Phadke, M. S.) However, according to Suresh Nageswaran(whom also explains another approach of estimation using Use Case Points) the possible overruns with this type of estimation can be 50%-75% basically because of lack of scientific platform for this approach. (Nageswaran, S.)

So two questions that come from previous paragraph is who is right and if the range is so wide how project managers are going to estimate the particular part of software testing?

In fact, all of the author are right in some cases of their own experience. The main reason is that these authors came from different background and with experience in different industries, their approaches of time-effort estimation are working for them at it best. In my country IT specialists have a joke: “If it work don’t touch it”. Which clearly explains why every specialist think that he is right – because his strategy work! In addition, some companies or managers may have their own special estimates based on the experience in particular area with particular people. For a simple example let’s consider the situation when the the test manager know the time which test is required based on his own experience but the team changed. Manager don’t know what to expect on this stage and can add more time for testing in order to make sure that team will accomplish testing as required. And this will lead in greater time effort estimation.

There are a plenty of other factors that forms the individual point of view about time-effort estimation. However, these factors do not play a greater role, in fact they just add some percents to the result time-effort in a special cases. Despite all of the dependencies, some big factors plays the major role in time-effort estimation and these factors will help to answer the second question – how managers are going to estimate?

First thing first, the area in which the software company works is very important. As said before, software touches all aspects of human being so the best way to start is to try to split all software into some simple categories.

The most important thing in concern is a human life. It’s not necessary to tell how big this concern is for software developing companies when they have to deal with it. It can be a plenty number of different areas which main concern is a life such as medicine, some military software and software for industry, public transport software (particularly for trains and plains because we already know some crashes caused by mistakes in software). The area touched by this concern is too wide, even in a modern cars the breaks system controls by a computer.(Wikipedia, NA) However, it has to be said, that there are two different aspects: one of them is when software deals directly with human life such as life support systems in a hospitals, other is a system of quality control on the firework factory. It’s evident with a first aspect that with a fault in this system doctors may loose a patient’s life, that’s why it has to be reliable and fault tolerant. Second aspect also concern about safety of the fireworks users but probably I bit less because factory may have a personal quality control brigade which may find the bad products, nonetheless, the system that controls the safety particular on this factory have to be reliable and fault tolerant as well. A good example is the incident happened in Russian in the mid 90-s when one of the major canons exploded and caused the fire in the armory. This is very dangerous situation because the explosion of the armory will lead to the catastrophe, but the emergency fire system was working well and just let the water fill the armory’s room and prevented the fire even when some modes of the fire system was broken by explosion. That really means 100% reliable.

According to written above, as for the most important factor – human life, I would spend the highest percentage of the range for the testing. Assuming listed above 30-70% it would be 60-70 percents depending on the size and I would not sign the paper until I will be sure that system is ready to use.

Next big concern is about money. Money plays the great role in modern world (sometimes more then even life) and that’s why the attention paid by financial software developers is too high. With creation of on-line banking systems, stores, web auctions people got the idea of the future financial lines in software – it became bigger and require more knowledge on all stages of software development process than never before. With such a complex system the testing became a solid part of software development life-cycle. The reason lies in the principal idea of the Internet Banking Systems: if someone lost something then someone got something. This idea is the most important in the financial software. Consider the situation when during transaction one of the side lost their money. Anyway it will cause the drain of money and bad reputation for that particular bank. That’s why all of the systems that deals with money or systems that supports first one (for example back up systems) have to be tested properly and the importance of in is very high because some transactions may involve hundreds of millions dollars.

I had the argue with a friend of mine who was trying to prove to me the correctness of using some technologies which allows to develop the new financial application very quickly. My argument was that companies who work with the money transactions do not concern the speed of development as a most important factor, instead they need just 100% reliable and powerful software. I still believe in my words as well as I believe that these kind of systems have to be tested properly – around 50-65 percents of a total software development effort.

Other important area is a creation of the functionality of the software which means if the software has to do something than it has to. When bugs preventing successful work of the software it causes more money expenditures for fixing bugs, loosing the reputation and often customers. I clearly remember from my experience that bug found on the stage of development cost 100 dollars but on the stage of release cost 10000$. There are a lot of jokes as well around Microsoft Windows stability which is the best evidence of a possible opportunities (in bad meaning of last expression). For testing this systems I would be given 50% and more, reputation is still valuable and the end user wants to get a software with a best quality as well.

We had a look on the different areas and saw the difference in time-effort estimation between those areas. However, in the beginning of this paper we were talking about 30 to 70 percents but covered only 50 or more. When it’s suitable to spend only 30-50 percents of time-effort for software testing? Nevertheless, there are some projects that do not require a lot of testing but usually these projects are relatively small. For example some web-sites based only on the HTML and CSS. According to my experience in this case the development of this kind of web-site takes more than a half of time which means that for testing it will take less then 50% of time-effort.

Other good example is the development of API. Usually all of the API tested with automated testing which prevents the manual GUI testing and saves in this way a couple of decades of percents of time effort.

Here we came to idea that time-effort estimation also depends on type of testing of software. For a big project when tests have to be executed time after time it’s good idea to automate some part of them otherwise it will take years to execute everything with a manual testing. But with a relatively small project it’s easier to perform the manual testing because it will lead in less percentage of time-effort then automated.

The last factor is the complexity of the project. The best example here is a small jellyfish in a comparison with a big animal such as elephant. Jellyfish don’t have a brain, blood, heart and other organs comparable to an elephant and that’s why jellyfish lives much and much longer, sometimes even infinite. The same way with a software – the more we add, the greater chance it will become broken or will cause a problems.

A good example that I found recently is about Enterprise Service Bus, particularly with JBossESB. On the official web-site developers said that their product work well with a list of databases but that doesn’t mean that their ESB doesn’t work with other databases which are not in a list, they are simply just don’t have a time to test it. Enterprise Service Bus is a really complex project which deals with mailing system, support of the legacy application, web services etc. But JBossESB is a most popular open source solution with a good financial base and even for this giant the testing is a problem and developers can not finish even test the compatibility. And I think that’s why it’s a good example of problems with testing a complex software projects. To conclude, the more functions and modules we add, the more we have to test. The moving from existing system to the system based on ESB is a good example when comparable to other stages, testing takes a huge amount of time (up to 70-80%) because developers were trying to create easy in development solution but each company has to test their own system by itself.

Last thing to be said is about re-development of the software product or creation of a new version based on existing source code. Sometimes it requires to test too a lot of different features and weakness of the software because the existing part may not combine with the newest perfectly. Let’s remember Windows operating system: Bill Gates had a nickname in college “con” and he was so upset about it that he decided to exclude it from his child – Windows OS. From earliest versions of Windows NT when user was trying to create a folder with a name “con” it caused an error. Till Windows XP it was a simple the same error and from Windows Vista Microsoft add handler for this error. This is the evidence that Microsoft copied their code from previous versions and simply re-develop the operating system. But who will prove that the new version of Windows doesn’t contain old bugs?

In this paper we had a look on the different areas of software testing and saw how the time-effort can change and in which cases. It’s very important to understand what’s the purpose of the software to be developed in order to give a correct estimation of the time-effort. Good thing to remember that this effort depends on such important factors as money and human life and also the complexity of the software. The more we get these factors involved the more we have to test and make sure that software is going to be reliable. Also we saw that in most cases software require at least 50% of software development time-effort and project with less requirements are too rare and even are not a part of an enterprise software development. Also time-effort can be higher in case of migration and re-development with usage of existing source code. In addition, managers have to deal with relatively smaller factors like team knowledge, special preferences of a customer etc. which may have influence on the the resulting time-effort estimation.

List of references

1. Pressman, R., S., 2005, “Software Engineering – A Practitioner’s Approach”, McGraw-Hill, New-York, edn. 6^Th, pp. 711–712
2. 2. Nageswaran, S., 2001, “Test Effort Estimation Using Use Case Points”, Quality Week 2001, San Francisco, USA, p.2

1. Pongas, P., “Practical Measurements for Reenginering the Software Testing Process”, EuroSPI99, viewed on-line on 5th, April 2010, http://www.iscn.at/select_newspaper/testing/singular.html
2. Rubin, H., 1995, “Worldwide benchmark project report”, Rubin Systems Inc
3. Grady, R., 1992, “Practical software metrics for project management and process improvement”, Prentice-Hall, Inc.
4. Phadke, M., S., 1997, “Planning Efficient Software Testing”, Phadke Associate Inc., p.1
5. Wikipedia, NA, viewed on-line on 6^th, April 2010, http://en.wikipedia.org/wiki/Anti-lock_braking_system