Monthly Archives: February 2015

Automation in the test preparation phase

Nowadays we don’t need to explain that functional testing is a standard component within the total development cycle. Functional testing is becoming even more important and more complex. Therefore we constantly need to improve the testing process. One of the most importantimprovementscould be to automate the development of test cases in the preparation phase. This because developing test cases is one of themost time consumingactivities within the whole test process. EspeciallyinAgileprojectswhere changes in the requirements are very common. Automating the development of test cases, cansave a lot of time and moneybutalso will ensure the qualityofthe test cases!

To be able to guarantee a stable and structured testing process in the future, were we can deliver high quality and also save time and money, we need to automatethe developing oftest cases. Model Based Testing (MBT) in combination with a functional MBT tool makes this possible. 

Model-based testing is a testing technique where the runtime behaviour of an implementation under test is checked against predictions made by a model.

- From: Colin Campbell, Microsoft Research

Model based testing

To perform MBT we first require models from which we can generate our test cases. There are many different type ofmodels available withinthe development process, mostly technical models or models that are generated from the code. For the test process, we don’t require these models.

For example, if we would use a model that is generated from code, we would only test what was built. In this case: the test result is always OK. Technical models are also difficult to maintain and therefore the test process also will be less maintainable and reproducible. Above all we want the models to be readable for everyone. A readable model is an excellent way of communication to clarify the business case as soon as possible.

So the models that we do require for the test process have to be made by the tester in close consultation with the business. These test models must be maintainable and from these test models we should be able to generate the test cases. If the generation of test cases could be automated, we would be able to organize our test process more effective and efficient.

Working with the DTM tool

The DTM tool is an easy-to-use MBT tool. Within a couple of hours anyone can start to draw a test model. Thisincontrast to many other MBT tools. The test model, based on business requirements and / or (additional) input from the business, must include all possible conditions and paths. A big advantage is that this test model can be drawn in the DTM tool itself by simply drag and drop the different symbols and connector lines into a canvas. There are just 6 symbols and with those 6 symbols we can draw any test model. It’s also possible to connect one test model to another test model by using a reference symbol. In the near future it will be also possible to import Visio models directly into the DTM tool and export them to Viso if required.

When the test model is drawn and we want to generate our test cases, just click on the button ‘Generate’. We can choose to generate the test cases based on medium test coverage or high test coverage. The medium test coverage will generate all possible combinations of 2 consecutive test paths (multiple condition coverage). The high test coverage will generate all possible paths from the start of the test model till the end of the test model. Now the DTM tool automatically places numbers to the arrows out from a decision symbol in the test model. All the possible test cases will be generated automatically, based on the chosen algorithm (medium or high test coverage).

If we generate our test cases based on an incorrect test model, we will be notified by an error pop-up message, explaining for example if the model is missing an ‘end’ symbol or a connector line. If a connector line is missing this will be highlighted in the model itself.

After the test cases are generated, it is possible to highlight an entire test path of a test case in the test model. Simply by clicking on the various test cases, the entire test path for the relevant test case becomes green. If we need we can prioritise the test cases so we can obtain the test cases related to the highest risk first, etc. (risk based testing). The different categories are low, medium, high and all.

Now it is possible to automatically generate the complete test design. This means that all the test cases inclusive the test results are specified in text, in a separate document (PDF or Excel). The test design can be generated by selecting a test case priority; low, medium, high orall.

The structure of the test design in Excel is based on requirements of the quality management tool HP Quality Center. Therefore it’s easy to export the test design into HP Quality Center.

All the test cases are generated from a specific model. If there is a changein the requirementswe only needs to update the test model and thetool will automatically and easily generate all the test cases and related test design again. This saves a lot of time!

Conclusion

The automation of developing test cases has great advantages compared with the manual preparation. Not only in terms of saving time, but also in terms of high quality. Because generating the test cases is done automatically and no mistakes are made.

The DTM Tool could save a rough estimate of 70% of the total preparation time.

The DTM test tool has been used successfully in various Agile projects. A frequentlyheard comment about the DTM tool is “simplicity is its strength”.

Learn more about the capabilities of DTM tool or do you want a demo of this product? Feel free to contact Silvio Cacace, silvio.cacace@xlteam.nl

 

Model Based Testing without assumptions

Model Based Testing nicely deals with modern development methodologies. But also in traditional development projects, this way of testing has many advantages over other testing methods. However, this is highly dependent on the way we organize our MBT testing process and what role is played by the tester. Should the tester blindly assume an already established model or should the tester be responsible for the preparation of the test model? In this paper, an explanation.

Increasingly, we see that Model-Based Testing (MBT) is used for software testing. This is a logical step to keep our value as a functional tester high. As a tester we simply can no longer carry out testing as we previously did which was accustomed to the traditional development processes. No, especially if we look at the contemporary agile development processes, as a tester we must anticipate the new ways of working. Project properties such as interactive, iterative, incremental and multidisciplinary must therefore be consistent with the current test approach. MBT deals nicely with that, provided that this new way of testing is applied correctly.

The benefits of applying MBT are now known. Thus we can, through the proper use of MBT, address any ‘open ends’, ambiguities, inconsistencies and errors in the requirements, at a very early stage. This will shorten the duration of the project and improve the quality of the software. Another great advantage of MBT is the flexibility and adaptability of the test set. If changes occur in the requirements, we merely need to adjust the model and then we can generate a new test set automatically. This automatic generation of the test set is another great advantage of MBT. The condition is that we have the proper tool.

With regards to the above, I would emphasize that MBT is the modern way of testing and has many advantages over other testing methods, but this is highly dependent on the way we organize our MBT testing process and what tool we are using.

If we look at the traditional development processes, we see that the tester has an independent view on the test basis. Think of the V-model, it shows that the basis for technical testing and development is the technical design and that the tester will use the functional design to set up the test set for the functional test. Precisely this will effectuate an independent view of the tester and any interpretation errors in the requirements will be observed. The tester can make no assumptions and all the required information to establish a thorough test set is collected by the tester himself.

But how does this work in MBT? Is a tester also able to maintain an independent view and does the tester have a clear field to complete the test basis, without making assumptions? The answer is yes, as long as the tester is able to draw the test models himself.

We too often see that within MBT the models that are used to generate the test set, are too technical, static or poorly readable and these models have not been produced by the tester itself. Who can say the author of these models has made no mistakes in the interpretation? Who takes care of updating the models after an update in the requirements? Do all project members, such as the tester, product owner, etc., understand these models? Are all test situations captured in these models? And perhaps more importantly, are these models so arranged that for the generation of the test set, a test algorithm can be used, based on the desired test coverage?

The solution to the above-described problems is simple. In MBT, the functional tester must draw the test models. The tester should then evaluate these models with the team members and business and where necessary adapt the model. Just as the functional tester composes the test set within traditional development processes. With this, the functional tester preserves an important independent perspective, these models are world-readable, these models are easy to adjust by the tester itself and these are really test models. From out of these models, now the test set can be generated automatically based on a test algorithm, taking into account the desired test coverage.

The tool, which will be used, must be adapted to this. The tester itself must be able to draw the test models in the tool and the tool must automatically generate the test set based on different test algorithms (test coverage). The DTMtool makes this possible!

The conclusion is that for functional testing, MBT perfectly fits to modern development processes, but that the value of the functional tester is even greater, with a correct insertion of this method.

For more information about our DTMtool, please visit www.dtmtool.com

Silvio Cacace, silvio.cacace@xlteam.nl

Test Consultant, Dialogues Technology

Happy Birthday, XL Team!

On a cold Monday morning, on 13th of February 2012, we opened our Bucharest office. With only two programmers and a 70-square office, but more determined to succeed than we had ever been before, we grew our team with two people every six weeks, reaching a total of 23 people after just three years.

Project after project, success after success, we grew stonger, learned valuable lessons from our own mistakes and gradually became a mature company, ready to turn its own business ideas into reality.

After these three years, we no longer use our phones while driving, we efficiently avoid any bumps in the road and feed our pets with online-bought food. In other words, we improved our lives and people’s lives in general by launching a series of personal projects, including Gropometrul, Taximetrul, Astropet and our beloved SafeDrive. For the next years, our core goal is not only to grow our current projects into successful businesses, but also to keep on innovating and to launch new projects, in an attempt to make the world a better, safer and more efficient place.

We came so far in these 3 years and would like to thank you guys, the ones that form XL Team today, but also the ones that have left us. Without the Team we would have not been here. We would also like to thank our customers and our supporters, who had faith in us since our early days in Bucharest.

Happy Birthday, XL Team!