To Software Software - To Software-Assessment - Software/Assessment
To Software Current Tests - Software/CurrentTest - To Software Testteam - Software/TestTeam
To Software Testers Welcome Pack (Intro) - Software/TestTeam/WelcomePack
Software Testers Welcome Pack
4. Reporting
Hallo Software-Testers,
after the first steps on the test server and the test server management system we now come to the most important part of software testing: The Reporting.
What does it help if you test and nobody, especially the Software-Assesor, gets informed about it? We can do a test over and over again but we would not be able to finish a test. That is the reason for reporting.
- Which browser are you using?
- Which way did you go to use a function?
- Was the bug that reported eliminated?
- Is there now a new bug?
For the Software Assessors it is vital to know how many testers tried a test if the bugs where eliminated. Only if enough feedback was provided it is reasonable that the patch can be reviewed and then moved to the productiv system. Therefore a patch that is not tested cannot be moved to the productive system.
In the bug tracking system (Mantis) the whole history of a bug/patch is visible. Starting from the reporting, hints about possible solutions and workarounds, test reports and so on.
The test starting page Software/CurrentTest is vital to see which patches that eliminate a bug are currently on the test server. There is a link for each Bug # to the bug tracking system.
When you start a test sequence open the bug # in the bug tracking system, document the steps of your test, menues, links, values that were entered and maybe the result of your doing. Sometimes this can be done in short (OK, not OK). Sometimes it might be useful to enter a longer description; especially if you found any errors or bugs.
Please abstain from lengthy, overly verbose comments as they usually only obscure test results and thus are of little value to software assessors and software developers. For Software Assessors only the OK/not OK and for developers the minimal steps to reproduce are of importance: Please avoid additional noise in the comments whereever and whenever possible.
The report has to be added at the end of the bug report as a note.
Only with a complete documentation the Software Assesors are able to perform an adequate review of a patch. Otherwise a patch cannot be published.
What to do if I find a bug?
Report
Most important is it to document how the bug occurs so that people can reproduce the error. Sometimes it is quite useful to start the documetation right at the beginning of your doing in the sytem and record every step in the bug report as you go through your process.
If it is a new bug, report it under the actual bug number that you work on. If it shows that this bug is not related to the test bug an new bug report should be filed.
Where to find a detailed description of a patch ?
The problem with describing a patch as two faces. On the one hand the description should be as detailed as possible. On the other hand a to detailed description might point the tester in certain direction so that the tester might not see all things that can be happen ...
The starting point is the tester portal
on which the activated test on the test environment are given. In most cases there is a bug# with a link and a short description of the problem.
The patch should solve the problem on the one way on another.
The aim of testing is to find if the patch works as desired. So test it with different browsers, play at differnt areas of the software, use different values, e.g. assurance level, report all your findings as noze in the bug#. There it does not matter if the bug is eliminated or still present. Also check if you can abuse certain inputs to do unintended things - if necessary report those findings privately in the bugtracker.
As reference you can use the productive system as long as you have access to the specific area. In case of doubt avoid tests on the live system - especially when testing involves persistent inconsistencies or things at the edge of policies ...
There are various patches from simple to complex.
A "simple" patch is the exchange of a web page with a link to the Wiki. Is the link working or are you still on the test server?
The simplest test is.
Go to the productive system, do what ever the bug report states and try to understand the problem. Change to the test system and try to find the bug there. Hopefully you find gone ...
The bug report is the documentation of the patch. Sometimes you find some solution hints, what to expect ...
I do not understand what to do with a patch?
Ask another tester or the test team leader. They will help you.
Can I miss a test ?
In principal yes, but as we do not have many testers it would be great if you could do every test.
When is a test finished ?
As long as the patch is visible on the starting page it is not fully tested. Do the test and report your findings.
The Software Assessment Team decides after reviewing the results if a test can be closed are if it still needs further testing.
Who decides if a patch is moved to the test server?
Again the Software Assessment Team does it. They choose the patches under obeyance of priority, quality and other preconditions which is the next to test.
Your
Testteam Teamleader
What is next ?
Software Testteam Welcome Pack