The evaluation of test results is not a trivial task. However, in order to achieve true automation of software testing, ways have to be found also to automate test evaluation. The evaluation of test is also known as the oracle problem in software testing. We are using necessary properties of the system under test in order to evaluate software tests (and partially also generate test inputs). Partially, we are working in the field of Metamorphic Testing. Furthermore, we have also applied statistical tests in this context. We have been able so far to propose some approaches to test real life image processing implementations.

Team:

Former members:

Publications:

  1. Ralph Guderlei, Johannes Mayer:
    Towards Automatic Testing of Imaging Software by Means of Random and Metamorphic Testing. International Journal on Software Engineering and Knowledge Engineering 17 (6), 2007, pp. 757-781.
  2. Ralph Guderlei, Johannes Mayer:
    Statistical Metamorphic Testing Testing Programs with Random Output by Means of Statistical Hypothesis Tests and Metamorphic Testing. In: Proceedings of the First International Workshop on Software Test Evaluation (STEV 2007), IEEE Computer Society, 2007, pp. 404-409.
  3. Ralph Guderlei, Johannes Mayer, Christoph Schneckenburger, Frank Fleischer:
    Testing Randomized Software by Means of Statistical Hypothesis Tests. In: Proceedings of the Fourth International Workshop on Software Quality Assurance (SOQUA 2007), ACM Press, New York, NY, USA, 2007, pp. 46-54.
  4. Johannes Mayer, Ralph Guderlei:
    On Random Testing of Image Processing Applications.
    In: Proceedings of the 6th International Conference on Quality Software (QSIC 2006), IEEE Computer Society, 2006, p. 85-92.
  5. Johannes Mayer, Ralph Guderlei:
    An Empirical Study on the Selection of Good Metamorphic Relations.
    In: Proceedings of the 30th Annual International Computer Software and Applications Conference (COMPSAC 2006), IEEE Computer Society, 2006, p. 475-484.
  6. Jürgen Bohrmann:
    On Random Testing and Test Oracles in the Context of Credit Risk.
    Diploma Thesis, Ulm University, January 2006.
  7. Johannes Mayer:
    Towards a Reliable Statistical Oracle and its Applications.
    In: Proceedings of 22nd GI-TAV Meeting, Softwaretechnik-Trends 25 (1), 2005, pp. 21-27.
    [pdf]
  8. Johannes Mayer:
    On Testing Image Processing Applications with Statistical Methods.
    In: Proceedings of Software Engineering 2005, Lecture Notes in Informatics P-64, Gesellschaft für Informatik, Köllen Druck+Verlag GmbH, 2005, pp. 69-78. © Gesellschaft für Informatik
  9. Johannes Mayer, Ralph Guderlei:
    Test Oracles Using Statistical Methods.
    In: Proceedings of the First International Workshop on Software Quality (SOQUA 2004) in Proceedings of the Net.ObjectDays 2004, tranSIT GmbH, Ilmenau, Germany, September 2004, pp. 85-95. (ISBN 3-9808628-3-6)
    Also in: Lecture Notes in Informatics P-58, Gesellschaft für Informatik, Köllen Druck+Verlag GmbH, 2004, pp. 179-189. © Gesellschaft für Informatik
    [pdf]