Article Preview
TopIntroduction
Unified Modelling Language (UML) sequence diagrams graphically describe system component interactions, arranged in time sequence (OMG Unified Modelling Language, 2017). The temporal nature of sequence diagrams makes them suitable for capturing functional scenarios. System components in sequence diagrams can vary from higher-level entities, such as users and software systems, to lower-level entities, such as individual objects and methods, allowing the specification of functional scenarios at all levels of design. Although sequence diagrams are widely used to describe both higher and lower-level functional scenarios, their informal and graphical format limits their usability in the software development life cycle. This work aims to increase their usability by proposing a method to utilize sequence diagrams in test sequence generation and functional coverage level measurement. A test sequence is an ordered list of elements, where elements are in the alphabet of the system’s set of states and events. They are used to express the desired transitions for the system under test. Test sequences are materialized by test inputs, where a test input is an ordered list of elements from the alphabet of possible system inputs. For a deterministic system in a given state, application of a test input results in a certain test sequence.
The proposed method shows that sequence diagrams can be transformed into equivalent formal and graph-based event sequence graphs (ESGs) (Belli, 2006), and their equivalent ESGs can be utilized to define formal coverage criteria for testing of functional scenarios and allow measurement of coverage levels of such criteria. This is one of the novelties presented in this work; another is the process of test sequence generation to satisfy specific coverage criteria. The proposed method utilizes existing methods for test sequence generation from ESGs on a per-sequence diagram approach to achieve a set of efficient and effective test inputs. For evaluation, the proposed method is applied to the one and only found public project with UML sequence diagrams and executable acceptance test cases. The novelty of this work enabled the evaluation to compare the test sequences that are generated by users to the ones that are automatically generated via a transformation from sequence diagrams in terms of effectiveness and efficiency.
This paper makes the following contributions: (1) Method – The proposed method transforms sequence diagrams into equivalent ESGs and utilizes them to define test coverage criteria for functional scenarios and to measure coverage levels of test inputs for these criteria. Equivalent ESGs are also used to generate efficient and effective test sequences to satisfy defined coverage criteria. This work validates the proposed transformation and test sequence generation method on more than 80 public repositories for a wide range of different types of projects. Additionally, this work compares the test sequences generated by the proposed method with the ones generated by a user on an existing project, contemplating its applicability in practice. (2) Tool – Two tools were developed to implement the method explained in (1). The tools are shared in public repositories.
The first part of the proposed approach, namely the transformation of sequence diagrams and generation of the test sequences, imposes no requirement on the inputs, other than they be in the PlantUML syntax (as it is a widely used format). This is further demonstrated by the newly added validations on over 80 projects. The second part of the approach, namely the measurement of coverage levels, requires function and class names used in the sequence diagrams and the code itself to match, to be able to map them accurately. Ideally that should be the case (as the opposite would imply a mismatch between the design and implementation, which would result in lower coverage levels); however how well this is followed in industrial projects is a valid point. The scope of the second part of the approach focuses on function-call-based coverage criteria (e.g., call coverage or call-pair coverage), as this pairs well with sequence diagrams using functions as their smallest and fundamental unit (within message bodies), and with parameter types and values often omitted. In the future, research will extend the approach with coverage types involving parameter values and perhaps utilizing frame guard expressions.
The manuscript is organized as follows: Section 2 presents the fundamentals of the concepts used throughout this work. Section 3 explores the related works in the literature. Section 4 and 5 explain the proposed method. Section 6 describes the developed software tools implementing the proposed method. Section 7 tests and evaluates the proposed method. Threats to validity are discussed in Section 8, and Section 9 concludes the paper.