29-30 March 2011, Area della Richerca CNR, Pisa.

Antonia Bertolino, ISTI CNR Pisa, Italy
Paul Clements, SEI, USA
Paola Inverardi, University of L'Aquila, Italy
Henry Muccini, University of L'Aquila, Italy

Contact: ast core mailing list

Workshop on Architecture-Based Testing: Moving Research into Practice

Architecture support for testing refers to using a system’s architecture to inform and guide its testing process. While there has been substantial research devoted to this topic, little, if any, of this research has filtered into practice. Hence, the promise of architecture support for testing – to use architecture to reduce the time and expense of testing and to increase its effectiveness – remains unfulfilled.

This workshop is the second of a series of events on architecture support for testing which we organize on the topic. While the first workshop (held at the SEI in Pittsburgh on February 1-2, 2011, and titled “Workshop in Architecture-Based Testing: Best Practices, Best Potential”) was oriented to practitioners and identified industrial challenges in architecture support for testing, the one organized in Pisa targets (academic) researchers and seek to identify research-driven solutions to the problems identified during the practitioners workshop.

Thus the primary goals of this workshop are to:

• Help answering this question: How can research help (improve) architecture support for testing our systems?
• Build an agenda for future work

The main input of this workshop is a set of 29 “model problems” in architecture support for testing produced during the practitioners' workshop hold in Pittsburg. A model problem is a problem that, if solved, would result in a significant decrease in project resources devoted to testing and/or a significant increase in system quality given an expenditure level.

Those 29 model problems are the output of the four working groups created during the previous workshop, that are: AST and Requirements (five model problems), AST and Software Product Line (seven model problems), Scope of Architecture and AST (two model problems), and AST and Integration testing (sixteen model problems). Model problems have been eventually prioritized by the industrial participants.

The prioritized list of model problems is available here

The Pisa workshop has been organized in this way:

1. some few presentations have been run to welcome the participants, and to describe the subject of this working meeting
2. some of the highly ranked model problems (see file) have been presented and discussed. A few representative model problems have been selected for each area, and more specifically: REQ 1, SPL 1, and INT 16.
3. three working groups (WG) have been formed, one for each model problem. The goal of each WG has been to define:
   • SOTA of the selected model problem
   • Research questions (novelty, challenges, issues)
   • An approach as concrete as possible
4. on day two, the three WGs have shared their findings, and found a common case study to be used to instantiate what discussed. The case study is provided by the SEI, and is available at the SEI ppl page

The workshop's output has been:

• a presentation from each WG, covering the SOTA, identification of challenges and issues, and initial definition of potential solutions of their respective model problem(s).
• a post-workshop report from each WG
• so far (May 05, 2011), two papers have been submitted for publication. Others are under development

If you may be interested to see those documents, please send an email to Henry Muccini.

• Workshop at the SEI (Febr. 2011)
• Charette session at AST 2011
• WICSA 2011 workshop on "Architecture-Based Testing and System Validation Workshop"
• A Special Issue on Architecture-Based Testing has been agreed with the Computing Journal of Springer.

A mailing list is being created for all of you interested to keep in touch on this topic. If you might be interested to be added, please contact Henry Muccini.

Ranking of Architecture Support for Testing (AST) Model Problems

Selected Model Problems

Paola Inverardi

Paul Clements

Henry Muccini

MJ Escalona

Charette Session at AST 2011