论文标题
关于降低测试套件与回归测试策略之间的相互作用
On the Interaction between Test-Suite Reduction and Regression-Test Selection Strategies
论文作者
论文摘要
单元测试是用于软件开发的最具成熟的质量助剂技术之一。单位测试的一个主要优点是效率（即测试工作）和有效性（即故障检测概率）之间的可调节权衡。为此，已经提出了各种策略来利用这种权衡。特别是，在测试单个程序版本时，测试套件减少（TSR）减少了（可能是冗余）测试用例的数量。回归测试选择（RTS）选择测试案例以测试连续的程序修订。但是，当组合使用时，TSR和RTS可能会影响甚至妨碍彼此的表现。例如，特定程序版本在TSR期间丢弃的测试用例可能再次与RT相关。但是，找到两种策略的结合，导致在计划的整个版本历史上进行合理的权衡是一个悬而未决的问题。本文的目的是在效率和有效性方面更好地了解TSR和RT之间的相互作用。为此，我们提出了一个可配置的框架，称为C程序的自动单位测试后悔。该框架包括针对TSR和RTS的不同策略以及可能的组合。我们将此框架应用于主题系统的集合，提供了几种关键见解。首先，TSR几乎总是对RT的有效性产生负面影响，但对效率产生了积极影响。其次，向测试人员揭示连续程序版本之间程序修改的影响要比仅涵盖修改的代码零件的测试用例，但造成更多测试工作的效果要效率得多。
Unit testing is one of the most established quality-assurance techniques for software development. One major advantage of unit testing is the adjustable trade-off between efficiency (i.e., testing effort) and effectiveness (i.e., fault-detection probability). To this end, various strategies have been proposed to exploit this trade-off. In particular, test-suite reduction (TSR) reduces the number of (presumably redundant) test cases while testing a single program version. Regression-test selection (RTS) selects test cases for testing consecutive program revisions. However, both TSR and RTS may influence -- or even obstruct -- each others' performance when used in combination. For instance, test cases discarded during TSR for a particular program version may become relevant again for RTS. However, finding a combination of both strategies leading to a reasonable trade-off throughout the version history of a program is an open question. The goal of this paper is to gain a better understanding of the interactions between TSR and RTS with respect to efficiency and effectiveness. To this end, we present a configurable framework called RegreTS for automated unit-testing of C programs. The framework comprises different strategies for TSR and RTS and possible combinations thereof. We apply this framework to a collection of subject systems, delivering several crucial insights. First, TSR has almost always a negative impact on the effectiveness of RTS, yet a positive impact on efficiency. Second, test cases revealing to testers the effect of program modifications between consecutive program versions are far more effective than test cases simply covering modified code parts, yet causing much more testing effort.
