Team’s New Tool Advances the Art of Busting Hidden Software Bugs
One of the biggest challenges to fixing software bugs can be finding them.
With support from the National Science Foundation, computer scientists at The University of Texas at Dallas are going after some of the hardest-to-find errors, called variability bugs, which appear only when software is configured to work with certain hardware.
The researchers presented a framework they developed to detect variability bugs at the recent Association for Computing Machinery (ACM) Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering in Estonia. Variability bugs cannot typically be detected using off-the-shelf software analysis tools, said Dr. Shiyi Wei, assistant professor of computer science in the Erik Jonsson School of Engineering and Computer Science.
“It is hard to test and analyze software for variability bugs because of all of the different hardware involved,” Wei said. “You have to test many software configurations to find these bugs, and it’s not possible to test all the configurations.”
A software program can have millions of configurations that allow it to operate among different models and brands of computers and devices. Typical software analysis tools test only one configuration at a time, said Austin Mordahl, a software engineering PhD student and computer science research assistant working on the project.
“The analogy I use is that it’s like ordering a pizza, where the initial code base for a program is the entire palette of topping options you have available at the beginning, and the final product contains selected elements. But off-the-shelf tools are only able to analyze the finished pizza,” Mordahl said. “So, if you don’t select the part of the code that has a bug in it to be included in the final product — let’s say you skipped the anchovies — then no matter how good your off-the-shelf tool is, it will never find the issue because the bug simply doesn’t exist in your executable.”
The researchers tested 1,000 configurations across three programs, for a total of 3,000 configurations, in an attempt to identify variability bugs. The project detected and confirmed 77 bugs, 52 of which were variability bugs.
Mordahl won the ACM Student Research Competition at the International Conference on Software Engineering in May for a paper describing this research titled, “Toward detection and characterization of variability bugs in configurable C software: an empirical study.”
Millions of people rely on highly configurable software, yet these systems lack adequate automated tools to keep them secure and reliable, Wei said. He and fellow researchers plan to continue to develop and improve their framework and hope their data set can support future research on highly configurable code analyses.
“Configured software is one of the most common types we use,” Wei said. “That’s why it’s so important to improve the quality of this software.”
The UT Dallas researchers collaborated on the project with computer scientists at The University of Texas at Austin, the University of Maryland and the University of Central Florida.
A version of this story also ran in News Center.