The State of Software Engineering Research

At the 2016 Foundations of Software Engineering conference, a panel discussed the state of software engineering research. The panelists were asked to reflect on three questions.

• Do you believe our community as a whole is achieving the right balance of science, engineering, and design?
• What new or existing areas of research do you think our community should pay more attention to?
• Do you have novel suggestions for how we could improve our research methods to increase the impact of software engineering research in the near and distant future.

Tao Xie (University of Illinois at Urbana-Champaign), said he would like to see more investment in architecture. He would also like to see more of software engineering research impact on research communities beyond software engineering. Past examples of this include symbolic execution, delta debugging, and REST; all originated within software engineering but their effect extends beyond it.

Laurie Williams (North Carolina State University) discussed two areas of research that impact us all: cybersecurity and continuous deployment. She asked the audience to consider its work in a sea of software insecurity and in an environment where software is deployed daily.

Peri Tarr (IBM Research) started by presenting the business lifetime of a good idea. It starts as a competitive differentiator, which yields increased profit. There investment results in value creation. Then the idea gets commoditized and becomes a cost of doing business. There spending lowers profits as a necessary expense. She claimed that software has crossed the line from being a competitive differentiator to the cost of doing business. On the other hand, many high-impact, societally important, value-creation areas are coming of age. These include cognitive computing, heath care, environment, education, quality of life, economics, security. In all these areas critical problems and solutions are in their intersection with software engineering. As researchers we are privileged to be in a position where we can take big risks in order to claim big rewards, she concluded.

Prem Devanbu (University of California at Davis) identified three lines of research: Rationalism (following the tradition of Gottfried Leibniz), language games (Ludwig Wittgenstein), and Empiricism (John Locke). Under rationalism software is evaluated using formal methods, whereas under empiricism the correctness of software is evaluated in practice. Lately progress is being achieved by combining the two approaches, for example looking at combinations of static and dynamic typing.

Lionel Briand (University of Luxembourg) lamented the gap between academic research and the needs of the practitioners. This is evidenced by the few practitioners that attend academic conferences, such as ICSE. He suggested that this can be addressed by more collaborative research. By treating software engineering as a self-standing discipline, rather than a branch of computer science, we should be looking at practicality and scalability considerations right from the start.

The moderator, Margaret-Anne Storey, asked whether industry should solve its own problems. Briand responded that industry lacks the freedom and time to take risks. Devanbu added that researchers can take a broader view, looking e.g. at whole ecosystems, rather than an individual project. Tarr mentioned that it takes time to gain industry’s trust; people in industry put their company at risk when they work with academics, and they object to being treated like lab rats. Academic researchers collaborating with industry should deliver solutions, even if these are not done in an innovative way, rather than abandoning them.

Storey then mentioned the risk of having engineering work rejected due to the lack of scientific rigour. Williams’s take was that the problem can often be addressed simply by tweaking the presentation to show the work in a scientific manner. Others added as aspects generalization and following the scientific method.

The discussion then opened to questions from the floor. Speaking from the floor, Daniel Jackson (MIT) argued that what has influenced the industry more is great software as exemplified by many small systems, such as Unix, Simula, SQL, and Visicalc.

Tim Menzies commented that any evolving organism has to have lots of parts in order to influence its futures. We can influence industrial practice through graduate students who go on to work in industry.

Another speaker from the floor suggested three barriers in collaboration with industry: the drive to publish, methods that a research can use, and access to data.

Briand commented that knowledge does not flow linearly from basic research, to applied research, to innovation, but also that innovation should inform applied research and that basic research.

Steve McKeever (Uppsala University) proposed that we can influence industry by teaching students in industrial masters programs. In that context students don’t ask “what do I need to pass the exam”, but demand knowledge that is really useful in an industrial context.

Tarr, in response to a tweet by Jane Cleland-Huang (University of Notre Dame) asking for ways we can foster more diverse areas of software engineering research, suggested that we should as a community establish different norms and bars for researchers who take the risk to collaborate with industry and can’t therefore offer a replication package or disclose the data. Such work should be part of the research track and not be delegated to the industry track.

Comments   Toot! Tweet Share