IST Researchers Aim To Help
Companies Fortify Cyber Defenses

In a threat report published by U.S. Director of National Intelligence, James Clapper last year, cyber attacks were listed first among global threats, above both terrorism and weapons of mass destruction. To combat those threats, companies have increasingly turned to ‘white hats’ – ethical hackers who expose vulnerabilities in computer systems to improve cybersecurity rather than compromise it. Researchers at Penn State’s College of Information Sciences and Technology (IST) are investigating the dynamics of these ‘bug bounty’ programs with the intention of helping organizations amp up their defenses against malevolent hackers.

“The major goal is to better understand this ecosystem,” said Jens Grossklags, Assistant Professor at IST.

Grossklags, along with Mingyi Zhao, a doctoral candidate at the College of IST, and Peng Liu, professor of IST, presented their paper ‘An Empirical Study of Web Vulnerability Discovery Ecosystems‘ last fall at the ACM Conference on Computer and Communications Security (CCS), the flagship annual conference of the Special Interest Group on Security, Audit and Control (SIGSAC) of the Association for Computing Machinery (ACM). Last January, they also presented their results to a public policy audience at the Federal Trade Commission’s PrivacyCon event to raise awareness about the challenges and opportunities of bug bounty programs.

According to the IST researchers, web security has become critically important for most organizations. Logically, the prevention of security compromises enabled by web vulnerabilities is increasingly gaining the attention of company leadership and the broader security community. Web vulnerabilities are the likely causes of many recent security breaches leading to massive disclosure of user data, leakage of business information and other losses.

While the researchers had previously investigated white hat behaviors, their current study is more comprehensive and examines web vulnerability discovery ecosystems. These ecosystems include businesses, organizations and white hats and third party vulnerability disclosure reward/bounty programs.

“These ecosystems have been growing rapidly and are becoming more prominent in the battle against malicious actors on the Internet,” Grossklags said. “We are trying to determine how to shape these ecosystems in a way that diminishes potential disadvantages and provides the maximum benefit.”

Research Results

In their work, Grossklags, Zhao and Liu conducted the first empirical study of two major web vulnerability discovery ecosystems, basing their analyses on publicly available data. The first data set stems from Wooyun, the predominant and likely the oldest web vulnerability discovery ecosystem in China. The second data set was collected from HackerOne, a US-based startup company that hosts bug bounty programs for hundreds of organizations, including Yahoo, Mail.ru and Twitter.

An important difference between Wooyun and HackerOne is the number of organizations that are involved. While HackerOne has 99 organizations running public bounty programs, Wooyun affects 70 times more organizations.

“This drastic difference can be explained by the difference in organization participation models,” he said. “For HackerOne and other similar US-based platforms such as BugCrowd and Cobalt, an organization has full control of its bounty program. It determines when to start, what are the rules and scope of vulnerability discovery, and whether to disclose the reports. We refer to this model as the white-hat-initiated model.”

On the other hand Wooyun and some other Chinese platforms give more control to the vulnerability researchers. White hats can submit vulnerability reports for almost any organization. The organization can claim the report and work with the white hat to fix the issue. However, if the organization fails to do so in 45 days, the report will go public.

Grossklags and Zhao said that the two types of ecosystems, HackerOne (organization-initiated), and Wooyun (white-hat-initiated) have different pros and cons. They found that the white-hat-initiated participation model covers a wider range of organizations, including government sites, and financial and educational institutions.

“To improve the coverage of bug bounty, we might want to give more control to the white hats,” Zhao said.

Zhao and Grossklags also discovered that many organizations in the Wooyun ecosystem are not prepared to deal with the reported vulnerabilities, particularly smaller websites. The white-hat-initiated model may increase risk for unprepared organizations since vulnerabilities with no response likely remain exploitable. Once a vulnerability becomes public knowledge, companies may feel mounting pressure to handle the situation.

“Eventually, it might build enough pressure that the companies are forced to react,” Zhao said.

In contrast, the researchers found that the organizations that run bounty programs on HackerOne are able to resolve vulnerability reports in most cases in a reasonable amount of time. Further, a large portion of public bounty programs on HackerOne show a decreasing trend of vulnerability reports over time, which suggests continuous improvements of cybersecurity of these participating organizations.

While Zhao and Grossklags discovered that monetary compensation of white hats on HackerOne increases their productivity significantly, they still observed many contributions to programs without bounties. Contributing to vulnerability reports increases white hats’ standing in their community, while also making the Internet more secure. In addition, the software engineering community and peer organizations can learn valuable lessons from vulnerability reports to avoid similar mistakes in the future.

One motivation of their study is to inform the public policy process as it affects vulnerability discovery. According to Zhao, the proposed new treaty of the Wassenaar Arrangement – a voluntary agreement among 41 countries that calls for regulating the knowledge of how to create “intrusion software” – can significantly limit vulnerability research and impede the development of bug bounty programs.

“Our study effectively shows that white hats make important contributions to cybersecurity, but they can be significantly limited by such initiatives,” Zhao said.

Moving forward he and Zhao will conduct more in-depth empirical and theoretical research on discovered vulnerabilities. A further key objective is to help consumers to make choices between website services based on their security. Vulnerability research can provide important input for such guidance.

Additional results from the research were presented at the TEDxPSU event.