Rapidly scanning the Internet has become vital to efforts to keep it secure.
When a major flaw in the encryption that secures websites was revealed this March, Zakir Durumeric,
a research fellow at the University of Michigan, was the first person
to know how serious it was. By performing a scan of every device on the
Internet, he realized its full potential even before the researchers who
had first identified the flaw, known as FREAK.
“There were questions as to the correct way to respond before we did the scan,” says Durumeric.
The
scan showed that more than five million sites were affected, including
those operated by the FBI, Apple, and Google. Facebook’s like button, a
fixture on many popular sites, was also vulnerable. The results prompted
an urgent, careful effort to inform key companies and organizations
before the problem was announced publicly.
The
FREAK flaw allows an attacker to break a secure connection between a
Web browser and a vulnerable site, gaining access to encrypted data sent
between the two. The attack works by forcing a site to fall back to a
weak form of encryption mandated by the U.S. government in the 1990s.
Durumeric leads a team of researchers at the University of Michigan that has developed scanning software called ZMap.
This tool can probe the whole public Internet in under an hour,
revealing information about the roughly four billion devices online. The
scan results can show which sites are vulnerable to particular security
flaws. In the case of FREAK, a scan was used to measure the scale of
the threat before the bug was publicly announced.
The ZMap team was contacted by Matthew Green,
an assistant professor at Johns Hopkins University who had been alerted
to FREAK by its discoverers, a team of researchers from Microsoft, the
French Institute for Research in Computer Science and Automation, and
Madrid’s IMDEA Software Institute.
Green
says the scan results helped him decide who needed to be tipped off,
ensuring the announcement wouldn’t leave large swaths of the Internet at
risk. “We haven’t had really good data like this before,” says Green.
“You can find out exactly who’s broken, and tell people exactly how bad
something is. It was when Zakir did that scan I knew this was bad.”
Durumeric
and colleagues developed ZMap late in 2013. Before that, the software
used to scan the Internet took weeks or months to finish the job.
“Existing tools were a thousand times too slow,” says Durumeric.
The
first high-profile project for ZMap was tracking the impact of the
Heartbleed bug, a flaw in a widely used piece of Web encryption software
found in April 2014 (see “Many Devices Will Never Be Patched to Fix Heartbleed”). The researchers scanned regularly for systems vulnerable to the bug, and published a site listing the most popular unpatched websites along with information on how to fix the problem.
Durumeric
says this effort helped pressure companies into fixing their systems.
The group even sent automated e-mails informing companies that they had
vulnerable infrastructure and offered guidance on what they should do.
Controlled experiments showed that the notifications made a measureable
difference, says Michael Bailey, a professor at the University of Michigan who also works on the project.
The
team plans to issue similar notifications for FREAK soon. It is also
using scans to track how long it takes for FREAK and similar major flaws
to be mopped up. Almost a year after Heartbleed’s disclosure, says
Durumeric, about 1 percent of the top one million websites are still
vulnerable to it.
One reason well-known bugs linger is that companies fail to realize the extent of the problem, says HD Moore, chief research officer with security company Rapid7.
Moore uses ZMap for his own scans. “Most enterprises are completely
unaware of at least 10 percent of their assets on the public Internet,”
he says. ZMap scans can help companies find vulnerable infrastructure.
Moore began scanning the Internet using software of his own design in 2012 (see “What Happened When One Man Pinged the Whole Internet”). He now runs a more formal scanning project at Rapid7, using ZMap as well as tools developed inside the company.
Green
says that Google has also begun to perform its own Internet scans. The
results are used to program the Chrome browser to connect more
cautiously with sites that pose potential security risks, he says.
However,
tools like ZMap can’t find everything. The software works by
systematically contacting every possible numerical address for Internet
devices using the most commonly used protocol, called IPv4. That misses
the tiny but growing fraction of devices using addresses under a newer
system called IPv6, which has too many possible addresses to scan
comprehensively. ZMap’s scans also can’t reach inside private networks,
such as corporate intranet sites, or devices on mobile networks.
Still,
Green says, ZMap and other scanning software provides a much needed, if
sometimes gloomy, picture of the state of Internet infrastructure.
“We’re getting better all the time, but from a very bad place,” he says.