Rapidly scanning the Internet has become vital to efforts to keep it secure.
When a major flaw in the encryption that secures websites was revealed this March, Zakir Durumeric,
a research fellow at the University of Michigan, was the first person
to know how serious it was. By performing a scan of every device on the
Internet he realized its full potential, before even the researchers who
had first identified the flaw, known as FREAK.
“There were questions as to the correct way to respond before we did the scan,” says Durumeric.
Afterwards
there weren’t. The scan showed that more than 5 million sites were
affected, including those operated by the FBI, Apple, and Google.
Facebook’s like button, a fixture on many popular sites, was also
vulnerable. The results prompted an urgent, careful effort to inform key
companies and organizations before the problem was announced publicly.
The
FREAK flaw allows an attacker to break a secure connection between a
web browsers and a vulnerable site, gaining access to encrypted data
sent between the two. The attack works by forcing a site to fall back to
a weak form of encryption mandated by the U.S. government in the 1990s.
Durumeric leads a team of researchers at the University of Michigan that has developed scanning software called ZMap.
This tool can probe the whole public Internet in under an hour,
revealing information about the roughly four billion devices online. The
scan results can show which sites are vulnerable to particular security
flaws. In the case of FREAK, a scan was used to measure the scale of
the threat before the bug was publicly announced.
The ZMap team was contacted by Matthew Green,
an assistant professor at Johns Hopkins University who had been alerted
to FREAK by its discoverers, a team of researchers from Microsoft, the
French Institute for Research in Computer Science and Automation, and
Madrid’s IMDEA Software Institute.
Green
says the scan results helped him decide who needed tipping off,
ensuring the announcement wouldn’t leave large swathes of the Internet
at risk. “We haven’t had really good data like this before,” says Green.
“You can find out exactly who’s broken, and tell people exactly how bad
something is. It was when Zakir did that scan I knew this was bad.”
Durumeric
and colleagues developed ZMap late in 2013. Before that the software
used to scan the Internet took weeks or months to finish the job.
“Existing tools were a thousand times too slow,” says Durumeric.
The
first high-profile project for ZMap was tracking the impact of the
Heartbleed bug, a flaw in a widely used piece of Web encryption software
found in April 2014 (see “Many Devices Will Never Be Patched to Fix Heartbleed”). The researchers scanned regularly for systems vulnerable to the bug, and published a site listing the most popular unpatched websites along with information on how to fix the problem.
Durumeric
says this effort helped pressure companies into fixing their systems.
The group even tried sending automated emails informing companies they
had vulnerable infrastructure and offering guidance on what they should
do. Controlled experiments showed the notifications made a measureable
difference, says Michael Bailey, a professor at the University of Michigan who also works on the project.
The
team plans to start issuing similar notifications for FREAK soon. It is
also using scans to track how long it takes for FREAK and similar major
flaws to be mopped up. Today about one percent of the top one million
websites are still vulnerable to Heartbleed, says Durumeric, almost a
year after its disclosure.
One reason well-known bugs linger is that companies fail to realize the extent of the problem they have, says HD Moore, chief research officer with security company Rapid7,
who uses ZMap for his own scans. “Most enterprises are completely
unaware of at least ten percent of their assets on the public Internet,”
he says. ZMap scans can help companies find vulnerable infrastructure.
Moore began scanning the Internet using software of his own design in 2012 (see “What Happened When One Man Pinged the Whole Internet”). He now runs a more formal scanning project at Rapid7, using ZMap as well as tools developed inside the company.
Green
says that Google has also begun to perform its own Internet scans. The
results are used to program the Chrome browser to connect with sites
that pose potential security risks more cautiously, he says.
However,
tools like ZMap can’t find everything. The software works by
systematically contacting every possible numerical address for Internet
devices using the most commonly-used protocol, called IPv4. That misses
the tiny but growing fraction of devices using addresses under a newer
system called IPv6, which has too many possible addresses to scan
comprehensively. ZMap’s scans also can’t reach inside private networks,
such as corporate intranet sites, or devices on mobile networks.
Still,
Green says ZMap and other scanning software provides a much needed, if
sometimes gloomy, picture of the state of Internet infrastructure.
“We’re getting better all the time, but from a very bad place,” he says.