What we do
We are a research group in Aberdeen, Scotland, interested In performing large-scale network measurements of the Internet. Network measurement is a common tool for network research, ether to directly gather knowledge about network or traffic behavior in the wild or as a way to determine realistic input parameters for the evaluation of new proposed network and protocol mechanisms.
How we measure
This website provides resolved lists corresponding to the Alexa Topsites and Cisco Umbrella top 1M domains, the latest Citizen Labs and OpenDNS lists as well as scans of all IPv4 webservers in the Internet. These lists provide useful information for various network measurement tools, and can be used directly with PATHspider.
For domain lists, we use a parallelised DNS resolver, hellfire, located in Digital Ocean. The IPv4 scans are done with zmap, a fast single packet network scanner designed for Internet-wide network surveys, using ports 80 and 443 only. The resulting lists are in JSON format, and contain IP addresses (qne IPv4 address and 1 IPv6 address, if applicable), AS, Country code, prefix, name and rank for each domain.
The info associated with each domain is obtain from RIPEstat, using canid. The lists are updated every Tuesday evening.