Botnet Metrics: Learning from Meteorology

As ISP’s continue to spin up their anti-botnet defenses and begin taking a more active role in dealing with the botnet menace, more and more interested parties are looking for statistics that help define both the scale of the threat and the success of the various tactics being deployed. But, as I discussed earlier in the year (see “Household Botnet Infections“), it’s not quite so easy to come up with accurate infection (and subsequent remediation) rates across multiple ISP’s.

To overcome this problem there are several initiatives trying to grapple with this problem at the moment – and a number of perspectives, observations and opinions have been offered. Obviously, if every ISP was using the same detection technology, in the same way, at the same time, it wouldn’t be such a difficult task. Unfortunately, that’s not the case.

One of the methods I’m particularly keen on is leveraging DNS observations to enumerate the start-up of conversations between the victim’s infected device and the bad guys command and control (C&C) servers. There are of course a number of pros & cons to the method – such as:

• DNS monitoring is passive, scalable, and doesn’t require deep-packet inspection (DPI) to work [Positive],
• Can differentiate between and monitor multiple botnets simultaneously from one location without alerting the bad guys [Positive],
• Is limited to botnets that employ malware that make use of domain names for locating and communicating with the bad guys C&C [Negative],
• Not all DNS lookups for a C&C domain are for C&C purposes [Negative].

On the top of all this lies the added complexity that such observations are conducted at the IP address level (and things like DHCP churn can be troublesome). This isn’t really a problem for the ISP of course – since they can uniquely tie the IP address to a particular subscriber’s network at any time.
One problem that persists though is that a “subscriber’s network” is increasingly different from “a subscriber’s infected device”. For example, a subscriber may have a dozen IP-enabled devices operating behind their cable modem – and it’s practically impossible for an external observer to separate one infected device from another operating within the same small network without analyzing traffic with intrusive DPI-based systems.

Does that effectively mean that remote monitoring and enumeration of bot-infected devices isn’t going to yield the accurate statistics everyone wants? Without being omnipresent, then the answer will have to be yes – but that shouldn’t stop us. What it means is that we need to use a combination of observation techniques to arrive at a “best estimate” of what’s going on.

In reality we have a similarly complex monitoring (and prediction) system that everyone is happy with – one that parallels the measurement problems faced with botnets – even if they don’t understand it. When it comes to monitoring the botnet threat the security industry could learn a great deal from the atmospheric physicists and professional meteorologists. Let me explain…

When you lookup the weather for yesterday, last week or last year for the 4th July, you’ll be presented with numerous statistics – hours of sunshine, inches of rainfall, wind velocities, pollen counts, etc. – for a particular geographic region of interest. The numbers being presented to you are composite values of sparse measurements.

To arrive at the conclusion that 0.55 inches of rainfall fell in Atlanta yesterday and 0.38 inches fell in Washington DC over the same period, it’s important to note that there wasn’t any measurement device sitting between the sky and land that accurately measured that rainfall throughout those areas. Instead, a number of sparsely distributed land-based point observations specific to the liquid volume of the rain were made (e.g. rain gauges), combined with a number of indirect methods (e.g. flow meter gauges within major storm drain systems), and broad “water effect” methods (e.g. radar) were used in concert to determine an average for the rainfall. This process was also conducted throughout the country, using similar (but not necessarily identical) techniques and an “average” was derived for the event.

That all sounds interesting, but what are the lessons we can take away from the last 50 years of modern meteorology? First and foremost, the use of accurate point measurements as a calibration tool for broad, indirect monitoring techniques.

For example, one of the most valuable and accurate tools modern meteorology uses for monitoring rainfall doesn’t even monitor rain or even the liquid component of the droplets – instead it monitors magnetic wave reflectivity. Yes, you guessed it – it’s the radar of course! I could get all technical on the topic, but essentially meteorological radar measure the reflection of energy waves from (partially) reflective objects in the sky. By picking the right wavelength of the magnetic wave from a radar, it gets better at detecting different sized objects in the sky (e.g. planets, aircraft, water droplets, pollutant particulates, etc.). So, when it comes to measuring rain (well, lots of individual raindrops simultaneously to be more precise), the radar system measures how much energy of a radar pulse was returned and at what time (the time component helps to determine distance).

Now radar is a fantastic tool – but by itself it doesn’t measure rainfall. Without getting all mathematical on you, the larger an individual raindrop the substantially bigger the energy reflection – which means that a few slightly larger raindrops in the sky will completely skew the energy measurements of the radar – meanwhile, the physical state of the “raindrop” also affects reflectivity. For example, a wet hailstone reflects much more energy than an all-liquid drop. There are a whole bunch of non-trivial artifacts of rainfall (you should checkout things like “hail spikes” for example) that have to be accounted for if the radar observations can be used to derive the rainfall at ground level.

In order to overcome much of this, point measurements at ground level are required to calibrate the overall radar observations. In the meteorological world there are two key technologies – rain gauges and disdrometers. Rain gauges measure the volume of water observed at a single point, while disdrometers measure the size and shape of the raindrops (or hail, or snow) that are falling. Disdrometers are pretty cool inventions really – and the last 15 years have seen some amazing advancements, but I digress…

How does this apply to Internet security and botnet metrics? From my perspective DNS observations are very similar to radar systems – they cover a lot of ground, to a high resolution, but they measure artifacts of the threat. However those artifacts can be measured to a high precision and, when calibrated with sparse ground truths, become a highly economical and accurate system.

In order to “calibrate” the system we need to use a number of point observations. By analogy, C&C sinkholes could be considered rain gauges. Sinkholes provide accurate measurements of victims of a specific botnet (albeit, only a botnet C&C that has already been “defeated” or replaced) – and can be used to calibrate the DNS observations across multiple ISP’s. A botnet that has victims within multiple ISP’s that each observe DNS slightly differently (e.g. using different static reputation systems, outdated blacklists, or advanced dynamic reputation systems), could use third-party sinkhole data for a specific botnet that they’re already capable of detecting via DNS, as a calibration point (i.e. scaling and deriving victim populations for all the other botnets within their networks).

Within their own networks ISP’s could also employ limited scale and highly targeted DPI systems to gauge a specific threat within a specific set of circumstances. This is a little analogous to the disdrometer within meteorology – determining the average size and shape of events at a specific point, but not measuring the liquid content of the rainfall directly either. Limited DPI techniques could target a specific botnet’s traffic – concluding that the bot agent installs 5 additional malware packages upon installation that each in turn attempt to resolve 25 different domain names, and yet are all part of the same botnet infection.

Going forward, as ISP’s face increased pressure not only to alert but to protect their subscribers from botnets, there will be increased pressure to disclose metrics relating to their infection and remediation rates. Given the consumer choice of three local ISP’s offering the same bandwidth for the same price per month, the tendency is to go for providers that offer the most online protection. In the past that may have been how many dollars of free security software they bundled in. Already people are looking for proof that one ISP is better than another in securing them – and this is where botnet metrics will become not only important, but also public.

Unfortunately it’s still early days for accurately measuring the botnet threat across multiple ISP’s – but that will change. Meteorology is a considerably more complex problem, but meteorologists and atmospheric physicists have developed a number of systems and methods to derive the numbers that the majority of us are more than happy with. There is a lot to be learned from the calibration techniques used and perfected in the meteorological field for deriving accurate and useful botnet metrics.