Static vs. Dynamic Reputation (or, why Blacklists suck more each day)

If you look deep enough, hidden at the darkest recesses of most security technologies deployed within enterprise networks today, you’ll find static reputation systems chugging away doing the grunt work of threat protection. They’re not glamorous and vendors have had a propensity to instruct their sales force (and resellers) to refrain from mentioning them to customers and prospects in recent years. They’re a legacy hangover from the days when cutting-edge security consisted of blacklists and regex signatures.

Static reputation systems are effectively frameworks for managing lists of previously classified goodness or badness – i.e. blacklists and whitelists. Their basic concepts are thoroughly understood and they tend to perform tremendously well as a first-pass filter for many of the most prevalent threat categories. So, despite their aged stature, they are an incredibly valuable tool. In fact, for many threat categories, modern protection products wouldn’t be able to handle traffic volumes if static reputation systems didn’t perform the first pruning of inbound threats. For example, in the world of Anti-Spam up-to-date blacklists of just a few hundred known bad IP addresses can reduce the spam volume that more sophisticated technologies must parse by 90+ percent.

There are however many limitations to static reputation systems. In a world of increasingly agile threats and a fundamentally dynamic (and some would say ‘chaotic’) Internet infrastructure, static reputation systems are simply incapable of keeping pace. Some short-term fixes have been applied – for example, releasing and importing updated blacklists more frequently, or pruning overly long blacklists to the most reliably static data in an attempt to remove “false positives”. Whilst these quick fixes have extended the life of some static reputation systems, the frayed edges have been exposed and are being constantly picked at.

In response to the failures and reducing viability of static reputation systems, a number of dynamic reputation system approaches have come to the fore in recent years. These new approaches seek to be more accurate in discerning goodness and badness, and to dynamically keep pace with agile threats and continuous Internet change.

Dynamic reputation systems aren’t a one-for-one replacement for systems currently dependent upon static reputation. While their protection objectives are similar, their output and delivery are quite different. Static reputation systems are effectively Boolean list technologies; the IP/Domain/URL/etc. is either on the list or it isn’t. Dynamic reputation systems typically operate as a queryable API and provide answers in a “score” format.

These scores can change at a moment’s notice as new intelligence relating to the IP/Domain/URL/etc. are received, features extracted and classified, and are derived in real-time. The scores themselves can often be interpreted as probabilities or confidence in a particular threat classification – and are delivered as values between zero and one, or as a percentage.

If you’re interested in learning more about the limitations of static reputation systems and how dynamic reputation systems have begun to replace them (and why), I’ve released a new reference paper on the topic – “Blacklists & Dynamic Reputation – Understanding Why the Evolving Threat Eludes Blacklists“- and it can be found on the Damballa website.