DGA-based Botnet Detection

There are essentially two types of magic in the world – those that deceive an audience in to believing something impossible has occurred, and that which is best defined by Arthur C. Clarke as “any sufficiently advanced technology is indistinguishable from magic”.

The antivirus industry is rife with the former – there’s no shortage of smoke and mirror explanations when it comes to double-talking “dynamically generated signature signature-less engines” and “zero false-positive anomaly detection systems”.

I’m going to introduce you to the later kind of magic. Technological approaches so different from traditional approaches that, for many folks out there in Internet-land, it’s indistinguishable from magic. More than that though, I’m going to try to explain how such techniques are reversing the way in which threat discovery has occurred in the past. However what I’m not going to do is to even try to explain a fraction of the math and analytics that lies behind that magic – at least not in a blog!
Oh where, oh where should we start?

Let’s begin, for arguments sake, by classifying malware as a tool; a weapon to be more precise. In the physical world it would be easy to associate “malware” with the bullets from a gun, and the gun in turn likened to perhaps a drive-by download site or a phishing email. In response to that particular physical threat, there are a number of technological approaches that have been deployed in order to counter the threat – we have metal detectors and x-ray machines to alert us to the presence of guns, sniffing technologies to identify the presence of explosive materials, CCTV and behavioral analysis systems to identify the suspects who may be hiding the gun.

A fundamental premise of this layered detection approach is that we’ve encountered the threat in the past and already classified it as bad – i.e. as a “weapon”. Gun equals bad, knife equals bad, metal corkscrew equals bad, and so on. Meanwhile everything else is assumed to be good – like an ostrich egg – until it happens to be used as a weapon (such as when “Russell pleaded guilty to assault using an ostrich egg as a weapon, assault, and breaching a protection order“) and inevitably some new detection technique is proposed to detect it.

Traditionally the focus has been on “preventing” the threat. In particular, detecting the presence of a known threat and stopping it from reaching its target. In the physical world, in general, the detection technologies are pretty robust – however (and it’s a big “however”), the assumption is that the technology needed to provide this prevention capability is ubiquitous, deployed everywhere it could potentially be needed, and that it works every time. Sure, at high value targets (such as airports) you’ll find such technology employing its optimal capability, elsewhere though (such as the doorway into your home) it’ll not be encountered. There are obvious parallels with the cyber-world here – except arguably the Internet-equivalent technologies are a little more ubiquitous, but considerably less capable in preventing the malware threat.

For the mob hitman, serial killer, or other kind of mass murderer, the threat of sparsely deployed metal detectors is an easily avoided problem. Subversion or avoidance of the detection systems is pretty easy and, more importantly, an appropriate choice of location negates the problem entirely. Even then, such a detection strategy, operated in isolation, isn’t a serious inhibitor for new murders. If such a technology exists to only detect the guns and bullets, but is not capable of providing attribution (e.g. this gun was used in 16 different murders over the last 2 weeks and the owner of this gun is the murderer), then the criminal only ever loses a tool each time they get caught – since the prevention technology is divorced from association with the victims (past or prospective).

But there’s an entirely different side to dealing with that kind of threat – and that’s the forensics element. While our not-so-friendly murderer can avoid the detection technologies, they’re much less capable of avoiding the evidence of violent past actions. Starting from the first murder, it is possible to build a case that points to a specific criminal by analyzing the components of the crime scene(s).
I know, the argument is that everything should be done to prevent the crime in the first place. That’s clearly very difficult in the physical world, but you’re basically living in a fantasy land of Goblins and Unicorns if you’re expecting it to work better in the cyber-world.

Which basically brings me to the discussion (and subsequent detection) of the latest generation of sophisticated malware – malware that uses domain generation algorithms (DGA) to locate their command and control infrastructure and upload their stolen data. Malware with this capability are designed to evade all those conventional prevention technologies and, once successfully deployed within their victim populace, evade all other methods of traffic filtering and takedown processes. Even if a malware sample is accidentally captured, its DGA capabilities will go undetected.
Detecting DGA-based malware is, as I implied earlier, both “magic” and a reversal of conventional prevention approaches. In order to detect DGA-based threats early on, you start with the victims first…

DGA-based malware use an algorithm to pick out candidate domain names in order to hunt for their prospective C&C servers. The vast majority of the domain names they’re looking for simply don’t exist. In the world of DNS, attempting to resolve a domain name that doesn’t exist will result in a “no such domain” (i.e. an NX) response from an authoritative DNS server somewhere down the line. So, in essence, DGA’s are noisy if you’re watching DNS activity – and lots of NX responses are a key feature of an infected host. Unfortunately, the average Internet-connected device typically tries to look up lots of things that don’t exist, and there’s often a lot of legitimate NX traffic which can disguise the flapping of the malware.

Assuming some kind of algorithmic basis to the domain candidates being created by the malware, you could suppose that it would be possible to develop a unique signature for them. If only it was that easy – the criminals are smarter than that. And you’re also assuming that you’ve already encountered a copy of the malware before in order to create a signature for that particular DGA-based malware.

Instead there’s a much better way – you monitor all your DNS traffic and NX responses, and you identify clusters of devices (or IP addresses) that are generating roughly similar domain name requests. This first pass will provide a hint that those devices share some kind of common ailment (without ever needing to know what the malware is or have ever encountered a sample before). In a second pass you focus upon identifying just the domain names that are structurally very similar across all the afflicted assets and classify that DGA based upon a number of features. Classifying the DGA makes it easier for attribution and uniquely tracking a new threat.

At this point you’re pretty much sure that you have a new malware outbreak and you know who all the victims are, and you can easily track the growth (or demise) of the botnet within your network. Unfortunately you’re also dealing with a brand new threat and you probably don’t have a malware sample… and that’s where an additional layer of analytics comes into play and more “magic” happens as you automatically begin to identify the domain names that are tried by the DGA-based malware that actually succeed and engage with the criminals server.

Let’s work through a simple example uncovered last week by Damballa Labs. Over the weekend one of our NX monitoring tools identified a few thousand IP addresses within our customer base that were generating clusters of NX DNS traffic very similar to the following:

9ftor9srimbna-q.com, 0sso151a47nztrxld6.com, 0ubpccgkvzrnng.com, dq2yl1zxcmvko2.com, wfj-5i5p4uhq8tylhiz.com, 1sj4rh1i5l8-4tca.com, zv7dfcgtusnttpl.com, om2air5ah5lisj7.com, qmyuvaftpgve2tzwhcjr.com, fndskzqmsob5r2bzby.com, f5p2vxn7dcdkujhbguqb.com, 6q5sigkfu3fl3q.com, zpbm3emcuosopagdttxi.com , trsmw3wceik79krk5.com, qtxfmfyulnnjpxwqo.com, ikh9w-3vdmlndafja.com, udf-szhubujmuhp1jj.com, 4v8hohrphizcas.com, nhb72anz8ifdyzgckqf.com, rcu4n0lzoghuj2.com, dklfebjexiabttkwvgos.com, fjg56xwoupqpdxr.com, drxyuezdllerpd.com, t407bqgh56jbkv4ua.com, 49ubufqnjzvhct.com, n7c3qfpslcjosy-5.com, nvnzihl6krkyfo8zp.com 

While you’d be hard pressed to write a “signature” for the domain names that wouldn’t cause false positives out the wazoo, the features of the combined victim’s traffic (frequency, distribution, commonality, etc.) work fine as a way of associating them to a shared new threat.

Armed with this knowledge, it is then possible to identify similarly structured domain names that were successfully resolved by the victims that also shared timing elements or appeared to alter the flow of NX domain responses. For example, if the DGA-based malware is designed to locate a “live” C&C server, once it’s found a server it probably doesn’t need to keep on looking for more and will likely stop generating NX domain traffic for a period of time.

Based upon our observations of this particular botnet outbreak, it was possible to identify the following C&C servers being operated by the criminals:

• ###.133. ###.247 – idpd1y###-elywj.com
• ###.133. ###.247 – k7-cwubgsrqj###rb.com
• ###.133. ###.247 – omz###1k1vgrqf.com
• ###.133. ###.247 – taqwucpzj###an.com
• ###.133. ###.247 – vhrey###-ooz6ig.com
• ###.133. ###.75 – o###pp1k1vgrqf.com
• ###.133. ###.75 – rm6dol7###cxje-ajl.com
• ###.133. ###.191 – id###yzib-e###j.com
• ###.133. ###.191 – k7-c###gsrqjebzrb.com
• ###.133. ###.191 – ###gpp1k1vgrqf.com
• ###.133. ###.191 – rm6dol###wcxje-ajl.com
• ###.133. ###.191 – taqwucpzj###an.com
• ###.133. ###.191 – vhreyveh-ooz###.com

[NOTE: We've temporarily obfuscated some of this data while we continue to investigate and enumerate the global pool of victims. We'll release the technical details of this particular DGA-based botnet soon…]


So, by this point we know who the victims are, how many C&C servers the criminals are operating, what their IP addresses are and, subsequently, which hosting facilities they are operating from:

• AS13237 LAMBDANET-AS Lambdanet Communications Deutschland GmbH

What about the malware piece? While we know who the victims are, it would be nice to know more about the tool that the criminals behind this botnet prefer and ideally to get to know them more “personally” – if you know what I mean…

As it happens, there are some malware samples that have been discovered in the very recent past that also like to contact C&C’s running upon the same hosts at this facility. For example:

• d977ebff137fa97424740554595b9###

Fortunately, while the malware sample wasn’t detected by any antivirus products, it had previously been automatically executed within our dynamic analysis environment and we’d already extracted all of its observable network features, including an additional (successful) C&C engagement:

• m8###ecc9lsnks6kbcrv7.com using ###.133. ###.75, ###.133. ###.191, and ###.23. ###.139

This, in turn, helped identify the following additional hosting facility based in the Netherlands:

• AS49981 WORLDSTREAM WorldStream

There’s obviously much more to this particular threat, and if you’d like to get involved digging into it and helping with the attribution please let me know…

But, getting back on track, this approach in identifying brand new threats – while sounding like magic to many – works really well! We’ve found it to be immensely scalable and fantastically accurate.

However, there’s one fly in the ointment (as it were)… the approach identifies new threats long before the malware component is typically uncovered by the security community and is independent of the the vector used to infect the victims,the malware tool that was deployed and ultimately the actual endpoint device type.

Think of it this way. Based upon the forensics of the blood splattered bodies and evidence left in the room that the murderer left behind, we know that the victims were bludgeoned to death with an ovoid object approximately 6 inches in diameter, weighing 3 pounds and composed of a calcium shell. We also know that the murderer was 5 foot 11 inches, weighed 240 pounds and wears size 11 Reeboks.

You can keep your metal detectors for all the good that’ll do in this case…

Security: The Art of Compromise

“Security is all about protecting the user.” That’s the comment that came up the other week in the twittersphere that kicked off a not-unexpected trail of pro and con tweets.

Being limited to 140 characters makes it rather difficult to have a deep and meaningful discussion on the topic and the micro-blogging apparatus isn’t particularly conducive to the communicating the nuances of a more detailed thought. So I thought I’d address the topic here in blog format instead.

I suppose my first thought is that Internet security isn’t necessarily about protecting the user, in fact I’d go as far as saying that modern security approaches increasingly assume that the user themselves is the threat. In the most optimistic case security is about protecting assets (digital or physical) from harm or theft. Failing that, Internet security is about detecting change to assets that were deemed to merit protection.

As a community we tend to over use safety analogies when we’re trying to communicate the significance of a threat and the value of protection – which is why I believe there’s a prevailing assumption that Internet security is about protecting the user. For example, I’ve often heard (and abused it myself) the car analogy for putting defense in depth in to perspective – i.e. airbags, safety belts, crumple zones, etc. being metaphors for anti-virus, IDS and firewalls.

I think a more appropriate analogy for modern Internet security practices is that of protecting a bicycle. The cyclist is the user, and by protecting the bike itself we’re not actually doing much for the safety of the rider. In fact I’ll argue that over-protecting the bike may end up decreasing the safety of the cyclist – as we too often see in the cyber world (e.g. the lock’s so big & heavy that it affects the cyclists ability to actually ride the bike). By way of problem statement, we can consider the cyclist as a consumer of the technology (i.e. the bicycle) and, for him to be able to ride, he needs to ensure that his bike hasn’t been stolen or damaged.

When it comes to “protecting” the bike, there are a number of factors the would-be cyclist needs to take into account. Likely the most important concern is going to be how to lock-up the bike when not in use – especially when away from home. The most obvious solution is to purchase a dedicated bicycle lock.

Now this is where I think the analogy works better than most for the Internet security world… what are some of the deliberations the cyclist must make in selecting an appropriate protection solution?

• How big a lock do I need? A small lock can be trivially overcome, but is light and easy to carry on longer rides. A big heavy lock will likely be much harder to overcome, but is going to be troublesome to carry.
• How long a chain? A short chain is easier to carry. Meanwhile a longer chain offers me the flexibility to also lock up the wheels and wrap around bigger objects.
• How much do I want to spend? Some top-quality locks are almost as expensive as the bicycle they’re destined to protect. Meanwhile a more expensive lock may be lighter and more proficient at keeping thieves away.

Deciding upon a protection solution is a practice in compromise – comparing the risk likelihood with the inhibitors of the proposed solution. There’s also awareness that no matter how big and badass the lock may be, there’ll always be someone out there with more powerful bolt-cutters or a more imaginative way of subverting the lock. It may be a compromise, but hopefully it is an informed decision. The cyclist opts for a solution, forks out the money, and lives with the decision. If all goes to plan, their bicycle will be present the next time they go to use it.

The same applies to the Internet security world. You can’t protect against all the threats and, even if you could, you’d likely end up making the system you’re trying to protect unusable for the folks that need to use it.

But “protection” is only one side of the security coin. “Detection” is a critical element of modern security. Some may argue that detection is something you aim to do if you can’t protect. I’d have to politely disagree.

Locking up your bike is a realistic security solution if you’re looking to protect it – but ensuring that your bike is locked up somewhere highly visible (good lighting, etc.) and located where a potential thief is likely to be noticed by the cyclist and other passerby’s is a critical “detection” component. The threat of detection becomes part of the security strategy. Even if that deterrent fails, and the protection was also insufficient, the sooner the cyclist knows whether their bicycle has been stolen or tampered with, the quicker they can respond to the threat and take the corresponding actions.

Detection within the Internet security realm is as important as protection. For the last decade or so the emphasis upon security has been protection – despite acknowledging the limits of the compromise situations and product vulnerabilities. Knowing precisely when an asset has become the focus of a would-be thief or eventually succumbing to the threat is critical in how an organization must respond to the incident.
As anyone who has had a bike stolen will tell you, the quicker you notice it’s gone, the higher the probability you have of getting it back.

Corrupted Files for Sale

I stumbled across an interesting site earlier today - ideal for all those folks needing a little extra time beyond a hard-deadline for submission. The premise is an improvement over the "my dog ate my homework", in that you submit a corrupted file on the submission date - it takes the receiver some time to figure out something was wrong - and you gain a few extra days as the recipient asks you to "resend" the submission file. Corrupted-Files.com offers these corrupted files for a small fee.

How it works:

1. After purchasing a file, rename the file e.g. Mike_Final-Paper.
2. Email the file to your professor along with your "here's my assignment" email.
3. It will take your professor several hours if not days to notice your file is "unfortunately" corrupted. Use the time this website just bought you wisely and finish that paper!!!
   

So, for a fee of $3.95 you can purchase a corrupted file (DOC, XLS, PPT). But, in the words of the sites author...

"Please Note: I had deadlines with professors too, but I still got my sh*t done on time - its called Red Bull. If you need an extension, just be honest and ask your professor before you use a corrupted file."

If you can't protect it, you'd better be able to detect it!

The security trend over the last half-decade has been towards "protection" and we've seen technologies such as IDS morph in to IPS and network sniffing evolve in to DLP.

What I find amusing/worrying is that this laser focus on protection means that organizations have increasingly dropped the ball where it comes to threats that currently have no protection solution on the market. Basically, an attitude of "if I can't protect against it, then I don't want to know about it" has become prevalent within the security industry.

So, on that note, I found it refreshing to read the brief story over at Dark Reading How To Protect Your Organization From Malicious Insiders by Michael Davis. It's been a long standing mantra of mine that "If you can't protect it, you'd better be able to detect it!"

The 'Insider Threat' is one of the more insidious threats facing corporates today (especially in economic turmoil) and there really are so many ways for a knowledgeable employee to screw things up if they wanted to. I've had to do a mix of forensics and internal pentests within these areas in the past and it's always a potential playground of carnage.

But it's a little distressing to me that with the global sales push on DLP solutions many organizations have essentially thrown away their common sense. What I've observed is that enterprises that were initially deeply concerned about the potential of insider threat jumped heavily on to the DLP bandwagon and see this class of security technology as a way of over coming the threat. Then once they've deployed the DLP solution it's as if a mental box is ticked - "insider threat = solved" - and they move on to their next priority.

The problem is that DLP sucks as a protection system against the real insider threat and its rollout within an enterprise can be a substantial distraction to security & audit teams responsible for tracking the threat. Add to that the fact the executive support for further insider threat protection strategies quickly wanes after DLP has been rolled out -- "DLP = job done".

DLP will help identify (and block) many clear-text data leakage routes from an enterprise, however it'll do nothing against an insider that backdoors a server or Easter-eggs a DB to self destruct in a couple of weeks time - yet the mindset is that an investment has been made in DLP, and that since these kinds of insider threats can't be "protected" against, it's a problem too tough to solve (even though it may have been "solved" previously to the DLP solution - but that budget has now been used up - and DLP is supposed to reduce costs).

What ever happened to "detection"? As far as the insider threat goes, if you can't protect against it, you'd damn-well better ensure you can detect it. Failing that, I hope you're budgeting enough for post-attack disaster recovery and forensics.

Think of it this way. Say you're running a public library. You can bag check everyone that leaves the library to make sure they aren't stealing your books - and that's a wise precaution. But that doesn't mean you should skimp on the smoke detectors. The threat is "book loss" but there are clear differences between protection and detection strategies.