Persistent Threat Detection (on a Budget)

If there’s one simple – high impact – thing you could do to quickly check whether your network has been taken over by a criminal entity, or uncover whether some nefarious character is rummaging through your organizations most sensitive intellectual property out of business hours, what would it be? In a nutshell, I’d look to my DNS logs.

It’s staggering to me how few security teams have gotten wise to regularly interrogating the logs from their recursive DNS servers. In many ways DNS logging can be considered sprinkling flour on the floor to track the footsteps of the culprit who’s been raiding the family fridge. Each step leaves a visible impression of where and how the intruder navigated the kitchen, and their shoe size.

Whenever an electronic intruder employs their tools to navigate your network, tries to connect back to their command and control server, or attempts to automatically update the malicious binaries they've installed upon the system they have control over (or wish to control), those victim devices tend to repeatedly resolve the domain names that that attacker is operating from. Therefore, armed with a list of known bad domain names and/or IP addresses, it’s a trivial task to mine the DNS logs and identify any successful intrusions.

Depending upon how authoritative your “blacklist” of criminal domains is, and how picky you are about the IP destinations that the domain names are resolving to, you can rapidly spot those nefarious shoe impressions in the flour.

One word of caution though, this isn’t a comprehensive technique for detecting persistent threats operating within your network – but it is one of the simplest! It also has the highest impact – particularly if you’re operating on a shoestring budget.

An obvious limitation of DNS log mining is the depth and accuracy of the blacklist you’re matching DNS events to – so you’ll want to ensure that the list you’re using covers the types and classes of threats you’re most interested in detecting. While there are plenty of free blacklists out there, the vast majority of them deal with spam, phishing and drive-by hosts… so you’ll want to invest some time shopping around a little.

Here are a few tips to using DNS as a means of detecting persistent threats (advanced or otherwise):

• Turn DNS logging on! Seriously, do it now… you can read the rest of this blog after you've turned it on.
• Select a bunch of relevant blacklists that contain malicious domains associated with the threats (and criminal actors) you’re most interested in.
• Create a list of IP address ranges for countries, companies or regions that computer systems within your organization shouldn’t be communicating with, and use this as a second-pass filter for spotting other unwanted or likely malicious traffic.
• Scrape your DNS logs frequently – ideally at least once per week.
• If you’re worried about a handful of specific threats (e.g. a criminal operator or state that is likely targeting your organization), scrape your DNS logs for relevant domain names hourly – and alert upon their discovery.
• Even if you’re only scraping you’re DNS logs weekly, don’t throw them away after you’re done. Try to keep your DNS logs for multiple years at least. (Note: DNS logs compress to almost nothing – so they’re not going to take up much space).
• Consider scraping all the older logs (going back up to 5 years) once a month or so. New domains will be added to the blacklists you’re using over time, and new intelligence can shed new insight in to past intrusions. It’ll also help to establish when the intruder first compromised your network when you do find one.
• If your DNS server allows it, turn on logging of “failed” lookups – i.e. NX domain requests. While these events won’t help in your blacklist lookups, they will help identify malware families that make use of domain generation algorithms (DGA) as well as “broken” applications within your network that need some tuning in finding their legitimate destinations.
• DNS log scraping can be conveniently done off-line through simple batch script processing. So the impact on the team responsible for securing the corporate infrastructure is minimal after a nominal development investment.

If you’re not happy with the quality of the blacklist you’ll be able to bring to bear in uncovering the persistent threats likely already operating within your environment, or if it would be helpful to do a “one-off” check of your DNS logs and to help build the internal business case for investing in a more permanent detection solution, let me know.

IP's and the APT

Most of the good thrillers I seem to have watched in recent years have spies and assassins in them for some diabolical reason. In those movies you’ll often find their target, the Archduke of Villainess, holed up in some remote local and the spy has to fake an identity in order to penetrate the layers of defense. Almost without exception the spy enters the country using a fake passport; relying upon a passport from any country other than their own.

Like any good story, there’s enough truth to the fiction to make it believable. Take the real-life example of the hit squad that carried out the assassination of a Hamas official in Dubai early 2010. That squad (supposedly Israeli) used forged passports from the United Kingdom, Ireland, France and Germany.

So, with that bit of non-fiction in mind, why do so many people automatically assume that cyber-attacks sourced from IP addresses within China are targeted, state-sponsored, attacks? Are people missing the plot? Has the Chinese APT leapfrogged fact and splatted in to the realm of mythology already?

If you’re manning a firewall or inspecting IPS log files, you can’t have missed noticing that there’s a whole bunch of attacks being launched against your organization from devices hosted in China on a near continuous basis. A sizable fraction of those attacks would be deemed “advanced”; meaning that as long as they’re more advanced than the detection technology you happen to be reliant upon, they’re as advanced as they need to be to get the job done.

Are these the APT’s of lore? Are these the same things that government defense departments and contractors alike quake in their boots from? There’s a simple way to tell. If what you’re observing in your own logs shows the source as being from a Chinese IP address it almost certainly isn’t.
Yes, there’s a tremendous amount of attack traffic coming from China, but this should really be categorized as the background hum of the modern Internet nowadays. China, as the most populous country on the planet, isn’t exempt from having more than its fair share of Internet scoundrels, wastrels, hackers and cyber-criminals — spanning the full spectrum of technical capability and motivations. Even then, the traffic originating from China may not be wholly from criminals based there — instead it may also contain attack traffic tunneled through open proxies and bot infected hosts within China by other international cyber-criminals.

Mind you, when we’re talking about cyber-warfare and state-sponsored espionage, we’re not talking about a bunch of under-graduate hackers.

Just about every country I can think of with a full-time professional military force has been investing in their cyber capabilities – both defense and attack. While they’re not employing the crème de la crème of professional hacking talent, they are professional and have tremendous resources behind them, and they follow a pretty strict and well thought-out doctrine. If you’re in the Chinese Army and have been tasked with facilitating a particular espionage campaign or to aid a spy mission, the last thing on earth you’re going to do is to launch or control your assets from an IP address that can be easily traced back to China. Anywhere else in the world is good, and an IP address in a country that your foe is already suspicious of (or fully trusting of) is way better.

Don’t get me wrong though, I’m not singling out the Chinese for any particular reason other than most readers will be familiar with the hoopla and epic proportions of Chinese APTs in the media. Any marginally competent adversary is going to similarly launch their attacks from a foreign source if they’re planning on maintaining deniability should the attack ever be noticed – just like those spy tactic of using foreign passports.

So, if you’re inclined, how are you going to get access to foreign resources that can proxy and mask your attacks? Elementary my dear Watson, there’s a market for that. First of all there’s a whole bunch of free and commercial anonymizing proxies , routers and VPN’s out there – but they may not be stable enough for conducting a prolonged campaign (and besides, they’re probably already penetrated by a number of government entities already). Alternatively you could buy access to already compromised systems and hijack them for your own use.

Over the last five years there have been a bunch of boutique threat monitoring and threat feed companies springing up catering almost exclusively to the needs of various national defense departments. While they may offer 0-day vulnerabilities, reliable weaponized exploits and stealthy remote access Trojans, their most valuable offering in the world of state-sponsored espionage is arguably the feed of intelligence harvested from the sinkholes they control. Depending upon the type of sinkhole they’re fortunate to be operating, and which botnet or malware campaign that happened to utilize the hijacked domain, they’re going to have access to a real-time feed of known victim devices from around the world, copies of all the data leached from the victims by the malware and, in some cases, the ability to remotely control the victim device. Everything a cyber-warfare unit is going to need to hijack and usurp control of a foreign host, and launch their stealthy attack from.

Now, if I was say working within the cyber-warfare team of the French Foreign Legion or perhaps the DGSE (General Directorate for External Security) and interested in gathering secret intelligence about the investment Chinese companies are making in sub-Sahara mineral resources, I’d probably launch my attack from a collection of bot-infected hosts located within US or Australian universities. The security analysts and incident response folks working at those Chinese companies are probably already seeing attack traffic from these sources off-and-on, so my more specialized and targeted attack would unlikely raise suspicion. Should the targeted attack eventually be discovered, the Chinese would simply blame the US and Australian governments – rather than the French.

Having said all that, you’ve probably seen movies with double-agents in them too. And it’s entirely possible that someone hair-brained enough would argue that China launches attacks from their own IP space because everyone knows that you shouldn’t, and therefore an assumption would be made that attacks launched from China are clearly not from the Chinese government – while they are in fact. How very cunning. Now there’s a twist for the next spy movie.

Military Grade Malware (Part 1)

Not all malware is created equal. Of the 50k-80k new and unique malware samples received daily by the mainstream anti-virus companies, there's a lot of scope for variety. Most of the samples are merely serial variants being pumped out as part of a barrage of criminal campaigns, and then there's a sizable handful of custom crafted malware that (for the most part) is generally unsophisticated botherware and spyware, but occasionally you'll uncover a few very crafty and sophisticated malware samples mixed in there.

In a lot of cases, these particularly sophisticated malware samples only manage to get caught up in the wash of general malware samples because of some circuitous and "unlucky" compromise paths - or because they're several months old and the "discoverers" have finished reaping the reward of having investigated them. Most of the really interesting bespoke malware samples rarely come via mainstream discovery and sample sharing systems though - in fact the majority of them rarely go beyond the virtual walls of the organization or government department that were targeted or victimized by them.

Given all the discussions about Advanced Persistent Threats (APT), Advanced Malware and Next Generation Malware (NG Malware), I thought it was about time to disclose some of the techniques being used within the commercial world in the production of such sophisticated malware... hence this blog entry being the first in a series covering "Military Grade Malware".

Military Grade Malware
I use the term "Military Grade Malware" to encompass the following key concepts:

1. A legal contractual agreement exists between the professional software development team and the purchasing organization.
   
2. The expectation is that the "product" will be used for purposes beyond financial and criminal fraud.
3. The intended distribution of the malware will be limited in scope and typically only be deployed in very specific environments.
4. The malware is designed to be stealthy and continue to operate for extended periods of time - typically against a sophisticated adversary.
   

Why are these important? The vast majority of malware circulating around the Internet and infecting both home and corporate systems are clearly designed for criminal purposes. More often than not, they're heavily weighted towards data theft and financial fraud. While the authors of the malware may or may not be criminals themselves (e.g. many of the popular DIY construction kits are sold commercially, by licensed companies, as "Remote Administration" tools) - they are designed to operate on popular operating systems and commodity hardware.

In the past I've used the term "weaponized" to encompass malware that makes use of exploit material as part of its critical operations - but this term only extends so far.

Exploit Weaponization
There are plenty of boutique security consulting organizations out there that offer "weaponization" services. They will typically review and study the latest vulnerability disclosures, develop reliable exploits for use against specific operating systems (e.g. an exploit for a popular Vietnamese instant messaging client running on Microsoft Windows XP SP3 with the Vietnamese language pack installed), and pass the final QA-checked exploit on to their client.

Most of the organizations I've come across that provide this kind of service have strong affiliations with their local government. That said though, a handful of them are more mercenary and will provide their weaponized exploits to other "friendly" governments. I'll point out at this stage though that this is a wholly different arrangement compared to vulnerability research teams working within companies that develop commercial vulnerability scanning and exploitation tools.

The provisioning of (reliable) weaponized exploits will generally be governed by formal legal contracts. It's not easy work though. Many people see the plethora of public vulnerability disclosures and hear about the odd zero-day exploit doing the rounds, but the development of reliable exploits that meet the contractual demands of the client is not a simple task. A company that can deliver a half-dozen ruggedly reliable weaponized exploits each year is doing very well - and will be compensated accordingly.

Malware Weaponization
The weaponization of malware in my opinion generally only encompasses the binding of a "standard" malware component to a particularly good/reliable/weaponized exploit.

For example, a client may have a preferred Remote Access Trojan (RAT). This RAT is consequently bound to the latest weaponized exploit - i.e. the RAT is merely the payload of the successful exploitation.

In another example, a versatile malware agent may support a library of exploits that it can use to worm and propagate around a targeted network. In this case, the weaponized exploit is constructed to be compatible with the malware agent and is added as an "update".

Both examples would fulfill the generic term "weaponized malware", but there is a difference between this type of malware and what I'd tend to term "Military Grade" malware, since military grade malware may or may not actually make use of weaponized exploit materials.

What are the features and techniques of military grade malware? I'll begin to cover those details in subsequent blog posts...

APT Dilemmas

The last month has seen a plague of comments and "expert" opinions materialize related to the Advanced Persistent Threat (APT). In the majority of cases, I'd have to class those very same comments and opinions as ambulance chasing tripe - by people that either have over active imaginations or are just simply looking to capitalize on the confusion generated by the media.

Sure, we're all entitled to our opinions, but there's more to all this. If many of these comments and expert opinions had been directed at an individual or corporation, those "experts" would have found themselves in court over slander charges many times over by now. So perhaps they're personally lucky that their ignorant and ill-educated comments haven't resulted in such actions. On the other hand though, they would appear to be adding kindling to a growing wildfire which will likely affect us all.

There are of course multiple camps of thought in every argument. For many (former) military types, it often appears to be about Nation States driving and incentivising hacking teams to target the assets of a foreign entity. That's they way they were trained to think. Similarly, Nationalism comes in many shapes and forms - and varying degrees of dedication - ranging from wearing a lapel pin through to chanting a pledge of allegiance to a flag (or deity, or prophet) each day. Every country, population or group has different levels and ways of showing nationalistic pride or reverence.

I believe that this applies greatly to APT's. The ability to acquire, retain and motivate a team of hackers capable of orchestrating and executing an APT campaign against a target (global conglomerate, strategic technology provider or government department, etc.) goes beyond meeting a specified financial compensation plan. APT campaigns aren't for the feint of heart. They require a degree of dedication not normally seen in most cyber criminal attacks.

That is not to say that someone can't simply go online and hire a bunch of hackers and build out a team to launch an APT campaign. That's not particularly hard - especially if you've got the cash. However, to keep a campaign flowing and obtain the level of persistence needed to keep the cross-hairs on a target for a year or more - well, that requires something more.

For one thing, running such a long campaign is probably going to need a core team that shares similar (if not identical) core values - nationalistic, political, religious, etc. - and is willing to dedicate the time needed. The dedication element can be brought easily enough, while the core values aspect means that the hacking team will likely have shared many experiences in the past. This of course doesn't prevent the campaign from engaging other external entities and subcontracting out either more specialist attacks or delivery options, but it does mean that tactical elements of the campaign can be passed on to third-parties as and where needed.

So, the dilemma with APT's is that they're a campaign strategy rather than an exploit, hack or attack vector. Which of course confuses many people who think of things solely in terms of attackers and tools - rather than objectives and motivations.

Would I class APT's as nation-state strategies serving as a precursor or reconnaissance for cyber-war? In some extreme cases, yes. I've met and probably helped train (in some fashion or other) several of the individuals that work this angle and are prepared to engage in these kinds of activities. However, many more of these people would refuse to engage in these activities out of nationalistic pride or prejuidice - but are only too happy to offer their persistent attention and services for a fee; being ideal candidates for longer term corporate espionage (e.g. back-dooring of oil pipeline control systems, targeting pharmaceutical research laboratories, accessing patent filing and tracking systems, etc.).

Then again, motivations for engaging and conducting an APT campaign can vary a lot - searching for UFO evidence, saving the whales or even targeting car manufacturers that attempt to hijack and steal other peoples Internet domain names - are all past causes capable of wedding a team together and working towards a common objective.

So, a word of advice then. It's dangerous to think of APT's as being wielded solely by nation-states. Unfortunately APT's are a fact of life - and have been so for well over a decade now. It's just that they've only been spoken about in hushed voices within closely closeted communities before Google said enough is enough to the secrecy.

Network ADS - Playing at Botnet Detection

With botnets and more recently APT's plastered across the news, you'll struggle to find a security vendor that hasn't spent a furious couple of weeks repositioning their preemptive detection technologies as "anti-botnet" or "anti-APT".

Needless to say, digging a little deeper in to these technologies - beyond their cursory marketing spin - is probably a good idea, especially if your company executives are thrashing about looking to take any steps that'll keep them from being the next Google-like victim and making headline news.

Yesterday I pulled together my thoughts on the use of network-based Anomaly Detection Systems (ADS) in their capacity as botnet detection tools. In a nutshell, NADS is fine for dealing with those big and noisy Internet botnets that everyone writes about in the news, but not much chop against the types of botnets normally found successfully operating within enterprise networks.

My thoughts and analysis can be found on the Damballa blog - Detecting Botnets with Network ADS - and is also cross-posted below...

-------

Many businesses have already deployed Anomaly Detection Systems (ADS) within their enterprises whether they know it or not. Most ADS technologies can be discovered operating at the host-level – typically integrated in to the popular desktop antivirus suites of the major security vendors – where they can be often be found functioning in a hybrid detection mode somewhere between a personal firewall and a behavioral analysis engine.

Network-based ADS (NADS) on the other hand serve a different purpose within large enterprises. Their deployments are far fewer than host-based ADS, and are often used by security teams to detect major changes in network activity – typically analyzing and regulating traffic flow. Optimized to view high volumes of network traffic across an enterprise as fast as possible (in real-time in many cases), they rely upon specially crafted abstract protocols of the traffic – such as NetFlow, JFLow, NetStream, etc. – where content visibility is sacrificed for analysis speed.

Over the last 2 years NADS technologies have increasingly been positioned as having an anti-botnet capability – which has caused much confusion amongst those responsible for managing ADS deployments and those responsible for enterprise-wide security. NADS do in fact have some value as an enterprise-level botnet detection tool, but their capabilities are all too often misrepresented.

How capable are NADS in detecting and mitigating botnets? What are their strengths and weaknesses? The following is a summary of my observations and experiences (garnered over the last 5+ years) in using NADS as an enterprise security technology – warts and all.

Botnet detection & mitigation strengths:

1. A correctly configured and baselined NADS deployment is capable of detecting the high-volume attack output from certain classes of botnets. By identifying voluminous email sending (e.g. spam agent or spam proxy operation) or crafted port-specific traffic (e.g. DDoS agent operation) and tracking that back to specific hosts, the infected systems operating in this manner can often be classed as being a member of a botnet.
2. Many general-purpose Internet botnet malware make use of worming capabilities to propagate around enterprise networks by exploiting unpatched software flaws in vulnerable hosts. If a NADS solution has been correctly baselined, it can be relatively easy to spot the anomalous traffic this propagating threat creates – thereby alerting security teams to a new malware outbreak.
3. By configuring the NADS system to account for “normal working hours”, different detection thresholds can be utilized to aid in the detection of hosts that are actively communicating with botnet Command and Control (CnC) nodes. For example, if a host suddenly commences HTTP or IRC communication with an IP address located in Tibet at 3:00am in the morning – this will likely be very suspicious.
4. If an organization has suffered a sudden and large botnet infection, the constant polling of some botnet malware variants will quickly become apparent and will aid in the identification of those bot infected hosts.
5. If the NADS system supports the use of blacklists, a list of known botnet CnC’s can be used as a means of tracking the volume of data that has been sent or received by enterprise hosts that are part of the botnet. Study of this logging can help reveal the scope of a breach and the types of information the criminal botnet operator is targeting.

Botnet detection & mitigation weaknesses:

1. Very few botnets encountered within enterprises nowadays are noisy and spew copious volumes of spam or participate in devastating DDoS attacks. Botnet masters have largely moved on from this activity – and will only order bot infected hosts to operate this way if they’ve already exhausted all other value from the compromised hosts, or if they never bothered to figure out the infected hosts were actually located within an enterprise (in which case, if they had, they would have probably sold them to someone else for a good price). Therefore, basing detection of a botnet infection on copious volumes of attack traffic is either too late in the botnet lifecycle or was just bothersome (and not a security risk) to begin with.
2. By basing botnet detection upon the identification of outbound malicious traffic, the enterprise security team have failed to preempt the malicious operation of the botnet and are forced to deal with the voluminous output of an ongoing attack. Detection and mitigation of the botnet control instructions that instructed the attack to begin would have been more efficient and less damaging.
3. Baselining an enterprise NADS deployment – and keeping that baseline current – is almost impossible in the vast majority of businesses. New application deployments and software updates, along with roaming users and peer-to-peer communications, mean that enterprise network traffic is not as consistent and predictable as it was even only half-a-decade ago. As such, it becomes increasingly difficult to spot the worming traffic generated by botnets attempting to propagate around the network. This has become further complicated by the fact that the malware authors themselves have learned much from past attacks and have intentionally become more stealthy and deliberately slowed their propagation pace to avoid anomaly detection systems.
4. Botnet malware is more often than not designed to function only when the infected host is actually in operational use by its authorized user. As such, it is increasingly difficult to identify anomalous traffic from real traffic as the user goes about their regular Internet surfing.
5. Most botnet malware found within enterprise networks are proxy aware. This means that they borrow the users credentials and funnel all their CnC traffic through the corporate proxy servers – i.e. they do not use non-standard ports or protocols to navigate out to the Internet or between internal systems. The vast majority of botnet malware rely upon HTTP or HTTPS for communication.
6. Timing is everything for botnet operators nowadays. No sooner has a host been compromised through a drive-by-download vector or Trojan file download, that it connects back to its CnC ready to receive both a updated malware package and a new set of instructions. As such, unless the NADS solution is configured to react in real-time to the identification of a new botnet infection, the threat will have either moved on or already become more severe.
7. Many of the more commercially-minded botnet operators invest in fast-fluxing and domain-fluxing CnC technologies. The unrelenting changes in CnC IP addresses and hosts names can quickly overwhelm NADS systems.

So, in summary, I’d say that NADS does have a role to play in botnet detection – but only a very minor one, and even that’s diminishing all the time. NADS deployments make for capable enterprise-wide network health monitoring systems, but have faltered against advanced threats like botnets and even more stealthy threats such as Advanced Persistent Threats (APT’s). I’d liken NADS to a school nurse – constantly overseeing the health of the entire student population and dealing with the odd knee scrape and cut lip – but not trained or equipped to deal with major head trauma or the results of a shooting spree.

Whats more important - preemptive, or post-preemptive?

Preemptive security technologies - they're great, and you can't beat them. Well, that's how it's supposed to work anyway. If only life was so simple.

The core idea behind preemptive protection technologies is to detect and stop entire classes of threat from successfully compromising the integrity of a host, network or application. Sales and marketing teams are only too eager to throw around the "preemptive" term - which can lead to rather embarrassing discussions between customers and technical engineers as the two have work out why a particular threat that was supposed to have been stopped, managed to get through the defenses. Its very rarely because the "preemptive" technology failed to to what it was designed to do - and almost exclusively because of nuances in the attack.

For example, there are litterally dozens of technologies out there being touted as "preemptive" protection against drive-by-download attacks. Some of these technologies focus upon detecting the presence of a shellcode payload, with others may hone in on JavaScript obfuscation techniques. However, despite these preemptive detection technologies, there is a growing list of vectors that bypass each technology. For example, drive-by-download attacks that use Flash scripting instead of JavaScript, or embed the shellcode in a different file - rather than within the JavaScript. The net result is customers scratching there heads and looking for answers. The nuances will often escape them - and the vendors R&D team will have a few late nights adding detection capabilities for the new evasion technique or encoding scheme (if they're lucky).

Don't get me wrong, "Preemptive" protection is damned important. You need it a lot more than some just-in-time signature update. But you've also got to realize that the more ground-breaking the "preemptive" protection is, the narrower its focus is in threat mitigation.

Whats more important than "preemptive" protection? In my mind its post-preemptive protection detection - i.e. being able to rapidly detect when all your combination's of "pre" protection didn't quite work, and your network got nailed despite the effort (and resources) you expended. If you focus exclusively on trying to prevent hosts, networks and applications from being compromised, you're going to have a damned hard time detecting when your systems do in fact get p0w3d by some Internet criminals.

This is particular important when you're facing a more organized and motivated opponent - such as those running an APT operation against your organization.

I discussed this in more detail the other day in my Damballa blog - “Preemptive Protection” Isn’t – If You’re Battling APT’s - and cross posted below...

------>

There’s been no shortage of press covering Advanced Persistent Threats (APTs) this week. While there have been plenty of post-hack discussions over the past few years following the big public breaches, this one’s different – there’s almost a kind of relief that this one’s made it out in the open. I can liken it the relief and revelations that followed that first major tobacco manufacturer’s decision to admit that smoking actually probably wasn’t so good for you after all…

Unfortunately, the revelation of several dozen major organizations being the victim of this particular APT example has just about every security vendor on the planet clamoring to extol and position their latest nicotine patch equivalent. Or, perhaps more appropriately, a lock-box to prevent you from reaching for another cigarette.

In the hussle-bussle of vendors claiming “First” or “Preemptive”, there’s a lot of weighted wordage flying about. But if that’s all true, if a particular vendor was “First” in its discovery, why didn’t they stop the threat or protect the currently known victims? Didn’t they understand the significance of what they had already discovered? Did they choose to keep the information to themselves for competitive advantage? I can’t answer those questions – and frankly any answers I’d likely receive in return from these “First” vendors would probably be carefully word-smithed by a gaggle of marketing folks.

What about “Preemptive”? I like that word – it’s important. Having developed and invented many security technologies that fall in to that bucket over the last decade, I can categorically state that “Preemptive” is good. But (and you know there’d be a “but”), it’s not good enough…

Those nicotine patch equivalent vendors are going on about how they could/would/will/have/might preemptively…

• …detect the fact that the user is visiting a URL that’s probably dangerous
• …detect the malicious JavaScript or HTML that delivered the exploit
• …detect the exploit shellcode
• …detect the buffer overflow
• …detect the memory manipulation of the exploit
• …detect the malicious payload
• …detect the malware component
• …detect the malicious behaviors of the compromised application
• …detect the inappropriate behaviors of the compromised host
• …detect the malicious network behaviors

…and by “detecting” the APT, they’d have been able to protect against it (or an aspect of it). But at the end of the day, all those technologies, for one reason or another, failed to protect these organization from being a (very public) victim of the APT.

Why? Because APT’s aren’t like average-Joe malware, botnets, script-kiddies, hackers, fraud artists and cybercriminal attacks. The thing that makes APT attacks different from the other forms of cyber-attack can best be summed up with the mantra “if at first you don’t succeed – try, try and try again.”

The vast majority of Internet attacks – especially mass Internet botnets – are opportunistic attacks. The bad guys have a broad objective in mind along with a number of tools they specialize in and have a ceiling to the amount of effort they’re willing to expend. They will optimize a particular attack vector, select the preferred delivery method, and pound the Internet (and everyone on it) with that toolset until they’re acquired enough victims. So, while many of the attacks may appear to be “targeted” (e.g. Spear Phishing), their objectives are rather limited (e.g. immediate financial fraud), and if they don’t succeed against the currently highlighted target they’ll simply move on to the next.

APT’s don’t follow this model. If a particular attack vector, tool, technology or exploit didn’t (or is unlikely to) work, they switch to another – never changing targets nor focus.

What does that mean in practice? Regardless of the perimeter or host security technology you deploy, and how “preemptive” it is, it isn’t going to stop an APT. Sure, each “preemptive” technology worked just fine – stopping each and every attack vector, malicious payload or strange behavior it was supposed to – but the criminal operators targeting your organization just move on to the next tool or vector until they find one that works. And lets not forget (or kid ourselves), this probing of network defenses and “preemptive” protection doesn’t happen as an overnight barrage of simultaneous attacks from a small cluster of IP addresses tracked down to the Chinese Army. No, this is low and slow stuff spread over many days, weeks or months, routed via a variety of sources and proxies from around the world – or even through your business partners.

So, can all of these nicotine patch sellers protect your organization against APT’s? No, of course not. They can protect against many of the vectors that may be tried and probably identify the particular exploit or malware they end up using, but at the end of the day APT’s will win.

Which brings me to my final point. I don’t care how you got infected or became the latest APT victim – because you will be – so get over it and do something already. If a criminal operations team is willing to spend the time, effort and monies to target your organization, they will win! So, how do you defeat APT’s? Simple, you detect their presence as fast as you possibly can and remediate the victims almost as fast.

OK, so “preemptive” protection is important – but being able to know when that “preemptive” protection has failed is even more important!

FailSafe

Let me put on my Damballa hat for the moment. I’ve been getting a bunch of queries about whether the Damballa FailSafe solution detects the “Google APT thing in the news”. The answer is Yes, and many of the other APT’s that you haven’t heard about (and are unlikely to hear about). You see, from our technology perspective, we don’t care how you became a victim either (you can debate that’s my influence or cynicism leaking through). Lying at the heart of our technology is the ability to identify the suspicious and unauthorised remote control of systems within the enterprise. All this is done at the network level and an APT’s command and control (CnC) is generally no different from a successful mass-Internet botnet, an insider threat or even a remote access trojan hand placed by a criminal operative. The motivations behind a botnet, insider threat and APT may be wildly different – but the CnC communications do not.

It gets a little tougher distinguishing between a brand new targeted botnet, an insider threat or an APT purely from their CnC traffic. But in reality the trick is to identify those threats that have already navigated your layers of corporate defenses and shut them down. Deciding which particular threat was politically/financially/ethics motivated comes afterwards.

Was this “Google APT thing in the news” the first APT to place Google under it’s cross-hairs? No. Is it the only APT targeting Google? No. Will it be the last APT to be targeting Google? No. Will targeted enterprises be able to prevent APT’s from getting in? No. Is it possible to detect when an APT has successfully bypassed all your “preemptive” protection technologies and compromised your systems? Yes.

Advanced Persistent Threats

I've been getting lots of questions on what precisely is an Advanced Persistent Threat - or APT for short - from all kinds of angles.

As such, the Damballa team have created the executive two-pager that helps answer "What is an Advanced Persistent Threat?"

Tethered Espionage

News of corporate espionage amongst the Fortune-100 - with targets like Google and Adobe - has been breaking all day. It's interesting to note the thoughts of the different commentators and their take on the China slant.

Earlier today I blogged (rather extensively) on my take of the news. You can find those comments posted here - Corporate Espionage and Tethered Criminal Actions - and copied below...

--------------------

The media is buzzing with the latest news concerning Google and Adobe and the targeted attacks directed at their corporate systems. While it’s news, it’s important to understand that this isn’t something that’s only just happened – rather it’s been something that both these organizations (and dozens more) have been subjected to for quite some time; it’s just become public, and they’re admitting to be the victims. But this is important.

I’ve been providing security consultancy advice for a couple of decades. I’ve been pulled in to do post attack forensics along with specialized pentesting, bug-hunting and reverse engineering for the majority of the Fortune 500 companies and in all that time, unless they were required to by law, not one have gone public about the attacks they were subjected to and the losses they have incurred. That’s why this Google/Adobe/etc. news is so significant – some Fortune-500 companies are actually saying “hey, enough already, we’re under constant attack – we need to do something collectively about this!”

Whats the primary vehicle for these (ongoing) attacks? You’ll hear plenty of discussion portraying viruses and malware as being the problem, and plenty of implications that the Chinese government lies behind the attack(s). But let’s be clear – that’s a fantastically simplistic view of the threat. Implying that the threat lies with targeted malware and China is like saying that drunk driving deaths are due to poor car design, and that the underlying cause is a particular beer brewery.

Malware is just a tool. The fundamental element to these (and any espionage attack) lies with the tether that connects the victim with the attacker. Advanced Persistent Threats (APT), like their bigger and more visible brother “botnets”, are meaningless without that tether – which is more often labeled as Command and Control (CnC).

The methods for getting a malware agent into an organization and on to key/critical hosts are incredibly diverse but, most importantly, can best be phrased as “trivial”. If someone wants to infect systems within a targeted organization and is willing to spend more than a few thousand dollars worth of effort to do so, it’ll happen – simple as that. Just as importantly, the malware being distributed and used in these kinds of attacks can be thought of as a Swiss Army knife with Klingon cloaking capabilities.

I jest only in part about the Klingon cloaking part – but it actually works well as a visual metaphor. Just as the Klingon Warbirds must decloak in order to launch their attack with photon torpedoes etc., APT’s and botnets must decloak themselves at the network level in order to maintain their CnC connections and be successful in harvesting espionage data. While APT’s are more surreptitious when it comes to CnC connectivity, their weakness lies in their network communications. At the host level, the probability of detecting an installation prior to actual financial/legal damage lies largely in the realm of dragons and mermaids.

Looking at the botnets we identify and track at Damballa that target enterprise networks, many of them fall in to the classification realm of APT’s. The malware component is under constant change – often being updated on a daily basis. Meanwhile the low-and-slow stealthy CnC traffic navigates the corporate network, weaves it’s way through fast fluxing networks and stratified levels of command relays, and makes it back to the team who’s really in control of the compromised assets – a bunch of contracted criminals located somewhere safe and far away. I use the term “team” on purpose because this is an organized collective of professional operators – each with their own skills and specialties.

I see a lot of discussions about preventing systems from being compromised – in fact most of the security business today is exclusively focused on threat prevention. But, you know what, every year (for the last two decades at least) as antivirus vendors release their annual threat reports the percentage of hosts known (or suspected) of being a victim and running malware has increased. As we launch in to 2010, I think the percentage most industry experts and veterans would throw about would be 35-40 percent of all Internet connected systems are compromised and currently running malware. Despite the terrific advances in detection, mitigation and cleanup – the numbers continue to go up. Despite the new detection technologies, the bad guys retain their lead. APT’s related malware lie in a particular niche, but they aren’t being prevented from getting in to an targeted organization. Let’s just face facts – if someone wants in on your organization and are willing to invest time and resources to do so, the probability that they will be successful in doing so certainly favors them.

Detecting and mitigating the CnC – breaking that tether of control – lies at the heart of dealing with this threat. By blocking those CnC channels, the bad guys can’t remotely control your enterprise systems, and they can’t extract the secret data they want. Tracing back who lies at the end of the CnC communication ultimately leads to he contracted criminals running the operation. The fact that those criminals happen to be located in a particular country is only part of identifying the instigators of the threat – but it’s probably as far as we’ll get.

Like I said earlier, I’ve had to deal with many of these threats before. In the UK, it appeared that many of the corporate espionage attacks were masterminded by French or US entities. In Taiwan it appeared to be China and South Korea. In China it appeared to be Taiwan and Australia. In Greece it appeared to be Turkey and Egypt. And so on… but those are only my specific experiences. [unfortunately, not a single corporate victim ever went public about the attacks they fell victim to - and probably never will... sigh]

With regards to the APT’s and botnets that Damballa tracks, detects and mitigates… well, those CnC’s are spread around all over the world and most likely reflect the locations of the professional teams that contract out there services, rather than the location of their their ultimate customers.

My advice to organizations being targeted with APT’s, botnets and unauthorized remote control of corporate resources? Focus on the network CnC – and mitigate there. By all means protect your perimeter and clean up your hosts – that’ll keep the unsophisticated script-kiddies and rif-raf off your systems – but it means very little to the pros. Success in dealing with this threat – the threat that Google, Adobe, and most global businesses (and governments) face constantly – is to identify which assets are currently compromised and “nuke-and-pave” them asap. I.e. identify systems that are trying to connect to their remote CnC, immediately cut that tether, and rapidly rebuild that system from a known good state (which is increasingly looking like a bare-metal state). If you can get that notification-to-rebuilt process down to 20 minutes or less, you’ll be in a good position to deal with this class of threat long term. Until then, you’re just messing around at playing detective.