Monday, March 29, 2010

Recruiting: Threat Analyst @ Damballa

OK readers, I've got a roll open right now in the Damballa research team for a Threat Analyst.

If you think you know your Bot's from your APT's, and your script-kiddies from your cyber criminals, then it's time to take the plunge and join the coolest threat research team out there and make a real difference to Internet security.

Drop me an email if you're interested in the role...


Job Position: Threat Analyst
Job Area: Research

Internet security is evolving at an increasingly rapid pace. As the thrust and parry of attack vectors and defensive tactics force technologies to advance, the biggest security threat now facing enterprise organizations lies with botnets. The Damballa Research team spearheads global threat research and botnet detection innovation.

Damballa’s dedicated research team is responsible for botnet threat analysis and detection innovation. From our Internet observation portals, and using the latest investigative technologies to intercept and capture samples, the research team studies the techniques employed by criminal botnet operators to command and control their zombie hordes – mapping their spread and evolution – and developing new technologies to both detect and thwart the threat.

As a Threat Analyst you would be part of the team responsible for providing the threat intelligence that powers the core technologies of Damballa’s products – working with massive threat intelligence collections and cutting-edge botnet detection technologies.

The rapid evolution of the threat means that, as a Threat Analyst, you will also need to be able to deep-dive in to the botnet masters lair – turning over the rocks they hide under and visiting the online portals they do their business in – and be capable of analyzing the evidence of their passing. A key to being successful in this role is the ability to provide internal departments and customers with comprehensive intelligence on newly uncovered botnets and other targeted threats – and to be able to communicate the threat in a clear and concise manner.

Collaborating with the marketing and engineering teams, the Threat Analyst will often need to craft scripts to automate the extraction of botnet intelligence and make it available to the company’s other technologies and its knowledgebase as well as responding to ad-hoc requests for malware analysis driven by business and client needs to determine characteristics, functionality, and/or recommend countermeasures.

The position may entail interaction with the media following the successful outcome of directed research or response activities.

• Intelligence gathering and updating of Damballa threat knowledgebases
• Responding to customer queries for deep-dive information on particular botnets and malware
• Independent threat analysis and data mining of new botnet instances
• Investigation of new botnet command and control tactics and subsequent enumeration of botnet operators
• Focused analysis of botnet outbreaks within enterprise and ISP networks
• Contribution to research and commercial papers describing the evolving botnet threat

Skills & Experience:

• Experience as a cyber-threat analyst, or similar technical consulting role
• Good understanding of TCP/IP networking and security
• Strong script building and automation skills
• Database query formulation and stored procedure manipulation
• Ability to troll underground Internet forums and criminal sites/portals for new botnet intelligence

• BS or MS in Computer Science, Engineering or Physical Sciences
• 3+ years of IT industry experience with 2+ years of Internet security experience
• Proficient in multiple compiled and scripting languages (Perl, Python, Ruby, Java, C, etc.)
• Proficient query design in relational databases (Postgres/pgsql preferred)
• Excellent formal communication and presentation skills
• Ability to read and translate multiple international languages a bonus

Friday, March 26, 2010

Worthless Digital Security Advice

Some advice isn't worth the paper it's written on - more so if it happens to written in digital ink. Sure, security software tends to eat up a sizable chunk of your desktop's processing capabilities and can be downright annoying when the antivirus engine decides on an impromptu full-disk scan in the middle of the video editing you were doing... but sure;y we can do without advice like the following:

This is from CNNMoney and their story on how to "Speed up your sluggish computer".

Granted there are many sucky protection suites out there (and many more fake-antivirus products that criminals are peddling), but this particular advice entry is unhelpful and funny at the same time.

Firstly,this particular advice is ill informed. Sure, there are some overlaps in protection capabilities like anti-popup blockers and firewalls, but only on paper. They're complementary overlaps, as their capabilities to perform (and be managed) as pop-up blockers and firewalls tend to be quite different and increase overall. Defense in depth etc. Sure - like I said earlier - desktop protection is a dog on system resources.

Secondly, while I have nothing against ESET's Nod32 Antivirus product (I even use it on a couple of my computers at home - along with a handful of other av products), reference in this "guide" for speeding up sluggish computers smacks of a paid-for advertisement. Further depreciating the advice.

Third and final? "The Mac Fix" funnily enough is true - Mac users tend to not use security software. Like motorcycle riders swerving amongst rush hour traffic on the highway without a helmet, I'd class these Mac users as "temporary citizens" of the Internet.

Sunday, March 21, 2010

Botnet Prevention with DLP Technologies

Last week I was asked a couple of times how good Data Leakage Prevention (DLP) products are at protecting against botnets. Before I get started describing the pro's and con's of DLP in combating professional botnet operations, there are a few things I probably need to make clear - as it'll help add some perspective to the angle I'm coming from.

As you probably already know, I spent a fair amount of time developing and improving Intrusion Prevention System (IPS) technologies in my tenure with ISS (and then later, under IBM). During that time there were a number of market dynamics that required me to spend quite a bit of time reviewing, analyzing and evaluating the various DLP technologies - both at the host and then network levels. In general though, I was not impressed with the technology - and still aren't. From my perspective, DLP is a bit of a white elephant and is probably going to go down in the annuls of Security History in the chapter next to NAC. Don't get me wrong, as a concept DLP has its place, but in practice it fails to provide any compelling features that aren't (or can't be) delivered using other more common (and existing) enterprise security technologies.

Now, being a networking kind of guy, the thing I find most interesting about network-based DLP is the show and dance the various DLP vendors make about Deep Packet Inspection (DPI) - you'd almost think that they invented the technology and that it only to DLP. Lets get this straight from the beginning - DPI existed within IPS (and IDS) for 5+ years before even the first DLP companies became incorporated and, whats more, products like ISS' Proventia fully parse many hundreds of networking and content-level protocols - many times more than even the most mature dedicated DLP product out there.

So, if you're thinking DLP is a new and vital technology to roll out in your enterprise (particularly at the network layer), my advice would be to look to a top-tier IPS appliance instead because you'll find better protocol and content inspection coverage, and higher capabilities in inspecting traffic for critical data leakage. One day I'd love to see a head-to-head appliance review of the various vendors products detecting and defending against all the most common data leakage techniques/tactics.

DLP and Botnets
So, how useful is DLP in combating botnets? First of all, we obviously need some degree of clarification about "combating botnets". Lets break this down in to three separate botnet attack phases:
  1. Preventing hosts from becoming botnet victims,
  2. Detecting and stopping the leakage of confidential corporate information,
  3. Cleanup and remediation of bot infected victims.
Preventing hosts from becoming botnet victims
In order to understand DLP capabilities in preventing hosts from becoming botnet victims (from a network perspective), we need to bear in mind the limitations of DPI and the most common mechanisms hosts succumb to being compromised and joining a botnet.
  • Criminals leverage a broad spectrum of attack vectors in order to compromise their target - with the most common being spam/phishing emails that convince the user to infecting themselves, malicious drive-by-download sites the exploit vulnerabilities in the Web browser and removable media worming (e.g. USB devices). Unless the DLP solution is configured to watch inbound network traffic and scrutinize URL's (perhaps using a URL blacklist for checking against), the probability of detecting the malicious payloads is remote - and anti-spam and perimeter Web gateway technologies would be a much more effective solution here. IPS technologies would also excel in dealing with the exploits being used to compromise the Web browser vulnerabilities.
  • Inspection of the HTTP/FTP/etc. downloads or email attachments is of course possible - but it will be a struggle to to identify the malicious intent of the binary files, but should best be dealt with using anti-virus technologies - particularly products with good behavioral analysis engines and, in a pinch, virtual/sandbox dynamic-analysis of malware.
I think you would be hard pressed to use DLP technologies as an effective tool for preventing hosts from becoming botnet victims.

Detecting and stopping the leakage of confidential corporate information
Detecting the information leakage from bot infected hosts should be an easy task - after all, that's supposed to be DLP's bread and butter. Unfortunately it's not quite as easy as it sounds.
  • The signatures (or "fingerprints") DLP devices use are generally tuned to specific forms of structured data. For example, SSN's, credit card details and address details have a specific structure. As such, DLP solutions are generally good at spotting this kind data being transmitted across a network and leaking from the enterprise (just as IPS's can too). As such, DLP appliances can easily detect the "clear text" transport of these kinds of data.
  • Unfortunately, botnet operators tend not to transport/extract confidential data past perimeter inspection/detection technologies in "clear text". Obviously, if the bot agent chooses to transport the data to a remote server over HTTPS, then all the traffic will be encrypted. But botnet operators don't even need to do that...
  • Purchasing, managing and configuring Web server certificates for HTTPS can be tedious and can often result in "invalid certificate" alerts - which would in turn alert the user of any folks inspecting the system logs. As such, many botnet operators have decided to not use HTTPS - instead they extract their stolen data over un-encrypted HTTP, but they compress and encrypt the data they're stealing from on the victims machine before sending. I.e. the transport is unencrypted, by the file being transferred is itself encrypted and cannot be inspected by DLP (or any other DPI technology).
  • Armed with a blacklist of known botnet Command and Control (CnC) channels or file drop-boxes, the DLP solution could keep watch over who the victim system is communicating with and block those - but there are already plenty of IP/Domain/URL blocking technologies already out there that are more efficient.
  • It's important to understand that many professional botnet operators have moved away from stealing classic datasets (e.g. credit card details, SSN's, etc.), and towards more valuable datasets (e.g. software source code, CFO banking credentials, prototype designs) - which happen to be considerably more difficult to detect with DLP technologies (especially if the data is encrypted of course).
  • DLP is limited to specific protocols and specific file/attachment types for inspection. To evade detection, the criminal botnet operator just needs to use an "unsupported" protocol/format.
Cleanup and remediation of bot infected victims
Well, I can't think of anything that DLP offers in this realm.

Clobbering Botnets with DLP
In general, DLP makes for a very poor anti-botnet technology. DLP is adequate enough detecting the simple stuff - e.g. a user sending an email with 10,000 credit card details - but is ill positioned to detect an automated bot agent obfuscating or encrypting a compressed file of corporate secrets.

In fact, as far as I'm concerned, I can't really see a reason for it existing as a separate security technology anyway. Existing IPS technologies and signatures include just about all of the data leakage detection features already.

That all said, DLP is probably adequate enough for detecting stupid user mistakes, but useless for combating professional criminals - whether they're botnet operators or insider threats.

Friday, March 19, 2010

Comment Spam and SEO Campaign Apology

By way of an update to yesterdays blog covering my concerns over a comment spam and SEO campaign by Sophos (of which this blog was one such target), I received an apologetic email from Sophos early this morning and we exchanged a couple of followup responses.

Here's some of this morning's email apology:

I am mortified, as is everyone in our marketing team, that this has happened.

The messages were not posted on that guy's blog by an employee of Sophos, but by a worker at an external company hired by our marketing department.

We have called the company concerned in for a meeting today, and will be reading the riot act to them. Furthermore, we will be ensuring that this kind of activity stops immediately, as it runs counter to everything we believe in as a computer security company.

There's enough junk on the internet already - we don't need firms representing computer security companies adding to the problem with such inane and unprofessional posts.

We strive to be much much better than this, and on this occasion things went badly wrong. I'm genuinely sorry.

Just so you know, we are going to put better processes in place so that third party agencies understand what Sophos does and doesn't find acceptable in promoting our brand.
Thanks for the quick response Sophos. Apology accepted.

Thursday, March 18, 2010

Protecting Your Malware IP Investment

Competition between malware authors and botnet operators can be fierce at times. Opponents are constantly squaring up and trying to build bigger, better and more "advanced" everything. As such, they're keen to make sure that their latest advances and IP isn't ripped off by a competitor or, heaven forbid, some pesky malware analyst working at an antivirus company.

Earlier this week, a customer asked me what was the smartest and most sophisticated thing I’d seen malware authors doing recently. He was probably expecting me to mention some new toolset feature such as auto-cracking CAPTCHA’s for webmail spamming or the custom advertiser routines for redirecting in-browser advertising… instead, I discussed the new host-locked malware versions that are being experimented with by a number of professional botnet operators.

Three years ago I wrote a paper covering the one-of-a-kind exploitation techniques that were being adopted by drive-by-download distributors and exploit delivery systems. The paper – X-Morphic Exploitation – covers the generation of one-off “custom” exploits and malware that are created for each potential victim visiting the attackers malicious Web site. One of the techniques covered related to the creation and delivery of serial variant malware and how each unique sample was only ever served to a single victim – all as a means of defeating signature-based protection technologies (and, to a smaller extent, bulk analysis of malware samples).

Well, as you’d expect, the threat has moved on. While the X-Morphic exploit delivery platforms have grown more and more popular over the last three years, it would seem that the botnet builders have adopted an additional new (and rather powerful) technique that makes it even more difficult for malware analysts and bulk analysis tools to deal with their malicious bot agents – and it taken right out of the commercial anti-piracy cookbook.

To explain whats going on, it’s probably easiest to step through a botnet infection that makes use of the new technique:

  1. The would-be victim/user is browsing the Internet and stumbles upon a drive-by-download Web page. The page cycles through a number of Web browser vulnerabilities – locates an exploit that will work against the users browser – exploits the vulnerability – inserts a shellcode payload and causes the newly introduced (and hidden) process(es) to execute.
  2. A hidden process downloads a “dropper” file on to the victims computer, and causes it to execute. The dropper may be a custom package created just for this victim (i.e. X-Morphic generated) or one that is being served to all potential victims for that day/week.
  3. The dropper unpacks itself – unraveling all of the tools, scripts and malware agents it needs on to the victims computer – and then proceeds to hide the malicious payload components (e.g. disabling the hosts antivirus protection, turning off auto-updates, modifying startup processes, root-kitting the botnet agent), cleans itself up by removing all redundant files and evidence of the installation activities, and finally starts up the actual botnet agent.
  4. The first time the botnet agent starts up, it does a number of checks to see whether or not it has Internet access (e.g. deciding whether a corporate proxy is in use) and whether or not its running on a “real” victims computer (i.e. that it’s not running in a sandbox or virtualized environment – which would indicate that someone is trying to analyze and study the malware itself). If everything looks good and the coast is clear (so to speak), the botnet agent does a quick system-level inventory of the victims computer (e.g. CPU ID, HDD serial number, network card MAC, BIOS version, etc.) and then makes its first connection to the botnet’s Command and Control (CnC) – registering the victims computer as a member of the botnet, and sending through the unique system inventory data.
  5. In response, the botnet CnC immediately sends an updated bot agent to the victims computer – uninstalling the old agent, and installing the new agent. However, this new agent is specifically created and “locked” to the victims computer – i.e. it is unique to this particular victim and will not run on any other computer.
  6. Once the new “locked” bot agent is installed, it connects to a different CnC server – and the victim’s computer is now fully incorporated in to the criminals botnet, and under their remote control.

Those last three steps are what’s new and innovative, and what’s going to spell the ruin for many of the most important malware analysis tools and techniques antivirus vendors use to combat the malware plague.

By infecting their victims computer with a unique and “locked” version of bot agent (or malware), and ensuring that it will only ever run on that particular victims computer, it means that any samples that may eventually be acquired by the antivirus vendor(s) wont actually be useful to them. Automated analysis systems that take in malware samples from spam traps, web crawlers, etc. and execute them in virtual environments or sandboxes etc. will not yield the real botnet agent for study nor details of the true botnet CnC. Meanwhile, malware samples obtained from forensic retrieval processes or submitted by antivirus customers will not work (e.g. they will either not function maliciously or not execute at all in an analysis environemnt) – because they are encoded and locked specifically to the victims machine.

This “locking” process isn’t new in itself. Many commercial software vendors use this technique – for example, Microsoft uses the same technique for detecting pirated versions of their operating system and enforcing their licensing policy.In fact many manufacturers of DIY malware construction kits use the same techniques to protect their toolkits from being both pirated and falling in to the hands of security vendors. However, in this case the botnet operators are using it as a technique to ensure that samples of their malicious bot agents are useless to antivirus vendors.

Sure, a skilled malware reverse engineer could manually work around this kind of software locking mechanism, but its a slow and tedious process even for the most experienced folks – and manual analysis done in this way doesn’t remotely scale in any meaningful way to counter this threat. That said, if the (real) botnet agent also sends through an updated system inventory to the botnet CnC server each time it connects, and the “signature” no longer matches the one that the bonet operator originally associated with that particular botnet agent, then the botnet operator will know that someone is tampering with their software and disconnect the victim from the botnet (or perhaps launch an attack at the investigators/analysts computer)

As botnet operators (and general malware authors) further adopt this kind of victim-specific locking practice to protect their malware investment, and as the sophistication of the locking increases (as it inevitably will), the antivirus industry is going to have to rethink many of the techniques it currently relies upon for sample analysis and signature generation. There is no easy option for countering this new criminal practice.

Sophos - Stop Spamming Me and End Your SEO Campaign

Spam takes on many different forms. Sure, we're all familiar with the crap that makes it in to our inbox, but what about the other stuff - like the stuff that appears as comments in our blog entries?

Blog comment spam is on the rise, particularly when it's used less as a direct advertising tool and more for Search Engine Optimization (SEO) attacks/manipulation. In most cases I've observed, the SEO-orientated blog spam has been initiated by the bad guys - looking to escalate their infectious drive-by Web sites to the top of search engine results.

Lately though, I've noticed that a well-known security vendor - Sophos - has been employing this tactic. For example, check out the following blog comment submissions (pending moderation):

For the last few weeks there have been similarly themed comment submissions, typically initiated by the same accounts and targeting the same blog entries (based upon keywords).

This tactic is common, and there are a number of tools designed to automated this kind of spam and SEO attack.

What's interesting (and annoying at the same time) is that this repeated spam appears to be initiated by Sophos. As you'll see in the three comments above, the word "malware" is hyperlinked and in all cases points back to

I find this a pretty unsavory tactic, especially if it's initiated by a security company looking to be trusted by its customers.

Sophos - if you're listening - stop your comment spam campaign and end your SEO attacks. It's unprofessional.

Tuesday, March 16, 2010

San Francisco Security B-Sides

A couple of weeks ago it was my pleasure to present at the Security B-Sides event in San Francisco - in between all the comings and goings of the main RSA show. For those of you who are interested, the presentation deck is now available.

"Your Computer Is Worth 30¢ - This Battle for Control of Your Computer Isn't Personal, Its Business"

Abstract: The botnet ecosystem is evolving at a rapid pace. Specialized services have come to fill every niche of the hacking world. The frontline is rarely the mechanical process of exploitation and infection - instead it lies with innovative 24x7 support and helpdesk ticketing systems - quality of service is the competitive edge. How much is your computer worth to them? The price point is dropping day-by-day, but 30 cents is a pretty average trade value. Why is it so low? Because your computer is only part of the ecosystem - and a commodity one at that.