"Attackers collaborate, defenders are drowning in a sea of data."

I’m a little behind on getting my weekly blog posted this week, relaxing after a week in altitude in Colorado Springs at the AFCEA Cyber Symposium. My body just isn’t used to spending that much time at 6000+ feet! It was an ok conference, but a couple of speakers struck chords. On a day one panel were Brett Hartman, now with Cisco, but also the former CTO at RSA (during their much publicized breach), and Dr. Phillis Schneck from McAfee. Two quotes from the panel that I found especially on target:

Attackers collaborate, defenders are drowning in a sea of data.

-Brett Harttman, Cisco Systems

It’s pretty safe to believe that everything is owned.”
“Can’t take the humans out of the loop.”  Human and machine.

-Dr. Phillis Schneck (McAfee)

These are key points. Here’s why.

This week we analyzed malware that was believed to be one piece of code was used in the NY Times attacks. It’s not automated analysis, it’s automated assisted analysis. Code is pulled from a machine after enterprise-wide searches for unique ‘indicators of compromise’. This is done in one of a few ways, but the most common is through software that examines each computer for file names, sizes, MD5 hash matches, or other pieces of metadata about the files on the system. If there’s a match, a file, piece of running memory, and/or a forensic image of the computer is pulled --either locally, or remotely.

Here’s the problem. In any given enterprise, when you run those host based tools to look for these indicators of compromise (IOCs), any given company is going to be inundated with results --and most will not be false positives.

So here’s what happens...

1. You load a system that can inventory computers, their file systems, and the infrastructure. There are some really nice tools out there that do this work for us --Mandiant’s MIR, Carbon Black, RSA ECAT, and others. (Note that I didn’t call out anti-virus. AV alone is simply not in the same class as these other tools. The use of signature-based anti-virus alone as a defense is no longer enough in a world attacks and threats are changing daily.)
1. Every file on every device is identified, and the meta-data about those files are collected and sent to a centralized analysis device for aggregation and correlation. (Not all require movement of data to the centralized analysis machine. Some perform the work locally on the host, others send data back.)
2. At the same time, a good team will have some form of network analysis device running on their Internet Point of Presence (iPoP). Whether a netflow analysis tool (i.e.: Arbor, NetWitness, etc.), or full packet capture... thereby aggregating even more data (that must later be analyzed).
3. Now comes the hard part... every piece of that data must be examined, separating the known good from the suspected bad --our indicators of compromise. Done manually, correlations of this data could take months. Thankfully we don’t have to do this manually.
1. The results? You’re not going to like this... expect big numbers. In a small or medium sized company, without a formalized information security structure, expect every computer to be reported as compromised. In a larger enterprises where formal information security processes exist, expect a large percentage of your systems to be compromised at first, and then as you work the issues, the numbers will level out. And, in enterprises where formal infosec processes exist, even those with great process, training, etc., it is not uncommon to experience a 2-5% daily detection (probable compromise) rate. Many companies (companies with GREAT infosec team) have stated that they are attacked hundreds of thousands of times per week. These numbers, and a 2-5% targeted compromise rate is not uncommon. In fact, it’s probably a pretty good number. So imagine this.. your company owns 1,000 computers. Giving you the benefit of the doubt, Let’s say you probably have a great, highly trained, very mature infosec team. 20-50 compromised computers per day (7 days/week) could (should) be expected.
4. Assuming you have a great team (I’ll assume that you do!), and you have 20 compromised computers every day (140 per week), and each computer is a 100 Gb hard drive (yes, I know this is small), not including shares, SAN devices, network attached storage, cloud, etc., You need to be prepared to perform incident response on 140 systems and 1.4 Tb of host based data per week, or roughly 6 Tb per month, plus whatever you’ve collected at the iPoP. That’s like examining more than six Libraries of Congress every month!  Now, how might you pull out the good (bad) stuff? Automation (and hopefully on systems that are not compromised!) You’re going to want to run tools against those systems to help you assess which ones are really compromised, or at a minimum, prioritize your work. How do you do that?

So, both Brett and Dr. Schneck are absolutely correct.

Defenders are drowning in data --both enterprise data that must be analyzed, and potential sources of good intelligence and indicators. Defenders MUST gather information from as many good sources as possible. You need sources of known goods, sources of high confidence bads; and, you must share information between your peers, other industries. Look for sources of information that will give you high quality indicators that can be placed in your networks that can proactively block, drop, and stop those attacks before they successfully penetrate your environment.  An interesting observation - smaller infosec shops generally tend to try and save money by seeking out open source lists of indicators. Their teams are often times smart technically, and they choose to spend time reading the open source lists of bugs. While they can obtain a lot of good information, to find those nuggets requires a lot of time reading, analyzing, and evaluating the information before actual implementation. This is a counterproductive. Research times can be reduced significantly, by purchasing a membership in an information sharing group like Red Sky (or others), where many members are reading those same lists, and talking about them in a private environment. This is a game changer. In fact, a July 2012 McKinsey Report states the average worker spends 28 hours per week reading email. The report suggests that knowledge workers (I’d call an infosec pros knowledge workers) using a social environment (like Red Sky?) to exchange knowledge can double the benefits received, and increase productivity by 25%!

This goes to the heart of why Red Sky. How long before your incident responders burn out? What if you could reduce their workload by 25% by participating in our social environment? What if they could be twice as productive by reducing cycle times needed to research cause and effect?  You can.

In conclusion.

• Defenders are drowning in data, and losing the fight. (Harttman)
• Humans are required in the loop to understand the nuances of daily changing threats and attacks. (Schneck)
• Current thinking on how to capture intelligence and information isn’t working. Red Sky Alliance and its public sector portal, Beadwindow, are working. (Stutzman)

Call today for an introduction to our community. With every Red Sky demo, we’ll give you our latest white paper “How Great Companies Fight Targeted Attacks and APT”.  This paper outlines a roadmap, at an executive level, in less than 10 pages, seven items companies who’ve dealt with, survived, and thrived in the face of Targeted attacks and APT have done effectively to defend themselves against targeted and advanced persistent threats.

Until next time,

Have a great week!
Jeff

Red Sky is growing!

Red Sky Alliance is happy to announce the addition of two new senior members of our technical staff.

Bill Billings is a retired Navy Commander (Cryptologic, now Information Dominance) Officer who recently served as the CISO of a Reston, VA startup --Blackridge Technologies. Prior to that, Bill was the CISO for Microsoft's Federal Business where he lead information security for products surrounding projects underway for the Federal Government. Bill brings both a deep technical and executive level skill set to Red Sky. He will be working in the Federal space supporting Red Sky's Beadwindow organization, and with our newly forming research and lab organization.

AJ Brown is our first Business Development Executive, charged with new member acquisition. AJ is  multi-lingual, and spent the last ten years at PC Connections and service connections. AJ has both an IT and sales organizations, and will be working with Red Sky to more formalize it's ability to acquire, process, and welcome new members. AJ will be handling new member requests in the Red Sky Alliance.

We're excited. Red Sky Alliance is growing. Please help me welcome Bill and AJ to the Red Sky team!

Jeff

FR13-003: A string of 0-days; NY Times, and a bit of a rant

The news is buzzing today with talk of the New York Times. I know how they feel, having gone through similar (many similar) experiences, I’d rather take a moment and blog about something not being talked about --the colleges and universities that are being used as hop points for many of these attacks.

For those of you who are surprised by attacks on our news and media outlets, (I read a comment yesterday suggesting “Our pillars of civilization are under attack!”), I’m sorry. You’ve been asleep for far too long.  These attacks are not new. The thought of our university system being used as the jump point may, however, be. Press (and Richard B at Mandiant) are reporting that the NY Times was victimized by attackers using a college or university (I suspect several) in carrying out the attacks which occurred, according to the press, four months ago, although suspicion is, it’s probably a lot more like a couple of years, not four months as reported.

...So, I asked the question of our members. “How many of you are seeing colleges and universities used in attacks against your environment?”

I received feedback from two members today --they included a list of universities who’d been used as attack relay points over the course of the last two years. One was a DDoS that used the college as a proxy, with an estimated 5000 users on campus compromised and broadcasting packets uprange! Mine was a fast turn-around query, but it’s evident that whatever these colleges and universities are receiving for security intelligence isn’t giving them the whole picture.  The schools in this list were, absolutely, without a doubt, used in focused attacks at least this one defense contractor, but my suspicion is they’ve been used in others. It’s scary. These are not small, no-name universities. They’re large, with great budgets. 
 
So why is it that we are content to watch our colleges and universities be consistently p0wned? Watch their intellectual property (our future prosperity) roll out the door? Why is it that we see that the NY Times is hacked and the world turns upside down, but don’t seem to care if someone uses our colleges and universities as their proxy networks to break in and grab data (or worse!) from someone else?

I’m not sure we’ll ever know, except that every college we’ve talked to on behalf of either Beadwindow or Red Sky seemed happy (content), knowing that they receive inputs from an information sharing and analysis center (ISAC), and feel they don’t need any more information. I’m a believer in REN-ISAC. I’m a huge fan of the Collective Intelligence Framework. They’ve done wonderful work in automating analytics... heck, I used some of their model in building DCISE. So the question is this: with such a wonderful engine as the REN-ISAC has put together (other ISACs can be included in this too), is it that they’re not receiving the ‘right’ information? Do they not get information that is analyzed and prioritized as either APT or Targeted? Why, with such a nice analytic engine, are the schools still not protected? I have two thoughts. Either the information they receive doesn’t include breakouts of prioritized threats and protective measures, or I suspect feeds from the ISAC may just be overwhelming the small information security teams at some of these schools. There’s no doubt that the CISOs I’ve met want to protect their networks, but at the same time, like everyone else, they’re strapped for resources --manpower and money.

CISOs everywhere must have two things to be successful:

1. The right information which allows them to make the best possible decisions.
2. The ability to actually act on that information.

So let’s get off this roller coaster. It’s time. Since 1996, as a Navy Officer at the (then) Fleet Information Warfare Center, I (personally) have witnessed colleges and universities become the best jump points --they’re generally fairly open network (students must share information) where experimentation is encouraged and the student networking is at best, heterogeneous, if not changing daily.

So how do we get off the roller coaster? 

Better, smarter, faster information. CISOs and risk managers need better information, formattein in such a way will allow them to prioritize work based on the wolves closest to the sled... and they need it right now

Red Sky Alliance (and Beadwindow) facilitates forums for unique, peer, and voluntary, private to private information sharing to help companies protect themselves - not meet a USG requirement to "share info”. If someone observes suspicious or confirmed malicious activity, it's uploaded to the community, and within minutes, others offer their own comments, often times with explicit instructions for remediation and risk reduction.  I'd also say that Red Sky members' expertise, information & incident response capabilities are bigger, better & more adept than what currently exists within the construct of "one sector" information sharing that exists today in much of the ISAC structure. Red Sky members leverage the expertise of cross-sector peers with better capabilities, funding, tools, solutions & knowledge gleaned from paid conferences, premium feeds, and high-quality organic analysis.

If you don’t care about the strict privacy requirements of the Red Sky environment, the Beadwindow portal may be right for you. Beadwindow allows government users to share information private companies (or colleges) to share real time information about happenings in their environment. The rules limit data usage to information protection (not regulatory, or investigative), so members speak openly.

Interested? Drop us a note.

BT BT (this means Break Break for those of you without a military comms background)

Inside Red Sky this week:

• This week we released our 3rd report for 2013 which provided campaign analytics on a recent string of attacks leveraging 0 day exploit code. This consisted of details on malware protocols and a large amount of related malicious infrastructure. Our analysts also provided tailored mitigations including SNORT signatures and YARA rules. On the portal side, we are bringing in new members daily and our user adoption has reached an all-time high.
• This week we added a TON of new folks to the portals, and have increased our participation to record levels. What does this mean to you? It means you have a much broader group of people that can give you real time comments and analysis from a peer reviewed group. It means you make new friends, connect with new people, and share information in the member-only portals, now, not three days from now (when it might be too late).
• Last, we scored some great space in Manchester, NH this week. We’ve been peppered with research requests, some related to Red Sky, many not. To separate these from Red Sky, we took a lease on space in the in the historic mills along the river. The new organization will operate as Wapack Labs (as a Red Sky Alliance company), and will do contract analysis/forensics, research, and other kinds of things that don’t fall nicely into information sharing construct of Red Sky Alliance. We’re looking for a Lab Director, so if you’re interested, don’t mind living in NH, and can live on next to nothing for a while (it is a startup!), drop us a note. Having a badge is a definite plus.

Enough for now.
Until next time, have a great week!
Jeff

Extortion of a 16 billion dollar company...

I tell this story often. It’s a story of a company -- a $16 billion dollar company, who in 2006, bet the farm on some specific technology. The company wanted to use the High Definition (HD) codec to replace a massive network of movie distribution. If it worked, the company would make billions.

You may recall, at the time, some of us owned HD players, others Blu-Ray. In this case, the HD company came under attack by folks backing Blu-Ray. Patent pending technologies stored in the “HD Company” (we’ll call them HDCo) were harvested systematically from their servers all over the world. HDCo executives were targeted individually --physically, cyber, psychologically. Their email accounts were taken over. They were forced to use throw-away cell phones. One exec even reported multiple occasions of harassment in the parking lot of her grocery store! This is what some of us know as ‘asyncronous warfare’. In the corporate world, it’s considered corporate espionage. I’d call it extortion.

The bullying, harassment, and theft went on for roughly two years. The company couldn’t survive.  In the end (Feb last year), this company rebranded. They’d lost nearly all of their stock value, dropping from a whopping $16 billion global diversified company in 2006 to $160 million company in 2012!

Why do I tell this story?

What’s old is definitely not new again... While many things are, cyber isn’t.

Fifteen years ago when I started my career in information security, I’d watch the FIRST list. We had roughly thirty of us sharing real time information through PGP encrypted emails. The information we shared among us was amazing. When I left the Navy, my FIRST membership went with it.

In 2005 I rejoined, remembering the successes I enjoyed in 1997. This time, there were hundreds of folks participating. The landscape had shifted. Moore’s law applies not only to CPU speed, it also applies to growth of the network. In that short period, FIRST grew from roughly 30 to over 300 --highly indicative of the number of victim computers, and the requisite number of incident responders needed to handle the massive, exponential growth of cyber victimization. Nearly ten times more people needed information, and were smart enough to seek it out.

What many didn’t know at the time, was not only had the number of machines compromised grow, requiring more incident responders, but the very nature of those attacks was shifting under our feet. In 2006 when I first was exposed to APT, it wasn’t like my early days in the Navy when one attacker broke into one computer. In 2006, one attacker might compromise hundreds of computers over the course of only a few days. Today, thousands would be compromised using simple exploits with complex chaining of events. Attackers (even today) will use just enough tech and know-how to get what they want. When a defender ups the game, attackers do too.

• Have you ever heard of a company who has to change the credentials on their domain controllers on a weekly basis? One CISO tells me that if he could change the credentials every 10 minutes, it still wouldn’t keep them out!
• What would you do if you knew every week, every one of your servers running IIS was going to have their credentials harvested, and there’s not a damn thing you can do about it. (Read back in my blog. I described the Windows Credential Editor problem earlier.)
• Have you ever changed a rule in your Intrusion Prevention Systems only to find it changed back twenty minutes later?
• What would you do, if on a weekly basis, 3% of all of your company’s computers were found compromised, with nearly all sending data home? In the companies I work with, 3% means at least 3000 computers --every single week.
• How would you feel, as CISO, if you were informed by the FBI today that you’ve been compromised, and when you did the analysis, found you’ve been completely owned --man in the mailbox, full exfiltration, and attacker control over any box in your network for years?

I talk with CISO’s nearly every day. I’m amazed that even today, with the massive advances in attacker TTPs (Tactics, Techniques and Procedures) in the last few years, that a CISO can look me, square in the eye, and believe with every fiber in their being that twenty year old information security practices --going it alone, simple firewalls, relying solely on antivirus, are going to save their companies from the complex chaining of these simple events --just enough, to breach their companies. I talked to one recently who had no idea what a wateringhole attack was. He was floored when I told him that his corporate webserver may well have been compromised, serving up poison water to every employee who visits.

BT BT

This week was fantastic for the Red Sky and Beadwindow Alliances.
 

• Two new companies joined Red Sky last week. Our provisioning guy has taken to drinking (heavily). I know in one day he did 28 new accounts!
• This month alone (and it’s not over), our accounts receivable are more than all of the revenue we collected last year. It’s not a result of my good looks, or of our sales prowess, it’s a result of members telling others to join... and they are!
• I’m happy to announce AJ Brown and Bob Hillery have all joined the Red Sky | Beadwindow team as Senior Members of the Technical Staff. I’ll announce the third next week. Bob is a retired Navy Commander, a long time SANS Instructor, and a founder of InGuardians. AJ came out of PC Connections as an IT Account Executive. Bob will be handling our intern program, and has already kicked off one major, (provisional) patent pending project that we hope to be using in our portal at some point in the future. AJ is pounding the pavement, charged with new member acquisition from the commercial sector. Both are great guys!
• Our Annual Report FINALLY went to the printer. I’m hoping to have a stack of them on Rick when he goes to RSA.

What’s old is definitely not what’s new. If you’re a CEO, CIO, or CISO, and you’ve not stayed current, drop us a note. We’re scheduling threat briefs. We’d be happy to schedule an online or in-person threat brief for you and your team. You’ll see first-hand the kind of amazing work that comes from a group such as ours.  Live action. Real Time cyber intelligence through crowdsourcing, smart, easy to work with people, and good tools.

I’d like to invite you to join Red Sky Alliance. 

Please contact AJ Brown today.

“Red Sky at night, sailors’ delight... ”

Until next time,
Have a great week!
Jeff

Red Sky Weekly: 0-day, Intel Analysis Report, and new Associate Member...

Cyber Security Intelligence - Live, Real Time, Right now.

I love this. Clear, succinct. It’s what we are. It’s what Red Sky Alliance does all day, every day.

I’m the geek in this operation. My sales skills aren’t everything they could be (not necessarily a bad thing), but we pushed through 2012 successfully. Yesterday I was doing the third interview with a new Business Development Executive that we were looking at for new member recruitment.

As part of the interview, I asked him “In your words, what’s the value?” He replied, without missing a beat --it’s live. Red Sky isn’t a movie that was recorded live but played over and over, it’s live. It’s conversations and actions that members can use now.. in real time, talking to others, seeing what’s happening in real time. I could see it in his eyes. The light bulb had gone off and was fueled by the contagion we all feel when we realize just how powerful a community such as our two (Red Sky and Beadwindow) can be...

I think our new membership guy defined our new company slogan..  

Cyber Security Intelligence - Live, Real Time, Right now!

BT BT

Intelligence Analysis Report 13-001 (IAR 13-001) released: Over the course of the last couple of months we’ve been working with one of our members in analyzing and authoring an in-depth analysis of one of the most prolific and damaging APT groups out there today. The group claims thousands of jump points into and out of thousands of commercial, defense, and government targets, including we believe, much of the chemical sector last year, well known IT security companies, and dozens of others, stealing enormous amounts valuable intellectual property from each as they’ve ravaged their way through cyberspace over the course of the last couple of years. We labeled this report ‘Intelligence Analysis Report 13-001’. It’s a little different than one of our Fusion Reports. The IAR focuses more on the people, how they work, and what they want. The report consisted of over 20 pages of high-level analysis on tools, targeting, infrastructure and identifying information on suspected actors.

Other happenings:

• 0-day: Red Sky analyzed recent 0 day. Feedback from one member confirmed that analysis from Red Sky enabled this member to mitigate the activity from this 0 day on his company’s network.  

• New Associate Member: This week we’re joined by a newcomer to the security intelligence space - Exodus Intelligence. Exodus is this cool little company that does 0-day research, selling subscriptions to finished reporting to their subscribers. For those of you who don’t know what an 0-day is, an 0-day (zero day, or oh day) is a new vulnerability that isn’t yet published in the wild. The Exodus team is now in Red Sky, and will be interacting directly with our Alliance, providing real time 0-day discovery, discussions and participating in our crowdsourced analytic intelligence engine.

• New folks: As mentioned above, we’d posted three positions on UpLadders last week. In the few days the ad was running, we had probably 25 applicants before we turned the ads off. In the end, I’m happy to report, we have extended offers, and all three have accepted. Two of these new folks are linguists and one deep technical. Our first two start on 2/4, and the third, during the first week in March.

• Beadwindow: We’re working contractual language with our first Federal Government Beadwindow member, and we’re hoping to have them in very soon.

A quick admin note: I’m sad to say, Dave Chauvette, our Director of Academic Services has left Red Sky to pursue activities more inline with his long term interests. Please direct any messaging regarding internships to me.

Oh, and before I forget --an update on my piece from last week. Remember that CIO with his head in the sand? I gave him a threat brief... went to his office, sat side by side with him, and gave him a threat brief to show him what's going on around him. The outcome? he's agreed to use an outsider for incident response and triage analysis. His Carbon Black server should be arriving tomorrow. 

If you're interested in having your CIO, CEO, or management team receive our threat brief, please drop us a note. We'd be happy to set something up online, or in person. We've got qualified people in New England, DC and St. Louis, MO areas and would be happy to arrange a time. 

So, another fantastic week in Red Sky Alliance!

Until next time,
Have a great week!
Jeff