Digital Forensics

I've mentioned before that I'm not a forensics guy by any means. I've never done any "real" forensics, at least not anything beyond simple looking for fairly obvious evidence of a breach or problem. I enjoy reading about digital forensics because it fascinates me. The way that data can be extracted from media after it has been deleted, hidden, and even when the disk has been formatted. Not to mention how someone who is trained can look at the system and determine what happened, how it happened, who did it, how they gained access to the system, etc....

Last week I read this post by Harlan Carvey here. This quote that he made got me to thinking:

My personal thought on this is that ideally what an organization would want to do is develop an in-house capability for tier 1 response...trained folks whose job it is to respond to, triage, and diagnose a technical IT incident. By "trained", I mean in the basics, such as NSM, incident response, troubleshooting, etc...enough to be able to triage and accurately diagnose level 1 and 2 incidents, as well as preserve data until outside professionals can respond to level 3 or 4 incidents.

What is it that companies really need? What are the basics to ensure that triage is done in a manner that doesn't compromise "the crime scene". I decided to post that question to my friends in the Security Catalysts Community here. As I expected I have gotten some good responses.

On Thursday of this week I attended a one day event put on by ISC2 called SecureAtlanta 2008. I had forgotten what the topic was and it turned out to be Digital Forensics. It was a high level discussion that covers a lot of the basics of what DF is and why companies need to be informed and concerned about it. Not much of the content was technical but it was informative. One of the things that grabbed my attention was the topic of DF and the law. We need to keep in mind that what we are doing in incident response and forensics needs to keep in mind the possibility of going to court. Our findings may need to be presented in court to convict or defend. Therefore we need ensure that our teams are trained in the basics but also trained in how to not contaminate the crime scene.

One last thing to consider is that just as all things related to security there has to be a balance. We have to balance IR and DF with ensuring that we get (or keep) the company running. We can't forget that our company probably relies on these systems running in order for them to make money. So if your company doesn't have proper policies and procedures in place for this that you start the conversation with your boss. Then work with management to put in place the proper program and training get put in place.

Reacting or Thinking

Yesterday I drove to work which isn't something that I typically do. I like my sanity too much (what little is left) to fight Atlanta traffic on a regular basis. I woke up late and missed the one bus that will get me to the office in a decent amount of time so I decided to work from home for a couple of hours and then drive in after rush hour was over. I had the same thought process for my commute home. Leave before rush hour and work remotely for a couple of hours. So I left early and went to my favorite coffee house and set up office for a while. I let my wife know that I was close by in case something happened and she needed me in an emergency.

Some would say that I was setting myself up for this but about an hour later my cell phone rang and it was her. "You've got to come home right now! Bella drank about 1/4 cup of Hydrogen Peroxide!" CLICK My phone went dead just as I was about to tell her to call Poison Control. So, I packed up quickly and hit the road. I called back to calm my wife down and to have her call Poison Control. When I arrived home my wife informed me that our youngest daughter may have also drank some of the peroxide also.

My wife was rushing around getting ready to take the girls to the doctor and getting upset with me because I wasn't panicking. I knew that peroxide could be dangerous to a child if enough was ingested but I also knew that it would cause them to throw up soon. So I convinced her to wait a while and see what happens. I also asked my daughters about how much they had actually drunk and called Poison Control myself to talk to them. It turns out that the oldest only had a "good swallow" and that the youngest just tasted it. The oldest did throw up and Poison Control told me not to worry.

That got me to thinking about how IS/IT teams often react to emergencies at work. Do they panic and rush into a plan that hasn't been thought out or do they take a deep breath and look at what is going on and try to learn the facts of what has happened and what their options are? If you don't have an incident response plan I can tell you that more than likely people are reacting instead of thinking. Even if you have an IR Plan if it hasn't been tested and the team isn't familiar with the plan and their role in the incident they will usually just do whatever comes to mind first. Sometimes that works well and sometimes not so much. You can't take that chance.

Incident Response Poll Results

The Incident Response Poll closed last week and I was out of town over the weekend so I didn't get a chance to write up the summary. Here are the results:

When it comes to Incident Response does Your Company

Have a formal and tested plan	8 (25%)
Have a plan that hasn't been tested	2 (6%)
Has a general idea what they will do	9 (29%)
Not have a plan	12 (38%)

67% of you answered either "Has a general idea what they will do" or "Not have a plan". That's not very encouraging. It shows that we have not done a good job in conveying the need to management. Perhaps you don't think that the need is that great. I live in a world filled with compliance and most regulations out there require an IR plan. That alone should be enough for you to take to management. Not to mention the sheer lack of understanding of what needs to happen to respond to a breach. If you don't have a plan then how will you know what to do? Do you disconnect the system from the network or leave it connected? Do you power it off or leave it on? Do you have to notify the police? The FBI? A financial institution? Your Customers? Your employees? The media? If you don't know now how do you think you will know when the time comes and you are in the heat of the moment?

A IR Plan details all of this. It tells you what to do and what not to do. It tells you who you need to notify and how to do so. It tells you how to stop breach from continuing and how to clean it up. All of these things and much more are included. Things that can make the difference in a successful incident response and one that is a dismal failure. A successful one is one that your company survives and continues on with little impact. A failure may mean that the company has to shut their doors and go out of business. It may mean that the company survives you you don't. It may drastically alter the way your company does business. That may be good or it may be bad.

If you yourself don't understand the need in a IR plan PLEASE, PLEASE, PLEASE!!!!!! do some research and discover the need. If you do understand the need but haven't been able to communicate it effectively to management PLEASE, PLEASE, PLEASE!!!!!! do some research and find someone who will help you be able to do that. The Security Catalyst Community is a great place to start with that. There are people there who will be able to help you understand the need and be able to communicate it effectively.

For the rest of you that have a plan I only have a little to say. First, congrats to you and your companies for seeing the need and doing something about it. Second, please ensure that it is kept up to date. An outdated plan is almost as bad as not having a plan. Third, if it hasn't been tested please talk to management about testing it. Even if it's just having several people review it and ensure that it makes sense that is better than nothing.

New Poll on Incident Response Plan

I've just posted a new poll about company Incident Response Plans. This is an area that is often over looked and under planned. Many companies don't even realize that there is a need for an IR plan and have no real idea what they would do if an incident occurred. In this day of legal and compliance issues having a plan is no longer just a good idea. The lack of one could cost your company lots more than the cost of clean up. You need to have a plan of attack for a variety of different incidents. The way you would handle a virus outbreak is different than how you would have a server compromise that exposed financial or customer data.

If you don't know where your company stands in regards to an IR Plan don't just take it for granted that they have one. Ask your boss and if there isn't one inform them of the necessity and importance of one. Be prepared to either volunteer to help or be volunteered. :) Do your homework and you may come out smelling like a rose.

Here is the question and the possible answers to choose from. You can find the poll itself here.

When it comes to Incident Response does Your Company

	A. Have a formal and tested plan
	B. Have a plan that hasn't been tested
	C. Has a general idea what they will do
	D. Not have a plan

When It Rains It Pours

It's always good to be prepared. I try to be prepared for situations that may arise both in my professional and personal life. I look at potential threats and issues that may arise and see what I need to do to be prepared. That's part of any good security program. Having controls in place to prevent a breach or reduce it's impact are important in securing a network, web site or application. Doing preventive maintenance on your systems (patching, monitoring, etc) will help ensure that they are in shape to prevent holes that can be compromised.

Similarly at home I check for leaks around windows and doors. Change my HVAC filters, maintain my vehicles to ensure that they run properly. There are many, many things that need to be taken into consideration to ensure that you prevent problems and are prepared in case they occur.

On Friday I had an incident happen that I wasn't prepared for. I had left work about 2:45 in the afternoon to ensure that I missed the bulk of the terrible Atlanta Friday afternoon traffic. I'm traveling up the interstate when I hear a loud roar coming from my Jeep. I immediately turn of my radio and start checking for smoke or flying parts in my rear view mirror. I quickly pull to the side of the road and discover that I have a flat tire. So I get out my jack and lug wrench and start removing the flat tire. I get it off and grab the spare (one of those crappy temporary tires) and the first thing I notice is that it is also flat. Oh yeah, I forgot to mention that by this time it is now raining VERY hard. So I'm getting soaked while changing one flat for another.

I go ahead and put the spare on and then walk back to the drivers side of the car to get in. I had forgotten that my windows was down so now my drivers seat is also soaked. I then call the Georgia Department of Transportation to inform them of my situation. One of the good things about living in Atlanta is that they have incident response trucks to assist stranded motorist.
After about a 45 minute wait my HERO (Highway Emergency Response Operations) arrives and puts air in my tire and I'm on my way home. Of course now I have to sit in traffic because by this time it's about 4:30.

So, what did I learn? It's the little things that can bite you. Not keeping an eye on the air pressure in my spare cost me time and a headache. What is there at work that is possible being overlooked that may come back to bite me in the butt? Unfortunately I don't know what it is right off but I'm going to start looking at those little things a little closer.

Incident Response belongs to everyone

Harlan asks "who decides what best practices are" in regards to Incident Response. Harlan is a forensics guy and has written an excellent book (I've only read 1 chapter but many others have told me how good it is) on Windows Forensics Analysis. Obviously forensics plays a part in many Incident Response scenarios. His answer to the question of who decided best practices is "It depends". And I agree.

Dr. Anton asked how PCI can be both complex and basic security. He asked that based on the fact that my PCI poll (as scientific as it was) said that 40% of you said that PCI is complex and 40% said that it was basic security. My take on that was "It depends".

On the Security Catalysts forums someone asked if NAC had any real value due to the fact that there are ways around it. My response again was "It depends".

Security depends on your company, your environment, your level of risk and risk acceptance. It depends on the level of competency of your IT and security staff. The level of competency of your end user employees. What partners, contractors, visitors, etc that are allowed to connect to your network. What controls you are willing and able to put in place. What policies you have and enforce. What level of buy-in you have from management. What does your IT environment look like. Is it new, old or a mix? It is small, medium or large? Is it complex or simple? Do you have lots of different apps or only the core ones required to do business. How big is your Internet facing presence.

This list could go on and on and on and on. There are too many variables to give a concrete answer to these and other similar questions. So the real answer is that it doesn't matter what your idea of the answer is. Your job, as a Information Security Professional is to do the best you can with what you have and plan for the worst. That is where the concept that IR belongs to everyone comes in.

Many companies have IR teams that jump into action at a moments notice. But what happens between the time a incident is discovered and the team is able to take action can make all the difference in the outcome of the teams work. The rest of the company, from end user to IT/IS needs to know what to do in the event of a incident. If they don't then they will invariably do something wrong that will hinder the investigation and fixing of the issue.

I've written too much so I don't have time to go into details here but suffice it to say that IR goes way beyond the team. It has to be dealt with at ALL levels if success in dealing with an incident is your goal.

The best laid plans....

I was talking to someone briefly the other day about the CIA triad and it got me to thinking. Most security books teach it and many security professionals will agree that it is foundational to Information Security. As you all know the 3 legs are Confidentiality, Integrity and Availability. We all work hard to ensure that our data stays confidential, that it's integrity is maintained and that it is available to authorized users when it is needed.

What I want to talk about is Availability. What does it involve and what are we doing to ensure that data truly is available. Availability can be affected by the following (and more that I'm sure I will miss).

• Denial of Service Attacks
• Hardware failure
• Improper device configuration
• Man-in-the-middle attacks
• Corruption of data
• Removal/deletion of data (intentional and unintentional)
• Route poisoning (ARP,DNS, etc)
• Software bugs

These things affect Information Security yet are often looked at as either belonging to another group (Network, Servers, Firewall, etc) or not being a big deal. When this happens you are setting yourself up for failure.

The best way to assure the availability of information is to have a plan and to test it.

• What is your plan to prevent MitM attacks, Route poisoning, DoS attacks? Do you test your systems to ensure that these types of attacks can be fended off? Do you have a plan to mitigate them? What about an incident response plan? Has it been tested and carefully thought through?
• What about data corruption or deletion? You have backups but are they any good? When was the last time you did a test restore? What happens if your tape drive goes bad? Can you restore on a different model if necessary?
  
• What steps are in place to ensure that devices are configured properly? Do you have procedures to ensure that they are configured and tested? Is the configuration backed up and documented in case of hardware failure? How quickly can you get the device back up and running or replaced? Say you lose a server with all your user files. You have a spare that you can restore to quickly, but what about ensuring that the users can connect to the new device. It likely has a different IP address and name than the original box. What are you procedures for uninstalling applications and patches that cause problems?

These are the types of things that can easily be over looked if you have not done your homework. You need to do a Risk Assessment and ensure that the basics are covered. You need to then put a plan in place and test, test, test. It's not always the most fun thing to do, but in the process you will learn a lot about yourself, your network, your coworker and your company. It might even keep you out of the unemployment line.