Friday, November 2, 2007

What do you use for an incident resposne console?

As security practitioners, what do you use for an incident response console? Every security vendor has their own management interface for their products, like Symantec Security Center or Cisco's Pix Device Manager. Are you consolidating alerts and data from your security devices? Is it going into a SIM / SEM? Are you integrating in with a ticketing system (like Remedy) for tracking incidents? I am trying to understand what products / technologies people are using to manage security incidents.

Excellent question. Security responses should be correlated with network and service events and presented as a unified situational response to operators. Then the service engineers in each domain should have technology tools that let them coordinate their response and the response to impacted customers. No tools existing today fully accomplish this. However, I have found the as a general purpose response system Remedy has merit; the newest entry I like is Frontline which incorporates the ITIL processes and integrates well to other systems. I currently writing an article on automation in responses. What is your take on this?

I’m in the middle of a SOC implementation for a client and we decided to go with a SEM centric architecture which allows the consolidation, filtering and correlation of events coming from various sources such as network components, switches, firewalls, IPSs ...etc.

Such architecture has the merit of providing a consolidated view of the security posture based on multiple event sources which is the ultimate goal of a SOC. Also, a SEM provides and great assistance in dealing with the huge volume of events which is a pre-requisite in achieving the maximum value of a SOC.

From an operational perspective, although we think the integration of the SEM to a ticketing systems (i.e. Remedy) is a must in order to ensure the proper handling of all the alerts produced by the SEM and produce meaningful KPIs, we elected to postpone this integration to the next phase. For now, a SOC officer role will be responsible of analyzing SEM alerts and deciding whether to create an incident ticket and which severity/criticality to assign. This is appropriate in the context because the operations organization doesn’t have any prior experience in SOC operations and processes and we will need to pilot the whole thing for a couple of months before automating the ticket creation process.

Our near future plan is to integrate the SEM to an ITIL compliant operational platform (i.e. Maximo) that we’re already implementing for the network operations to allow for the NOC/SOC organizations convergence.

Hope this helps

A previous employer built out the SEM using NeuSecure which became what is now the Tivoli Security Operations Mangement (TSOM). This platform was chosen for it's ability to collect logs from mulitple vendor devices (firewalls, IDS, etc..). I can tell you that it was never able to handle the load of 1000 plus security devices. The interface was slow to unusable.

They started re-evaluating the Enterasys console. Which is much faster and is doing a better job at gather and alerting.

The SEM system is being integrated with our Remedy ticketing system for tracking and metrics

We have been building information security management infrastructure for our customers at several sites. Incident response can be a part of several other tasks so it is hard to have a single console (incident response tasks listed @ http://www.cert.org/csirts/services.html). But in daily operation we do use SEM and ticketing consoles simultaneously. Depending on the reliability of the automatic correlation of events, you may even use a single ticketing console and dig down the events when needed. For me, the basic IR components are as follows:
1- Process Framework – You need a methodology for building the incident response system... Depending on your requirements, resources you may choose ISO 27001, ISACA, NIST based risk management models, or IETF, CERT, OGSF, type CSIRT procedures... Whatever you do, you need to define the incident response process well. There are a lot of resources, books, articles, guides on the technical and operational side. Let me know if you have any questions on that side.

2- Unified Log Collection and Event Correlation – Once you define your processes, it is time to choose the tools. If your infrastructure is not single vendor, you will need a centralized way of collection and correlating events... There is no silver bullet, but there are a lot of tools. Architecture wise you need to define agent based or agentless systems, remote log collectors, aggregation points, traffic forecasts, processing requirements etc. You may choose generic network management powerhouses like HP Openview, CA Unicenter, IBM Tivoli, Micromuse Netcool or specific security SEM players like RSA Envision, Arcsight, netforensics etc .. If you have homogeneous single vendor environment, Cisco Mars, Novell, Check Point Eventia, Symantec type solutions work as well. You do not need to spend big money on SEM if you have limited budget, there are open source log managers or low cost tools like what’s up.


3- Ticket Management/Escalation: For Incident Response, a solid ticketing system is very useful. Regardless of the SEM, NMC tools deployed, you need a helpdesk system. Gold standard is Remedy , but it is for the large enterprises with solid customization capabilities, once the events are correlated on SEM , and marked as incidents you can manage the whole escalation in your ticketing system. There are 1000s of alternatives for ticketing systems. You need to integrate the SEM systems with ticketing systems.

4- 3rd Party Communication and Integration: Messaging with other Computer Security Incident Response Team (CSIRT)s , private vulnerability research centers, managed security services providers, in the cloud vulnerability management services requires integration of your escalation procedures and tools, during the design phase

At our own operation, we have built our own log collectors, agents, receivers, correlation engines, agent consoles, correlation and business rules engines because of the specific requirements of the operation, the main drivers were to have a single console for operators and increase efficiency, capacity and security. We still utilize Remedy for asset, change and issue management as well as regular escalation.

Let me know if you have a specific question.
I think the answer depends on a number factors but most importantly cost, from a customers prospective. We asked ourselves the same question before we made the decision to build our own enterprise management system.

A number of the solutions mentioned are extremely expensive to purchase and maintain, relegating them to use by Fortune 1000 companies and the Federal government. That leaves a lot of market to be addressed. Our approach was to do "both". Let me explain...

Due to wide variety of SIM implementations we found it was difficult to narrow it down to a select few, although some obviously standout like ArcSight and Tivoli. In the end our approach was to build our own SIM based on open source (for flexibility and fast engineering modification) with an eye towards being both a central, standalone SIM but also feeding into a larger system via SNMP/RMON like agents or through data mining on the backend (i.e through a SQL interface).

Only recently has there been activity around standardizing SIM information into a format that could be used industry wide. Adoption and use remain to be seen although the conversation is promising.

In the current environment, I don't see how you can get around having an architecture that allows you to be both the "master" and potentially a "slave". Unfortunately the days of "plugging in" like the old days of HP OpenView and the like don't totally apply to security events. Standards should fix this..

Hope that helps....feel free to drop a note if you want to brainstorm more...
Yes, we consolidate all our security alerts from the distinctive monitoring systems, some of them custom, into one security console which throws alerts to the network monitoring system and flags particular events for ticketing in our help desk system. It's a work in progress, but it's mostly functional. The names of the vendors are not important, but if you're developing something at your house, you can come by and talk shop. Which should imply that vendors and resellers are not encouraged to drop by. We'll call you; or else show up at a SANS conference. Real world implementers are welcome though. If I should put together a presentation on this for someplace, suggest away.

Clarification added 3 months ago:
I think obviously that size matters. We've got hundreds of core systems over an inter-city area. With that much disparity over that much distance, it's impossible to consolidate into a single platform. Smaller or simpler networks can hit that goal, but we can't. I love our system, it consistently gets praise during our penn tests

An excellent question and one I battled with for a number of weeks. After looking at numerous products on the market, we failed to find anything that fitted our needs 100%. We designed and built our own custom IR console in PHP with an Oracle backend using Remedy for ticketing purposes. This allowed us to tie in features such as our grey net monitoring system, honey pot analysis, sourcefire defence center and Cisco's Pix Device Manager into one single and simple system.

Hope this helps you.

1 comment:

danlighter said...

I found this blog very helpful. Incident response process is a collection of procedures aimed at identifying, investigating and responding to potential security incidents.