Better safe than sorry – How we get prepared for D-Day

by Felix Rust / October 23rd 2018

Believe it or not: It takes more than you see in the movies when it comes to handling a fire close to the playout center or a bomb threat on the set of a TV show. Therefore, emergency plans and security concepts are prepared well ahead to secure sites and react accordingly in time.

As our business model and services shift more and more into the digital world, it became obvious that the same amount of preparation is needed to provide reliable services. We need to secure broadcast infrastructure and streaming platforms for multiple channels to ensure services for our viewers. Moreover, ProSiebenSat.1 Group holds many assets conducting B2C e-commerce involving lots of customer data, which must be protected adequately.

That is why we decided to prepare for D-Day and laid out a plan to be carried out in case of serious breaches and information security incidents. It also enables our small and largely independent units to benefit from the infrastructure and expertise a large company can provide.

With this article we want to share our ideas about how to prepare a highly diversified network of companies within ProSiebenSat.1 Group for serious information security incidents. The idea is to offer a service to our units which should help them in handling breaches by instantly getting access to resources from the headquarters to manage these harmful situations.

Once the root cause of a breach is tracked down, the know-how needed to fix a problem is available in the respective DevOps and operations teams of the individual units. However, investigation, judicial consideration, technical or procedural assessment or other areas of special expertise may be required to address breaches effectively. Also, to distribute information to all necessary stakeholders there is need for sensitive and well-orchestrated communication.

Thus, the key concept to handle cyber security breaches in a network of media and e-commerce companies is a reliable process to ensure quick communication and well-defined responsibilities. This may sound straight forward. However, at ProSiebenSat.1 we live in an ecosystem of various companies, which are sometimes more and sometimes less interconnected. That’s why we came up with the approach to provide a communication hub for all our units and assets which they may address in case of a serious information security incident or breach. The following processes are agreed on with our units and ensure quick response and remediation in the event of a breach.

Define roles and responsibilities

Each unit provides a “cyber breach response owner” and makes sure he or she is part of the incident management process within this unit. This may be a manager or central contact person within a unit.

These response owners keep oversight of the situation at their unit. Minor problems may be dealt with solely within a unit. But we agree on conditions which make it mandatory for the response owners to contact the group’s central cyber breach response management. In that case, they stay in contact with the headquarters until the breach can be considered closed and conclusions are drawn.

ABCD_units

Now, who should be contacted? We provide a single point of contact (SPOC) for the units’ response owners. This is a group of managers experienced in IT security, information security, or data protection. They assess the severity and nature of a breach immediately. They have enough overview over the company to quickly establish contact to all required persons. In case the SPOCs recognize a breach that might develop into an emergency or even a crisis (which are defined by impact on our business processes and revenues), they can activate the group’s crisis management processes.

SPOC

In any case, as soon as the SPOCs have been approached and they recognize a breach, they coordinate the necessary resources to solve the problem and immediately set up a breach management team. This team may invoke activation of the security operation center to provide analysts and technical expertise or to consult the data protection officer. The persons involved agree on how to perform the following chain of steps.

chain_of_steps

1. Plan & Prepare

Any organized handling of breaches requires to have a framework in place, such as defined responsibilities, documented response and communication plans as well as trained personal. Therefore, roles must be assigned, escalation processes within individual units have to be agreed on, the processes must be trained, and all material must be kept up to date.

2. Detection & Reporting

As soon as a breach is reported or otherwise recognized, evidence is collected, analyzed, and documented. Then the SPOC is contacted and briefed. The experts necessary to handle the breach are contacted and start to assess the situation.

3. Assessment & Engagement

After indicators for breaches or evidence have been collected and reviewed, all necessary information for further steps is collected and relevant parties will be involved. These may be internal or external experts to carry out analyses, forensics, or implement changes in the infrastructure.

The cyber breach response team now assesses the severity of the breach and the possible impact on business processes, privacy, and compliance. The result determines how to respond to the breach and who must be informed.

4. Response – Containment & Mitigation

In this step, the possible damage must be reduced to a minimal extent. The necessary steps to mitigate negative impact and recover from the incident are determined according to the previous assessment. As an example, think of a roll back to a safe state of an affected server or limiting access to critical components which are potentially impacted by the breach.

5. Response – Elimination & Remediation

The affected systems have to be analyzed and the root cause of the breach will be eliminated or at least mitigated. Then, the effectiveness of all measures taken has to be checked thoroughly. This step may include tedious modifications, tests, cleanups, or fresh installations of affected systems.

6. Response – Recovery & Closing

The parties concerned agree on how and when to bring back all systems to normal service. This is carried out under close monitoring.

7. Lessons Learned

Looking back and validating actions, processes, documents is key to learn and improve. Thus, before calling it a day the response team has to ensure that the cause for the breach is identified, measures for mitigation are in place and proposals for future enhancements of technical or procedural nature are submitted to all relevant parties.

Yes, this may sound straight forward. But to carry out a plan reliably and quickly it is essential to brief all parties involved, to stay in contact, maintain lists of people in charge of the different steps and prepare to-do lists for each step. This reduces stress and takes pressure from the persons involved. It also decreases the amount of errors or oversights significantly. (Would you have thought about disabling log rotation during incident analysis to prevent loss of evidence? Do you know how to prepare evidence for court?).

The process itself should be generic enough to deal with as many situations as possible. It is designed to handle non-standard situations that may occur despite the preventive measures we implemented.

Therefore, staff must be experienced enough to act deliberately and competent in any event. Still, some scenarios are more likely than others. Therefore, we prepare some dedicated lists of steps for likely events and for those that may cause significant trouble.

At the end it boils down to practicing the paths of communication, reviewing the plan regularly and bringing together the right people at the right time. Fortunately, we have a working culture that allows for all of this.

MORE BLOG POSTS