Six Sigma for Security, Part 5: Staying in Control

After you have moved through the define, measure, analyze, and improve stages of your Six Sigma for Security project, you are ready to move into the control phase. Your security operations should always operate in the control phase. If so, the control phase for Six Sigma is merely a matter of adding your improvements into your existing controls.

Here are some tools that you may find helpful in the control phase:

• Control matrices
• CTQ dashboards
• Control charts
• Performance graphs

The more of your metrics you can automate, the more successful you will be in the control phase. If you have to spend time every week or every month to pull data, manipulate it, and report on it, you will quickly stop measuring. Use Excel and Access extensively to make routine measurement and reporting a snap.

The control phase does not use the detailed analysis tools used earlier in the Six Sigma project. It only reports on the results. If you find that you are no longer meeting your targets in any CTQ goal, go back through the Six Sigma process again to see what caused the change. You may have something influencing the process causing variation, or the quantity of work may require you to re-examine resource allocation.

Have you made it this far through the Six Sigma for Security process? How has it worked for you?

Read More...

Six Sigma for Security, Part 4: Making Improvements

You have made your security program measurable. You have identified problem areas that keep you from meeting targets in your critical to quality (CTQ) goals. Now it is time to move on to making improvements and measuring their effectiveness.

One of the great things about Six Sigma is that improvements are tied to specific metrics, with a goal to reduce variation and improve your average. To make this work, continue with your CTQ goals, tied to root causes for problems within those goals. Each improvement project should be tied to these root causes. This is the first stage of the project in which you get to look at solutions.

One of the easiest ways to identify the best improvement opportunities is brainstorming. For each root cause problem, work with a team to list as many solutions as possible. They don't have to be practical or affordable. Your goal is to find as many creative solutions as possible. When working with a team, ask the team not to place any value judgment on any of the solutions. Instead, rapidly throw out ideas and keep moving.

Next rank each improvement idea in each root cause area according to the following attributes, such as on a scale from 1 to 4:

• Ease - Rate each improvement on how easy it is to implement. You are looking for quick wins.
• Impact - Rate the likeliness of the solution solving the root cause problem. You may find good solutions, but they may not be for the problems you are trying to address. Make sure you are tying it to the root cause.
• Affordability - Rate how affordable this solution is.

Multiply the above factors together to create a score for each improvement idea. The top scores are your best choices to make measurable improvements. They combine positive impact with ease of implementation and affordability. By tackling improvement ideas starting with the highest score, you can use your limited resources in the most effective way.

After implementing each improvement, measure again and see if your CTQ metrics have improved. If they have improved dramatically, congratulations. You have found the best solution. If they have improved slightly, try moving down your list of improvements to see if the next solution will help.

It is important to note that not every idea on your improvement list should be implemented. Only the top scoring ideas. The other ideas have less impact, take too long, or are too expensive. After trying your top improvement ideas, discard your list and start over to find valuable solutions.

Finding the best, measurable improvement opportunities for security management departments is important. Companies keep security lean. Six Sigma for Security will help you to make the best use of your time and resources.

Read More...

Six Sigma for Security, Part 3: Analyzing the Data

Making security measurable is only the beginning of Six Sigma for Security. The purpose of security metrics is not the metrics themselves but what you do with them. The next phase is to analyze the data and find opportunities for improvement.

The analysis phase is what makes Six Sigma stand out. In it you will use statistics to find where you can make improvements. There are two improvements in your measurements that are important: 1) average performance; and 2) variation.

Your average performance is an important measurement because it allows you to track and trend how well you are doing at protecting information assets. But performance may vary widely around an average. For example if your average turn around on a user termination is 15 days, it may be that half of your requests are 30 days. Some may even linger for 90 days. This is very wide variation around the 15 day average.

Your goal in the analysis phase is to quantify variation and identify factors that can positively influence your performance. This is accomplished by using a variety of statistical analysis tools.

One of the most popular and easy-to-use tools is Pareto analysis, which allows you to quickly find the biggest problem areas in your metrics. It also results in a distinctive chart that shows you frequency and a cumulative percentage to visually quantify the impact of improvements.

Analysis is one of the most challenging phases of Six Sigma. Until now the process has been very consistent and predictable. In this phase, you may try analyzing data one way, find nothing worthwhile, and try it again in several different ways. It is a trial and error process in which you don't stop until you find something of significance that can lead to improvement.

At the end of the analysis phase you should end up with root causes for problem areas in your security metrics. If you are confident that you have arrived at the root cause, you are ready to move on to the improvement phase.

Learn more about Six Sigma tools on Wikipedia.

Read More...

Six Sigma for Security, Part 2: Measuring

After you have defined the Critical to Quality (CTQ) goals of your security program, the next step is to determine how to measure them. Creating a measurement plan for security is like creating a scoreboard. It allows you to gauge at any time how successful you are and your level of protection.

Keep in mind that at the outset of this Six Sigma project we determined that we would not just measure the results of security, which should be few and far between. Instead, we are focusing on measuring the level of protection.

In the measurement phase, we take the four CTQ goals and create a measurement plan for each. Here are sample metrics you may decide to capture.

Policy

• Annual review/approval of policy completed
• Percentage of policies with corresponding controls
• Number of policy exceptions documented
  

Process

• Percentage of controls followed as scheduled
• User access management SLAs
• Incident response times

Partnerships

• Number of security awareness sessions completed
• Number of monthly cross-department meetings completed
• Number of customer contacts made

Payoff

• Internal policy violations
• Security incidents resulting in loss
• Virus/malware incidents
• Compliance gaps

These measurements should get you started. To determine your measurements, brainstorm to create a list of everything that would be nice to measure. Then decide which three or four are the essential metrics. If you have more than three or four measurements for each CTQ, you will quickly be overwhelmed with data.

The next step in building your measurement plan is to identify the data source for each of the metrics. You may find that you don't have a good source of data for some. In these cases, consider ways to begin capturing it now. You may simply use a data collection worksheet to keep track. Keep it simple, but automate it whenever possible.

With your CTQ goals defined, your metrics for each determined, and your data sources identified, you are ready to begin collecting data. For some measurements, you will be able to retrieve historical data and begin your analysis immediately. For new data sources, you may need to capture data for a month or two before you have enough information for analysis.

Complete your measurement plan and capture data as soon as possible. This should be the quickest phase of the Six Sigma for Security project. It will establish a baseline that allows you to set your goals for improvement. It will also allow you to move into one of the most interesting phases -- analysis.

What are some of the metrics you are using to keep score for your CTQ goals?

Read More...

Six Sigma for Security, Part 1: Defining Success

How do you measure the effectiveness of your security program? Unlike business metrics which have tangible goals, such as revenue growth, inventory reduction, and sales force effectiveness, security's goal is to prevent internal and external breaches. Your goal in security should be to have nothing to measure - no virus attacks, no external breaches, no successful social engineering attacks, no insider exploits.

Although measurement of security incidents and related losses has value, it is not sufficient for measurement of security effectiveness or for justifying the cost of security investment. It is important to have a different standard for making security measurable.

Instead of measuring loss, this article proposes measuring prevention. Security management changes over time, but it is generally accepted to consist of a set of best practices for prevention of business loss caused by security breaches. If you do well at following the standards established by ISO27001, ITIL, ISM3, or CoBIT, you will have exercised due care and will have have provided excellent protection for your information.

But how do you know how well you are doing in following these standards? This is where Six Sigma comes in. Six Sigma is an improvement process that is commonly used in manufacturing. In recent years it has been applied to various types of businesses. The goal of Six Sigma is to define how to measure success, measure the current performance, analyze the data to find root causes of problems, implement improvement projects and test their effectiveness, and move newly implemented processes and practices into a controlled phase. To remember this project life cycle, the acronym DMAIC is used. It stands for:

• Define
• Measure
• Analyze
• Improve
• Control

Applying Six Sigma to security management means walking through the DMAIC process. The most critical stage is the Define stage, where the Critical to Quality (CTQ) goals are defined. These goals are carried through the process and are used to measure success. For most uses of Six Sigma, CTQ goals are defined by finding the voice of the customer. In security management, because of the growing body of knowledge and best practices, they are easier to define.

Consider these four CTQ goals for measuring your security program. They are especially geared toward measuring how well you are performing standard best practices in security management.

1. Policy - Measuring the quality and enforceability of your security policies
2. Processes - Measuring how well you follow defined practices according to the standards you adopt
3. Partnerships - Measuring your efforts to build relationships with other departments and customers
4. Payoff - Measuring incidents and resulting business loss

With the CTQ goals of security defined, the next step is to create a plan for measuring them and begin capturing this data.

These CTQ goals for security are generic. Do they work for your security organization? Please share your comments.

Read More...

Security Process Maturity: Level 5

Security managers are often plagued by the question, “How do I make security measurable?” Since security does not produce a product or have a positive impact on the cash flow of a company, creating meaningful measurements that justify an organizations expenditure on security is challenging. This is where ISM³ provides a great tool for measuring security.

ISM³'s level five is about taking all of the processes of levels one through four and using them to communicate the coverage and effectiveness of security.

ISM³ defines seven types of metrics that work well within this maturity model:

Process Metrics

• Number of times a security process was performed in a period
• Scope of protection as a percentage of assets protected by the process
• Time since the last update of process outputs
• The time since a security process has produced the expected output

Performance Metrics

• Return on Security Investment (ROSI) as the percentage of losses avoided compared to the cost of the process
• Comparison of the process output to a baseline or benchmark
• Ratio of available resources in actual use

Measuring each of the ISM³ processes implies that there is a system that easily captures metrics as part of normal operation. Without a centralized metrics reporting system, ISM³ level five will be unsustainable.

The most important reason for measuring your security processes is to identify how well you are operating and work on continuous improvement. At this level, managing the process of continuous improvement is important. By measuring, automating, improving, and communicating your security metrics, you will create a sustainable, continuously improving security operation.

Read More...

Security Process Maturity: Level 4

Level four of ISM³ adds the remaining security processes in the model. This level requires the highest investment, but provides the highest level of protection against technical and internal threats. The security processes in this level are necessary if your organization operates in a highly regulated environments with information assets that are targets for attackers. Examples include stock exchanges, financial institutions, and utilities.

The operational security processes for level 4 are:

• Enhanced reliability and availability management
• Information archiving
• Information quality and compliance probing
• Events detection and analysis
• Forensics

If level four fills in the remaining security processes, what's left for level five? Tomorrow's post will move us from managing security processes to measuring and continuously improving them.

Read More...

Security Process Maturity: Level 3

The third level of ISM³ requires significant investment, but it provides a high level of protection against technical security threats. This level is important for organizations that have high security risks and many critical assets, especially externally facing applications.

The jump from level two to level three does not add many processes, but the processes require more people and time commitment. In addition, these processes require more participation from business function owners, IT operations, project management, and software development. To achieve this level, your security staff will need skills to create a collaborative, participative environment.

The level three operational security processes are:

• Asset classification and management
• SDLC control
• Operations continuity management
• Incident response
• Incident emulation

The level 3 processes build on the previous levels. Building a solid foundation at levels one and two before tackling level 3 will provide broad security coverage of systems, processes, and people.

Read More...

Security Process Maturity: Level 2

For most businesses, level one will not provide enough security to protect their information assets. If a company needs to demonstrate due care to its business partners, has many users of server-based applications, and sensitive information, it needs to move up to the security processes listed in ISM³'s second level.

Level two provides additional protection against technical security threats. It requires more security investment, but not significantly higher. The processes in level two build on the level one processes. To achieve this level, you will need to define and manage all of the level one and level two tasks.

The level one processes may be able to be accomplished by IT technical staff, but level two requires a combination of dedicated security professionals and facilitation with IT departments responsible for server and application administration.

The operational security processes for level two are:

• Select security tools
• Environment change control
• Data clearing
• System hardening
• Security measures change control
• Access control
• User registration
• Physical security
• Internal technical audit
• Alerts monitoring

Level 2 security is a good starting point for securing your organization, but it may not meed compliance standards, such as SAS70, SOX 404, PCI, and NERC-CIP. If you are required to comply with these standards, you'll need to move up to level 3 or 4.

Read More...

Security Process Maturity: Level 1

How mature is security in your organization? Thanks to the ISM³ Consortium, we have a framework for measuring the security maturity of any organization. ISM³ looks at security as a set of defined processes. Each level of maturity has its own processes. Organizations can decide how much security is enough for their type of business and ensure that the processes at that level are defined and implemented.

This series provides a high level view of the operational processes of ISM³. Learn more about ISM³ and its strategic and tactical processes at http://www.ism3.com.

Level one security is suitable for organizations that have very low risk of security threats. These companies have few or no servers and operate primarily using personal computers. At this level, the operational security processes require little investment yet reduce the risk of security threats.

The operational processes for level one are:

• Patch management
• Segmentation and filtering
• Malware protection
• Backup management
• Reporting to tactical management

These processes may be performed by existing IT operations personnel and do not require a dedicated security staff.

Read More...