Six Sigma for Security, Part 2: Measuring

by Steven McElwee, CISSP

After you have defined the Critical to Quality (CTQ) goals of your security program, the next step is to determine how to measure them. Creating a measurement plan for security is like creating a scoreboard. It allows you to gauge at any time how successful you are and your level of protection.

Keep in mind that at the outset of this Six Sigma project we determined that we would not just measure the results of security, which should be few and far between. Instead, we are focusing on measuring the level of protection.

In the measurement phase, we take the four CTQ goals and create a measurement plan for each. Here are sample metrics you may decide to capture.

Policy

• Annual review/approval of policy completed
• Percentage of policies with corresponding controls
• Number of policy exceptions documented
  

Process

• Percentage of controls followed as scheduled
• User access management SLAs
• Incident response times

Partnerships

• Number of security awareness sessions completed
• Number of monthly cross-department meetings completed
• Number of customer contacts made

Payoff

• Internal policy violations
• Security incidents resulting in loss
• Virus/malware incidents
• Compliance gaps

These measurements should get you started. To determine your measurements, brainstorm to create a list of everything that would be nice to measure. Then decide which three or four are the essential metrics. If you have more than three or four measurements for each CTQ, you will quickly be overwhelmed with data.

The next step in building your measurement plan is to identify the data source for each of the metrics. You may find that you don't have a good source of data for some. In these cases, consider ways to begin capturing it now. You may simply use a data collection worksheet to keep track. Keep it simple, but automate it whenever possible.

With your CTQ goals defined, your metrics for each determined, and your data sources identified, you are ready to begin collecting data. For some measurements, you will be able to retrieve historical data and begin your analysis immediately. For new data sources, you may need to capture data for a month or two before you have enough information for analysis.

Complete your measurement plan and capture data as soon as possible. This should be the quickest phase of the Six Sigma for Security project. It will establish a baseline that allows you to set your goals for improvement. It will also allow you to move into one of the most interesting phases -- analysis.

What are some of the metrics you are using to keep score for your CTQ goals?