Risk Management: Challenges and Guidelines

NIACAP defines risk as:

A combination of the likelihood that a threat will occur, the likelihood that a threat occurrence will result in an adverse impact and the severity of the resulting impact.

So we decompose the computation in: the likelihood that a threat will occur and the likelihood that a threat will become and incident. This model result in more advanced sophisticated statistical analysis.

Now likelihood is decomposed in two parts: evaluation of the threat and evaluation of the vulnerability, the threat is directly linked to the asset, where the vulnerability is linked to the relation between threat and asset. So the vulnerability as to be related to an asset and enable a threat. The consequences is also related to both the asset and the threat; because the asset itself does not tell anything about the impact.

In a situation in which there are detailed information about the vulnerabilities this model will fit better and will provide additional information to be used in the risk mitigation phase. It is particularly useful when doing risk estimation of an IT infrastructure:

• Very detailed information that can be acquired via vulnerability scanners;
• Use repositories to identify attack patterns and related threats;
• Acquire information about possible related threats and likelihood

When reasoning at the network level this model will produce a refined low level analysis.

It is a generalization of the Three factor measure in which also the consequences are split in technical-consequences and business-consequences. For example an asset can be important at the technical level but insignificant at the business level. In general we can add as many variables as we want:

An high number of factor represent an higher level of detail, but a lot of high quality data is needed, otherwise if a high quantity of low quality data can lead to high uncertainty.

Which approach you should use and how you should use it depends on the context and your risk assessment situation

Every measure will depends on the type of analysis and on the environment in which we are working. For example in the network level we need much fine granularity; at the other hand at the strategic level an aggregation has to be performed to provide an high level overview of the risk. The aspects to consider when choosing a measurement model are:

• The availability of data and its quality: if from the environment we can extract only qualitative information then we can’t use a quantitative method. Trade off between level, quality and measurability of data.
• If it is possible to extract information directly regarded to the incident force the selection of a model. The selection of the model is not independent from the type of data.
• In the cybersecurity domain the multi factor measure is very used because the public repositories a lot of information are provided.
• Most of the time the problem is not the lack of data, but the lack of the right kind of data needed by the model used. The Communication and Consultation phase is important to declare which information are important to gather and measure. This situations arise because most of the time we rely on default logging configuration, that produce a lot of data that most of the time can be noisy or lacking of useful feature. Reasoning on the tye of information needed is fundamental to increase the accuracy.
• A different approach consist on starting from the data instead that from the model. Of course this activity as to be performed carefully, if the processes are unknown it is easy to underestimate/overestimate some aspects of the processes. Data Mining and Machine Learning techniques can be used.
• If some information are provided by stakeholders it is important to consider that the data gathered can have different level of granularity and quality; so the information is not structured or detailed. It is important to use less factors if much of the information is provided by stakeholders.
• It is crucial to use the right scale to represent each variable; the scale has to be appropriate to capture the important information.

Once the model is fixed, for each variable we need to define how to quantify it.

Quantitative	Qualitative
Information that need to be quantifies are homogeneous, tends to work better at a technical level, and requires a fine level of granularity	Information that need to be quantified are not homogeneous. In the same class can be heterogeneous elements introducing some imprecision
Easy to pass from quantitative to qualitative	Difficult to convert qualitative scale to quantitative data.
It easy in theory, but it is not necessarily practical; numbers do not always capture the meaning of a measurement.	A purely qualitative approach is not satisfactory on its own. Comparing something in the same class is difficult and not always viable: how can we prioritize the elements inside the Medium class?

The likelihood is a measure of the probability of the occurrence of an incident. If we use a numerical scale is important to consider that there is an uncertainty aspect. Another thing to consider is the fact that sometimes historical data is present, so the scale has to capture also them; if historical data is not present a quantitative scale may be not usable. The third element to consider is the fact that if we use a quantitative scale that does not fit with the situation that we want to represent can lead to huge errors, leading to bias.

It is not recommended to use quantitative scales when the information comes from people. It is better to look for intervals of frequencies or qualitative scales instead that using absolute frequencies.

A risk evaluation may end up with a cost-benefit analysis, so depending on the asset there may be very different characteristics to consider.

• For some assets we can have a 1:1 proportion between the incident and the monetary value of the asset. If a bag of diamond is stolen then the cost consequence of the incident is equal to the cost of the diamonds.
• For other assets, there is not a 1:1 proportion. When talking about the integrity of a database, can be easy to characterize the number of records affected, but hard to say what is the cost of each record affected.
• Some assets do not have any quantitative scale, like the reputation.

The scale depends on the asset; so a general scale can’t be defined; so it is recommended to define a consequence scale for each asset of relevance, so each scale has to fits its intended usage. On the strategic level the risk is better to be quantified in money terms, on the technical level is better to quantify information on the practical level.

NOTE: The suitability of a consequence scale depends on the asset in question

In the cyber scale, there is a complication that comes from the fact that when dealing with the ICT part the impact of an incident at the technical level can be quite different from the impact at the strategic level.

It is due to the lack of information, clean data, or right data. In case of perfect knowledge there is no epistemic uncertainty, and there is no further knowledge to be gained from additional empirical investigations. In case of imperfect knowledge there is always some epistemic uncertainty.

Comes from inherent randomness in some processes.

Quantitative scales can be used, maybe representing uncertainty by using intervals, where the width of the interval represent the level of uncertainty. This reflects on the representation of a risk in the risk matrix not in a well defined cell but as an area in the matrix which depend on the level of uncertainty:

The level of acceptable uncertainty depends on its impact over the decision procedure. To deal with this aspect a qualitative can be used, it gives an overview of mistakes done during the evaluation, for example it can be expressed as a separate natural language expression for each measurement in accordance to an ordinal scale, like: some, none and considerable to express uncertainty. Sometimes the same level of consequence without considering the uncertainty can give a bad perception of the risk.

Different methods can be used:

• Fuzzy logic
• Iterating data collection
• Comparative Analysis
• Test the risk model against historical data. If the model behaves correctly then, the model is validated also if the level of uncertainty is high, otherwise the model as to be changed to work with less factors.