SANSFIRE 2006

ALL PDFs (1.25MB ZIP)

Sharing the Unverifiable: Prediction Exchange        Get the PDF

infectionvectors.com

July 2006

This research was presented at the SANS Institute's SANSFIRE 2006 conference. The slides for that presentation are available in as PDF files:

Sharing the Unverifiable: SANSFIRE Slides

Unverifiable: Additional Slides

The core research is available: Sharing the Unverifiable & related works, see Net Demographics & Evening the Score. 

As security professionals we are keenly aware of the myriad of predictions that permeate newsletters, media reports, product sales briefs and just about every piece of security literature – however, there is rarely a discussion of how we can make better predictions, how those predictions came to be, and how to use predictions – in sum, there is no focus on “the how” of forecasting – just the “what and when” (when an attack will occur and what the impact will be). We all consume predictions – not just within our discipline – weather forecasts are pumped out of every media outlet and have a profound impact on our day. We create predictions, when talking to the decision makers at our respective organizations. In some cases predictions are all we have when trying to justify new purchases. For the purposes of this talk we examined predictions from a familiar vantage point – malware attacks. Automated attacks are of concern to every organization as malicious code has the potential to become an expensive clean-up very quickly. We tried to analyze malware, more correctly the flaws that serve as the malware’s entry point, in terms of “dangerousness” – just like one may use when profiling a criminal.

The rigor is not applied to IT security for a number of reasons – the most notable is the mysticism that guards IA. The “mysticism” that surrounds IT security did not appear by accident; IT itself still enjoys a certain degree of respect as a practice requiring innate talent to succeed at – although this is quickly fading. But, even technically-inclined people regard security as an advanced practice – surrounded with obscure coding tricks. Because of this, predictions are often given more attention or have the ability to grab more attention – especially those pointing to dangerous worms or identity theft. Now, certainly, there are some models that exist that intend to help predict attacks, score vulnerabilities, and the like – but they ask for a test to be applied in a very specific manner, with subjective scoring systems that cannot be shared across organizational boundaries.

Our research on this topic actually began with the “mass mailers are dead” predictions that started a few years ago. Unfortunately, the initial direction of the effort was to try and analyze predictions as they happened, examining the logic and evidence of individual attack (or non-attack) forecasts. Although the research turned out some interesting works, and were rather entertaining to work on, it became apparent that we needed to focus on making predictions as a science itself instead of the infinite regression of taking on specific forecasts.

Existing models, such as CVSS and the US CERT scoring system are effective at providing threats with criticality measures – but only within the subjective and specific context of the host organization. That is, the respective analyst has to build a subjective opinion about the flaw based completely on a subjective score. Current criteria include things like “how easy is the flaw to exploit?” and “how vital is the data that is being threatened?” Moreover, there is a push to judge the amount of impact a flaw has on the confidentiality, integrity, and availability of a system. These are valid concerns for a company, however, criteria and answers like this are an impediment to sharing data amongst people within the same organization, much less across the Internet. The presentation here presents forecasting, and its practice, as a process that should be tempered with objective criteria focused on a dangerousness score, much like that found in criminology.

The first paper in the series introduces the research: Sharing the Unverifiable

(The following is excerpted from the overview.)

In late 2005, one of the most widespread Internet “call-to-arms” predictions circulated among security and non-security groups: the danger of the Microsoft WMF (Windows Meta File) vulnerability. The damage from the resultant attacks (which can be considered relatively mild) was analyzed in depth; the predictions of danger were not. This is a common course for vulnerability concerns. Industry and technology experts take various pieces of data and craft forecasts of danger. With no standard way of formulating or evaluating predictions, especially with regard to applying them to specific environments, the community ends up with a “guru”-run discipline. That style of knowledge management fosters environments where Internet security is considered an imprecise or even near-magical practice. This examination is an attempt to make the exercise of security forecasting into a more scientific endeavor.

At its heart, a cyber attack of any kind is a criminal act. Predicting Internet crime may require the same types of skills and processes used by law enforcement to forecast crime in the physical world. The connection between cyber criminals and “real world” offenses has been established and continues to grow with the profile of the professional malware coder. What is lacking in the IT security discipline is an accepted means of crafting, sharing, and analyzing predictions. This paper looks at possible rationales for such an undertaking as well as the possibility of creating a model for making Internet attack predictions. Although a sample model will be presented, the use of a standard process is examined over making estimates simply based on an expert’s feeling.

The table of Microsoft vulnerabilities referenced in the presentation is available as an MS Excel document.