Understanding and managing model risk for reinsurance and ILS

In recent years, insurers and regulators have sought improved transparency of risk modeling practices. For example, recent changes to insurer capital requirements outlined by Solvency II explicitly promote goals of transparency and risk model assessment.

Catastrophe model vendors supply clients and regulators with extensive documentation on methodology. Yet, risk managers are increasingly interested in the uncertainty inherent in model methodology.

This type of uncertainty comes from the theoretical underpinnings of the model construction. In the sciences this type of uncertainty often goes by the term epistemic uncertainty. In insurance, it is often regarded as model risk and generally defined as, “the uncertainty that arises from having the wrong models to start with.”

In response to client demand, catastrophe modelers are offering improved access to model components and ease of model blending, morphing, fusing, etc. Most notable of efforts are those of RMS, Karen Clark and Company (KCC) and Lloyd’s. RMS(one) promises to provide users with access to over 300 probabilistic models, whereas KCC’s RiskInsight enables users access to internal assumptions. Lloyd’s Oasis offers users choice in “a set of plug-and-play components.”

These efforts are aimed at resolving concerns about model risk but do not actually help to reduce or control model risk. Improved ability to manipulate vendors’ models may buffer companies from volatility produced by model updates. But that volatility is produced by changes in the decision making by the model vendors and their judgments about how best to create a model.

The ability to create one’s own theory on how best to estimate a given risk does not make that theory an accurate representation of reality.

By chance alone, some views of risk will demonstrate better skill than others. However, model risk remains persistent because, as Karen Clark long ago explained, “[m]odel validation is also problematic.” The time scales involved for observing catastrophic events make it difficult if not impossible to evaluate catastrophe models for skill and accuracy on the time scales within which decisions must be made (see also here).

More likely, improved ability of the risk manager to manipulate model components and thereby cognitively evaluate the process of risk characterization appeases the user’s feelings about model risk (for some literature on this see here, here and here).

In the words of political scientist, Charles Lindblom, risk managers are no less “muddling through” today than before. Today, they just feel better about the process.

ILS decision makers have an interesting vantage point useful for illustration. Consider looking over the entire insurance industry and all the decisions used in building catastrophe models.

The myriad of decisions may appear as an arbitrary process though they are not. As market conditions, experience and knowledge changes so do perceptions of catastrophic risk and the judgments used to guide decision making about models. No enduring shared understanding of the risk is ever achieved because it is always variable according to context.

In addition, market judgments about risk and decision making about model construction are interactive. A better appreciation for this complex feedback process provides insight to how the practice of quantitatively estimating risk is part of a far larger social (and even political) process.

It is again, simply chance that ILS participants do well or lose their investments. That is, of course, unless the game is stacked. But, then, that’s a whole other discussion.

The proliferation of modeled risk estimates carries other challenges for the ILS industry. Risk transfer requires negotiation at several points for investors and risk managers of different types to come to an agreement about a given risk. Rating agencies, with their own model estimates, also play an influential role. Reaching a shared perspective can be difficult when each participant has a unique and scientifically valid view of the risk directing decision makers towards different actions in pricing and investing.

The ILS and overall risk transfer industry would not be the first to run into the obstacles predictive modeling presents for complex decision making. Public policymakers have long struggled with the “excess of objectivity” created by bountiful scientific information and the resulting gridlocked debate over which side has the right scientific facts.

Complete agreement about a given catastrophic risk may also be undesirable. Not only would it obstruct healthy market competition but groupthink poses systemic risk.

With the new modeling capabilities coming to the market the challenge for the risk transfer industry as a whole, including ILS, will be to find an optimal level of agreement and disagreement about catastrophic risks. The matter will be further complicated by public demand for stable risk pricing that does not obstruct economic and general well-being policy goals.

When dealing with irreducible uncertainty at the limits of human knowledge, risk managers should keep in mind higher order objectives such as economic sustainability of the company.

Common sense underwriting tactics that lead decision makers back to the basics of exposure management and local knowledge of building stocks coupled with developing a risk management culture that encourages curiosity, critical thinking and thoughtful consideration of near-misses helps hedge the risk that unique and elegantly complex model estimates are still, simply wrong.

Guest author information:

Jessica Weinkle, PhD.
Postdoctoral Research Associate, Center for Science and Technology Policy Research, University of Colorado Boulder.
Catastrophe Model Researcher and Analyst, ICAT, Lloyd’s Syndicate 4242.