Scenario Planning Revisited–Can You Predict the Future Precisely?

In a blog last spring, I argued that universities needed to respond to the uncertainties created by the COVID-19 pandemic by utilizing scenario planning to identify possible futures for their institutions and to develop strategic and operational responses to those futures. I described a six-step process for identifying possible scenarios and planning organizational responses for each.

In the current issue of Foreign Affairs, Peter Scoblic of Harvard and Philip Tetlock of the University of Pennsylvania describe a process for combining scenario planning and probabilistic forecasting that can get us beyond the false assumption that “tomorrow’s dangers will look like yesterday’s” while providing more accurate predictions of the challenges that we will face in the near term. As universities move beyond operational planning for the new normal of the post-Corona pandemic academic world, utilizing a method like that suggested by Scoblic and Tetlock can improve strategic planning and institutional effectiveness. Readers seeking greater detail should explore the article in Foreign Affairs as well as a series of publications by Scoblic and Tetlock. (1) What I will undertake here is to provide a brief outline of their methodology and to illustrate how it can be applied to college and university planning.

“Typically, forecasting is highly unreliable even when provided by experts. There are techniques, however, that can result in significant improvements in forecasts.”

Scoblic and Tetlock divide their “planning for uncertainty” process into two parts. Generating scenarios is the first part. They note that a two-by-two matrix focused on two critical uncertainties and highlighting the extreme values of each yielding four possible futures is a typical strategy within business organizations. After developing the four possibilities, managers “backcast” a story that describes how that particular state came into being and how it operates. The purpose is to challenge traditional assumptions and to spur debate as to the implications of the various states for organizational functioning. The scenarios, however, are not predictive. They merely describe a particular state. Thus, Scoblic and Tetlock suggest a second part of the process—probabilistic forecasting. Typically, forecasting is highly unreliable even when provided by experts. There are techniques, however, that can result in significant improvements in forecasts. Think for a moment of the progress that has been made in hurricane forecasting. In recent years, we have gone from unreliable guesses as to hurricane paths to highly predictive forecasts of paths and storm strength. You see the process on your television weather reports. Multiple forecasting models from the European Model to numerous United States models are pulled together to yield, in traditional Bayesian fashion, a most likely occurrence. What you do not see on television is the behind-the-scenes interactions of meteorologists that produce this “most likely” scenario. Scoblic and Tetlock argue for the use of heterogeneous groups where participants provide “probabilistic answers to sharply defined questions.” By adding an element of competition among the groups increasingly accurate predictions can be obtained. If the groups combine a broad range of expertise and experience, the results can be impressive.

How could this two-staged scenario process work for university planning?

Developing 2×2 Matrices of the Future

Scoblic and Tetlock would argue that, in developing possible future scenarios, university planners should identify two critical contingencies that will define success or failure in the envisioned time period. From the perspective of this fall, for example, university planners might have identified the two contingencies as the proportion of students who would return to campus for the spring semester and the success of preventative measures to hold down the incidence of coronavirus among the student body. This spring as planners anticipate the 2022 academic year, the contingencies might shift to the size of overall student enrollment in fall 2021 and the proportion of students who are residents on the campus. Each of these sets of contingencies yields four possible scenarios. The planning process then shifts to articulating the implications of the scenarios for university operations—for example, what are the consequences of overall enrollment increasing but students being unwilling to reside on campus in numbers comparable to previous years? Once these implications are identified for each scenario, planning an operational and budgetary response can take place. This is the traditional scenario planning process.

Developing Forecasts of the Future

What this process does not provide, however, is guidance as to the likelihood of the occurrence of any of the particular scenarios. Which scenario is the university likely to face? This is the question that perplexed university administrators both during the summer and the fall. It led them to hoard cash delaying capital projects, freezing open positions, and, in general, making what appeared to be Draconian budget cuts to all parts of the university. Furthermore, it led to strategies to get students back on campus that were often perceived by faculty members as dangerous and counterproductive. When the fall semester arrived the enrollment picture, for most universities, was much less dire than expected—enrollments at the undergraduate level were down but much less than expected, and, for many universities, graduate enrollments were actually up. Could these outcomes have been forecast? Possibly not. However, any degree of precision with regard to the likelihood of possible outcomes could have improved organizational responses.

The second part of Scoblic’s and Tetlock’s process addresses this need for predictions. They provide evidence that organized processes can provide reliable predictions allowing planners to substitute risk for uncertainty. The key is to identify “clusters of questions that give early, forecastable indications of which envisioned future is likely to emerge.” Question clusters allow the university to break down scenarios into clear/forecastable elements. These clusters should consist of questions that are clear and precise— “Will enrollment decrease by 5%?” as opposed to “Will enrollment decrease?” Second, to protect against confirmation biases, questions should ideally be directed toward disproving as opposed to proving hypotheses—“Will the state decrease funding to universities by 5%” as opposed to “Will the state maintain current university funding levels?” Finally, just as is the case with investment portfolios, asking a diversity of questions will yield better forecasts.

It is also important to structure appropriate and effective forecasting groups. Expertise is important but equally important is the diversity of membership. Investors often make the mistake of thinking that they can achieve diversity in their investment portfolios by buying into funds offered by different investment groups; but if those groups are operating under similar assumptions and investing in the same equity groups, no diversity is achieved. Similarly, college and university forecasting groups need to be composed of a wide variety of individuals representing various parts of the administration, faculty disciplines, students, and external constituencies.

Implementing Scenario Planning with Predictive Forecasting

While I have not observed a precise implementation of the planning process as described by Scoblic and Tetlock, many universities with which I have experience have structures that would facilitate such a process. At a large private university in the northeast, two structures were a regular part of university planning. The first was a budget advisory committee that had broad membership from administrators to faculty and student leaders. This group met each winter to review the university budget in conjunction with the assumptions that drove the budget’s modeling. As a result of their analysis, budget recommendations were made to the president and executive leadership. These recommendations informed the development of the final university budget. A second group was a presidential council including vice presidents, deans, and key department heads. This group met prior to the start of each semester to review performance in the previous semester and priorities for the upcoming semester. It would be straightforward and relatively simple to expand the expectations of these two groups to include scenario development and predictive forecasting. For the purposes of this process, membership in the two groups could be expanded to provide greater diversity in background, experience, and perspective. Comparing the accuracy of the forecasts of each of the groups on a regular basis would, according to Scoblic and Tetlock, improve the accuracy and usefulness of the forecasts.

Conclusion

Never before has the future for institutions of higher education been more uncertain. Obviously, uncertainty makes planning much more difficult. In our planning processes, we cannot attain certainty. We can, however, turn uncertainty into risk—a probabilistic assessment of the future. Risk can be managed—uncertainty cannot. Processes such as scenario planning are viable risk management strategies for higher education.

At Optimal, we help colleges thrive! Institutions face severe issues threatening their ability to survive. Which students to recruit, how to retain them, and how to deliver a high-quality education at a reasonable cost must all be part of the institution’s “systems thinking” approach. Learn more about our strategy consulting solutions tailored specifically to your institution’s needs.

1) Peter Tetlock & Dan Gardner, The Art and Science of Prediction. New York: Broadway Books, 2015; Peter Scoblic, “Learning from the Future,” Harvard Business Review, July-August, 2020 (https://hbr.org/2020/07/learning-from-the-future);

Peter Scoblic, “Strategic Foresight as Dynamic Capability: A New Lens on Knightian Uncertainty,” Harvard Business School Working Paper 20-093 (https://www.hbs.edu/faculty/Publication%20Files/20-093_7e70d4a3-aab8-449c-82e9-62cf143d6413.pdf);

Philip Tetlock & Peter Scoblic, “The Power of Precise Predictions,” The New York Times, Oct. 5, 2015 (https://www.nytimes.com/2015/10/04/opinion/the-power-of-precise-predictions.html);

Philip Tetlock, Barbara Mellers, & Peter Scoblic, “Bringing Probability Judgments into Policy Debates Via Forecasting Tournaments,” Science, Feb. 3, 2017 (https://science.sciencemag.org/content/355/6324/481).