The role of supervisory stress tests in banking supervision has increased dramatically since the aftermath of the 2007–2009 financial crisis. In particular, the Federal Reserve’s stress tests are a key driver of large U.S. banks’ capital requirements and play a crucial role in determining their ability to return capital to shareholders.1 In the stress-testing exercise, the Fed staff projects the loan losses and revenues of each bank in a severe recession to determine their post-stress regulatory capital ratios. (Note: Broadly, there are two types of U.S. stress tests – Dodd-Frank Act stress tests (DFAST) and comprehensive capital assessment review (CCAR). Under DFAST, both Federal Reserve’s and banks’ own models are used, while CCAR uses only Federal Reserve’s own models. This article is about the use of the Fed’s models in stress tests, so it refers to both DFAST and CCAR. See: The Capital Allocation Inherent in the Federal reserve’s Capital Stress Test.
Banks’ own models are used only in the qualitative assessment portion of the stress tests. This regime gives banks an incentive to manage their own capital according to the results of the Fed’s models, which may decrease the efficiency of the financial system if the projections generated by those models are inaccurate. Alternatively, this process could increase systemic risk if the Fed’s models are vulnerable to a particular source of risk. That the Fed’s models are inaccurate with respect to individual banks is a real possibility, because those models generally are not estimated to best fit each individual bank’s stressed projection, while banks’ models are more granular and fine-tuned to their own business models. (Note: For indirect evidence and discussion on the lack of accuracy of the Fed’s models see: Previewing This Year’s Stress Tests Using the Bank Call Reports Forecasts; and Inside the Black Box: The Accuracy of Alternative Stress Test Models).
Banks have data on the historical performance of their own loan portfolios, which lets them develop accurate models for such portfolios.
The opaqueness and imprecision of the Fed’s models lead to uncertainty among institutions as to what level of capital they will be required to hold, and thus may cause banks to reduce credit availability or prevent them from making loans in anticipation of knowing the results of the stress tests. Therefore, the efficient allocation of credit in the U.S. financial system could be improved significantly by having banks’ own models play a greater role in determining banks’ post-stress regulatory capital ratios and having the Fed’s models used only to ensure the consistency of stress-test results across banks, similar to the current approach employed by the Bank of England (BoE).
Problems with Using the Fed’s Models
The U.S. supervisory stress tests have the reputation of being tougher compared to the stress tests conducted in other advanced economies’ jurisdictions, such as those administered in the United Kingdom and the European Union. The key drivers of the toughness of U.S. stress tests are the stringency of the supervisory scenarios, the assumption that banks’ balance sheets expand under stress, and the requirement that equity distributions be prefunded as if economic conditions were normal. Yet another unique feature of the U.S. supervisory stress tests is that they rely almost exclusively on the Fed’s own models to generate the projections of banks’ post-stress regulatory capital ratios. In contrast, in the stress tests conducted by the BoE and the European Banking Authority, banks’ own models play a key role in generating the projections of stressed loan losses and net revenues, which are key inputs required to calculate banks’ post-stress regulatory capital ratios.
There are several problems that arise when a banking regulator relies exclusively on the projections provided by supervisory models in stress tests. First, if the projections generated using the Fed’s models are overly pessimistic, capital requirements are excessively tight and, as a result, bank credit is too expensive and likely to be unavailable to riskier borrowers (i.e., those loans that require banks to hold relatively more capital). As shown in our own research, the Fed’s supervisory stress tests impose dramatically higher capital requirements for small business loans, residential mortgage loans, and trading assets relative to the capital the banks’ own internal models and the Basel standardized approach would require. Conversely, if the Fed’s models underestimate a particular source of risk and banks manage their own capital according to the results of the Fed’s models, the whole U.S. financial system could be undercapitalized during a time of stress. This scenario unfolded during the crisis, as the Fed’s models underestimated the risks inherent in private-label residential mortgage-backed securities.
Second, it is very difficult to validate the Fed’s models against banks’ own loss experiences because the Fed’s models take a “one-size-fits-all” approach. The use of a common set of models across all banks participating in the stress tests requires the Fed to make important simplifying assumptions, such as glossing over variations in bank-specific business practices, which leads to inaccurate projections for loan losses, and stressed revenues, as described below:
- With respect to loan losses, the Fed does not incorporate bank-specific effects in its projections, because there will be unique characteristics of each bank’s loan portfolio that cannot be captured using the variables included in the Fed’s models. For example, it’s almost impossible for the Fed to project expected losses of loans extended to borrowers in foreign jurisdictions because the Fed lacks the default and loss data required to develop accurate models for these portfolios. In contrast, banks have data on the historical performance of their own loan portfolios, which lets them develop more accurate models for such portfolios. In addition, almost none of the simplifying assumptions made by the Fed are publicly disclosed, which makes it impossible to evaluate their appropriateness or compare them in any detail to banks’ own models.
- With respect to stressed revenues, the Fed projects pre-provision net revenue using models that link the subcomponents of revenues and expenses to macroeconomic variables. All subcomponents of revenues and expenses are taken from the regulatory reports. However, to model bank profitability accurately requires a bank-specific approach, and the reliance on the limited level of granularity available on regulatory reports poses considerable challenges. For example, the net income subcomponents available on the regulatory reports are not sufficiently granular to capture the diversity of business activities of a given bank. Also, other revenue items that are known to move in opposite directions in response to macro shocks are reported in the same line item on the regulatory reports (most notably, fees from mortgage prepayments are reported in the same line item as interest income). This leads to inaccurate bank-specific projections of stressed revenues because it hides important differences in banks’ revenue sensitivities to the stress scenarios. In contrast, banks’ internal models are much more granular than those reported in the regulatory reports categories and are likely to yield higher-quality models for net interest income and subcomponents of non-interest income compared with the Fed’s models.
The third problem that arises when a banking regulator relies exclusively on the projections provided by supervisory models in stress tests is the lack of transparency. Without this transparency, banks have a very limited ability to compare their loan loss projections across loan products or geography to the Fed’s because the Fed publishes only loss rates for major loan portfolios. The Fed could provide greater transparency by providing additional detail regarding the statistical specifications of loss and revenue models, and also if it disclosed the results of statistical tests on the performance of its own models. The Fed also could improve transparency, albeit to a lesser degree, by reporting loss rates of U.S. versus non-U.S. portfolios, loan loss provisions by portfolio type, and stress revenues for each of its major subcomponents.
Advantages of Using Banks’ Models
Moving to a regime where banks’ own models play a key role in U.S. supervisory stress tests would improve the accuracy of bank-specific projections for loan losses, stressed revenues, and balance sheet and post-stress capital ratios. But why is accuracy so important? One reason is because it reduces uncertainty in post-stress regulatory capital ratios and allows banks to make better choices on how to allocate their capital across business lines.
The U.S. supervisory stress tests have the reputation of being tougher compared to the stress tests conducted in other advanced economies’ jurisdictions, such as those administered in the United Kingdom.
In a seminal academic paper, Ben Bernanke was one of the first to formally analyze the impact of uncertainty on business investment. The paper showed that an increase in uncertainty depresses current investment, especially for investment projects that are long-lived and that are economically costly to reverse. Because the capital requirements of the majority of large banks are effectively determined by the supervisory stress tests, uncertainty about banks’ post-stress regulatory capital ratios caused by the lack of accuracy of the Fed’s models is expected to lead to an underinvestment in lending activities. Note that banks’ main role is to transform short-term liquid investments such as deposits into long-term illiquid assets such as loans. Banks’ maturity transformation role is by definition a “long-lived investment project” that is illiquid, so uncertainty induced by the U.S. stress tests is likely to depress bank lending. The use of banks’ own models in stress tests would reduce uncertainty regarding banks’ post-stress regulatory capital ratios and, according to the academic literature, boost lending, improve banking efficiency, and lead to a better allocation of capital.
Allowing banks’ to use their own models would also give banks an incentive to continue to invest additional resources to improve the gathering of internal data on loan defaults and recoveries and to continue to develop their own internal risk models, which would further stimulate innovation in risk management at banks. At the same time, it would also reduce the systemic risk that arises in the current single-model regime, wherein all banks have the incentive to increase their exposure to the types of assets that have lower capital requirements under the Fed’s models, which creates significant one-way risk. Similarly, banks’ use of their own models would also motivate them to develop robust stress scenarios more tailored to their own business models, as the development of a bank-specific scenario is a mandatory requirement in the U.S. stress tests. Ultimately, the Fed wants banks to run effective stress tests, and in order for the banks to anticipate which challenges lie ahead for their bespoke portfolios, they need to run their own models.
Challenges of Relying on Banks’ Own Models
A common argument in defense of the use of the Fed’s models in the supervisory stress tests is that the use of common models provides a level playing field and consistency of stress-test results across banks. In particular, it allows investors to compare results across banks in terms of the risk they’re taking. If banks were allowed to use their own models, the concern is that differences in post-stress regulatory capital ratios would also be driven by differences in modeling approaches. Hence, if a new regime in which banks’ own models play a key role in stress tests is implemented, the Fed should ensure some degree of consistency in the post-stress capital ratios and maintain a level playing field across banks, as discussed below.
Allowing banks’ to use their own models would give banks an incentive to continue to invest additional resources to improve the gathering of internal data on loan defaults and recoveries.
Another popular argument is that allowing banks to use their own models in stress tests encourages bad behavior because banks would have an incentive to make more optimistic assumptions in order to increase equity distributions to its shareholders. Although we can’t rule out this possibility, banks’ own projections are not necessarily more optimistic than those obtained using the Fed’s models in past stress-testing exercises. As shown by the yellow bars in Figure 1, banks’ own projections of pre-tax net losses cumulatively over the planning horizon are 20% and 80% higher than those obtained under the Fed’s models in the Dodd-Frank Act Stress Test (DFAST) for 2016 and 2017, respectively. Conversely, for DFAST 2013 through 2015, the projections obtained using the Fed’s models were more pessimistic than those obtained using banks’ own models. This evidence suggests that the concern that banks’ always will arrive at less-pessimistic results than those obtained using the Fed’s models could be somewhat exaggerated.
Lastly, there is also the possibility that the Fed will increase significantly examination efforts to approve banks’ own models for stress tests. Currently, as part of the qualitative review of the stress tests, the Fed’s examiners extensively review banks’ own models but as if they were refereeing an academic paper submitted for publication in a scholarly journal and not so concerned in ensuring that all assumptions in banks’ own models are conservative. Moving to a stress-testing regime where banks’ own models play a key role could give examiners an incentive to demand that more conservative assumptions be embedded in banks’ models. Moreover, examination teams can have a sizable impact on model outcomes, and there is also a concern that different exam teams’ approaches to modeling could lead to disparate capital requirements across banks. More broadly, the biggest concern of moving to a regime where banks’ own models play a key role in stress tests is that the Fed will micromanage banks’ models. For instance, under the Advanced Approaches framework, large, internationally active banks already are allowed to use internal models to calculate their point-in-time capital requirements; however, banks’ own models are so altered in response to requests from bank examiners that they have reportedly become unusable for internal risk management purposes.7 Such an outcome in supervisory stress testing models would negate the purpose of using banks’ own models in the first place. (Note: In the U.S., the 13 bank holding companies subject to the Advanced Capital Adequacy Framework 12 CFR Parts 208 and 225 (Advanced Approaches) are American Express Company, Bank of America Corporation, Bank of New York Mellon Corporation, Capital One Financial Corporation, Citigroup, Inc., Goldman Sachs Group, Inc., JPMorgan Chase & Co., Morgan Stanley, Northern Trust Corporation, PNC Financial Services Group, Inc., State Street Corporation, U.S. Bancorp, and Well Fargo & Company).
The U.S. is the only advanced economy in which post-stress regulatory capital ratios are exclusively determined using supervisory models. However, some of the challenges presented by allowing banks to use their own models in stress testing are currently being tackled in other jurisdictions. For instance, the Bank of England uses its own models to do peer-benchmarking and ensure a level playing field and consistency of stress-test results across banks. The BoE does not request that banks change their own models; rather, it lets the banks know how much capital they need to hold above minimum requirements. Banks can infer if their projections are excessively optimistic or pessimistic by comparing their own estimates with the BoE’s results. This is a great incentive mechanism for banks to be realistic, as opposed to overly optimistic, in their loss and stressed revenue projections. Assuming that banks try to avoid uncertainty, they have an incentive to hew closely to the BoE’s results to determine with certainty whether and how much they can make equity distributions to their shareholders. Similarly, in the EU-wide stress tests, banks estimate the impact of the supervisory scenarios on risk-based capital ratios using their own internal models, which must be prepared in accordance with a common set of assumptions and an analytical framework that are specified by the European Banking Authority in cooperation with the European Systemic Risk Board, the European Central Bank, and the European Commission.
Conclusion
Given the problems associated with the use of the Fed’s models outlined in this article, the Fed should seriously consider moving to a regime where banks’ own models play a key role in the supervisory stress tests. This would significantly reduce the uncertainty around capital planning at each bank, increase efficiency, and expand credit availability to bank-dependent borrowers. The Fed’s models would still be important in supervisory stress tests to conduct peer benchmarking and ensure consistency and a level playing field across participating banks. This approach would also improve financial stability by eliminating the risk of the industry coalescing around the same models as those used by the Fed, and it would require the Fed’s examiners to focus their efforts on portfolios where the differences between the Fed’s models and banks’ models are the greatest.
About the Author:
Francisco Covas is Senior Vice President and Deputy Head of Research, The Clearing House Association. Covas contributes research and analysis to support the advocacy of the Association on behalf of the owner banks.
Prior to joining The Clearing House in 2016, he was an assistant director of the Division of Monetary Affairs at the Federal Reserve Board, where he supervised a team focused on the role of banks in the transmission of monetary policy as well as on the effects of changes in bank regulation on monetary policy. Covas joined the Board in 2007 as an economist in the Quantitative Risk Management section of Banking Supervision & Regulation focusing on a range of capital, liquidity, and other regulatory initiatives. He previously worked at the central banks of Portugal and Canada.
Covas earned a Ph.D. in economics from the University of California, San Diego, in 2004 and a B.A. from the Universidade Nova de Lisboa, Portugal, in 1997.
