From Viral V. Acharya, Robert Engle, and Diane Pierret.

Macroprudential stress tests have been employed by regulators in the United States and Europe to assess and address the solvency condition of financial firms in adverse macroeconomic scenarios. We provide a test of these stress tests by comparing their risk assessments and outcomes to those from a simple methodology that relies on publicly available market data and forecasts the capital shortfall of financial firms in severe market-wide downturns. We find that: (i) The losses projected on financial firm balance-sheets compare well between actual stress tests and the market-data based assessments, and both relate well to actual realized losses in case of future stress to the economy; (ii) In striking contrast, the required capitalization of financial firms in stress tests is found to be rather low, and inadequate ex post, compared to that implied by market data; (iii) This discrepancy arises due to the reliance on regulatory risk weights in determining required levels of capital once stress-test losses are taken into account. In particular, the continued reliance on regulatory risk weights in stress tests appears to have left financial sectors under-capitalized, especially during the European sovereign debt crisis, and likely also provided perverse incentives to build up exposures to low risk-weight assets.

UPDATE: A reader forwards a link to these slides, in which Mark Flannery points out that Basel risk weights are based on book values, rather than market values.