On June 27, 2022, the legislature authorized the Acting State Auditor to perform an audit of the reasons for major errors Department of Water Resource’s snow runoff forecasts in 2021. The Department of Water Resources’ Director, Karla Nemeth, told the legislature that “the forecasting work is undertaken exclusively by the Department of Water Resources. The State Water Board is not responsible for this action and as such should not be a party to the audit.”  The State Water Resource’s Control Board’s Executive Director, Eileen Sobeck, agreed.

But the Water Board has not been following the recommendations of its own expert panel for use of models in Water Board proceedings, including consideration of Temporary Urgency Change Petitions by the Department of Water Resources and the US Bureau of Reclamation.

As part of a 2012 workshop for the Bay-Delta Water Quality Control Plan, the Water Board convened a panel of leading scientists and experts on Bay-Delta modeling, chaired by Dr. Jay Lund of the Center for Watershed Sciences. The Board reviewed existing water supply and hydrodynamic models, and made recommendations for use of models and model results in Water Board proceedings. The report of the panel made the following “Near Term Recommendations” (emphasis added):

Existing hydrodynamics, operations, planning, power, and economics models can provide insights and information, but must be documented and interpreted more thoughtfully and critically for each application.

[…] General documentation and testing should be readily available prior to the use of the model so that 1) the best model is chosen for the stated problem and 2) the model and results are used correctly. [….]

Models and model results used in Board proceedings should be better documented and include a discussion of the strengths, weaknesses, and limitations for each application

 […] Documentation of model testing for conditions relevant to the problem gives the model and modelers credibility. Conversely, failure to provide sufficient details on model strengths and weaknesses should be seen as a sign of model weakness.

Model weaknesses can include major differences of results from field data, including an assessment of the causes of these discrepancies.  […] Knowledge of model weaknesses allows for better interpretations of results. […] Such an assessment is not possible, however, if weaknesses are not revealed, discussed, and documented.

[…] Use of a model outside the limits of tested field conditions (for example, for inflow conditions that have never existed, or with major new physical features beyond calibrated and tested conditions) requires alternative forms of testing and a more cautious interpretation of results. […]

The Water Board has failed to require such information for hearings or workshops since 2012, including hearings on Temporary Urgency Change Petitions in drought years. In 2021, there was no publicly available documentation or testing information for DWR’s Bulletin 120 snow runoff forecasts, nor was documentation made available on DWR’s 2022 revisions to the forecast methodology.

Elaine Archibald, the Executive Director of the California Water and Environmental Modeling Forum also wrote a letter to the Water Board in 2012, proposing technical workshops and peer review of models. The letter stated in part:

CWEMF was formed in 1994, as suggested by the State Water Board in its 1991 Water Quality Control Plan for Salinity, because of disagreements over models used to establish salinity objectives for the Bay Delta. CWEMF has remained actively engaged in water modeling since that time. Our members represent state and federal agencies, water purveyors, non-governmental organizations, academics, and consultants.  CWEMF can assist the State Water Board and other water policy processes in several ways:

1. Technical Workshops – We organize and conduct technical workshops in which stakeholders and experts are brought together in a consensus-building environment to provide organized background information and discuss technical aspects of issues, such as the appropriate use of modeling tools, determine which models best answer particular questions, and identify data needs and model refinements.
2. Model Peer Reviews – We conduct impartial reviews of models to document strengths and weaknesses, suggest improvements, and identify appropriate applications.
3. Modeling Plan – CWEMF called for development of a strategic analysis framework in 2005 to help guide the development of databases and models to provide better quantitative information for a broad range of conditions. The CWEMF document could be the starting point for developing a modeling plan for the Bay-Delta system, and ultimately for water management in California. CWEMF has a small operating budget, bolstered by many volunteer hours, to retain the independent nature of our mission. As such, we require modest funding to assist with these activities.  CWEMF recognizes that establishing flow standards for the Bay-Delta system is challenging. We would be happy to assist.

We asked Water Board staff about reasons for why the Board to never funded CWEMF to do any technical workshops or peer reviews of the model, but received no response.

Further Reading

Archibald, E. 2012. Letter re: CWEMF Assistance with Bay-Delta Plan Analytical Tools. October 25, 2012.

Lund, J. (Chair), Burau, J. Enright, C., DeGeorge, J., Durand, J., Gartrell, G., Guerin, M., Smith, P., Smith, W. Stacy, M. 2012. Analytical Tools for Evaluating Water Supply, Hydrodynamic and Hydropower Effects of the Bay-Delta Plan. October 26, 2012.

Related blog posts:

Fatal errors in DWR’s runoff forecasting: comments to the State Water Resources Control Board. March 18, 2022.

Legislature approves audit of water management by DWR and SWRCB during 2021. June 28, 2022.

Statements at hearing on audit request by Assemblymember Gray. June 28, 2022.

.