Simulation Tool Benchmarking and Verification

Brent Dixon

Idaho National Laboratory

Abstract
All developers of fuel cycle simulation tools have needed to grapple with the difficulty of verifying their performance. While individual equations can be checked for accuracy, the performance of the code as a whole is more difficult. Typical techniques range from running highly simplified models that can be manually verified to cross-comparison with other codes that have themselves been verified through similar means. Newer codes can also be verified against benchmarks and other results previously generated by established codes.

This presentation will discuss a number of the previous activities that have been undertaken to verify the performance of fuel cycle simulation tools, including unit tests [1], code-to-code comparisons [2], and a number of international benchmarking efforts [3,4,5]. The short presentation will summarize each of these efforts, including the approach, the tests or scenarios considered, codes evaluated, and other information as a lead-in to a general discussion on code verification and the potential for building a library of unit tests and benchmark cases. The presenter has been involved with all of these previous activities referenced here and will provide first-person information on the processes used and general lessons learned. This presentation may be paired with the proposed presentation of Bo Feng, “Valuable Lessons from Fuel Cycle Code Comparisons” and would also benefit from a brief presentation by Anthony Scopatz on material on developing standardized benchmarks [6].

References
[1] Brown, N.R., B.W. Carlsen, B.W. Dixon, B. Feng, H.R. Greenberg, R.D. Hays, S. Passerini, M. Todosow, A. Worrall, “Identification of fuel cycle simulator functionalities for analysis of transition to a new fuel cycle”, Annals of Nuclear Energy 96 (2016) 88-95, 2016.
[2] Feng, B., B. Dixon, E. Sunny, A. Cuadra, J. Jacobson, N.R. Brown, J. Powers, A. Worrall, S. Passerini, R. Gregg, “Standardized verification of fuel cycle modeling”, Annals of Nuclear Energy 94 (2016) 300-312, 2016.
[3] Guerin, L., B. Feng, P. Hajzlar, B. Forget, M.S. Kazimi, L. Van Den Durpel, A. Yacout, T. Taiwo, B.W. Dixon, G. Matthern, L. Boucher, M Delpech, R. Girieud, M. Meyer, “A Benchmark Study of Computer Codes for System Analysis of the Nuclear Fuel Cycle”, Center for Advanced Nuclear Energy Systems, Massachusetts Institute of Technology, MIT-NRC-TR-105, April 2009.
[4] “Benchmark Study on Nuclear Fuel Cycle Transition Scenarios Analysis Codes”, Expert Group on Fuel Cycle Transition Scenario Studies, Nuclear Energy Agency, NEA/NSC/WPFC/DOC(2012)16, Paris, June 2012.
[5] “Framework for Assessing Dynamic Nuclear Energy Systems for Sustainability: Final Report of the INPRO Collaborative Project GAINS”, International Atomic Energy Agency, NP-T-1.14, Vienna, 2013.
[6] Gidden, M., Scopatz, A., Wilson, P., “Developing standardized, open benchmarks and a corresponding specification language for the simulation of dynamic fuel cycle”, Trans. Am. Nucl. Soc. 108, 127–130, 2013.