Arithmetic Operations When used as a storage format only, the two CFLoat8 formats, CFloat8_1_4_3 and CFloat8_1_5_2, shall support convert operations to and from the BFloat16 and IEEE Float32 formats. Two modes of rounding should be supported to convert from BFloat16 and IEEE Float32 formats to the two CFLoat8 formats—round-to-nearest and stochastic rounding. Stochastic rounding is implemented with a Random Number Generator (RNG). The arithmetic performed with stochastic rounding should be consistent and reproducible when the same seed is used in the RNG. Stochastic rounding enables probabilistic rounding with a uniform random number generator and enables statistical parameter updates, such as computing stochastic gradient descent (SGD) during back propagation in training. Very small values are accumulated into larger values, and such updates would not happen with IEEE rounding modes. Stochastic rounding is also useful in various training computes to add random noise to prevent regularization The arithmetic operations that the CFloat8 formats should provide are implementation dependent. Typically, the CFloat8 formats are used in mixed precision arithmetic, where the operands stored in CFloat8 format in memory may be operated on and expanded to wider data types, such as Float32. 07 Tesla Dojo Technology — A Guide to Tesla’s Configurable Floating Point Formats & Arithmetic