Calculation of the RMSE

Calculation of the energy root mean squared error (RMSE), which is given in the 'ENERGY' line of the mode 2 output file:

$\mathrm{RMSE}(E)=\sqrt{\frac{1}{N_{\mathrm{train}}}\sum_{i=1}^{N_{\mathrm{train}}}(E_i^{\mathrm{NN}}-E_i^{\mathrm{ref}})^2}$

Please note that for the RMSE values of the forces there are two possible definitions of the RMSE, based on the force components or based on the force vectors.

The force component-based RMSE is given in the mode 2 output file in the 'FORCES' line:

$\mathrm{RMSE}(F_{\mathrm{comp}})=\sqrt{\frac{1}{N_{\mathrm{components}}}\sum_{i=1}^{N_{\mathrm{components}}}\left( F_i^{\mathrm{NN}} - F_i^{\mathrm{ref}}\right)^2}$

Alternatively, it is also possible to define the force RMSE as the root mean squared deviation of the difference force vector:

$\mathrm{RMSE}(F_{\mathrm{vec}})=\sqrt{\frac{1}{N_{\mathrm{atom}}} \sum_{i=1}^{N_{\mathrm{atom}}}\left( | \underline{F}_i - \underline{F}_i^{\mathrm{ref}} | \right)^2}$

Currently $$\mathrm{RMSE}(F_{\mathrm{vec}})$$ is not printed by RuNNer, but the relation between both force RMSEs can be estimated as

$\mathrm{RMSE}(F_{\mathrm{vec}})=\sqrt{3}\times \mathrm{RMSE}(F_{\mathrm{comp}})$

The relation is only exact if all force components are included in the RMSE calculation. This is not the case if a threshold for the maximum value of force components has been set in the training, because then large force components will not only be skipped in the training but also in the computation of the RMSE (since it is assumed that untrained regions of PES are not relevant and do not need to be monitored by the RMSE).