An inconvenience is that most indicators are to be considered simultaneously, complicating the comparison of the agreement among multiple datasets. The development of numerical models further stimulated the need for metrics that could serve for their calibration 1 or to evaluate their performance 2. For instance, the slope and the offset of a linear model fitting the data and a measure of the dispersion around this line may portrait accurately enough the agreement between two datasets. A set of indicators may thus be indeed sufficient to express the “distance” of the available data points from the 1:1 line. Graphically, any deviation of the experimental data points from the 1:1 line in the datasets scatterplot. Intuitively one may classify as disagreement between two datasets any difference from equality. Examples include commonly used indicators such as the Pearson product-moment correlation coefficient ( r), the coefficient of determination ( R 2) and the root mean square error (RMSE). The notion of closeness (or “agreement”) is a concept that many mathematical formulations try to capture. The objective may be to compare simulations coming from different models trying to portray a given phenomenon, to compare the same physical quantity measured by different instruments, or to assess if changes in a given data processing chain is resulting in considerably different results. Many applications in most branches of sciences require researchers, analysts and decision-makers to compare different datasets against each other and to judge how close they are from one another.
The use and value of the index is also illustrated on synthetic and real datasets. The paper also proposes an effective way to disentangle the systematic and the unsystematic contribution to this agreement based on eigen decompositions. We thus show that this index can be considered as a natural extension to r that downregulates the value of r according to the bias between analysed datasets. After a brief review and numerical tests of the metrics designed to accomplish this task, this paper shows how an index proposed by Mielke can, with a minor modification, satisfy a series of desired properties, namely to be adimensional, bounded, symmetric, easy to compute and directly interpretable with respect to r. Although a number of indexes have been proposed to compare a dataset with a reference, only few are available to compare two datasets of equivalent (or unknown) reliability. The Pearson product-moment correlation coefficient r is a widely used measure of the degree of linear dependence between two data series, but it gives no indication of how similar the values of these series are in magnitude. Quantifying how close two datasets are to each other is a common and necessary undertaking in scientific research.
0 kommentar(er)
