Apache OpenOffice (AOO) Bugzilla – Issue 107509
Wrong rank computation of sum values
Last modified: 2013-02-09 04:02:49 UTC
In two rows, I computed the simple sum of four numbers. Then I computed the rank of these two sum values. Although the sums are identical, the ranks come out as 1 and 2. When comparing the two sums with the "if" function, they are regarded as equal. When copying both rows and subtracting 10 from both of the first values, the ranks come out as 1 and 1 (as expected). I attached a simple sample document, where the sums are marked blue, the ranks yellow and the "if" formulas green. I don't know if it matters, but I'm using OO in its German version.
Created attachment 66544 [details] Wrong computation of rank
I just double checked this file on a x86_64 Linux system (Ubuntu 9.10) with the exact same OO build number (OOO310m19). The behaviour is equally wrong here.
Weird issue, probably because of rounding in one case, and no rounding in the other case. The 2 sums are probably different using floating point arithmetic, though Clac is likely to round them to the nearest "shortest number" when displaying the result. This result may however behave oddly in subsequent operations: the rounded value might be used by the IF() function, while the extended result is used by the RANK() function. The handling of floating point results in Calc needs to be confirmed by a someone familiar with the code, though (either Eike or Niklas).
At first glance this does seem like a numeric precision limitation, but when I display the numbers to fill display precision, the first set is the same: 48.70000000000000000000 According to the ODF specification, "If a number in Data occurs more than once it is given the same rank, but increments the rank for subsequent different numbers". So I'm going to mark this as confirmed, and recommend that a developer investigate to see what is going on here. It might be just a precision issue in the end, but it might be something else.