Issue 35485 - Arbitrary precision in fraction number formats
Summary: Arbitrary precision in fraction number formats
Status: ACCEPTED
Alias: None
Product: Calc
Classification: Application
Component: formatting (show other issues)
Version: OOo 1.1.3
Hardware: All All
: P4 Trivial with 2 votes (vote)
Target Milestone: ---
Assignee: AOO issues mailing list
QA Contact:
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-10-13 23:51 UTC by ntsxmn
Modified: 2017-05-20 11:13 UTC (History)
1 user (show)

See Also:
Issue Type: DEFECT
Latest Confirmation in: ---
Developer Difficulty: ---

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this issue.
Description ntsxmn 2004-10-13 23:51:29 UTC 
previously I have report this issue, but I think I did write the problem clearly
and correctly.  Hear is the problem:

When I enter =(1/60)^2 in spreadsheet, it come out 0.000277777777777778 in both
excell and spreadsheet(openoffice1.1.3) if I want it display in decimal. 
However, If I enter the formula =(1/60)^2 in excel and make a custom format on
the cell with # ?/????, it would come out 1/3600.   But if I make a custom
format in openoffice(1.1.3) with # ?/????, the outcome is 1/3333.  This is
incorrect because the square of (1/60) is 1/3600, but is not 1/3333.  I would
consider this is a serious bug in spreadsheet.
Comment 1 frank 2004-10-15 10:01:37 UTC 
Hi Niklas,

a known rounding error ?

Frank
Comment 2 niklas.nebel 2004-10-15 10:27:46 UTC 
No, a deliberate decision. The full iteration for the closest fraction is done
only up to 3 digits. Above that, an approximation is used. Maybe this should be
changed, but I don't consider it important at the moment.
Comment 3 ooo 2004-10-20 14:52:54 UTC 
Accepted. Adjusting summary.
Comment 4 ooo 2004-10-22 11:29:21 UTC 
P4 OOoLater.
Comment 5 Marcus 2017-05-20 11:13:38 UTC 
Reset assigne to the default "issues@openoffice.apache.org".