On Mar 24, 2004, at 12:36 AM, Grigory A. wrote:
I am again about accuracy. input parameters exp=0xffffff6d=-147 -> -(36.75) right? value d2=210
210^(4/3)*2^(-36.75)*2^28
ans = 2.89921171527700
my answer 3
MAD 0.15 - #define FPM_INTEL answer 2
What is the problem?
Keeping in mind that 2^(-36.75) = 2^(-36) × 2^(-3/4), you will find that III_requantize(210, -147) calculates:
210^(4/3) ⇒ 0x04e0393e × 2^12 0x04e0393e × 2^(-36+12) ⇒ 0x00000005 2^(-3/4) ⇒ 0x09837f05 0x00000005 × 0x09837f05 ⇒ 0x00000002
I think the difference is due to rounding errors from the limited precision of the fixed-point type. You're talking about a difference of 3.725e-9, pretty insignificant in the grand scheme of things.
If you configure with --enable-accuracy, you'll get a slightly better fixed-point multiplication in the last step that happens to yield the value you were expecting.