1. Forum
  2. Usenet
  3. COMP.LANG.APL

  • Nars floor of a big float

    From Rosario19@21:1/5 to All on Thu Feb 22 11:34:06 2024
    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    but here result 12

    (4) -> floor( log10 ( 9999999999999.9))
    (4) 12.0

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From James Cloos@21:1/5 to All on Thu Feb 22 10:44:18 2024
    "R" == Rosario19 <Ros@invalid.invalid> writes:

    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    I don't know nars, you are probably being hit by floating point
    precision issues.

    with arbitrary precision, the log10 of 9999999999999.9 is:

    +12.999999999999995657055180967460008764615648097801793308999084...

    Using ieee 64bit floats, you are likely to get 12.999999999999995,
    but with 32bit floats you'll get 13.0.

    Even with 64bit floats, rounding issues along the way could lead to a
    result of 13 rather than 12.99999999...

    Hense the difference in floor()'s output.

    -JimC
    --
    James Cloos <cloos@jhcloos.com>
    OpenPGP: https://jhcloos.com/0x997A9F17ED7DAEA6.asc

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Rosario19@21:1/5 to James Cloos on Thu Feb 22 22:41:34 2024
    On Thu, 22 Feb 2024 10:44:18 -0500, James Cloos wrote:

    "R" == Rosario19 <Ros@invalid.invalid> writes:

    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    I don't know nars, you are probably being hit by floating point
    precision issues.

    with arbitrary precision, the log10 of 9999999999999.9 is:

    +12.999999999999995657055180967460008764615648097801793308999084...

    Using ieee 64bit floats, you are likely to get 12.999999999999995,
    but with 32bit floats you'll get 13.0.

    Even with 64bit floats, rounding issues along the way could lead to a
    result of 13 rather than 12.99999999...

    Hense the difference in floor()'s output.

    -JimC

    ok pheraps you are right for 64 and 32 bit numbers, but
    9999999999999.9v with fpc of 128 it should mean it is one float of 128
    bits

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Bob Smith@21:1/5 to All on Thu Feb 22 19:41:28 2024
    On 2/22/2024 5:34 AM, Rosario19 wrote:
    In Nars is right that the floor of log10 of 9999999999999.9v with
    fpc=128 result 13?

    30??10 ? 9999999999999.9v
    13.000000000000000000000000000000

    but here result 12

    (4) -> floor( log10 ( 9999999999999.9))
    (4) 12.0


    The answer depends upon your setting for ⎕CT. If you are using the
    default value of 3E¯15, then yes the floor of that multi-precision
    number is 13. That's because the algorithm first looks for an integer
    within Comparison Tolerance of the number. If it finds one, then that's
    the answer, as in

    a←10 ⍟ 9999999999999.9v
    a+a×⎕CT
    13.0000000000000347

    --
    _________________________________________
    Bob Smith -- bsmith@sudleydeplacespam.com
    http://www.sudleyplace.com - http://www.nars2000.org

    To reply to me directly, delete "despam".

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)