Skip Menu |
 

This queue is for tickets about the Scalar-List-Utils CPAN distribution.

Report information
The Basics
Id: 130302
Status: open
Priority: 0/
Queue: Scalar-List-Utils

People
Owner: Nobody in particular
Requestors: sisyphus [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.51
Fixed in: (no value)



Subject: uniqnum() doesn't handle float strings correctly.
Download (untitled) / with headers
text/plain 1.4k
Hi, (I believe the below fix would also have fixed the failing test referred to in https://rt.perl.org/Ticket/Display.html?id=134358 and https://rt.cpan.org/Ticket/Display.html?id=130277 without the need to apply the github Issue 78 pull request.) The following one-liner should return 2: C:\>perl -Mblib -MList::Util -le "$n = List::Util::uniqnum'1.73205080756888', '1.7320508075688772'); print $n; " But for perls whose nvtype is double, it returns 1. This happens because of this line ListUtil.xs: sv_setpvf(keysv, "%.15" NVgf, SvNV(arg)); For reliability on such (nvtype = double) perls the "15" needs to be increased to at least "17". When that is done, the one-liner returns the correct value of 2. However, on perls where nvtype is other than double, the value of "17" is also likely to be insufficient. For perls whose NV is the extended precision long double, the value needs to be at least "21". And for perls whose NV is __float128 or IEEE long double, the value needs to be at least "36". A simple fix is to change the offending line to: sv_setpvf(keysv, "%.36" NVgf, SvNV(arg)); and I beleive that will yield correct results irrespective of perl's nvtype. Such a solution is, of course, not optimal because it will often be requesting more decimal digits than is actually needed. A better solution would be to determine the actual minimum value needed (be it 17, 21, or 36) during preprocessing, and install that value into that line of code. Cheers, Rob
Download (untitled) / with headers
text/plain 139b
Hi, Attached is a patch (SLU.diff.txt) that: a) enables t/uniq.t to detect this bug; b) fixes the bug in an optimal manner. Cheers, Rob
Subject: SLU.diff.txt
Download SLU.diff.txt
text/plain 4.4k
--- Makefile.PL_orig 2019-08-18 11:36:40 +1000 +++ Makefile.PL 2019-08-18 19:57:35 +1000 @@ -7,11 +7,22 @@ use ExtUtils::MakeMaker; my $PERL_CORE = grep { $_ eq 'PERL_CORE=1' } @ARGV; +# Determine the correct NV formatting required by uniq() in ListUtil.xs + +my $nvbits; + +if($Config{nvsize} == 8) { $nvbits = 64 } # double or 8-byte long double +elsif(length(sqrt 2) > 25) { $nvbits = 128 } # IEEE long double or __float128 +else { $nvbits = 80 } # extended precision long double + +my $defines = $ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]; +$defines .= " -DLU_NV_BITS=$nvbits"; + WriteMakefile( NAME => q[List::Util], ABSTRACT => q[Common Scalar and List utility subroutines], AUTHOR => q[Graham Barr <gbarr@cpan.org>], - DEFINE => ($ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]), + DEFINE => $defines, DISTNAME => q[Scalar-List-Utils], VERSION_FROM => 'lib/List/Util.pm', --- ListUtil.xs_orig 2019-08-18 11:25:45 +1000 +++ ListUtil.xs 2019-08-18 20:05:59 +1000 @@ -72,6 +72,16 @@ #define sv_catpvn_flags(b,n,l,f) sv_catpvn(b,n,l) #endif +#if LU_NV_BITS == 64 +#define UNIQ_NV_FORMAT "%.17" +#elif LU_NV_BITS == 80 +#define UNIQ_NV_FORMAT "%.21" +#elif LU_NV_BITS == 128 +#define UNIQ_NV_FORMAT "%.36" +#else +#error "LU_NV_BITS not set" +#endif + /* Some platforms have strict exports. And before 5.7.3 cxinc (or Perl_cxinc) was not exported. Therefore platforms like win32, VMS etc have problems so we redefine it here -- GMB @@ -1190,7 +1200,7 @@ else if(SvIOK(arg)) sv_setpvf(keysv, "%" IVdf, SvIV(arg)); else - sv_setpvf(keysv, "%.15" NVgf, SvNV(arg)); + sv_setpvf(keysv, UNIQ_NV_FORMAT NVgf, SvNV(arg)); #ifdef HV_FETCH_EMPTY_HE he = (HE*) hv_common(seen, NULL, SvPVX(keysv), SvCUR(keysv), 0, HV_FETCH_LVALUE | HV_FETCH_EMPTY_HE, NULL, 0); if (HeVAL(he)) --- t/uniq.t_orig 2019-08-18 11:03:30 +1000 +++ t/uniq.t 2019-08-18 11:07:46 +1000 @@ -2,8 +2,8 @@ use strict; use warnings; - -use Test::More tests => 33; +use Config; # to determine nvsize +use Test::More tests => 35; use List::Util qw( uniqnum uniqstr uniq ); use Tie::Array; @@ -87,6 +87,57 @@ 'uniqnum distinguishes large floats (stringified)' ); } +my ($uniq_count1, $uniq_count2, $equiv); + +if($Config{nvsize} == 8) { + # NV is either 'double' or 8-byte 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.4142135623730951 == 1.4142135623730954; + + $uniq_count1 = List::Util::uniqnum (1.4142135623730951, + 1.4142135623730954 ); + + $uniq_count2 = List::Util::uniqnum('1.4142135623730951', + '1.4142135623730954'); +} + +elsif(length(sqrt(2)) > 25) { + # NV is either IEEE 'long double' or '__float128' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.7320508075688772935274463415058722 == 1.73205080756887729352744634150587224; + + $uniq_count1 = List::Util::uniqnum (1.7320508075688772935274463415058722, + 1.73205080756887729352744634150587224 ); + + $uniq_count2 = List::Util::uniqnum('1.7320508075688772935274463415058722', + '1.73205080756887729352744634150587224'); +} + +else { + # NV is extended precision 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 2.2360679774997896963 == 2.23606797749978969634; + + $uniq_count1 = List::Util::uniqnum (2.2360679774997896963, + 2.23606797749978969634 ); + + $uniq_count2 = List::Util::uniqnum('2.2360679774997896963', + '2.23606797749978969634'); +} + +if($equiv) { + is($uniq_count1, 1, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 1, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + +else { + is($uniq_count1, 2, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 2, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + # Hard to know for sure what an Inf is going to be. Lets make one my $Inf = 0 + 1E1000; my $NaN;
Download (untitled) / with headers
text/plain 260b
Duh ... can't believe that I've overlooked the case of the DoubleDouble NV, even though it is relatively rare and obscure. I actually find that quite embarrassing :-( I'll shortly prepare (and post) another set of patches that include that NV. Cheers, Rob
I see a few problems with this patch. NVSIZE is already available in XS as the size of NVs. On newer perls, NVMANTBITS is also available. You can compute the decimal digits this represents with "2 + int( log10(2) * NVMANTBITS )" or "2 + int( log(2)/log(10)*$Config::Config{nvmantbits} )". This may be a better approach than hard coding the values, but you'd still have to fall back to hard coded values. But I'm not sure if using that length is really the right approach. Large UV values will still have problems since they will lose precision if converted to floats. I have a module that provides pure-perl variants of the List::Util functions for use in places where compiling new versions of List::Util isn't possible, so I've had to deal with these same issues. I've written a test to demonstrate what I think the behavior should be, along with an implementation in my module. The test is here: https://github.com/haarg/List-Util-MaybeXS/blob/master/t/uniqnum.t It hopefully includes enough comments to describe the problems I saw. It can be run against List::Util by passing the parameter 'xs'.
CC: "sisyphus [...] cpan.org via RT" <sisyphus [...] cpan.org>
Subject: Re: [rt.cpan.org #130302] uniqnum() doesn't handle float strings correctly.
Date: Fri, 23 Aug 2019 19:24:12 +1000
To: bug-Scalar-List-Utils [...] rt.cpan.org
From: sisyphus <sisyphus359 [...] gmail.com>
Download (untitled) / with headers
text/plain 837b
On Fri, Aug 23, 2019 at 10:34 AM sisyphus <sisyphus359@gmail.com> wrote Show quoted text
> I'll also take a look at the cause of those 7 failures (above), with a
view to eliminating them. With additional patching for doubledouble NV types, I've now got everything working fine except for those 7 failures in Graham's script at https://github.com/haarg/List-Util-MaybeXS/blob/master/t/uniqnum.t . It's only NVs whose size is 8 bytes that are subject to those failures. (No problems at all with extended precision long double, __float128 or doubledouble NVs.) But I'm having trouble envisaging (and locating) the values instrumental in the 7 failures. Are there relevant examples somewhere of List::Util::uniqnum() determining that 2 unique values are the same, or (alternatively) of determining that 2 identical values are different ? Cheers, Rob
CC: "sisyphus [...] cpan.org via RT" <sisyphus [...] cpan.org>
Subject: Re: [rt.cpan.org #130302] uniqnum() doesn't handle float strings correctly.
Date: Fri, 23 Aug 2019 10:34:51 +1000
To: bug-Scalar-List-Utils [...] rt.cpan.org
From: sisyphus <sisyphus359 [...] gmail.com>
Download (untitled) / with headers
text/plain 3.7k
On Thu, Aug 22, 2019 at 11:34 PM Graham Knop via RT < bug-Scalar-List-Utils@rt.cpan.org> wrote: <URL: https://rt.cpan.org/Ticket/Display.html?id=130302 > Show quoted text
> NVSIZE is already available in XS as the size of NVs.
I'm guessing that NVSIZE is set to the same value as $Config{nvsize}. The problem with that value is that, on 64-bit builds of perl, it doesn't distinguish between extended precision (80-bit) long doubles and the 128-bit IEEE long doubles. I know that $Config{longdblkind} is there to help, but it's only available on later perl versions (5.22 onwards IIRC). Show quoted text
> You can compute the decimal digits this represents with "2 + int(
log10(2) * NVMANTBITS )" or "2 + int( log(2)/log(10)*$Config::Config{nvmantbits} )". The formula I've used to work out how many digits are required is similar to yours: 1+ceil(P * log(2) / log(10)) where P is the precision (in bits) of the NV - ie either 53, 64, or 113, depending upon the type. Show quoted text
> On newer perls, NVMANTBITS is also available.
I didn't know about NVMANTBITS. Sadly, even on current perl-5.31.3 it's incorrect for Windows 'long double' builds that have been compiled by mingw ports of gcc. (It says '52', though the correct value is '64'). I'll file a bug report about this - 'twould be nice to at least get that fixed for 5.32. Show quoted text
> This may be a better approach than hard coding the values, but you'd
still have to fall back to hard coded values. I'm not actually hard coding any values. It's all handled in the pre-processing of the XS file. If the NVSIZE is 8, 17 decimal digits will be requested. If the NV is an 80-bit long double, 21 digits will be requested. If it's neither of those then NVMANTBITS will be 112 and 36 decimal digits will be requested. Show quoted text
> But I'm not sure if using that length is really the right approach.
Large UV values will still have problems since they will lose precision if converted to floats. I don't think my patch will introduce any *new* issues in that regard. All it's doing is changing the value of "%.15" in the line: sv_setpvf(keysv, "%.15" NVgf, SvNV(arg)); to either "%.17" or "%.21" or "%.36" - whichever is correct for the particular type of NV. Show quoted text
> I have a module that provides pure-perl variants of the List::Util
functions for use in places where compiling new versions of List::Util isn't possible, so I've had to deal with these same issues. Show quoted text
> I've written a test to demonstrate what I think the behavior should be,
along with an implementation in my module. Show quoted text
> The test is here:
https://github.com/haarg/List-Util-MaybeXS/blob/master/t/uniqnum.t It hopefully includes enough comments to describe the problems I saw. It can be run against List::Util by passing the parameter 'xs'. When nvtype is double: 1) your test script fails 89 tests of 181 when run against List-Util-1.52 (tests 1 and 94 onwards) 2) your test script fails 7 tests of 183 when run against my patched List-Util-1.52 (tests 1, 178-183) When nvtype is extended precision long double: 1) your test script fails 211 tests of 284 when run against List-Util-1.52 2) your test script fails 0 tests when run against my patched List-Util-1.52 - presumably partly because the (above mentioned) issue regarding loss of UV precision does not apply here. I haven't yet tested for the __float128 or doubledouble type. I'm still working on a way of handling the doubledouble. The stringification of the doubledouble values is flakey and unreliable for certain values, so I'm putting together a patch that looks instead at the 16 bytes (which will be unique for different values). It differs from the original patch only in the way that doubledouble NVs are handled. All other NVs are still being handled as per the original patch that I posted. I'll also take a look at the cause of those 7 failures (above), with a view to eliminating them. Cheers, Rob
Download (untitled) / with headers
text/plain 4.8k
Ok - revised patch (constructed against S-L-U-1.52 is attached. I've changed tack and,for all NV types, I've used the actual NV bytes rather than NV values. There's some special treatment handed out for the case where nvsize and ivsize are both 8. With this patch applied, the test suite (including the 2 tests that I added to t/uniq.t) passes, and all of the tests in the script provided by Graham also pass. On MS Windows 7, I've tested 2 builds of perl-5.30.0 - one had an nvtype of double, the other an nvtype of little-endian extended precision (80-bit) long double. On Ubuntu-18.04, I've tested 3 builds of perl-5.31.3 - one had an nvtype of double, the second had an nvtype of little endian extended precision (80-bit) long double, and the third had an nvtype of __float128. On FreeBSD-12, I've also tested 2 builds of perl-5.31.3 - one with an nvtype of double, the other with a little endian extended precision (80-bit) long double. On a ppc64 big endian box, running Debian wheezy, I've tested two builds of perl-5.31.2 - one with an nvtype of double and one with an nvtype of doubledouble. On the freebsd system, I notice the following warnings during compilation: ##################################### ListUtil.xs:1250:29: warning: passing 'unsigned char [38]' to parameter of type 'char *' converts between pointers to integer types with different sign [-Wpointer-sign] sprintf(buff, "%" UVuf, SvUV(arg)); ^~~~ /usr/include/stdio.h:287:31: note: passing argument to parameter here int sprintf(char * __restrict, const char * __restrict, ...); ^ ListUtil.xs:1253:29: warning: passing 'unsigned char [38]' to parameter of type 'char *' converts between pointers to integer types with different sign [-Wpointer-sign] sprintf(buff, "%" IVdf, SvIV(arg)); ^~~~ /usr/include/stdio.h:287:31: note: passing argument to parameter here int sprintf(char * __restrict, const char * __restrict, ...); ^ ListUtil.xs:1262:33: warning: passing 'unsigned char [38]' to parameter of type 'char *' converts between pointers to integer types with different sign [-Wpointer-sign] sprintf(buff, "%" IVdf, SvIV(arg)); ^~~~ /usr/include/stdio.h:287:31: note: passing argument to parameter here int sprintf(char * __restrict, const char * __restrict, ...); ^ ListUtil.xs:1264:33: warning: passing 'unsigned char [38]' to parameter of type 'char *' converts between pointers to integer types with different sign [-Wpointer-sign] sprintf(buff, "%" UVuf, SvUV(arg))... ^~~~ /usr/include/stdio.h:287:31: note: passing argument to parameter here int sprintf(char * __restrict, const char * __restrict, ...); ^ ListUtil.xs:1272:29: warning: passing 'unsigned char [4]' to parameter of type 'char *' converts between pointers to integer types with different sign [-Wpointer-sign] sprintf(tbuff, "%02x", ((unsigned char*)p)[i]); ^~~~~ /usr/include/stdio.h:287:31: note: passing argument to parameter here int sprintf(char * __restrict, const char * __restrict, ...); ^ ListUtil.xs:1273:28: warning: passing 'unsigned char [38]' to parameter of type 'char *' converts between pointers to integer types with different sign [-Wpointer-sign] strcat(buff, tbuff); ^~~~ /usr/include/string.h:77:31: note: passing argument to parameter here char *strcat(char * __restrict, const char * __restrict); ^ ListUtil.xs:1273:34: warning: passing 'unsigned char [4]' to parameter of type 'const char *' converts between pointers to integer types with different sign [-Wpointer-sign] strcat(buff, tbuff); ^~~~~ /usr/include/string.h:77:56: note: passing argument to parameter here char *strcat(char * __restrict, const char * __restrict); ########################################## I should probably try to silence them. The other systems issued no warnings. I don't have access to a big endian extended precision (80-bit) long double, and I don't know at which end of that structure one finds the unused bytes (which I don't want to access in case they contain some ad hoc non-zero values). Consequently, the Makefile.PL might need correcting, depending upon where those bytes are located. I also don't have access to the 128-bit IEEE long double, though I would expect no problems there as the __float128 build passed all tests. Not sure how optimal my C code that populates the buffer is. An informed critical eye might see room for improvement there. Cheers, Rob
Subject: SLU.diff2.txt
Download SLU.diff2.txt
text/plain 7.9k
--- Makefile.PL_orig 2019-08-24 11:46:15 +1000 +++ Makefile.PL 2019-08-24 18:58:59 +1000 @@ -7,11 +7,37 @@ use ExtUtils::MakeMaker; my $PERL_CORE = grep { $_ eq 'PERL_CORE=1' } @ARGV; + +my $defines = $ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]; + +# Determine the correct NV formatting required by uniq() in ListUtil.xs + +if($Config{nvsize} == 8) { # double or 8-byte long double + $defines .= " -DUNIQ_LOOP_START=0 -DUNIQ_LOOP_END=8"; + $defines .= " -DUNIQ_PREC_LOSS" if $Config{ivsize} >= 8; # precision loss is possible when + # IV/UV is converted to NV +} + +elsif(length(sqrt 2) > 25) { # IEEE long double or __float128 or doubledouble + $defines .= " -DUNIQ_LOOP_START=0 -DUNIQ_LOOP_END=16" +} + +else { # extended precision long double + if($Config{byteorder} =~ /^1234/) { + # little endian + $defines .= " -DUNIQ_LOOP_START=0 -DUNIQ_LOOP_END=10"; # exclude unused bytes + } + else { + # big endian + $defines .= " -DUNIQ_LOOP_START=6 -DUNIQ_LOOP_END=16"; # exclude unused bytes + } +} + WriteMakefile( NAME => q[List::Util], ABSTRACT => q[Common Scalar and List utility subroutines], AUTHOR => q[Graham Barr <gbarr@cpan.org>], - DEFINE => ($ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]), + DEFINE => $defines, DISTNAME => q[Scalar-List-Utils], VERSION_FROM => 'lib/List/Util.pm', --- ListUtil.xs_orig 2019-08-24 11:45:08 +1000 +++ ListUtil.xs 2019-08-25 21:29:37 +1000 @@ -1152,6 +1152,14 @@ int index; SV **args = &PL_stack_base[ax]; HV *seen; + int i; + NV nv_arg; + void *p = &nv_arg; + unsigned char buff[38]; + unsigned char tbuff[4]; +#ifdef UNIQ_PREC_LOSS + int uvstring, ivstring; +#endif if(items == 0 || (items == 1 && !SvGAMAGIC(args[0]) && SvOK(args[0]))) { /* Optimise for the case of the empty list or a defined nonmagic @@ -1185,12 +1193,88 @@ #endif } - if(!SvOK(arg) || SvUOK(arg)) - sv_setpvf(keysv, "%" UVuf, SvUV(arg)); - else if(SvIOK(arg)) - sv_setpvf(keysv, "%" IVdf, SvIV(arg)); - else - sv_setpvf(keysv, "%.15" NVgf, SvNV(arg)); + /* Assign appropriate value to nv_arg */ + + if(!SvOK(arg) || SvUOK(arg)) { + nv_arg = (NV)SvUV(arg); +#ifdef UNIQ_PREC_LOSS + uvstring = 1; +#endif + } + + else if(SvIOK(arg)) { + nv_arg = (NV)SvIV(arg); +#ifdef UNIQ_PREC_LOSS + ivstring = 1; +#endif + } + + else { + nv_arg = SvNV(arg); + } + + /* Handle NaN, zeros and Infs */ + + if(nv_arg != nv_arg) { + sv_setpvf(keysv, "%s", "NaN"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; + uvstring = 0; +#endif + } + + else if(nv_arg == 0) { + sv_setpvf(keysv, "%s", "0"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; + uvstring = 0; +#endif + } + + else if(nv_arg/nv_arg != 1) { + if(nv_arg < 0) sv_setpvf(keysv, "%s", "-Inf"); + else sv_setpvf(keysv, "%s", "Inf"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; + uvstring = 0; +#endif + } + + /* Handle all values other than NaN, zeros and Infs */ + + else { + + buff[0] = '\0'; +#ifdef UNIQ_PREC_LOSS + if(uvstring) + sprintf(buff, "%" UVuf, SvUV(arg)); + + else if(ivstring) + sprintf(buff, "%" IVdf, SvIV(arg)); + + /* Take appropriate action if nv_arg is within IV and UV bounds */ + + else if(ceil(nv_arg) == nv_arg + && nv_arg <= 1.8446744073709552e+19 + && nv_arg >= -9.2233720368547758e+18 ) { + + if(nv_arg < 0) + sprintf(buff, "%" IVdf, SvIV(arg)); + else + sprintf(buff, "%" UVuf, SvUV(arg)); + } + + ivstring = 0; + uvstring = 0; +#endif + + for(i = UNIQ_LOOP_START; i < UNIQ_LOOP_END; i++) { + sprintf(tbuff, "%02x", ((unsigned char*)p)[i]); + strcat(buff, tbuff); + } + + sv_setpvf(keysv, "%s", buff); + } #ifdef HV_FETCH_EMPTY_HE he = (HE*) hv_common(seen, NULL, SvPVX(keysv), SvCUR(keysv), 0, HV_FETCH_LVALUE | HV_FETCH_EMPTY_HE, NULL, 0); if (HeVAL(he)) --- t/uniq.t_orig 2019-08-24 11:46:43 +1000 +++ t/uniq.t 2019-08-25 20:33:59 +1000 @@ -2,8 +2,8 @@ use strict; use warnings; - -use Test::More tests => 33; +use Config; # to determine nvsize +use Test::More tests => 35; use List::Util qw( uniqnum uniqstr uniq ); use Tie::Array; @@ -87,6 +87,76 @@ 'uniqnum distinguishes large floats (stringified)' ); } +my ($uniq_count1, $uniq_count2, $equiv); + +if($Config{nvsize} == 8) { + # NV is either 'double' or 8-byte 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.4142135623730951 == 1.4142135623730954; + + $uniq_count1 = List::Util::uniqnum (1.4142135623730951, + 1.4142135623730954 ); + + $uniq_count2 = List::Util::uniqnum('1.4142135623730951', + '1.4142135623730954'); +} + +elsif(length(sqrt(2)) > 25) { + # NV is either IEEE 'long double' or '__float128' or doubledouble + + if(1 + (2 ** -1074) != 1) { + # NV is doubledouble + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1 + (2 ** -1074) == 1 + (2 ** - 1073); + + $uniq_count1 = List::Util::uniqnum (1 + (2 ** -1074), + 1 + (2 ** -1073) ); + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 4.0564819207303340847894502572035e31 == 4.0564819207303340847894502572034e31; + + $uniq_count2 = List::Util::uniqnum('4.0564819207303340847894502572035e31', + '4.0564819207303340847894502572034e31'); + } + + else { + # NV is either IEEE 'long double' or '__float128' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.7320508075688772935274463415058722 == 1.73205080756887729352744634150587224; + + $uniq_count1 = List::Util::uniqnum (1.7320508075688772935274463415058722, + 1.73205080756887729352744634150587224 ); + + $uniq_count2 = List::Util::uniqnum('1.7320508075688772935274463415058722', + '1.73205080756887729352744634150587224'); + } +} + +else { + # NV is extended precision 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 2.2360679774997896963 == 2.23606797749978969634; + + $uniq_count1 = List::Util::uniqnum (2.2360679774997896963, + 2.23606797749978969634 ); + + $uniq_count2 = List::Util::uniqnum('2.2360679774997896963', + '2.23606797749978969634'); +} + +if($equiv) { + is($uniq_count1, 1, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 1, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + +else { + is($uniq_count1, 2, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 2, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + # Hard to know for sure what an Inf is going to be. Lets make one my $Inf = 0 + 1E1000; my $NaN;
Download (untitled) / with headers
text/plain 1.3k
I've now silenced the warnings I was seeing on FreeBSD. I've also (hopefully) improved the way that the NV's internal structure is transferred to the string buffer as hex bytes. For perl-5.20.x and earlier on MS Windows I discovered that the 'uniqnum preserves uniqness of full integer range (stringified)' test needs to be skipped because of the bizarre way that infs and nans are stringified on those particular perls. AFAICT it's actually not possible to come up with a string that will numify to NaN, though strings that numify to Infs do exist (eg '1e1000'). Graham's script also needs to be skipped for those builds - for the same reason. I suppose it would be possible to avoid those skips by rewriting the tests such that Inf strings were written differently and testing involving NaN strings was avoided, but I don't really see that as being a useful exercise. I eventually found confirmation (at https://en.wikipedia.org/wiki/Comparison_of_instruction_set_architectures#Endianness) that both little and big endian types start at the lowest indexed byte. This called for a change in the Makefile.PL to the treatment of the 80-bit long double NVs. Attached is my latest patch - formulated against Scalar-List-Utils-1.52. I don't envisage making any further alterations, though that could change if other problems come to light. Cheers, Rob
Subject: SLU.diff3.txt
Download SLU.diff3.txt
text/plain 9.5k
--- Makefile.PL_orig 2019-08-24 11:46:15 +1000 +++ Makefile.PL 2019-08-27 23:21:26 +1000 @@ -7,11 +7,30 @@ use ExtUtils::MakeMaker; my $PERL_CORE = grep { $_ eq 'PERL_CORE=1' } @ARGV; + +my $defines = $ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]; + +# Determine the correct NV formatting required by uniq() in ListUtil.xs + +if($Config{nvsize} == 8) { # double or 8-byte long double + $defines .= " -DUNIQ_LOOP_START=0 -DUNIQ_LOOP_END=8"; + $defines .= " -DUNIQ_PREC_LOSS" if $Config{ivsize} >= 8; # precision loss is possible when + # IV/UV is converted to NV +} + +elsif(length(sqrt 2) > 25) { # IEEE long double or __float128 or doubledouble + $defines .= " -DUNIQ_LOOP_START=0 -DUNIQ_LOOP_END=16" +} + +else { # extended precision long double + $defines .= " -DUNIQ_LOOP_START=0 -DUNIQ_LOOP_END=10"; # ignore unused bytes +} + WriteMakefile( NAME => q[List::Util], ABSTRACT => q[Common Scalar and List utility subroutines], AUTHOR => q[Graham Barr <gbarr@cpan.org>], - DEFINE => ($ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]), + DEFINE => $defines, DISTNAME => q[Scalar-List-Utils], VERSION_FROM => 'lib/List/Util.pm', --- ListUtil.xs_orig 2019-08-24 11:45:08 +1000 +++ ListUtil.xs 2019-08-27 23:23:46 +1000 @@ -72,6 +72,19 @@ #define sv_catpvn_flags(b,n,l,f) sv_catpvn(b,n,l) #endif +#ifdef UNIQ_PREC_LOSS +#define UNIQ_BUFF_SIZE 37 /* nvsize == 8, ivsize == 8 */ + +#elif UNIQ_LOOP_END == 8 /* nvsize == 8, ivsize < 8 **/ +#define UNIQ_BUFF_SIZE 17 + +#elif UNIQ_LOOP_END - UNIQ_LOOP_START == 10 /* NV is 10 bytes ***********/ +#define UNIQ_BUFF_SIZE 21 + +#else +#define UNIQ_BUFF_SIZE 33 /* NV is 16 bytes ***********/ +#endif + /* Some platforms have strict exports. And before 5.7.3 cxinc (or Perl_cxinc) was not exported. Therefore platforms like win32, VMS etc have problems so we redefine it here -- GMB @@ -1152,6 +1165,14 @@ int index; SV **args = &PL_stack_base[ax]; HV *seen; + size_t i; + NV nv_arg; + void *p = &nv_arg; + char s[UNIQ_BUFF_SIZE]; + char * buff = s; +#ifdef UNIQ_PREC_LOSS + size_t uvstring = 0, ivstring = 0, offset = 0; +#endif if(items == 0 || (items == 1 && !SvGAMAGIC(args[0]) && SvOK(args[0]))) { /* Optimise for the case of the empty list or a defined nonmagic @@ -1185,12 +1206,105 @@ #endif } - if(!SvOK(arg) || SvUOK(arg)) - sv_setpvf(keysv, "%" UVuf, SvUV(arg)); - else if(SvIOK(arg)) - sv_setpvf(keysv, "%" IVdf, SvIV(arg)); - else - sv_setpvf(keysv, "%.15" NVgf, SvNV(arg)); + /* Assign appropriate value to nv_arg */ + + if(!SvOK(arg) || SvUOK(arg)) { + nv_arg = (NV)SvUV(arg); +#ifdef UNIQ_PREC_LOSS + uvstring = 1; +#endif + } + + else if(SvIOK(arg)) { + nv_arg = (NV)SvIV(arg); +#ifdef UNIQ_PREC_LOSS + ivstring = 1; +#endif + } + + else { + nv_arg = SvNV(arg); + } + + /* Handle NaN, zeros and Infs */ + + if(nv_arg != nv_arg) { + sv_setpvf(keysv, "%s", "NaN"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; + uvstring = 0; +#endif + } + + else if(nv_arg == 0) { + sv_setpvf(keysv, "%s", "0"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; + uvstring = 0; +#endif + } + + else if(nv_arg/nv_arg != 1) { + if(nv_arg < 0) sv_setpvf(keysv, "%s", "-Inf"); + else sv_setpvf(keysv, "%s", "Inf"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; + uvstring = 0; +#endif + } + + /* Handle all values other than NaN, zeros and Infs */ + + else { +#ifdef UNIQ_PREC_LOSS + if(uvstring) { + sprintf(buff, "%" UVuf, SvUV(arg)); + offset = strlen(buff); + } + + else if(ivstring) { + sprintf(buff, "%" IVdf, SvIV(arg)); + offset = strlen(buff); + } + + /* Take appropriate action if nv_arg is within IV and UV bounds */ + + else if(ceil(nv_arg) == nv_arg + && nv_arg <= 1.8446744073709552e+19 + && nv_arg >= -9.2233720368547758e+18 ) { + + if(nv_arg < 0) { + sprintf(buff, "%" IVdf, SvIV(arg)); + } + else { + sprintf(buff, "%" UVuf, SvUV(arg)); + } + + offset = strlen(buff); + } + + ivstring = 0; + uvstring = 0; + + buff += offset; + + for(i = 0; i < 8; i++) { + sprintf(buff, "%02x", ((unsigned char*)p)[i]); + buff += 2; + } + + buff -= 16 + offset; /* return pointer to original position */ + offset = 0; +#else + for(i = UNIQ_LOOP_START; i < UNIQ_LOOP_END; i++) { + sprintf(buff, "%02x", ((unsigned char*)p)[i]); + buff += 2; + } + + buff -= 2 * UNIQ_LOOP_END; /* return pointer to original position */ +#endif + sv_setpvf(keysv, "%s", buff); + } #ifdef HV_FETCH_EMPTY_HE he = (HE*) hv_common(seen, NULL, SvPVX(keysv), SvCUR(keysv), 0, HV_FETCH_LVALUE | HV_FETCH_EMPTY_HE, NULL, 0); if (HeVAL(he)) --- t/uniq.t_orig 2019-08-24 11:46:43 +1000 +++ t/uniq.t 2019-08-27 19:12:28 +1000 @@ -2,8 +2,8 @@ use strict; use warnings; - -use Test::More tests => 33; +use Config; # to determine nvsize +use Test::More tests => 35; use List::Util qw( uniqnum uniqstr uniq ); use Tie::Array; @@ -87,6 +87,76 @@ 'uniqnum distinguishes large floats (stringified)' ); } +my ($uniq_count1, $uniq_count2, $equiv); + +if($Config{nvsize} == 8) { + # NV is either 'double' or 8-byte 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.4142135623730951 == 1.4142135623730954; + + $uniq_count1 = List::Util::uniqnum (1.4142135623730951, + 1.4142135623730954 ); + + $uniq_count2 = List::Util::uniqnum('1.4142135623730951', + '1.4142135623730954'); +} + +elsif(length(sqrt(2)) > 25) { + # NV is either IEEE 'long double' or '__float128' or doubledouble + + if(1 + (2 ** -1074) != 1) { + # NV is doubledouble + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1 + (2 ** -1074) == 1 + (2 ** - 1073); + + $uniq_count1 = List::Util::uniqnum (1 + (2 ** -1074), + 1 + (2 ** -1073) ); + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 4.0564819207303340847894502572035e31 == 4.0564819207303340847894502572034e31; + + $uniq_count2 = List::Util::uniqnum('4.0564819207303340847894502572035e31', + '4.0564819207303340847894502572034e31'); + } + + else { + # NV is either IEEE 'long double' or '__float128' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.7320508075688772935274463415058722 == 1.73205080756887729352744634150587224; + + $uniq_count1 = List::Util::uniqnum (1.7320508075688772935274463415058722, + 1.73205080756887729352744634150587224 ); + + $uniq_count2 = List::Util::uniqnum('1.7320508075688772935274463415058722', + '1.73205080756887729352744634150587224'); + } +} + +else { + # NV is extended precision 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 2.2360679774997896963 == 2.23606797749978969634; + + $uniq_count1 = List::Util::uniqnum (2.2360679774997896963, + 2.23606797749978969634 ); + + $uniq_count2 = List::Util::uniqnum('2.2360679774997896963', + '2.23606797749978969634'); +} + +if($equiv) { + is($uniq_count1, 1, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 1, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + +else { + is($uniq_count1, 2, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 2, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + # Hard to know for sure what an Inf is going to be. Lets make one my $Inf = 0 + 1E1000; my $NaN; @@ -109,8 +179,13 @@ my @strs = map "$_", @nums; - skip( "Perl $] doesn't stringify UV_MAX right ($maxuint)", 1 ) - if $maxuint !~ /\A[0-9]+\z/; + if($maxuint !~ /\A[0-9]+\z/) { + skip( "Perl $] doesn't stringify UV_MAX right ($maxuint)", 1 ); + } + elsif($] < 5.022 && $^O =~ /MSWin32/i) { + skip( "On MS Windows,perl $] stringifies infs and nans into something unusable", 1 ); + } + is_deeply( [ uniqnum @strs, "1.0" ], [ @strs ],
Download (untitled) / with headers
text/plain 429b
On Tue Aug 27 09:48:16 2019, SISYPHUS wrote: Show quoted text
> I don't envisage making any further alterations, though that could > change if other problems come to light.
Within 5 minutes of posting that I spotted a couple of places in ListUtil.xs where things could be improved. Latest patch, as against S-L-U.1.52, is attached. I think this is now in good enough shape for a pull request, which I'll try to make tomorrow. Cheers, Rob
Subject: SLU.diff4.txt
Download SLU.diff4.txt
text/plain 9.9k
--- Makefile.PL_orig 2019-08-28 09:11:02 +1000 +++ Makefile.PL 2019-08-28 11:50:59 +1000 @@ -7,11 +7,30 @@ use ExtUtils::MakeMaker; my $PERL_CORE = grep { $_ eq 'PERL_CORE=1' } @ARGV; + +my $defines = $ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]; + +# Determine the correct NV formatting required by uniq() in ListUtil.xs + +if($Config{nvsize} == 8) { # double or 8-byte long double + $defines .= " -DUNIQ_LOOP_END=8"; + $defines .= " -DUNIQ_PREC_LOSS" if $Config{ivsize} >= 8; # precision loss is possible when + # IV/UV is converted to NV +} + +elsif(length(sqrt 2) > 25) { # IEEE long double or __float128 or doubledouble + $defines .= " -DUNIQ_LOOP_END=16" +} + +else { # extended precision long double (either 12-byte or 16-byte) + $defines .= " -DUNIQ_LOOP_END=10"; # ignore unused bytes +} + WriteMakefile( NAME => q[List::Util], ABSTRACT => q[Common Scalar and List utility subroutines], AUTHOR => q[Graham Barr <gbarr@cpan.org>], - DEFINE => ($ENV{PERL_CORE} ? q[-DPERL_EXT] : q[-DPERL_EXT -DUSE_PPPORT_H]), + DEFINE => $defines, DISTNAME => q[Scalar-List-Utils], VERSION_FROM => 'lib/List/Util.pm', --- ListUtil.xs_orig 2019-08-28 09:10:36 +1000 +++ ListUtil.xs 2019-08-28 21:14:28 +1000 @@ -72,6 +72,19 @@ #define sv_catpvn_flags(b,n,l,f) sv_catpvn(b,n,l) #endif +#ifdef UNIQ_PREC_LOSS +#define UNIQ_BUFF_SIZE 33 /* nvsize == 8, ivsize == 8 */ + +#elif UNIQ_LOOP_END == 8 /* nvsize == 8, ivsize < 8 **/ +#define UNIQ_BUFF_SIZE 17 + +#elif UNIQ_LOOP_END == 10 /* NV is 10 bytes ***********/ +#define UNIQ_BUFF_SIZE 21 + +#else +#define UNIQ_BUFF_SIZE 33 /* NV is 16 bytes ***********/ +#endif + /* Some platforms have strict exports. And before 5.7.3 cxinc (or Perl_cxinc) was not exported. Therefore platforms like win32, VMS etc have problems so we redefine it here -- GMB @@ -1152,6 +1165,15 @@ int index; SV **args = &PL_stack_base[ax]; HV *seen; + size_t i; + NV nv_arg; + void *p = &nv_arg; + char s[UNIQ_BUFF_SIZE]; + char * buff = s; +#ifdef UNIQ_PREC_LOSS + size_t ivstring = 0, offset = 0; + IV int_arg; +#endif if(items == 0 || (items == 1 && !SvGAMAGIC(args[0]) && SvOK(args[0]))) { /* Optimise for the case of the empty list or a defined nonmagic @@ -1185,12 +1207,106 @@ #endif } - if(!SvOK(arg) || SvUOK(arg)) - sv_setpvf(keysv, "%" UVuf, SvUV(arg)); - else if(SvIOK(arg)) - sv_setpvf(keysv, "%" IVdf, SvIV(arg)); - else - sv_setpvf(keysv, "%.15" NVgf, SvNV(arg)); + /* Assign appropriate value to nv_arg */ + + if(!SvOK(arg) || SvUOK(arg)) { + nv_arg = (NV)SvUV(arg); +#ifdef UNIQ_PREC_LOSS + if(SvUV(arg) > 9007199254740992) { /* UV to NV conversion could lose precision */ + ivstring = 1; + int_arg = SvIV(arg); /* SvIV(arg) and SvuV(arg) have the same byte structure * + * and it's only the byte structure that interests us */ + } +#endif + } + + else if(SvIOK(arg)) { + nv_arg = (NV)SvIV(arg); +#ifdef UNIQ_PREC_LOSS + int_arg = SvIV(arg); + if(int_arg < -4503599627370496) /* IV to NV conversion could lose precision */ + ivstring = 1; +#endif + } + + else { + nv_arg = SvNV(arg); + } + + /* Handle NaN, zeros and Infs */ + + if(nv_arg != nv_arg) { + sv_setpvf(keysv, "%s", "NaN"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; +#endif + } + + else if(nv_arg == 0) { + sv_setpvf(keysv, "%s", "0"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; +#endif + } + + else if(nv_arg/nv_arg != 1) { + if(nv_arg < 0) sv_setpvf(keysv, "%s", "-Inf"); + else sv_setpvf(keysv, "%s", "Inf"); +#ifdef UNIQ_PREC_LOSS + ivstring = 0; +#endif + } + + /* Handle all values other than NaN, zeros and Infs */ + + else { +#ifdef UNIQ_PREC_LOSS + if(ivstring) { + + /* Read the bytes of iv_arg into the string buffer, in hex */ + + sprintf(buff, "%016" UVxf, int_arg); + buff += 16; + offset = 16; + } + + + /* Take appropriate action if nv_arg holds an integer value * + * within the IV and UV range that can lose precision as an NV */ + + else if(ceil(nv_arg) == nv_arg + && ((nv_arg <= 1.8446744073709552e+19 && nv_arg > 9007199254740992) + || + (nv_arg < -4503599627370496 && nv_arg >= -9.2233720368547758e+18)) ) { + + int_arg = SvIV(arg); + + /* Read the bytes of int_arg into the string buffer, in hex */ + + sprintf(buff, "%016" UVxf, int_arg); + buff += 16; + offset = 16; /* the buff pointer has been incremented by 16 */ + } + + ivstring = 0; + + for(i = 0; i < 8; i++) { + sprintf(buff, "%02x", ((unsigned char*)p)[i]); + buff += 2; + } + + buff -= 16 + offset; /* return pointer to original position */ + offset = 0; +#else + for(i = 0; i < UNIQ_LOOP_END; i++) { + sprintf(buff, "%02x", ((unsigned char*)p)[i]); + buff += 2; + } + + buff -= 2 * UNIQ_LOOP_END; /* return pointer to original position */ +#endif + sv_setpvf(keysv, "%s", buff); + } #ifdef HV_FETCH_EMPTY_HE he = (HE*) hv_common(seen, NULL, SvPVX(keysv), SvCUR(keysv), 0, HV_FETCH_LVALUE | HV_FETCH_EMPTY_HE, NULL, 0); if (HeVAL(he)) --- t/uniq.t_orig 2019-08-28 09:11:41 +1000 +++ t/uniq.t 2019-08-27 19:12:28 +1000 @@ -2,8 +2,8 @@ use strict; use warnings; - -use Test::More tests => 33; +use Config; # to determine nvsize +use Test::More tests => 35; use List::Util qw( uniqnum uniqstr uniq ); use Tie::Array; @@ -87,6 +87,76 @@ 'uniqnum distinguishes large floats (stringified)' ); } +my ($uniq_count1, $uniq_count2, $equiv); + +if($Config{nvsize} == 8) { + # NV is either 'double' or 8-byte 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.4142135623730951 == 1.4142135623730954; + + $uniq_count1 = List::Util::uniqnum (1.4142135623730951, + 1.4142135623730954 ); + + $uniq_count2 = List::Util::uniqnum('1.4142135623730951', + '1.4142135623730954'); +} + +elsif(length(sqrt(2)) > 25) { + # NV is either IEEE 'long double' or '__float128' or doubledouble + + if(1 + (2 ** -1074) != 1) { + # NV is doubledouble + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1 + (2 ** -1074) == 1 + (2 ** - 1073); + + $uniq_count1 = List::Util::uniqnum (1 + (2 ** -1074), + 1 + (2 ** -1073) ); + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 4.0564819207303340847894502572035e31 == 4.0564819207303340847894502572034e31; + + $uniq_count2 = List::Util::uniqnum('4.0564819207303340847894502572035e31', + '4.0564819207303340847894502572034e31'); + } + + else { + # NV is either IEEE 'long double' or '__float128' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 1.7320508075688772935274463415058722 == 1.73205080756887729352744634150587224; + + $uniq_count1 = List::Util::uniqnum (1.7320508075688772935274463415058722, + 1.73205080756887729352744634150587224 ); + + $uniq_count2 = List::Util::uniqnum('1.7320508075688772935274463415058722', + '1.73205080756887729352744634150587224'); + } +} + +else { + # NV is extended precision 'long double' + + # The 2 values should be unequal - but just in case perl is buggy: + $equiv = 1 if 2.2360679774997896963 == 2.23606797749978969634; + + $uniq_count1 = List::Util::uniqnum (2.2360679774997896963, + 2.23606797749978969634 ); + + $uniq_count2 = List::Util::uniqnum('2.2360679774997896963', + '2.23606797749978969634'); +} + +if($equiv) { + is($uniq_count1, 1, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 1, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + +else { + is($uniq_count1, 2, 'uniqnum preserves uniqness of high precision floats'); + is($uniq_count2, 2, 'uniqnum preserves uniqness of high precision floats (stringified)'); +} + # Hard to know for sure what an Inf is going to be. Lets make one my $Inf = 0 + 1E1000; my $NaN; @@ -109,8 +179,13 @@ my @strs = map "$_", @nums; - skip( "Perl $] doesn't stringify UV_MAX right ($maxuint)", 1 ) - if $maxuint !~ /\A[0-9]+\z/; + if($maxuint !~ /\A[0-9]+\z/) { + skip( "Perl $] doesn't stringify UV_MAX right ($maxuint)", 1 ); + } + elsif($] < 5.022 && $^O =~ /MSWin32/i) { + skip( "On MS Windows,perl $] stringifies infs and nans into something unusable", 1 ); + } + is_deeply( [ uniqnum @strs, "1.0" ], [ @strs ],
Download (untitled) / with headers
text/plain 743b
On Tue Aug 27 09:48:16 2019, SISYPHUS wrote: Show quoted text
> > For perl-5.20.x and earlier on MS Windows I discovered that the > 'uniqnum preserves uniqness of full integer range (stringified)' test > needs to be skipped because of the bizarre way that infs and nans are > stringified on those particular perls. > AFAICT it's actually not possible to come up with a string that will > numify to NaN, though strings that numify to Infs do exist (eg > '1e1000'). > > Graham's script also needs to be skipped for those builds - for the > same reason.
I don't have a windows machine to test this on right now, but could you test the latest master of List-Util-MaybeXS on those perls? The script shouldn't rely on Infs and NaNs being numifiable from strings.
Download (untitled) / with headers
text/plain 2.1k
On Fri Aug 30 06:30:04 2019, haarg wrote: Show quoted text
> The script shouldn't rely on Infs and NaNs being numifiable from strings.
But it means that you can't test the numification of NaN from a string. I'm not sure that it matters all that much, but it's something that both scripts were attempting to do. That aside, I tried a "cpan -fi List::Util::MaybeXS" on my Windows build of x64 perl-5.20.0 where both ivsize and nvsize are 8. You'll be pleased to know that List-Util-MaybeXS-1.500003 built and installed just fine, with all tests passing. In installing that module List::Util was updated from version 1.38 to 1.52, with its t/uniq.t failing one test: t/uniq.t .............. 1/33 Argument "1.#INF" isn't numeric in subroutine entry at t/uniq.t line 115. Argument "-1.#IND" isn't numeric in subroutine entry at t/uniq.t line 115. # Failed test 'uniqnum preserves uniqness of full integer range (stringified)' # at t/uniq.t line 115. # Structures begin differing at: # $got->[4] = '9223372036854775807' # $expected->[4] = '-1.#IND' # Looks like you failed 1 test of 33. t/uniq.t .............. Dubious, test returned 1 (wstat 256, 0x100) Things became a little annoying when I tried to build the current git version of List-Util-MaybeXS on the same build of perl: C:\sisyphusion\list-util-maybexs>perl Makefile.PL 'git' is not recognized as an internal or external command,operable program or batch file. Can't locate Distar.pm in @INC (you may need to install the Distar module) (@INC contains: Distar/lib inc C:/_64/perl520_482/site/lib C:/_64/perl520_482/lib .) at ./maint/Makefile.PL.include line 3. BEGIN failed--compilation aborted at ./maint/Makefile.PL.include line 3. C:\sisyphusion\list-util-maybexs>cpan -i Distar Reading 'C:\Users\sisyphus\.cpan\Metadata' Database was generated on Fri, 30 Aug 2019 12:41:02 GMT Warning: Cannot install Distar, don't know what it is. Try the command i /Distar/ to find objects with matching identifiers. At that point I gave up. I don't see why the 'git' command should need to be found to build a perl module - though I'm certainly quite happy to pursue this further if there's something to be achieved (&& if you tell me what steps I need to take ;-) Cheers, Rob
Download (untitled) / with headers
text/plain 2.6k
On Fri Aug 30 10:06:21 2019, SISYPHUS wrote: Show quoted text
> On Fri Aug 30 06:30:04 2019, haarg wrote: >
> > The script shouldn't rely on Infs and NaNs being numifiable from > > strings.
> > But it means that you can't test the numification of NaN from a > string. > I'm not sure that it matters all that much, but it's something that > both scripts were attempting to do.
I've adjusted the test to avoid doing that where it won't work. It wasn't a behavior I was intentionally relying on. Show quoted text
> > That aside, I tried a "cpan -fi List::Util::MaybeXS" on my Windows > build of x64 perl-5.20.0 where both ivsize and nvsize are 8. > You'll be pleased to know that List-Util-MaybeXS-1.500003 built and > installed just fine, with all tests passing. > In installing that module List::Util was updated from version 1.38 to > 1.52, with its t/uniq.t failing one test: > > t/uniq.t .............. 1/33 Argument "1.#INF" isn't numeric in > subroutine entry > at t/uniq.t line 115. > Argument "-1.#IND" isn't numeric in subroutine entry at t/uniq.t line > 115. > > # Failed test 'uniqnum preserves uniqness of full integer range > (stringified)' > > # at t/uniq.t line 115. > # Structures begin differing at: > # $got->[4] = '9223372036854775807' > # $expected->[4] = '-1.#IND' > # Looks like you failed 1 test of 33. > t/uniq.t .............. Dubious, test returned 1 (wstat 256, 0x100) > > Things became a little annoying when I tried to build the current git > version of List-Util-MaybeXS on the same build of perl: > > C:\sisyphusion\list-util-maybexs>perl Makefile.PL > 'git' is not recognized as an internal or external command,operable > program or batch file. > Can't locate Distar.pm in @INC (you may need to install the Distar > module) (@INC contains: Distar/lib inc C:/_64/perl520_482/site/lib > C:/_64/perl520_482/lib .) at ./maint/Makefile.PL.include line 3. > BEGIN failed--compilation aborted at ./maint/Makefile.PL.include line > 3. > > C:\sisyphusion\list-util-maybexs>cpan -i Distar > Reading 'C:\Users\sisyphus\.cpan\Metadata' > Database was generated on Fri, 30 Aug 2019 12:41:02 GMT > Warning: Cannot install Distar, don't know what it is. > Try the command > > i /Distar/ > > to find objects with matching identifiers. > > At that point I gave up. I don't see why the 'git' command should need > to be found to build a perl module - though I'm certainly quite happy > to pursue this further if there's something to be achieved (&& if you > tell me what steps I need to take ;-) > > Cheers, > Rob >
Distar is the dist authoring tool I'm using, and it is distributed via git. You shouldn't need it to run the tests though, since it's a pure perl module. You can run the tests using `prove -l t`.
Download (untitled) / with headers
text/plain 590b
Incidentally, with List-Util-1.52 and List-Util-MaybeXS-1.500003 I see the following output: C:\>perl -MList::Util::MaybeXS="uniqnum" -le "@x=uniqnum(9223372036854775808, 9.2233720368547758e+18); print \"@x\";" 9223372036854775808 9.2233720368547758e+18 This should be reproducible on any perl whose nvsize and ivsize are 8. I believe it's incorrect (as both values in binary are simply 1 followed by 63 zeroes), but it's what List-Util-1.52 produces. The accurate_uniqnum() sub provided in the script earlier supplied by Graham recognizes that the 2 values are equivalent. Cheers, Rob
CC: "sisyphus [...] cpan.org via RT" <sisyphus [...] cpan.org>
Subject: Re: [rt.cpan.org #130302] uniqnum() doesn't handle float strings correctly.
Date: Sat, 31 Aug 2019 21:23:29 +1000
To: bug-Scalar-List-Utils [...] rt.cpan.org
From: sisyphus <sisyphus359 [...] gmail.com>
Download (untitled) / with headers
text/plain 3.5k
Graham, I can get the git version of List::Util::MaybeXS to build by appending the location of Cygwin's git.exe to my PATH. However, on all of those windows versions of perl that stringify infs and nans as something absurd (ie pre 5.22), t/uniqnum.t cannot be run: t/uniqnum.t ........... "use" not allowed in expression at t/uniqnum.t line 94, at end of line BEGIN not safe after errors--compilation aborted at t/uniqnum.t line 94. t/uniqnum.t ........... Dubious, test returned 34 (wstat 8704, 0x2200) Line 94 is: use warnings FATAL => 'all'; Let me know when you've fixed that, and I'll gladly give it another go. Cheers, Rob On Sat, Aug 31, 2019 at 12:35 AM Graham Knop via RT < bug-Scalar-List-Utils@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=130302 > > > On Fri Aug 30 10:06:21 2019, SISYPHUS wrote:
> > On Fri Aug 30 06:30:04 2019, haarg wrote: > >
> > > The script shouldn't rely on Infs and NaNs being numifiable from > > > strings.
> > > > But it means that you can't test the numification of NaN from a > > string. > > I'm not sure that it matters all that much, but it's something that > > both scripts were attempting to do.
> > I've adjusted the test to avoid doing that where it won't work. It wasn't > a behavior I was intentionally relying on. >
> > > > That aside, I tried a "cpan -fi List::Util::MaybeXS" on my Windows > > build of x64 perl-5.20.0 where both ivsize and nvsize are 8. > > You'll be pleased to know that List-Util-MaybeXS-1.500003 built and > > installed just fine, with all tests passing. > > In installing that module List::Util was updated from version 1.38 to > > 1.52, with its t/uniq.t failing one test: > > > > t/uniq.t .............. 1/33 Argument "1.#INF" isn't numeric in > > subroutine entry > > at t/uniq.t line 115. > > Argument "-1.#IND" isn't numeric in subroutine entry at t/uniq.t line > > 115. > > > > # Failed test 'uniqnum preserves uniqness of full integer range > > (stringified)' > > > > # at t/uniq.t line 115. > > # Structures begin differing at: > > # $got->[4] = '9223372036854775807' > > # $expected->[4] = '-1.#IND' > > # Looks like you failed 1 test of 33. > > t/uniq.t .............. Dubious, test returned 1 (wstat 256, 0x100) > > > > Things became a little annoying when I tried to build the current git > > version of List-Util-MaybeXS on the same build of perl: > > > > C:\sisyphusion\list-util-maybexs>perl Makefile.PL > > 'git' is not recognized as an internal or external command,operable > > program or batch file. > > Can't locate Distar.pm in @INC (you may need to install the Distar > > module) (@INC contains: Distar/lib inc C:/_64/perl520_482/site/lib > > C:/_64/perl520_482/lib .) at ./maint/Makefile.PL.include line 3. > > BEGIN failed--compilation aborted at ./maint/Makefile.PL.include line > > 3. > > > > C:\sisyphusion\list-util-maybexs>cpan -i Distar > > Reading 'C:\Users\sisyphus\.cpan\Metadata' > > Database was generated on Fri, 30 Aug 2019 12:41:02 GMT > > Warning: Cannot install Distar, don't know what it is. > > Try the command > > > > i /Distar/ > > > > to find objects with matching identifiers. > > > > At that point I gave up. I don't see why the 'git' command should need > > to be found to build a perl module - though I'm certainly quite happy > > to pursue this further if there's something to be achieved (&& if you > > tell me what steps I need to take ;-) > > > > Cheers, > > Rob > >
> > Distar is the dist authoring tool I'm using, and it is distributed via git. > > You shouldn't need it to run the tests though, since it's a pure perl > module. You can run the tests using `prove -l t`. >
CC: "sisyphus [...] cpan.org via RT" <sisyphus [...] cpan.org>
Subject: Re: [rt.cpan.org #130302] uniqnum() doesn't handle float strings correctly.
Date: Sun, 1 Sep 2019 00:07:43 +1000
To: bug-Scalar-List-Utils [...] rt.cpan.org
From: sisyphus <sisyphus359 [...] gmail.com>
Download (untitled) / with headers
text/plain 5.5k
Taking a punt that removal of the offending "use warnings FATAL => 'all';" is a valid fix, all tests then pass on my Windows perl-5.20.0 (nvsize=8, ivsize=8). However, I see multiple warnings during the running of t/uniqnum.t: ...... t/uniq.t .............. ok t/uniqnum.t ........... Argument "1.#INF" isn't numeric in addition (+) at t/uni qnum.t line 83. Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. t/uniqnum.t ........... ok ...... Cheers, Rob On Sat, Aug 31, 2019 at 9:23 PM sisyphus <sisyphus359@gmail.com> wrote: Show quoted text
> Graham, > > I can get the git version of List::Util::MaybeXS to build by appending the > location of Cygwin's git.exe to my PATH. > > However, on all of those windows versions of perl that stringify infs and > nans as something absurd (ie pre 5.22), t/uniqnum.t cannot be run: > > t/uniqnum.t ........... "use" not allowed in expression at t/uniqnum.t > line 94, at end of line > BEGIN not safe after errors--compilation aborted at t/uniqnum.t line 94. > t/uniqnum.t ........... Dubious, test returned 34 (wstat 8704, 0x2200) > > Line 94 is: > use warnings FATAL => 'all'; > > Let me know when you've fixed that, and I'll gladly give it another go. > > Cheers, > Rob > > On Sat, Aug 31, 2019 at 12:35 AM Graham Knop via RT < > bug-Scalar-List-Utils@rt.cpan.org> wrote: >
>> <URL: https://rt.cpan.org/Ticket/Display.html?id=130302 > >> >> On Fri Aug 30 10:06:21 2019, SISYPHUS wrote:
>> > On Fri Aug 30 06:30:04 2019, haarg wrote: >> >
>> > > The script shouldn't rely on Infs and NaNs being numifiable from >> > > strings.
>> > >> > But it means that you can't test the numification of NaN from a >> > string. >> > I'm not sure that it matters all that much, but it's something that >> > both scripts were attempting to do.
>> >> I've adjusted the test to avoid doing that where it won't work. It >> wasn't a behavior I was intentionally relying on. >>
>> > >> > That aside, I tried a "cpan -fi List::Util::MaybeXS" on my Windows >> > build of x64 perl-5.20.0 where both ivsize and nvsize are 8. >> > You'll be pleased to know that List-Util-MaybeXS-1.500003 built and >> > installed just fine, with all tests passing. >> > In installing that module List::Util was updated from version 1.38 to >> > 1.52, with its t/uniq.t failing one test: >> > >> > t/uniq.t .............. 1/33 Argument "1.#INF" isn't numeric in >> > subroutine entry >> > at t/uniq.t line 115. >> > Argument "-1.#IND" isn't numeric in subroutine entry at t/uniq.t line >> > 115. >> > >> > # Failed test 'uniqnum preserves uniqness of full integer range >> > (stringified)' >> > >> > # at t/uniq.t line 115. >> > # Structures begin differing at: >> > # $got->[4] = '9223372036854775807' >> > # $expected->[4] = '-1.#IND' >> > # Looks like you failed 1 test of 33. >> > t/uniq.t .............. Dubious, test returned 1 (wstat 256, 0x100) >> > >> > Things became a little annoying when I tried to build the current git >> > version of List-Util-MaybeXS on the same build of perl: >> > >> > C:\sisyphusion\list-util-maybexs>perl Makefile.PL >> > 'git' is not recognized as an internal or external command,operable >> > program or batch file. >> > Can't locate Distar.pm in @INC (you may need to install the Distar >> > module) (@INC contains: Distar/lib inc C:/_64/perl520_482/site/lib >> > C:/_64/perl520_482/lib .) at ./maint/Makefile.PL.include line 3. >> > BEGIN failed--compilation aborted at ./maint/Makefile.PL.include line >> > 3. >> > >> > C:\sisyphusion\list-util-maybexs>cpan -i Distar >> > Reading 'C:\Users\sisyphus\.cpan\Metadata' >> > Database was generated on Fri, 30 Aug 2019 12:41:02 GMT >> > Warning: Cannot install Distar, don't know what it is. >> > Try the command >> > >> > i /Distar/ >> > >> > to find objects with matching identifiers. >> > >> > At that point I gave up. I don't see why the 'git' command should need >> > to be found to build a perl module - though I'm certainly quite happy >> > to pursue this further if there's something to be achieved (&& if you >> > tell me what steps I need to take ;-) >> > >> > Cheers, >> > Rob >> >
>> >> Distar is the dist authoring tool I'm using, and it is distributed via >> git. >> >> You shouldn't need it to run the tests though, since it's a pure perl >> module. You can run the tests using `prove -l t`. >>
>
On Sat Aug 31 10:08:30 2019, sisyphus359@gmail.com wrote: Show quoted text
> Taking a punt that removal of the offending "use warnings FATAL => 'all';" > is a valid fix, all tests then pass on my Windows perl-5.20.0 (nvsize=8, > ivsize=8). > However, I see multiple warnings during the running of t/uniqnum.t: > > ...... > t/uniq.t .............. ok > t/uniqnum.t ........... Argument "1.#INF" isn't numeric in addition (+) at > t/uni > qnum.t line 83. > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > t/uniqnum.t ........... ok > ...... > > Cheers, > Rob >
The test should be fixed now. It's meant to filter out those strings that can't be converted to numbers, but I had the order of some things wrong.
CC: "sisyphus [...] cpan.org via RT" <sisyphus [...] cpan.org>
Subject: Re: [rt.cpan.org #130302] uniqnum() doesn't handle float strings correctly.
Date: Sun, 1 Sep 2019 23:52:26 +1000
To: bug-Scalar-List-Utils [...] rt.cpan.org
From: sisyphus <sisyphus359 [...] gmail.com>
Download (untitled) / with headers
text/plain 3.1k
Latest git version of List::Util::MaybeXS is looking fine on windows - both on those versions of perl that stringify infs and nans absurdly, and those that stringify them sanely. I also tested my Linux quadmath build of 5.31.3 which was fine. It doesn't fare so well on my PowerPC machine where the nvtype is DoubleDouble, but you're probably not interested in that (nor would I assert that there's any reason you ought to be concerned about it). FYI, with that DoubleDouble build of perl-5.31.3 the first test fails: # Failed test 'correct count of unique numbers' # at t/uniqnum.t line 151. # got: '187' # expected: '184' and then more tests start failing at: # Failed test 'Found correct 1e5 in uniqnum output' # at t/uniqnum.t line 166. # got: 100000 # expected: -1 # Failed test 'Found correct -1e5 in uniqnum output' # at t/uniqnum.t line 166. # got: -100000 # expected: -10 Cheers, Rob On Sun, Sep 1, 2019 at 7:55 PM Graham Knop via RT < bug-Scalar-List-Utils@rt.cpan.org> wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=130302 > > > On Sat Aug 31 10:08:30 2019, sisyphus359@gmail.com wrote:
> > Taking a punt that removal of the offending "use warnings FATAL =>
> 'all';"
> > is a valid fix, all tests then pass on my Windows perl-5.20.0 (nvsize=8, > > ivsize=8). > > However, I see multiple warnings during the running of t/uniqnum.t: > > > > ...... > > t/uniq.t .............. ok > > t/uniqnum.t ........... Argument "1.#INF" isn't numeric in addition (+)
> at
> > t/uni > > qnum.t line 83. > > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#INF" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "-1.#IND" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > > Argument "1.#QNAN" isn't numeric in addition (+) at t/uniqnum.t line 83. > > t/uniqnum.t ........... ok > > ...... > > > > Cheers, > > Rob > >
> > The test should be fixed now. It's meant to filter out those strings that > can't be converted to numbers, but I had the order of some things wrong. >
CC: "sisyphus [...] cpan.org via RT" <sisyphus [...] cpan.org>
Subject: Re: [rt.cpan.org #130302] uniqnum() doesn't handle float strings correctly.
Date: Wed, 4 Sep 2019 00:06:38 +1000
To: bug-Scalar-List-Utils [...] rt.cpan.org
From: sisyphus <sisyphus359 [...] gmail.com>
Download (untitled) / with headers
text/plain 130b
I probably should mention that I've created a pull request: https://github.com/Dual-Life/Scalar-List-Utils/pull/79 Cheers, Rob


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.