Skip Menu |
 

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Storable CPAN distribution.

Report information
The Basics
Id: 31794
Status: open
Priority: 0/
Queue: Storable

People
Owner: Nobody in particular
Requestors: mark_young [...] hotmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



X-Originalarrivaltime: 21 Dec 2007 11:09:42.0288 (UTC) FILETIME=[FC6E1100:01C843C1]
MIME-Version: 1.0
X-Spam-Status: No, hits=-2.6 required=8.0 tests=BAYES_00,HTML_MESSAGE,SPF_PASS
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
Importance: Normal
Content-Type: multipart/alternative; boundary="_93afc687-bf4f-42d3-8ac5-0442ce02f176_"
Received: from x1.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 019474D8015 for <bug-storable [...] rt.cpan.org>; Fri, 21 Dec 2007 06:09:59 -0500 (EST)
Received: (qmail 31591 invoked from network); 21 Dec 2007 11:09:59 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 21 Dec 2007 11:09:59 -0000
Received: from bay0-omc2-s38.bay0.hotmail.com (HELO bay0-omc2-s38.bay0.hotmail.com) (65.54.246.174) by 16.mx.develooper.com (qpsmtpd/0.40-dev) with ESMTP; Fri, 21 Dec 2007 03:09:46 -0800
Received: from BAY143-W6 ([65.55.154.41]) by bay0-omc2-s38.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 21 Dec 2007 03:09:42 -0800
Delivered-To: cpan-bug+storable [...] diesel.bestpractical.com
Subject: Reading 32-bit Storable files on 64-bit system
Return-Path: <mark_young [...] hotmail.com>
X-Original-To: bug-storable [...] rt.cpan.org
X-Spam-Check-BY: 16.mx.develooper.com
Date: Fri, 21 Dec 2007 11:09:42 +0000
X-Spam-Level: *
Message-Id: <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl>
X-Originating-Ip: [195.212.29.67]
To: <bug-storable [...] rt.cpan.org>
From: Mark Young <mark_young [...] hotmail.com>
Content-Length: 0
content-type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-RT-Original-Encoding: iso-8859-1
Content-Length: 2217
Download (untitled) / with headers
text/plain 2.1k
Hi, I have a perl application that has been running for many years on a 32-bit Debian system using perl v5.8.4 and Storable v2.14. That application has stored thousands of files in Storable format using the store() command. I'm attempting to migrate to a new 64-bit system using perl v5.8.8 and Storable v2.18. I've found that retrieve() on the 64-bit system can not read the 32-bit systems' Storables. The error it produces is Byte order is not compatible at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/_retrieve.al) line 380, at ./try line 37 The problem appears to be that retrieve() on the 64-bit system is not happy with the Storables longsize, ptrsize and byteorder. If I write out a Storable on each system and use the Storable::file_magic to read the details of the header I get these values that differ between the two systems. 32 bit system: 'hdrsize' => 15, 'version' => '2.7', 'longsize' => 4, 'ptrsize' => 4, 'byteorder' => '1234', 64 bit system: 'hdrsize' => 19, 'version' => '2.7', 'longsize' => 8, 'ptrsize' => 8, 'byteorder' => '12345678', I have subsequently found out, thanks to the perlmonks, that if I write a storable using nstore() then the resultant file can be written by either then 32-bit or 64-bit system and can be read by either the 32-bit or 64-bit system. That's useful, but unfortunately I have many thousands of files that would need to be converted. So the reason I am writing to explain this is to ask whether you would consider that the 64-bit system should be able to read the store() file written out by the 32-bit system? Shouldn't the 64-bit system be able to read the header of the Storable file and thus know how to read the contents? My ideal situation would be to be able to enable the 64-bit system to read the 32-bit systems' storables, preferably in a manner that means it could read Storables written out by either a 32-bit system or a 64-bit system, in the same way you get that capability if you'd used nstore all along. Please let me know what you think? Best Regards,Mark Young BTW - The incompatibility is the same between a 32-bit perl v5.8.8 Storable v2.18 system and a 64-bit perl v5.8.8 Storable v2.18 system.
content-type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-RT-Original-Encoding: iso-8859-1
Content-Length: 2633
MIME-Version: 1.0
In-Reply-To: <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-2367-1279729173-569.31794-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
From: walkeraj [...] gmail.com
X-RT-Original-Encoding: utf-8
Content-Length: 2848
Download (untitled) / with headers
text/plain 2.7k
This bug also affects me. I receive the exact same error when attempting to retrieve() a file that was created on a 32-bit system. ['Byte order is not compatible at blib/lib/Storable.pm (autosplit into blib/lib/auto/Storable/_retrieve.al) line 380'] Setting $Storable::interwork_56_64bit=1 does not solve the issue.Please let me know how I can provide further information to resolve this. On Fri Dec 21 06:10:13 2007, mark_young@hotmail.com wrote: Show quoted text
> > Hi, > > I have a perl application that has been running for many years on a > 32-bit Debian system using perl v5.8.4 and Storable v2.14. That > application has stored thousands of files in Storable format using > the store() command. I'm attempting to migrate to a new 64-bit > system using perl v5.8.8 and Storable v2.18. I've found that > retrieve() on the 64-bit system can not read the 32-bit systems' > Storables. The error it produces is > > Byte order is not compatible at blib/lib/Storable.pm (autosplit into > blib/lib/auto/Storable/_retrieve.al) line 380, at ./try line 37 > > The problem appears to be that retrieve() on the 64-bit system is not > happy with the Storables longsize, ptrsize and byteorder. If I > write out a Storable on each system and use the > Storable::file_magic to read the details of the header I get these > values that differ between the two systems. > > 32 bit system: 'hdrsize' => 15, 'version' => '2.7', 'longsize' > => 4, 'ptrsize' => 4, 'byteorder' => '1234', > > 64 bit system: 'hdrsize' => 19, 'version' => '2.7', 'longsize' > => 8, 'ptrsize' => 8, 'byteorder' => '12345678', > > I have subsequently found out, thanks to the perlmonks, that if I > write a storable using nstore() then the resultant file can be > written by either then 32-bit or 64-bit system and can be read by > either the 32-bit or 64-bit system. That's useful, but > unfortunately I have many thousands of files that would need to be > converted. So the reason I am writing to explain this is to ask > whether you would consider that the 64-bit system should be able to > read the store() file written out by the 32-bit system? Shouldn't > the 64-bit system be able to read the header of the Storable file > and thus know how to read the contents? > > My ideal situation would be to be able to enable the 64-bit system to > read the 32-bit systems' storables, preferably in a manner that > means it could read Storables written out by either a 32-bit system > or a 64-bit system, in the same way you get that capability if > you'd used nstore all along. > > Please let me know what you think? > > Best Regards,Mark Young > BTW - The incompatibility is the same between a 32-bit perl v5.8.8 > Storable v2.18 system and a 64-bit perl v5.8.8 Storable v2.18 > system.
From mark_young [...] hotmail.com Thu Jul 22 04: 41:07 2010
X-Originalarrivaltime: 22 Jul 2010 08:42:46.0898 (UTC) FILETIME=[DC004D20:01CB2979]
MIME-Version: 1.0
X-Spam-Status: No, score=-10.002 tagged_above=-99.9 required=10 tests=[BAYES_00=-2.599, CONFIRMED_FORGED=0, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-8, SPF_SOFTFAIL=0.596] autolearn=ham
In-Reply-To: <rt-3.8.HEAD-2367-1279729173-958.31794-6-0 [...] rt.cpan.org>
X-Spam-Flag: NO
Importance: Normal
References: <RT-Ticket-31794 [...] rt.cpan.org> <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl>,<rt-3.8.HEAD-2367-1279729173-958.31794-6-0 [...] rt.cpan.org>
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <BAY143-W2147A71E79A1859F9CA20EEFA20 [...] phx.gbl>
Content-Type: multipart/alternative; boundary="_a8544ca6-321f-46c1-99fe-448eec85b126_"
X-Spam-Score: -10.002
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id 275E4240AA6 for <cpan-bug+storable [...] hipster.bestpractical.com>; Thu, 22 Jul 2010 04:41:07 -0400 (EDT)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lFn--cMI2HMA for <cpan-bug+storable [...] hipster.bestpractical.com>; Thu, 22 Jul 2010 04:41:05 -0400 (EDT)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id 95CA1240A2F for <bug-storable [...] rt.cpan.org>; Thu, 22 Jul 2010 04:41:04 -0400 (EDT)
Received: (qmail 1015 invoked by uid 103); 22 Jul 2010 08:42:53 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 22 Jul 2010 08:42:53 -0000
Received: from bay0-omc1-s14.bay0.hotmail.com (HELO bay0-omc1-s14.bay0.hotmail.com) (65.54.190.25) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Thu, 22 Jul 2010 01:42:50 -0700
Received: from BAY143-W21 ([65.54.190.61]) by bay0-omc1-s14.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.4675); Thu, 22 Jul 2010 01:42:46 -0700
Delivered-To: cpan-bug+storable [...] hipster.bestpractical.com
Subject: RE: [rt.cpan.org #31794] Reading 32-bit Storable files on 64-bit system
Return-Path: <mark_young [...] hotmail.com>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: cpan-bug+storable [...] hipster.bestpractical.com
X-RT-Mail-Extension: storable
Date: Thu, 22 Jul 2010 08:42:46 +0000
X-Spam-Level:
X-Originating-Ip: [195.212.29.67]
To: <bug-storable [...] rt.cpan.org>
From: Mark Young <mark_young [...] hotmail.com>
RT-Message-ID: <rt-3.8.HEAD-2369-1279788177-739.31794-0-0 [...] rt.cpan.org>
Content-Length: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-RT-Original-Encoding: utf-8
Content-Length: 3516
Download (untitled) / with headers
text/plain 3.4k
Hi Andrew, I think I may have inadvertently deleted a mail from you. Did you mail me directly about this problem? I ended up writing a script that found and converted all of my perl programs & storables to nstore format. I never heard any response to filing that bug. Cheers, Mark Show quoted text
> Subject: [rt.cpan.org #31794] Reading 32-bit Storable files on 64-bit system > From: bug-Storable@rt.cpan.org > To: mark_young@hotmail.com > Date: Wed, 21 Jul 2010 12:19:33 -0400 > > <URL: https://rt.cpan.org/Ticket/Display.html?id=31794 > > > This bug also affects me. I receive the exact same error when > attempting to retrieve() a file that was created on a 32-bit system. > ['Byte order is not compatible at blib/lib/Storable.pm (autosplit into > blib/lib/auto/Storable/_retrieve.al) line 380'] > > Setting $Storable::interwork_56_64bit=1 does not solve the issue.Please > let me know how I can provide further information to resolve this. > > On Fri Dec 21 06:10:13 2007, mark_young@hotmail.com wrote:
> > > > Hi, > > > > I have a perl application that has been running for many years on a > > 32-bit Debian system using perl v5.8.4 and Storable v2.14. That > > application has stored thousands of files in Storable format using > > the store() command. I'm attempting to migrate to a new 64-bit > > system using perl v5.8.8 and Storable v2.18. I've found that > > retrieve() on the 64-bit system can not read the 32-bit systems' > > Storables. The error it produces is > > > > Byte order is not compatible at blib/lib/Storable.pm (autosplit into > > blib/lib/auto/Storable/_retrieve.al) line 380, at ./try line 37 > > > > The problem appears to be that retrieve() on the 64-bit system is not > > happy with the Storables longsize, ptrsize and byteorder. If I > > write out a Storable on each system and use the > > Storable::file_magic to read the details of the header I get these > > values that differ between the two systems. > > > > 32 bit system: 'hdrsize' => 15, 'version' => '2.7', 'longsize' > > => 4, 'ptrsize' => 4, 'byteorder' => '1234', > > > > 64 bit system: 'hdrsize' => 19, 'version' => '2.7', 'longsize' > > => 8, 'ptrsize' => 8, 'byteorder' => '12345678', > > > > I have subsequently found out, thanks to the perlmonks, that if I > > write a storable using nstore() then the resultant file can be > > written by either then 32-bit or 64-bit system and can be read by > > either the 32-bit or 64-bit system. That's useful, but > > unfortunately I have many thousands of files that would need to be > > converted. So the reason I am writing to explain this is to ask > > whether you would consider that the 64-bit system should be able to > > read the store() file written out by the 32-bit system? Shouldn't > > the 64-bit system be able to read the header of the Storable file > > and thus know how to read the contents? > > > > My ideal situation would be to be able to enable the 64-bit system to > > read the 32-bit systems' storables, preferably in a manner that > > means it could read Storables written out by either a 32-bit system > > or a 64-bit system, in the same way you get that capability if > > you'd used nstore all along. > > > > Please let me know what you think? > > > > Best Regards,Mark Young > > BTW - The incompatibility is the same between a 32-bit perl v5.8.8 > > Storable v2.18 system and a 64-bit perl v5.8.8 Storable v2.18 > > system.
> > >
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-RT-Original-Encoding: utf-8
Content-Length: 6371
MIME-Version: 1.0
In-Reply-To: <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-11057-1283210953-31.31794-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
From: bitcard_kiddm [...] ghctechnologies.com
X-RT-Original-Encoding: utf-8
Content-Length: 1346
Download (untitled) / with headers
text/plain 1.3k
I have the same problem as Mark on Perl 5.10 as I move into the 64-bit Linux world. At the very least the error message should be changed. Byte order means little-endian vs. big-endian. Both my Windows and Linux boxes are running on Intel chips and therefore have the same byte order. The issue is really 32-bit vs. 64-bit pointers and longs. Although network order is the proscribed workaround, switching to it degrades performance when operating on the same platform. And according to the Storeable documentation, doubles get stored as text instead of the exact machine representation with the consequence of losing +-Inf and NaN (often used to represent missing data). Since nearly all platforms now use the IEEE Standard 754 standard to store floating point numbers, it is annoying to unnecessarily lose precision when switching between platforms. It seems like reading 32-bit Storable files on 64-bit platforms would be relatively easy to implement and a great convenience. In fact I also think 32-bit platforms ought to make an effort to read 64-bit Storable files too. Yes, an exception will have to be thrown if a 64-bit integer that can not fit in 32-bits is encountered. And some very large data structures will use pointers that are too big. But those large structures aren't going to fit into the 32-bit Perl's memory space anyhow.
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-11057-1283210953-31.31794-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <BAY143-W6660575A90167C4A7C8C2EF5E0 [...] phx.gbl> <rt-3.8.HEAD-11057-1283210953-31.31794-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-10406-1486052689-877.31794-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 353
Download (untitled) / with headers
text/plain 353b
I'm working on a version which can read 32 from 64bit and vice versa on most platforms, when the double size and kind is the same. Only on BE (big-endian) you cannot read LE files, you need to store netorder via nstore there. It's still in work, but passes all the interload tests. https://github.com/rurban/Storable/commits/64read32 -- Reini Urban


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.