Skip Menu |
 

This queue is for tickets about the CHI CPAN distribution.

Report information
The Basics
Id: 78590
Status: open
Priority: 0/
Queue: CHI

People
Owner: Nobody in particular
Requestors: BBYRD [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: (no value)
Fixed in: (no value)



Subject: CHI::Driver::File should consult POSIX's PATH_MAX for max_key_length
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 573
Download (untitled) / with headers
text/plain 573b
Instead of requiring the user to eventually figure it out, the File driver should use a sane default for max_key_length based on POSIX's PATH_MAX, the root_dir, and expected length of the namespace, directories, etc. Windows only has a PATH_MAX of 260, so storage of any large keys will typically not work. Since there is no error message by default, it will be silently ignored. Here's a formula to use for setting a max_key_length: PATH_MAX - length( $root_dir->subdir($namespace, 0, 0) ) - 4 - 8 The 4 is the '.dat' extension, and the 8 is an extra margin of error.
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-31397-1343320864-1443.78590-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 131
Download (untitled) / with headers
text/plain 131b
Specifically, this works: min( POSIX::PATH_MAX - length( $root_dir->subdir($namespace, 0, 0)->absolute->stringify ) - 4 - 8, 248)
MIME-Version: 1.0
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-14806-1345941027-501.78590-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 339
Download (untitled) / with headers
text/plain 339b
CHI::Driver::File currently defaults max_key_length to 248, and should automatically digest keys that grow larger than that (after characters are escaped for file system safety). You say that larger keys typically "will not work" on Windows - have you actually observed that? If so it means that something about the above is not working.
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-14806-1345941027-501.78590-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <rt-3.8.HEAD-14806-1345941027-501.78590-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-12146-1347391402-1540.78590-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 1061
On Sat Aug 25 20:30:27 2012, JSWARTZ wrote: Show quoted text
> CHI::Driver::File currently defaults max_key_length to 248, and should > automatically digest keys that grow larger than that (after characters > are escaped for file system safety). > > You say that larger keys typically "will not work" on Windows - have you > actually observed that? If so it means that something about the above is > not working.
Yes, this came from an example of using Dist::Zilla::Role::MetaCPANInterfacer on my Windows laptop. While the 248 max is larger than 260, it doesn't present a lot of room for the root path; only 12 chars. So, something like "C:\Documents and Settings..." is already going to run past PATH_MAX. This would be present in my module here: CHI->new( namespace => 'MetaCPAN', driver => 'File', expires_in => '1d', root_dir => Path::Class::dir( File::HomeDir->my_home )->subdir('.dzil', '.webcache')->stringify, ) Once you go past PATH_MAX, you start running into "File not found" errors for the cache entries.
From swartz [...] pobox.com Tue Sep 11 15: 26:51 2012
MIME-Version: 1.0 (Apple Message framework v1278)
X-Spam-Status: No, score=-5.351 tagged_above=-99.9 required=10 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_SORBS_WEB=0.77, SPF_NEUTRAL=0.779] autolearn=ham
In-Reply-To: <rt-3.8.HEAD-12146-1347391403-867.78590-5-0 [...] rt.cpan.org>
X-Mailer: Apple Mail (2.1278)
X-Spam-Flag: NO
References: <RT-Ticket-78590 [...] rt.cpan.org> <rt-3.8.HEAD-14806-1345941027-501.78590-5-0 [...] rt.cpan.org> <rt-3.8.HEAD-12146-1347391403-867.78590-5-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Content-Type: text/plain; charset="utf-8"
Message-ID: <D0770D5C-ED36-4D8C-8909-DCC022C7B053 [...] pobox.com>
X-Sasl-Enc: JfwezSnv8hOTgEpgZ0GnVzWSKeWI/RlFEtaxxbwwnDXV 1347391598
X-RT-Original-Encoding: utf-8
X-Spam-Score: -5.351
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] messagingengine.com
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id F22F6240759 for <cpan-bug+CHI [...] hipster.bestpractical.com>; Tue, 11 Sep 2012 15:26:50 -0400 (EDT)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WWnlgX3PB1NC for <cpan-bug+CHI [...] hipster.bestpractical.com>; Tue, 11 Sep 2012 15:26:45 -0400 (EDT)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id 5FC35240745 for <bug-CHI [...] rt.cpan.org>; Tue, 11 Sep 2012 15:26:45 -0400 (EDT)
Received: (qmail 27818 invoked by uid 103); 11 Sep 2012 19:26:44 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 11 Sep 2012 19:26:44 -0000
Received: from out1-smtp.messagingengine.com (HELO out1-smtp.messagingengine.com) (66.111.4.25) by 16.mx.develooper.com (qpsmtpd/0.84/v0.84-167-g4ed6cab) with ESMTP; Tue, 11 Sep 2012 12:26:41 -0700
Received: from compute1.internal (compute1.nyi.mail.srv.osa [10.202.2.41]) by gateway1.nyi.mail.srv.osa (Postfix) with ESMTP id 9864E21406 for <bug-CHI [...] rt.cpan.org>; Tue, 11 Sep 2012 15:26:38 -0400 (EDT)
Received: from frontend1.nyi.mail.srv.osa ([10.202.2.160]) by compute1.internal (MEProxy); Tue, 11 Sep 2012 15:26:38 -0400
Received: from [10.119.22.149] (unknown [204.126.251.245]) by mail.messagingengine.com (Postfix) with ESMTPA id 625748E0212 for <bug-CHI [...] rt.cpan.org>; Tue, 11 Sep 2012 15:26:38 -0400 (EDT)
Delivered-To: cpan-bug+CHI [...] hipster.bestpractical.com
Subject: Re: [rt.cpan.org #78590] CHI::Driver::File should consult POSIX's PATH_MAX for max_key_length
Return-Path: <swartz [...] pobox.com>
Dkim-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-type:mime-version:subject:from :in-reply-to:date:content-transfer-encoding:message-id :references:to; s=smtpout; bh=BLz/5hdAYb2jBi91nE8/5B2id+Y=; b=cR IecVZWeYOVYRtSP0VW0ztvyXBhgEEqbdkYyiO8NmV0lpmqSzFoAqf2DTFjF95YBO 5xpkJYlZp/HtHmW/ww1umKcVSAdG2npj5Nsnr5U0UQRcmmDvWiywuEKE2X/WronE +aUGR0Cc1x4DKT3xhZobEmye+ED0fDqdDm1FdNoSw=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: cpan-bug+CHI [...] hipster.bestpractical.com
X-RT-Mail-Extension: chi
Date: Tue, 11 Sep 2012 15:26:38 -0400
X-Spam-Level:
To: bug-CHI [...] rt.cpan.org
Content-Transfer-Encoding: quoted-printable
From: Jonathan Swartz <swartz [...] pobox.com>
RT-Message-ID: <rt-3.8.HEAD-12146-1347391611-367.78590-0-0 [...] rt.cpan.org>
Content-Length: 1401
Download (untitled) / with headers
text/plain 1.3k
Oh, max_key_length will only affect the filename that gets created. You're talking about the maximum path including all the directories plus the file? On Sep 11, 2012, at 3:23 PM, Brendan Byrd via RT wrote: Show quoted text
> Queue: CHI > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=78590 > > > On Sat Aug 25 20:30:27 2012, JSWARTZ wrote:
>> CHI::Driver::File currently defaults max_key_length to 248, and should >> automatically digest keys that grow larger than that (after characters >> are escaped for file system safety). >> >> You say that larger keys typically "will not work" on Windows - have you >> actually observed that? If so it means that something about the above is >> not working.
> > Yes, this came from an example of using > Dist::Zilla::Role::MetaCPANInterfacer on my Windows laptop. While the > 248 max is larger than 260, it doesn't present a lot of room for the > root path; only 12 chars. So, something like "C:\Documents and > Settings..." is already going to run past PATH_MAX. > > This would be present in my module here: > > CHI->new( > namespace => 'MetaCPAN', > driver => 'File', > expires_in => '1d', > root_dir => Path::Class::dir( File::HomeDir->my_home > )->subdir('.dzil', '.webcache')->stringify, > ) > > Once you go past PATH_MAX, you start running into "File not found" > errors for the cache entries.
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-12146-1347391611-367.78590-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <RT-Ticket-78590 [...] rt.cpan.org> <rt-3.8.HEAD-14806-1345941027-501.78590-5-0 [...] rt.cpan.org> <rt-3.8.HEAD-12146-1347391403-867.78590-5-0 [...] rt.cpan.org> <D0770D5C-ED36-4D8C-8909-DCC022C7B053 [...] pobox.com> <rt-3.8.HEAD-12146-1347391611-367.78590-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-17655-1347446698-432.78590-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 331
Download (untitled) / with headers
text/plain 331b
On Tue Sep 11 15:26:51 2012, swartz@pobox.com wrote: Show quoted text
> Oh, max_key_length will only affect the filename that gets created. > You're talking about the maximum path including all the directories > plus the file?
Correct. In a grand majority of cases, it won't work on Win32 systems, because of that short amount of space in-between.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.