Bug #99230 for File-Path: Please document that File::Path::rmtree may not be used with threads

Tue Sep 30 14:24:36 2014 jim.avera [...] gmail.com - Ticket created

MIME-Version:	1.0
X-Spam-Status:	No, score=-3.699 tagged_above=-99.9 required=10 tests=[AWL=-1.000, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham
X-Spam-Flag:	NO
content-type:	text/plain; charset="utf-8"; format="flowed"
Reply-To:	jim.avera@gmail.com
Message-ID:	<542AF559.3010007@gmail.com>
X-Received:	by 10.68.216.35 with SMTP id on3mr71697295pbc.34.1412101467983; Tue, 30 Sep 2014 11:24:27 -0700 (PDT)
X-Virus-Scanned:	Debian amavisd-new at bestpractical.com
X-Spam-Score:	-3.699
Received:	from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id EF67824062C for <cpan-bug+File-Path@hipster.bestpractical.com>; Tue, 30 Sep 2014 14:24:35 -0400 (EDT)
Received:	from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Z5RdYFcdcva4 for <cpan-bug+File-Path@hipster.bestpractical.com>; Tue, 30 Sep 2014 14:24:35 -0400 (EDT)
Received:	from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id B6EB4240612 for <bug-File-Path@rt.cpan.org>; Tue, 30 Sep 2014 14:24:34 -0400 (EDT)
Received:	(qmail 27244 invoked by alias); 30 Sep 2014 18:24:33 -0000
Received:	from mail-pd0-f181.google.com (HELO mail-pd0-f181.google.com) (209.85.192.181) by la.mx.develooper.com (qpsmtpd/0.28) with ESMTP; Tue, 30 Sep 2014 11:24:31 -0700
Received:	by mail-pd0-f181.google.com with SMTP id z10so5623450pdj.40 for <bug-File-Path@rt.cpan.org>; Tue, 30 Sep 2014 11:24:28 -0700 (PDT)
Received:	from ?IPv6:2602:30a:2ccd:6950:a582:383d:990d:db72? ([2602:30a:2ccd:6950:a582:383d:990d:db72]) by mx.google.com with ESMTPSA id g13sm15901003pat.45.2014.09.30.11.24.26 for <bug-File-Path@rt.cpan.org> (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 30 Sep 2014 11:24:27 -0700 (PDT)
Authentication-Results:	hipster.bestpractical.com (amavisd-new); dkim=pass header.i=@gmail.com
Delivered-To:	cpan-bug+File-Path@hipster.bestpractical.com
Subject:	Please document that File::Path::rmtree may not be used with threads
User-Agent:	Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0
Return-Path:	<jim.avera@gmail.com>
X-RT-Mail-Extension:	file-path
X-Original-To:	cpan-bug+File-Path@hipster.bestpractical.com
X-Spam-Check-BY:	la.mx.develooper.com
Dkim-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:reply-to:user-agent:mime-version:to:subject :content-type:content-transfer-encoding; bh=TR10jp+10BCLVYOjcbvswBiNdShHzuKkKbzFDvw+chU=; b=VwwxnxGdaYEyBtUROCBwxTfAbd5FUMgLFgl7faXKpSm33E0v2M3l9dFmRopGYBCAvm jeOKVNBZwNyR38vYCVhqmJLNPU3VoDCGqMvURTjOAssoX5V9A3xNaNcaoJFPRfuqkcaV hnM+Rq1wKpOsHE+Wv23xuGajH3uf0J4LUmHzNmv72woC6OeFzdeaxXqmSNlZqnvlzRXV P3vv8n4q89Y/9RTTvokZRP0SvrzXW6wtjx1fsqAZ4c7AvNA9+/DyLUvr5LtdCdRnljim AsSDhhh8n40A0Sd4TSbrifqMe4OeqO8PINek/2HR7eq/ut1VS0i1lAHHmcAof+nK7/tD Y9yQ==
Date:	Tue, 30 Sep 2014 11:24:25 -0700
X-Spam-Level:
To:	bug-File-Path@rt.cpan.org
Content-Transfer-Encoding:	7bit
From:	Jim Avera <jim.avera@gmail.com>
X-RT-Original-Encoding:	iso-8859-1
X-RT-Interface:	Email
Content-Length:	967

File::Path::rmtree (v2.08) changes the working directory, which makes it incompatible with Perl threads (the cwd is process-global, shared by concurrent threads). PLEASE document this clearly in the pod to save people hours of unnecessary debugging. This was reported/requested last March in bug #94209. That bug received no comment from the maintainers. Presumably this means nobody has bandwidth to re-implement rmtree, which is perfectly understandable. But the request to at least document the limitation remains valid. I spent many long days debugging a multi-threaded app and would very much like to have been spared that useless exercise. Documenting the limitation would save many people lots of work. Thank you. P.S. bug#94209 has a comment "already been reported into perl RT without any reply" and referencing URL https://rt.perl.org/Public/Bug/Display.html?id=112008 but clicking that link gives error "Could not load ticket 112008".

Tue Sep 30 17:59:26 2014 ether [...] cpan.org - Correspondence added

MIME-Version:	1.0
In-Reply-To:	<542AF559.3010007@gmail.com>
X-Mailer:	MIME-tools 5.504 (Entity 5.504)
Content-Disposition:	inline
X-RT-Interface:	Web
References:	<542AF559.3010007@gmail.com>
Content-Type:	text/plain; charset="utf-8"
Message-ID:	<rt-4.0.18-15358-1412114366-1742.99230-0-0@rt.cpan.org>
Content-Transfer-Encoding:	binary
X-RT-Original-Encoding:	utf-8
X-RT-Encrypt:	0
X-RT-Sign:	0
Content-Length:	1211

On 2014-09-30 11:24:36, jim.avera@gmail.com wrote:

Show quoted text

> File::Path::rmtree (v2.08) changes the working directory, which makes it > incompatible with Perl threads (the cwd is process-global, shared by > concurrent threads). > > PLEASE document this clearly in the pod to save people hours of > unnecessary debugging.

I'd prefer we die in this situation. No one reads documentation! (But the doc note should be there too!)

Show quoted text

> This was reported/requested last March in bug #94209. That bug received > no comment from the maintainers. Presumably this means nobody has > bandwidth to re-implement rmtree, which is perfectly understandable. > But the request to at least document the limitation remains valid. I > spent many long days debugging a multi-threaded app and would very much > like to have been spared that useless exercise.

Fixing the implementation would be much preferable, of course, but in the meantime we can guard against accidents.

Show quoted text

> P.S. bug#94209 has a comment "already been reported into perl RT without > any reply" and referencing URL > https://rt.perl.org/Public/Bug/Display.html?id=112008 but clicking that > link gives error "Could not load ticket 112008".

Works for me.

Tue Sep 30 17:59:26 2014 The RT System itself - Status changed from 'new' to 'open'

Tue Sep 30 18:38:26 2014 jim.avera [...] gmail.com - Correspondence added

MIME-Version:	1.0
X-Spam-Status:	No, score=-5.199 tagged_above=-99.9 required=10 tests=[AWL=1.500, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001, FROM_OUR_RT=-4, RCVD_IN_DNSWL_LOW=-0.7] autolearn=ham
In-Reply-To:	<rt-4.0.18-15358-1412114366-726.99230-6-0@rt.cpan.org>
X-Spam-Flag:	NO
X-RT-Interface:	API
References:	<RT-Ticket-99230@rt.cpan.org> <542AF559.3010007@gmail.com> <rt-4.0.18-15358-1412114366-726.99230-6-0@rt.cpan.org>
X-Virus-Scanned:	Debian amavisd-new at bestpractical.com
X-Received:	by 10.67.30.200 with SMTP id kg8mr73021352pad.35.1412116698560; Tue, 30 Sep 2014 15:38:18 -0700 (PDT)
Message-ID:	<542B30D8.8020509@gmail.com>
Reply-To:	jim.avera@gmail.com
content-type:	text/plain; charset="utf-8"; format="flowed"
X-RT-Original-Encoding:	utf-8
X-Spam-Score:	-5.199
Authentication-Results:	hipster.bestpractical.com (amavisd-new); dkim=pass header.i=@gmail.com
Received:	from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id AE8D124041E for <cpan-bug+File-Path@hipster.bestpractical.com>; Tue, 30 Sep 2014 18:38:25 -0400 (EDT)
Received:	from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1-6vYIUqMGkP for <cpan-bug+File-Path@hipster.bestpractical.com>; Tue, 30 Sep 2014 18:38:25 -0400 (EDT)
Received:	from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id B9A1F240355 for <bug-File-Path@rt.cpan.org>; Tue, 30 Sep 2014 18:38:24 -0400 (EDT)
Received:	(qmail 18247 invoked by alias); 30 Sep 2014 22:38:24 -0000
Received:	from mail-pa0-f45.google.com (HELO mail-pa0-f45.google.com) (209.85.220.45) by la.mx.develooper.com (qpsmtpd/0.28) with ESMTP; Tue, 30 Sep 2014 15:38:22 -0700
Received:	by mail-pa0-f45.google.com with SMTP id rd3so10865783pab.4 for <bug-File-Path@rt.cpan.org>; Tue, 30 Sep 2014 15:38:18 -0700 (PDT)
Received:	from ?IPv6:2602:30a:2ccd:6950:a582:383d:990d:db72? ([2602:30a:2ccd:6950:a582:383d:990d:db72]) by mx.google.com with ESMTPSA id ni5sm16037022pbc.83.2014.09.30.15.38.17 for <bug-File-Path@rt.cpan.org> (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 30 Sep 2014 15:38:17 -0700 (PDT)
Delivered-To:	cpan-bug+File-Path@hipster.bestpractical.com
User-Agent:	Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0
Subject:	Re: [rt.cpan.org #99230] Please document that File::Path::rmtree may not be used with threads
Return-Path:	<jim.avera@gmail.com>
Dkim-Signature:	v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=message-id:date:from:reply-to:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=Ejch9UsGlOTA0HlKMF8q7FugbpE92mxGUOCXCnsDa4k=; b=bH/RXu+qHquz7q+/FC/+cgy3LWwkqNySrvxhD1ZKS38LK9KMQ6ZJ3rcW6RJ7FWif5D u8TqJ9MhXweuswP3NdzIArrUS57/dxyBtCMyNbGdyPo+0EQ2DKx5iAYuZP41CEcVZqjB Uc0T9vGvcVSZt53QgD8gWJYj9CXmFj67EFS48Bd/jK2owaGkkksJpBcfP9rLnSMhSTI7 Del/8BJemXaExnxmSuIyA6bAxOWO7Awxz5UdAK4P5ekkOCw2UNSZX3ZMLhMtAZmzJS55 oUU+t9X2IePYgG4QWtDqtylob0HmkXHAciL1QPKmPovqaonnP0N7TShD101+EiPchdil 7s5w==
X-Spam-Check-BY:	la.mx.develooper.com
X-Original-To:	cpan-bug+File-Path@hipster.bestpractical.com
X-RT-Mail-Extension:	file-path
Date:	Tue, 30 Sep 2014 15:38:16 -0700
X-Spam-Level:
To:	bug-File-Path@rt.cpan.org
Content-Transfer-Encoding:	7bit
From:	Jim Avera <jim.avera@gmail.com>
RT-Message-ID:	<rt-4.0.18-1402-1412116706-77.99230-0-0@rt.cpan.org>
Content-Length:	643

On 09/30/2014 02:59 PM, Karen Etheridge via RT wrote:

Show quoted text

> I'd prefer we die in this situation. No one reads documentation! (But > the doc note should be there too!)

Agree, dying when rmtree() is called with other threads would save a lot of pain. OTOH, an app _could_ arrange to block all but one thread while calling rmtree (arguably not a reasonable strategy, but possible). So maybe just warn: my $thread_warning_issued; # global And in rmtree() if (exists $INC{"threads.pm"}) { warn "WARNING: File::Path::rmtree is not thread safe (changes cwd)" if threads->list() > 0 && !$thread_warning_issued++; }

Thu Oct 02 20:52:25 2014 LEONT [...] cpan.org - Correspondence added

MIME-Version:	1.0
In-Reply-To:	<rt-4.0.18-15358-1412114366-1742.99230-0-0@rt.cpan.org>
X-Mailer:	MIME-tools 5.504 (Entity 5.504)
Content-Disposition:	inline
X-RT-Interface:	Web
References:	<542AF559.3010007@gmail.com> <rt-4.0.18-15358-1412114366-1742.99230-0-0@rt.cpan.org>
Content-Type:	text/plain; charset="utf-8"
Message-ID:	<rt-4.0.18-4594-1412297545-757.99230-0-0@rt.cpan.org>
Content-Transfer-Encoding:	binary
X-RT-Original-Encoding:	utf-8
X-RT-Encrypt:	0
X-RT-Sign:	0
Content-Length:	566

On Tue Sep 30 17:59:26 2014, ETHER wrote:

Show quoted text

> I'd prefer we die in this situation. No one reads documentation! (But > the doc note should be there too!)

I don't think that is particularly useful behavior, specially because I don't see why this problem would be unfixable.

Show quoted text

> Fixing the implementation would be much preferable, of course, but in > the meantime we can guard against accidents.

Given the backwards compatibility effect, I wouldn't call this a solution. It may have a race condition now, but that would not be an issue for all programs using this. Leon

Sat May 02 22:41:29 2015 BULKDD [...] cpan.org - Correspondence added

MIME-Version:	1.0
In-Reply-To:	<542AF559.3010007@gmail.com>
X-Mailer:	MIME-tools 5.504 (Entity 5.504)
Content-Disposition:	inline
X-RT-Interface:	Web
References:	<542AF559.3010007@gmail.com>
Content-Type:	text/plain; charset="utf-8"
Message-ID:	<rt-4.0.18-14644-1430620889-395.99230-0-0@rt.cpan.org>
Content-Transfer-Encoding:	binary
X-RT-Original-Encoding:	utf-8
X-RT-Encrypt:	0
X-RT-Sign:	0
Content-Length:	156

File::Path uses "." and File::Spec::Unix::curdir (aka ".") in many places. Maybe it can be rewritten to use only abs paths and then it never needs to chdir.

Fri Jun 26 12:13:10 2015 RICHE [...] cpan.org - Taken

Sat Jul 11 09:37:25 2015 jkeenan [...] cpan.org - Cc jkeenan [...] cpan.org added

Sat Jul 11 09:39:00 2015 jkeenan [...] cpan.org - Reference to #94209: remove_tree() is not thread-safe due to chdir added

Fri Jul 17 10:58:25 2015 RICHE [...] cpan.org - Correspondence added

MIME-Version:	1.0
In-Reply-To:	<rt-4.0.18-14644-1430620889-395.99230-0-0@rt.cpan.org>
X-Mailer:	MIME-tools 5.504 (Entity 5.504)
Content-Disposition:	inline
X-RT-Interface:	Web
References:	<542AF559.3010007@gmail.com> <rt-4.0.18-14644-1430620889-395.99230-0-0@rt.cpan.org>
Content-Type:	text/plain; charset="utf-8"
Message-ID:	<rt-4.0.18-16597-1437145105-1394.99230-0-0@rt.cpan.org>
Content-Transfer-Encoding:	binary
X-RT-Original-Encoding:	utf-8
X-RT-Encrypt:	0
X-RT-Sign:	0
Content-Length:	489

I agree with both ether and bulk88 on the direction to resolve this issue. At this time we won't change any code relating to this issue and will document it and target a near term fix (dying) and long term fix (removing chdir) for the next release. Please see the following doc patch to mitigate the issue for the 2.10 release. https://github.com/rpcme/File-Path/commit/60c14331cfdf30980adbcd61ec2ac1fb1a65c4e5 As a side note, this issue won't be closed until the bug is fixed in code.

Sat Jul 18 00:06:13 2015 RICHE [...] cpan.org - Correspondence added

MIME-Version:	1.0
In-Reply-To:	<rt-4.0.18-16597-1437145105-1394.99230-0-0@rt.cpan.org>
X-Mailer:	MIME-tools 5.504 (Entity 5.504)
Content-Disposition:	inline
X-RT-Interface:	Web
References:	<542AF559.3010007@gmail.com> <rt-4.0.18-14644-1430620889-395.99230-0-0@rt.cpan.org> <rt-4.0.18-16597-1437145105-1394.99230-0-0@rt.cpan.org>
Content-Type:	text/plain; charset="utf-8"
Message-ID:	<rt-4.0.18-9401-1437192373-1391.99230-0-0@rt.cpan.org>
Content-Transfer-Encoding:	binary
X-RT-Original-Encoding:	utf-8
X-RT-Encrypt:	0
X-RT-Sign:	0
Content-Length:	217

actually, we can resolve this as the 'documentation' problem as stated by OP, since 94209 is the ticket that really says it's not thread safe. This ticket will be referenced when we go to fix in code. Fixed in 2.11.

Sat Jul 18 00:06:15 2015 RICHE [...] cpan.org - Status changed from 'open' to 'resolved'