Skip Menu |
 

This queue is for tickets about the File-Path CPAN distribution.

Report information
The Basics
Id: 56509
Status: resolved
Priority: 0/
Queue: File-Path

People
Owner: jkeenan [...] cpan.org
Requestors: dnucera [...] ti.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 2.00_07
Fixed in: (no value)



Subject: Race condition
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 1758
Download (untitled) / with headers
text/plain 1.7k
Hi I'm currently using File::Path with perl5.8.3 on a 64 bits linux machine. I have some questions about mkpath function in the perl package File::path you support. I would like to know if you take into account race conditions for it. Let me give you more details. I have several scripts that are run in parallel and try to create a directory (kind of “mkdir –p” command). I have some issues when using unix command “mkdir –p” at the same time For example: %mkdir –p /a/b/c %mkdir –p /a/b/d If b doesn’t exist at run time, irst script will try to create it, while second script will try to write in it whereas it is not completely created. Indeed I found this article to explain the behavior: Second, is that `mkdir -p` does not work for parallel builds. This occurs when make tries to invoke, for example, `mkdir -p /a/b` and `mkdir -p /a/c` in parallel, with directory /a being non-existent. What `mkdir -p` does is that first it stats /a directory and if it does not exist it tries to create that. There is a race condition between stat'ing the directory and creating it, during which another parallel invocation of mkdir creates this same directory, which causes creating /a directory in the former mkdir to return EEXIST and fail. It must be mkdir bug (both gnu and Solaris), but I not sure if it can be fixed in mkdir, since if creating a directory returns EEXIST, that directory needs to be stat'ed again, to make sure it is a directory and not a file, leading to a race condition again. (no POSIX interface to create a directory and if that name already exists in the filesystem return its type). I wanted to know if mkpath function can solve this issue. Thanks for your answer. Didier
MIME-Version: 1.0
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-12186-1387277932-978.56509-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 2173
Download (untitled) / with headers
text/plain 2.1k
On 2010-04-12 03:44:01, dnucera wrote: Show quoted text
> Hi > I'm currently using File::Path with perl5.8.3 on a 64 bits linux machine. > I have some questions about mkpath function in the perl package > File::path you support. > > > > I would like to know if you take into account race conditions for it. > Let me give you more details. > > I have several scripts that are run in parallel and try to create a > directory (kind of “mkdir –p” command). > > > > I have some issues when using unix command “mkdir –p” at the same time > > For example: > > %mkdir –p /a/b/c > > %mkdir –p /a/b/d > > > > If b doesn’t exist at run time, irst script will try to create it, while > second script will try to write in it whereas it is not completely created. > > Indeed I found this article to explain the behavior: > > > > Second, is that `mkdir -p` does not work for parallel builds. This > occurs when make tries to invoke, for example, `mkdir -p /a/b` and > `mkdir -p /a/c` in parallel, with directory /a being non-existent. What > `mkdir -p` does is that first it stats /a directory and if it does not > exist it tries to create that. There is a race condition between > stat'ing the directory and creating it, during which another parallel > invocation of mkdir creates this same directory, which causes creating > /a directory in the former mkdir to return EEXIST and fail. It must be > mkdir bug (both gnu and Solaris), but I not sure if it can be fixed in > mkdir, since if creating a directory returns EEXIST, that directory > needs to be stat'ed again, to make sure it is a directory and not a > file, leading to a race condition again. (no POSIX interface to create a > directory and if that name already exists in the filesystem return its > type). > > > > > > I wanted to know if mkpath function can solve this issue. >
Looking at the source code of File::Path::_mkpath it seems that this case (multiple processes are doing mkpath at the same time) is handled. There's a comment explicitly saying "allow for another process to have created it meanwhile". I think this issue may be set to resolved. Regards, Slaven
MIME-Version: 1.0
In-Reply-To: <rt-4.0.18-12186-1387277932-978.56509-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <rt-4.0.18-12186-1387277932-978.56509-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-5699-1435363050-27.56509-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 2529
Download (untitled) / with headers
text/plain 2.4k
On Tue Dec 17 05:58:52 2013, SREZIC wrote: Show quoted text
> On 2010-04-12 03:44:01, dnucera wrote:
> > Hi > > I'm currently using File::Path with perl5.8.3 on a 64 bits linux > > machine. > > I have some questions about mkpath function in the perl package > > File::path you support. > > > > > > > > I would like to know if you take into account race conditions for it. > > Let me give you more details. > > > > I have several scripts that are run in parallel and try to create a > > directory (kind of “mkdir –p” command). > > > > > > > > I have some issues when using unix command “mkdir –p” at the same > > time > > > > For example: > > > > %mkdir –p /a/b/c > > > > %mkdir –p /a/b/d > > > > > > > > If b doesn’t exist at run time, irst script will try to create it, > > while > > second script will try to write in it whereas it is not completely > > created. > > > > Indeed I found this article to explain the behavior: > > > > > > > > Second, is that `mkdir -p` does not work for parallel builds. This > > occurs when make tries to invoke, for example, `mkdir -p /a/b` and > > `mkdir -p /a/c` in parallel, with directory /a being non-existent. > > What > > `mkdir -p` does is that first it stats /a directory and if it does > > not > > exist it tries to create that. There is a race condition between > > stat'ing the directory and creating it, during which another parallel > > invocation of mkdir creates this same directory, which causes > > creating > > /a directory in the former mkdir to return EEXIST and fail. It must > > be > > mkdir bug (both gnu and Solaris), but I not sure if it can be fixed > > in > > mkdir, since if creating a directory returns EEXIST, that directory > > needs to be stat'ed again, to make sure it is a directory and not a > > file, leading to a race condition again. (no POSIX interface to > > create a > > directory and if that name already exists in the filesystem return > > its > > type). > > > > > > > > > > > > I wanted to know if mkpath function can solve this issue. > >
> > Looking at the source code of File::Path::_mkpath it seems that this > case (multiple processes are doing mkpath at the same time) is > handled. There's a comment explicitly saying "allow for another > process to have created it meanwhile". > > I think this issue may be set to resolved. > > Regards, > Slaven
Since there has been no further communication from the original poster, and since I trust Slaven's judgement on this, I'm marking this ticket Resolved. Thank you very much. Jim Keenan


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.