Skip Menu |
 

This queue is for tickets about the File-ChangeNotify CPAN distribution.

Report information
The Basics
Id: 52325
Status: open
Priority: 0/
Queue: File-ChangeNotify

People
Owner: Nobody in particular
Requestors: curtis.fletcher [...] n3k.co.uk
pjs [...] cpan.org
steve [...] yewtc.demon.co.uk
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Bug in File::ChangeNotify::Watcher::Inotify __build__mask
Date: Tue, 1 Dec 2009 16:42:01 -0000
To: <bug-File-ChangeNotify [...] rt.cpan.org>
From: "Curtis Fletcher" <curtis.fletcher [...] n3k.co.uk>
Download (untitled) / with headers
text/plain 1.8k
Package: File-ChangeNotify-0.09 Perl: 5.10.0 os: Linux ubuntu 2.6.27-7-generic In File::ChangeNotify::Watcher::Inotify sub __build__mask I believe that IN_MOVED_TO should be included in $mask. Otherwise renaming/moving files into a watched directory are not picked up. Curtis Fletcher Software Developer tuscany networks Limited Tuscany House, White Hart Lane, Basingstoke, RG21 4AF Tel: 01256 303 726 Fax: 01256 303 701 www.tuscanynetworks.com Important Notice The information contained in or attached to this email is intended only for the use of the correct addressee(s). If you are not the intended recipient, or a person responsible for delivering it to the intended recipient, you are not authorised to and must not disclose, copy, distribute, or retain this message or any part of it. It may contain information which is confidential and/or covered by legal professional or other privilege (or other rules or laws with similar effect in jurisdictions outside England and Wales). The views expressed in this email are those of the author and not necessarily the views of the company, its directors, officers or employees and we make no representation or accept any liability for its accuracy or completeness unless expressly stated to the contrary. The company will not be liable for direct, special, indirect or consequential damages arising from alteration of the contents of this message by a third party or as a result of any virus being passed on. The company has implemented virus detection systems to minimise the risk of passing on viruses. The company records all e-mail messages sent to and from this address and reserves the right to monitor and examine content for the purposes of training, quality control and for investigating or detecting any unauthorised use of its system and for ensuring its effective operation. tuscany networks is registered in England, company number 320 3561.
Subject: Re: [rt.cpan.org #52325] Bug in File::ChangeNotify::Watcher::Inotify __build__mask
Date: Tue, 1 Dec 2009 12:12:42 -0600 (CST)
To: Curtis Fletcher via RT <bug-File-ChangeNotify [...] rt.cpan.org>
From: Dave Rolsky <autarch [...] urth.org>
Download (untitled) / with headers
text/plain 561b
On Tue, 1 Dec 2009, Curtis Fletcher via RT wrote: Show quoted text
> In File::ChangeNotify::Watcher::Inotify sub __build__mask I believe that > IN_MOVED_TO should be included in $mask. Otherwise renaming/moving files > into a watched directory are not picked up.
Can you come up with a test case? That'd be really helpful. -dave /*============================================================ http://VegGuide.org http://blog.urth.org Your guide to all that's veg House Absolute(ly Pointless) ============================================================*/
On Tue Dec 01 11:42:39 2009, curtis.fletcher@n3k.co.uk wrote: Show quoted text
> Package: File-ChangeNotify-0.09 > Perl: 5.10.0 > os: Linux ubuntu 2.6.27-7-generic > > In File::ChangeNotify::Watcher::Inotify sub __build__mask I believe
that Show quoted text
> IN_MOVED_TO should be included in $mask. Otherwise renaming/moving
files Show quoted text
> into a watched directory are not picked up. >
I'm seeing something very similar to this when trying to watch a directory of files which are maintained by an rsync process. The rsync process makes a temporary file elsewhere in the filesystem and then renames the file after the download has been completed. File::ChangeNotify does not spot any changes to the file contents. When I add IN_MOVED_TO into the mask for the Inotify watcher it spots the changed files. The event type was reported as "unknown" so I also altered the Event class. Attached is a patch which works for me. I started trying to add some tests but I wasn't clear on the best way to write them so that they worked with different event watchers. Regards, Stephen Quinney
Subject: File-ChangeNotify-movedto.diff
diff -ruNp File-ChangeNotify-0.16/lib/File/ChangeNotify/Event.pm File-ChangeNotify-0.16.new/lib/File/ChangeNotify/Event.pm --- File-ChangeNotify-0.16/lib/File/ChangeNotify/Event.pm 2010-07-12 17:07:03.000000000 +0100 +++ File-ChangeNotify-0.16.new/lib/File/ChangeNotify/Event.pm 2012-01-23 08:37:32.000000000 +0000 @@ -18,7 +18,7 @@ has path => ( has type => ( is => 'ro', - isa => enum( [qw( create modify delete unknown )] ), + isa => enum( [qw( create modify delete rename unknown )] ), required => 1, ); @@ -66,8 +66,8 @@ The full path to the file or directory t =item * type => $type -The type of event. This must be one of "create", "modify", "delete", or -"unknown". +The type of event. This must be one of "create", "modify", "delete", +"rename" or "unknown". =back diff -ruNp File-ChangeNotify-0.16/lib/File/ChangeNotify/Watcher/Inotify.pm File-ChangeNotify-0.16.new/lib/File/ChangeNotify/Watcher/Inotify.pm --- File-ChangeNotify-0.16/lib/File/ChangeNotify/Watcher/Inotify.pm 2010-07-12 17:07:03.000000000 +0100 +++ File-ChangeNotify-0.16.new/lib/File/ChangeNotify/Watcher/Inotify.pm 2012-01-23 08:30:40.000000000 +0000 @@ -102,7 +102,7 @@ sub _build__mask { my $self = shift; my $mask - = IN_MODIFY | IN_CREATE | IN_DELETE | IN_DELETE_SELF | IN_MOVE_SELF; + = IN_MODIFY | IN_CREATE | IN_DELETE | IN_DELETE_SELF | IN_MOVE_SELF | IN_MOVED_TO; $mask |= IN_DONT_FOLLOW unless $self->follow_symlinks(); return $mask; @@ -190,6 +190,7 @@ sub _convert_event { $event->IN_CREATE() ? 'create' : $event->IN_MODIFY() ? 'modify' : $event->IN_DELETE() ? 'delete' + : $event->IN_MOVED_TO() ? 'rename' : 'unknown' ), );
Subject: Re: [rt.cpan.org #52325] Bug in File::ChangeNotify::Watcher::Inotify __build__mask
Date: Wed, 25 Jan 2012 09:13:00 -0600 (CST)
To: Stephen Quinney via RT <bug-File-ChangeNotify [...] rt.cpan.org>
From: Dave Rolsky <autarch [...] urth.org>
Download (untitled) / with headers
text/plain 688b
On Mon, 23 Jan 2012, Stephen Quinney via RT wrote: Show quoted text
> Attached is a patch which works for me. I started trying to add some > tests but I wasn't clear on the best way to write them so that they > worked with different event watchers.
I really want the behavior of every watcher to be the same, so I think you should just write tests to test the behavior. If some of the watchers fail, that's ok, now we at least have a failing test. -dave /*============================================================ http://VegGuide.org http://blog.urth.org Your guide to all that's veg House Absolute(ly Pointless) ============================================================*/
Subject: Directory renames not tracked
Download (untitled) / with headers
text/plain 1010b
As best I can tell, this is inherent to File::ChangeNotify and not Linux::Inotify2. If I rename a directory, subsequent operations on it still have the original name. Program: use Data::Dumper; use File::ChangeNotify; my $watcher = File::ChangeNotify->instantiate_watcher( directories => [ '/tmp' ]); while ( my @events = $watcher->wait_for_events ) { print "Got events:\n"; print Dumper $_ for @events; } Actions taken (wd: /tmp): mkdir foo mv foo bar touch bar/baz Output: Got events: $VAR1 = bless( { 'type' => 'create', 'path' => '/tmp/foo' }, 'File::ChangeNotify::Event' ); Got events: $VAR1 = bless( { 'type' => 'unknown', 'path' => '/tmp/foo' }, 'File::ChangeNotify::Event' ); Got events: $VAR1 = bless( { 'type' => 'create', 'path' => '/tmp/foo/baz' }, 'File::ChangeNotify::Event' ); Last event should of course have a path of /tmp/bar/baz.
Download (untitled) / with headers
text/plain 153b
Just realized that 52325 is addressing the same issue - the subject line threw me off. Close this one by all means; it may be useful for the test case.
Download (untitled) / with headers
text/plain 1.1k
I found that patch incomplete for tracking moves; it recognized a move but didn't track the new location. I had to apply the following patch on top of it. I will see if I can come up with a test. +++ /usr/local/lib/perl5/site_perl/5.14.2/File/ChangeNotify/Watcher/Inotify.pm~2012-04-09 20:15:48.000000000 -0700 @@ -85,15 +85,16 @@ # excluded or when the exclusion excludes a file, not a dir. next if $self->_path_is_excluded( $event->fullname() ); - if ( ($event->IN_CREATE() || $event->IN_MOVED_TO()) && $event->IN_ISDIR() ) { + if ( $event->IN_CREATE() && $event->IN_ISDIR() ) { $self->_watch_directory( $event->fullname() ); push @interesting, $event; push @interesting, $self->_fake_events_for_new_dir( $event->fullname() ); } - elsif ( $event->IN_DELETE_SELF() || $event->IN_MOVE_SELF() ) { + elsif ( $event->IN_DELETE_SELF() ) { $self->_remove_directory( $event->fullname() ); } + # We just want to check the _file_ name elsif ( $event->name() =~ /$filter/ ) { push @interesting, $event;
Download (untitled) / with headers
text/plain 105b
Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.
Download (untitled) / with headers
text/plain 105b
Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.
Download (untitled) / with headers
text/plain 658b
Okay, this will test for correct behavior as provided by the combined patches above. I do not have KQueue, I have no idea how to patch that. use Test::More tests => 2; use File::Temp qw(tempdir); use File::ChangeNotify; my $dir = tempdir( CLEANUP => 1 ); my $watcher = File::ChangeNotify->instantiate_watcher( directories => [ $dir ] ); $watcher->new_events; mkdir "$dir/foo"; # Event 1: create rename "$dir/foo", "$dir/bar"; # Event 2: rename symlink "..", "$dir/bar/baz"; # Event 3: create my @events = $watcher->new_events; ok @events == 3, 'Saw all events'; is $events[2]->{path}, "$dir/bar/baz", 'Saw correct destination path';
Oops, got the patch backwards, sorry...
Download (untitled) / with headers
text/plain 105b
Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.
Download (untitled) / with headers
text/plain 105b
Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.
Subject: File Closed
Date: Fri, 19 Apr 2013 17:25:52 +0100
To: bug-File-ChangeNotify [...] rt.cpan.org
From: Steve Rogerson <steve [...] yewtc.demon.co.uk>
Download (untitled) / with headers
text/plain 1.1k
hi, I've just read 52325 and this affects me also. I have a specific need to check directories that have files added, typically by an ftp process. When files have finished being downloaded that are then processed in some way. I thought File::ChangeNotify would do what I want. In fact I have a working process that uses Linux::Inotify2 but I like the fact the File::ChangeNotify is os neutral. I can't use it for two reasons. 1. Essentially #52325 - it doesn't detect files moved to the directory, as is done sometimes and 2. It doesn't detect that the file has been closed, in my context the file transfer has been completed. We get the creates and modifies, but that doesn't tell me when the file has been closed and is ready to process. It seems to me that my usage is not that strange, and that what's needed is to expose the mask to the user [ ok and process the extra events]. In my case I don't care about modified, I just want to know when it's finished. I do have a "test" of sorts - but it fails sort of by definition as I know that the events I want are not generated. Actually it sits and waits forever, which isn't too good for a test, but I'm working on it. Steve
Download (untitled) / with headers
text/plain 1.1k
On Fri Apr 19 12:26:11 2013, steve@yewtc.demon.co.uk wrote: Show quoted text
> I've just read 52325 and this affects me also. I have a specific need to > check directories that have files added, typically by an ftp process. When > files have finished being downloaded that are then processed in some way. I > thought File::ChangeNotify would do what I want. In fact I have a working > process that uses Linux::Inotify2 but I like the fact the File::ChangeNotify > is os neutral.
I'm all for making this more featureful. There are two things that need to happen: 1. There needs to be tests for new features. 2. There needs to be a way to make it work with all implementations. However, #2 can be a huge PITA since without something like inotify I have no idea how you can watch for close events or renames. So ... for people who need that, it might be better to just use inotify or kqueue directly rather than trying to use this module. Alternately, if someone came up with a Win32 implementation that used something Win32-specific to do this, I could consider getting rid of the pure Perl implementation, since that platform is the main reason it exists.
From: steve.bitcard [...] yewtc.demon.co.uk
Download (untitled) / with headers
text/plain 1.6k
On Sun Apr 21 12:02:15 2013, DROLSKY wrote: Show quoted text
> On Fri Apr 19 12:26:11 2013, steve@yewtc.demon.co.uk wrote: >
> > I've just read 52325 and this affects me also. I have a specific
> need to
> > check directories that have files added, typically by an ftp
> process. When
> > files have finished being downloaded that are then processed in
> some way. I
> > thought File::ChangeNotify would do what I want. In fact I have a
> working
> > process that uses Linux::Inotify2 but I like the fact the
> File::ChangeNotify
> > is os neutral.
> > I'm all for making this more featureful. There are two things that > need to happen: > > 1. There needs to be tests for new features. > 2. There needs to be a way to make it work with all implementations. > > However, #2 can be a huge PITA since without something like inotify I > have no idea how you can watch for close events or renames. > > So ... for people who need that, it might be better to just use > inotify or kqueue directly rather than trying to use this module. > > Alternately, if someone came up with a Win32 implementation that used > something Win32-specific to do this, I could consider getting rid > of the pure Perl implementation, since that platform is the main > reason it exists.
I agree that probably I'm better off continue using inotify (Linux::Inotify2) directly. It doesn't look as though there is a simple Win32 alternative. Having said that, if the mask was exposed, it would allow linux users to check for move/close, though I agree that sort of contradicts the notion of os independence. Though the other choice is "lowest common denominator".


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.