Skip Menu |
 

This queue is for tickets about the Text-CSV_XS CPAN distribution.

Report information
The Basics
Id: 119312
Status: open
Priority: 0/
Queue: Text-CSV_XS

People
Owner: Nobody in particular
Requestors: Chris.Denley [...] experian.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: threading problem
Date: Mon, 19 Dec 2016 22:40:14 +0000
To: "bug-Text-CSV_XS [...] rt.cpan.org" <bug-Text-CSV_XS [...] rt.cpan.org>
From: "Denley, Chris" <Chris.Denley [...] experian.com>
Download (untitled) / with headers
text/plain 2.3k
There seems to be an issue with threading. If I have an input thread and an output thread, both separately loading Text::CSV_XS, the output thread intermittently crashes after the input thread completes. If I load the module before starting the threads, it seems to resolve it. Is that a safe workaround. Should the module be thread-safe? #!/usr/bin/perl use threads; use Thread::Queue; #require Text::CSV_XS; my $input = 'input.csv'; my $output = 'output.csv'; my @cols = qw(a b c d); my $q; my $count = 0; while(1) { $q = Thread::Queue->new; my $inth = threads->create(\&inputthread); my $outth = threads->create(\&outputthread); $inth->join(); $q->enqueue(undef); $outth->join() or die("thread crashed?\n"); $count++; print "$count\n"; } sub inputthread { require Text::CSV_XS; $csv = Text::CSV_XS->new ({ binary => 1, eol => "\n", sep_char => ",", quote_char => '"', escape_char => '"', empty_is_undef => 0, decode_utf8 => 0 }); $csv->column_names(@cols); my $fh; open($fh,'<',$input); binmode($fh); my @rows = (); while(1) { my $data; if($csv) { $data = $csv->getline_hr($fh); unless($data) { last if($csv->eof); die("Failed to parse CSV line: ".$csv->error_diag."\n"); } } else { die("Data parsed another way\n"); } my @arr = values(%$data); push(@rows,\@arr); } close($fh); $q->enqueue(\@rows); } sub outputthread { my $fh; open($fh,'>',$output); binmode($fh); my $csv; require Text::CSV_XS; $csv = Text::CSV_XS->new ({ binary => 1, eol => "\n", sep_char => ',', quote_char => '"', escape_char => '"' }); while(1) { my $rows = $q->dequeue; last unless(defined $rows); foreach my $data(@$rows) { if($csv) { $csv->print($fh,$data); } else { die("Data output another way\n"); } } } close($fh); return 1; }
Subject: Re: [rt.cpan.org #119312] threading problem
Date: Wed, 28 Dec 2016 09:47:34 +0100
To: bug-Text-CSV_XS [...] rt.cpan.org
From: "H.Merijn Brand" <h.m.brand [...] xs4all.nl>
Download (untitled) / with headers
text/plain 1.2k
On Mon, 19 Dec 2016 17:40:29 -0500, "Denley, Chris via RT" <bug-Text-CSV_XS@rt.cpan.org> wrote: Show quoted text
> There seems to be an issue with threading. If I have an input thread > and an output thread, both separately loading Text::CSV_XS, the > output thread intermittently crashes after the input thread > completes. If I load the module before starting the threads, it seems > to resolve it. Is that a safe workaround. Should the module be > thread-safe?
It is a safe workaround. The (shared) static data is - with one small exception - all static and constant. It is static constant for reasons of performance. I've been playing a bit with the standard way to make that data thread-safe in XS, but that has quite an impact on performance, so I need to think that through before I try again (if I try again at all) The only exception to the constant data is a static that holds the last error, which cannot be kept in the object or object cache, as it also reflects the error if the object instantiation failed. -- H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/ using perl5.00307 .. 5.25 porting perl5 on HP-UX, AIX, and openSUSE http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/ http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/
Download (untitled)
application/pgp-signature 473b

Message body not shown because it is not plain text.

Download (untitled) / with headers
text/plain 338b
Show quoted text
> Should the module be thread-safe?
The XS part should be thread-safer in the new git branch thread-safe, where I followed the perlxs guidelines for static data in XS code. I measured no noticeable speed difference, so that could go in, but as it did not pass the suggested code, there is more to be changed somewhere. Feedback welcome
Download (untitled) / with headers
text/plain 168b
Still failed on my laptop with version 1.36 after over 400 iterations, both with and without my threadsafe changes (branch thread-safe) Still no idea on how to proceed
Download (untitled) / with headers
text/plain 1.8k
Linux 4.12.14-lp150.12.16-default x86_64 11876 Mb This is perl 5, version 28, subversion 0 (v5.28.0) built for x86_64-linux-thread-multi-ld config_args='-Duseshrplib -Duse64bitall -Dusethreads -Duseithreads -Duselongdouble -des' 1715 1716 1717 Thread 3436 terminated abnormally: Bareword "gensym" not allowed while "strict subs" in use at /pro/lib/perl5/5.28.0/x86_64-linux-thread-multi-ld/IO/Handle.pm line 325. BEGIN not safe after errors--compilation aborted at /pro/lib/perl5/5.28.0/x86_64-linux-thread-multi-ld/IO/Handle.pm line 629. Compilation failed in require at /pro/3gl/CPAN/Text-CSV_XS/blib/lib/Text/CSV_XS.pm line 26. BEGIN failed--compilation aborted at /pro/3gl/CPAN/Text-CSV_XS/blib/lib/Text/CSV_XS.pm line 26. Compilation failed in require at sandbox/rt119312.pl line 59. thread crashed? Linux 4.18.6-1-default x86_64 15960 Mb This is perl 5, version 28, subversion 0 (v5.28.0) built for x86_64-linux-thread-multi-ld config_args='-Dusethreads -Duseithreads -Duse64bitall -Duselongdouble -des' 1309 1310 1311 String found where operator expected at /pro/lib/perl5/5.28.0/SelectSaver.pm line 42, near "croak 'usage: SelectSaver->new( [FILEHANDLE] )'" (Do you need to predeclare croak?) Thread 2624 terminated abnormally: syntax error at /pro/lib/perl5/5.28.0/SelectSaver.pm line 42, near "croak 'usage: SelectSaver->new( [FILEHANDLE] )'" Compilation failed in require at /pro/lib/perl5/5.28.0/x86_64-linux-thread-multi-ld/IO/Handle.pm line 267. BEGIN failed--compilation aborted at /pro/lib/perl5/5.28.0/x86_64-linux-thread-multi-ld/IO/Handle.pm line 267. Compilation failed in require at /data/pro/3gl/CPAN/Text-CSV_XS/blib/lib/Text/CSV_XS.pm line 26. BEGIN failed--compilation aborted at /data/pro/3gl/CPAN/Text-CSV_XS/blib/lib/Text/CSV_XS.pm line 26. Compilation failed in require at sandbox/rt119312.pl line 59. thread crashed? Still no idea


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.