Skip Menu |
 

This queue is for tickets about the Text-CSV_XS CPAN distribution.

Report information
The Basics
Id: 119312
Status: open
Priority: 0/
Queue: Text-CSV_XS

People
Owner: Nobody in particular
Requestors: Chris.Denley [...] experian.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: threading problem
Date: Mon, 19 Dec 2016 22:40:14 +0000
To: "bug-Text-CSV_XS [...] rt.cpan.org" <bug-Text-CSV_XS [...] rt.cpan.org>
From: "Denley, Chris" <Chris.Denley [...] experian.com>
Download (untitled) / with headers
text/plain 2.3k
There seems to be an issue with threading. If I have an input thread and an output thread, both separately loading Text::CSV_XS, the output thread intermittently crashes after the input thread completes. If I load the module before starting the threads, it seems to resolve it. Is that a safe workaround. Should the module be thread-safe? #!/usr/bin/perl use threads; use Thread::Queue; #require Text::CSV_XS; my $input = 'input.csv'; my $output = 'output.csv'; my @cols = qw(a b c d); my $q; my $count = 0; while(1) { $q = Thread::Queue->new; my $inth = threads->create(\&inputthread); my $outth = threads->create(\&outputthread); $inth->join(); $q->enqueue(undef); $outth->join() or die("thread crashed?\n"); $count++; print "$count\n"; } sub inputthread { require Text::CSV_XS; $csv = Text::CSV_XS->new ({ binary => 1, eol => "\n", sep_char => ",", quote_char => '"', escape_char => '"', empty_is_undef => 0, decode_utf8 => 0 }); $csv->column_names(@cols); my $fh; open($fh,'<',$input); binmode($fh); my @rows = (); while(1) { my $data; if($csv) { $data = $csv->getline_hr($fh); unless($data) { last if($csv->eof); die("Failed to parse CSV line: ".$csv->error_diag."\n"); } } else { die("Data parsed another way\n"); } my @arr = values(%$data); push(@rows,\@arr); } close($fh); $q->enqueue(\@rows); } sub outputthread { my $fh; open($fh,'>',$output); binmode($fh); my $csv; require Text::CSV_XS; $csv = Text::CSV_XS->new ({ binary => 1, eol => "\n", sep_char => ',', quote_char => '"', escape_char => '"' }); while(1) { my $rows = $q->dequeue; last unless(defined $rows); foreach my $data(@$rows) { if($csv) { $csv->print($fh,$data); } else { die("Data output another way\n"); } } } close($fh); return 1; }
Subject: Re: [rt.cpan.org #119312] threading problem
Date: Wed, 28 Dec 2016 09:47:34 +0100
To: bug-Text-CSV_XS [...] rt.cpan.org
From: "H.Merijn Brand" <h.m.brand [...] xs4all.nl>
Download (untitled) / with headers
text/plain 1.2k
On Mon, 19 Dec 2016 17:40:29 -0500, "Denley, Chris via RT" <bug-Text-CSV_XS@rt.cpan.org> wrote: Show quoted text
> There seems to be an issue with threading. If I have an input thread > and an output thread, both separately loading Text::CSV_XS, the > output thread intermittently crashes after the input thread > completes. If I load the module before starting the threads, it seems > to resolve it. Is that a safe workaround. Should the module be > thread-safe?
It is a safe workaround. The (shared) static data is - with one small exception - all static and constant. It is static constant for reasons of performance. I've been playing a bit with the standard way to make that data thread-safe in XS, but that has quite an impact on performance, so I need to think that through before I try again (if I try again at all) The only exception to the constant data is a static that holds the last error, which cannot be kept in the object or object cache, as it also reflects the error if the object instantiation failed. -- H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/ using perl5.00307 .. 5.25 porting perl5 on HP-UX, AIX, and openSUSE http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/ http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/
Download (untitled)
application/pgp-signature 473b

Message body not shown because it is not plain text.

Download (untitled) / with headers
text/plain 338b
Show quoted text
> Should the module be thread-safe?
The XS part should be thread-safer in the new git branch thread-safe, where I followed the perlxs guidelines for static data in XS code. I measured no noticeable speed difference, so that could go in, but as it did not pass the suggested code, there is more to be changed somewhere. Feedback welcome


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.