Skip Menu |
 

This queue is for tickets about the DBD-SQLite CPAN distribution.

Report information
The Basics
Id: 20286
Status: resolved
Priority: 0/
Queue: DBD-SQLite

People
Owner: Nobody in particular
Requestors: MUIR [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in:
  • 1.11
  • 1.12
  • 1.13
Fixed in: (no value)



Download (untitled) / with headers
text/plain 829b
DBD::SQLite fails the regression tests for my OOPS module. Normally when this happens, I go digging for the problem in my code. This time I'm not because DBD::SQLite2 passes my regression tests. How to repeat: Download OOPS 0.1004 from CPAN. You'll also need: a recent Data::Compare, Time::HiRes, Test::MultiFork, Clone::PP. The your OOPSTEST_DSN environment variable to: DBI:SQLite2:dbname=/tmp/sqlite2.$$.db" Run the tests. See that they work fine. Set your OOPSTEST_DSN environment variable to: DBI:SQLite:dbname=/tmp/sqlite.$$.db" Run the tests again. My guess is that they'll fail. The tests that will fail: t/blobsize.t around test #9499 t/slowtest.t around test #9792 t/db-slowtest.t around test #9832 t/nulls.t around test #11711 The failure message is "unable to open database file(1) at dbdimp.c"
Download (untitled) / with headers
text/plain 187b
I'm sorry, I can't fix a bug based on "My app breaks when I use this". Please feel free to file a bug report with a reasonable (small) example attached. Meanwhile I'm rejecting this bug.
Subject: Re: [rt.cpan.org #20286] DBD::SQLite can't do what DBD::SQLite2 can
Date: Thu, 7 Sep 2006 15:47:30 -0700
To: bug-DBD-SQLite [...] rt.cpan.org
From: "David Muir Sharnoff" <muir [...] idiom.com>
Download (untitled) / with headers
text/plain 657b
I do not have a small example. If I did, I would give it to you. What I do have is a repeatable test case that you can run yourself. I believe I provided instructions on how to reproduce it. If there is some sort of logging or debugging that I could turn on that would allow you to see what the problem is, I would be happy to do so. -Dave On 9/7/06, via RT <bug-DBD-SQLite@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=20286 > > > I'm sorry, I can't fix a bug based on "My app breaks when I use this". Please feel free to file a > bug report with a reasonable (small) example attached. Meanwhile I'm rejecting this bug. > >
Download (untitled) / with headers
text/plain 358b
You don't have a test case nor any idea of what is broken, just a potentially very large application that I have no idea what it's supposed to do, that may or may not have its own bugs. I'm sorry, but I'm just not going to dive into this. Closing as rejected. Please do not reply to this bug unless you have a test case and a statement of what is broken.
Subject: Re: [rt.cpan.org #20286] DBD::SQLite can't do what DBD::SQLite2 can
Date: Thu, 7 Sep 2006 16:37:58 -0700
To: bug-DBD-SQLite [...] rt.cpan.org
From: "David Muir Sharnoff" <muir [...] idiom.com>
Download (untitled) / with headers
text/plain 794b
I must sincerely disagree. Each of us needs to debug our own application. You are asking that I debug yours. I do have a test case. Your module always fails when I run the same set of queries against it. I have no idea if the problem is with your module or with SQLite. If DBI had a way to record all calls to DBI and write them out in a linear file, I am quite sure that the linear file of calls to DBD::SQLite would fail and that the same calls to DBD::SQLite2 would succeed. The DBI tracing capability isn't good enough for what we need. I can well imagine writing a debug module that would instrument another module to do exactly what I'm suggesting. I don't have the time to write such a debugging module. I apologize that you'll need to go change it to "rejected" again. -Dave
From: ask [...] develooper.com
Download (untitled) / with headers
text/plain 450b
On Thu Sep 07 19:38:12 2006, muir@idiom.com wrote: Show quoted text
> The DBI tracing capability isn't good enough for what we need. I can > well > imagine writing a debug module that would instrument another module to > do exactly what I'm suggesting. I don't have the time to write such a > debugging module.
Huh? How can you not make your test suite tell you where it fails and thus reproduce it? (or at least get some hints as to where to look). - ask
Subject: Re: [rt.cpan.org #20286] DBD::SQLite can't do what DBD::SQLite2 can
Date: Fri, 8 Sep 2006 11:49:30 -0700
To: bug-DBD-SQLite [...] rt.cpan.org
From: "David Muir Sharnoff" <muir [...] idiom.com>
Download (untitled) / with headers
text/plain 11.3k
On 9/7/06, ask@develooper.com via RT <bug-DBD-SQLite@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=20286 > > > On Thu Sep 07 19:38:12 2006, muir@idiom.com wrote: > >
> > The DBI tracing capability isn't good enough for what we need. I can > > well > > imagine writing a debug module that would instrument another module to > > do exactly what I'm suggesting. I don't have the time to write such a > > debugging module.
> > Huh? How can you not make your test suite tell you where it fails and thus reproduce it? > > (or at least get some hints as to where to look). > > > - ask > >
My test suite is testing a cross-product of operations. I took one of my tests that fails with DBI::SQLite and tried to shorten it. After removing a bunch of tests, the remaining cross-product cannot be shortened any more and still provode the DBI::SQLite failure. On my system, today, my revised suite always fails after test # 6377 and before # 6378. Interestingly, the values that I'm testing with don't seem to matter. What does seem to matter is the number of iterations through my test. My suite can skip any number of initial tests to start in the middle. If I do that, the failure point also moves. Each cycle through my test suite I'm probably making 150 calls to the DBI layer. I do know something about where it fails. When I try to do: UPDATE charm1attribute SET pval = ?, ptype = ? WHERE id = ? AND pkey = ? With values: 105, R, 104, nopkey It fails with: unable to open database file(1) at dbdimp.c line 398 To make debugging easier, I've added code to my test suite so that DBI tracing can be turned on not long before the failure rather than at the beginning. The revised suite is below. -Dave Show quoted text
------------------------ cut here --------------------- #!/usr/bin/perl -I../lib -I./.. my $dbi_trace_turnon = 6377; my $dbi_trace_level = 4; # # Revised version of nulls.t from OOPS 1.005 cut down to # show the DBI::SQLite problem. # # DBI::SQLite fails this after test # 6377 # BEGIN { if ($ENV{HARNESS_ACTIVE} && ! $ENV{OOPSTEST_SLOW}) { print "1..0 # Skipped: run this by hand or set \$ENV{OOPSTEST_SLOW}\n"; exit; } $OOPS::SelfFilter::defeat = 1 unless defined $OOPS::SelfFilter::defeat; for my $m (qw(Data::Dumper Clone::PP)) { unless ( eval " require $m " ) { print "1..0 # Skipped: this test requires the $m module\n"; exit; } $m->import(); } } use OOPS; require Carp::Heavy; use Carp qw(confess); use Scalar::Util qw(reftype); use strict; use warnings; use OOPS::TestCommon; BEGIN { unless ($dbms eq 'sqlite') { print "1..0 # Skipped: this test is for DBD::SQLite only\n"; exit; } } modern_data_compare(); import Clone::PP qw(clone); my $skipto = 0; # go directly to test number... print "1..7356\n"; my $debug2 = 0; my $debug3 = 0; resetall; # -------------------------------------------------- { my $realdebug = $debug; my $failures = <<'END'; END # # flags: # # V - try with virtual object and regular object # h - set $key to various keys a hash might use # a - set $key to various index an array might use # v - replace $pval with potential values # my $tests = <<'END'; Vhv- $root->{$key} = $pval END my %failures; for my $failure (split(/\n/, $failures)) { $failure =~ s/^\s+//; $failure =~ s/\s+$//; $failures{$failure} = 1; print "# adding '$failure'\n" if $debug2; } for my $test (split(/^\s*$/m, $tests)) { $test =~ s/\s*(\S*)-\s//s; my $flag = $1; my (@virt) = $flag =~ /V/ ? (qw( virtual )) : (0); my (@key) = (0); if ($flag =~ /h/) { @key = (qw( 'x' 'x' 'x' 'x' 'x' 'x' 'x' )); } elsif ($flag =~ /a/) { @key = (0..7); } no warnings qw(syntax); my (@val) = $flag =~ /v/ ? (qw( 'y' 'y' 'y' 'y' 'y' 'y' 'y' )) : ( '1' ); use warnings; my @skips = (qw(10 00 01 11)); my @groups = ('onegroup', '', 'manygroups'); my $nodata_per_loop = count(grep(! /root/, @val)) / scalar(@val); my $skippre_per_loop = count(grep(substr($_, 0, 1), @skips)) / scalar(@skips); my $skippost_per_loop = 2 * count(grep(substr($_, 1, 1), @skips)) / scalar(@skips); my $loops = scalar(@val) * scalar(@key) * scalar(@groups) * scalar(@virt) * scalar(@skips); my $base_per_loop = 3 # resetall + 2 * 2 # rcon + (($flag =~ /V/) ? 1 : 0) # test vobj + 1 # test unconditional + 1 # notied ; my $per_loop = $base_per_loop + $skippre_per_loop + $skippost_per_loop; # printf "# per_loop = %d ( 9 + flag:%s + pre:%s + post:%s + nodata:%s )\n", # $per_loop, # (($flag =~ /V/) ? 1 : 0), # $skippre_per_loop, # $skippost_per_loop, # $nodata_per_loop; my $expected = $okay + $loops * ($per_loop + $nodata_per_loop); if ($expected < $skipto) { $okay = $expected; next; } for my $val (@val) { $nodata_per_loop = ($val =~ /root/) ? 0 : 1; my $loops = scalar(@key) * scalar(@groups) * scalar(@virt) * scalar(@skips); my $expected2 = $okay + $loops * ($per_loop + $nodata_per_loop); if ($expected2 < $skipto) { $okay = $expected2; next; } for my $key (@key) { my $loops = scalar(@groups) * scalar(@virt) * scalar(@skips); my $expected3 = $okay + $loops * ($per_loop + $nodata_per_loop); if ($expected3 < $skipto) { $okay = $expected3; next; } my $sub; my $e = <<END; \$sub = sub { my \$z = 'ov09'x($ocut/4+1); my \$root = shift; my \$pval = $val; my \$key = $key; no warnings; $test } END eval $e; die "on $test/$val/$key ... <<$e>> ... $@" if $@; for my $skips (@skips) { my $skippre = substr($skips, 0, 1); my $skippost = substr($skips, 1, 1); my $loops = scalar(@groups) * scalar(@virt); my $per_loop2 = $base_per_loop + $nodata_per_loop + ($skippre ? 0 : 1) + ($skippost ? 0 : 2); my $expected4 = $okay + $loops * $per_loop2; if ($expected4 < $skipto) { $okay = $expected4; next; } for my $groupmangle (@groups) { my $loops = scalar(@virt); my $expected5 = $okay + $loops * $per_loop2; if ($expected5 < $skipto) { $okay = $expected5; next; } for my $vobj (@virt) { my $expected6 = $okay + $per_loop2; if ($expected6 < $skipto) { $okay = $expected6; next; } if ($expected6 >= $dbi_trace_turnon) { $OOPS::debug_dbi = $dbi_trace_level; } my $preok = $okay; resetall; die if $debug && $okay != $preok + 3; my $desc = "$flag- $test: key=$key val=$val V$vobj.S$skippre$skippost.G$groupmangle"; $desc =~ s/\A\s*(.*?)\s*\Z/$1/s; $desc =~ s/\n\s*/\\n /g; $debug = $failures{$desc} ? 0 : $realdebug; print "# desc='$desc' debug=$debug\n"; print "# $desc\n" if $debug; my $rv = "x\000x"; my $x = chr(0)x($ocut+1); my $mroot = { # the length of this array should match the flag =~ /a/ array size of @key (above). '' => { skey2 => 'sval2' }, undef => "0 but true", "0" => [ qw( '' undef "0\000" "0" chr(0) "With\000inside" "with\\back" ), '"0 but true"' ], "0 but true" => \$rv, "0\000" => \[ undef, "0", chr(0) ], chr(0) => \{ chr(0) => undef}, # "With\000inside" => \ (scalar(chr(0)x($ocut+1))), #XXX "with\\back" => \$x, #XXX "witH\\back" => \$x, }; $r1->{named_objects}{root} = clone($mroot); $r1->virtual_object($r1->{named_objects}{root}, $vobj) if $vobj; $r1->commit; nocon; if ($groupmangle) { groupmangle($groupmangle); } rcon; die if $debug && $okay != $preok + 3 + 2; print "#PROGRESS: BEFORE $desc\n" if $debug2; my $proot = $r1->{named_objects}{root}; test(docompare($mroot, $proot), $desc) unless $skippre; die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1); print "mroot before: ".Dumper($mroot)."\n" if $debug3; &$sub($mroot); print "mroot after: ".Dumper($mroot)."\n" if $debug3; print "#PROGRESS: PRE CHANGES: $desc\n" if $debug2; print "proot before: ".Dumper($proot)."\n" if $debug3; print "# EXECUTING: $desc\n" if $debug; &$sub($proot); print "#PROGRESS: POST CHANGES: $desc\n" if $debug2; print "proot after: ".Dumper($proot)."\n" if $debug3; print "#PROGRESS: PRE COMPARE: $desc\n" if $debug2; test(docompare($mroot, $proot), $desc) unless $skippost; die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 1); print "#PROGRESS: POST COMPARE, PRE COMMIT: $desc\n" if $debug2; $r1->commit; print "#PROGRESS: POST COMMIT, PRE COMPARE#2: $desc\n" if $debug2; test(docompare($mroot, $proot), $desc) unless $skippost; die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 2); print "#PROGRESS: POST COMPARE#2, PRE RECONNECT: $desc\n" if $debug2; undef $proot; rcon; # our $xy = 1; die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 2) + 2; my $qroot = $r1->{named_objects}{root}; print "#PROGRESS: POST RECONNECT, PRE COMPARE #3: $desc\n" if $debug2; test(docompare($mroot, $qroot), $desc); die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 2) + 2 +1; print "#PROGRESS: POST COMPARE #3, PRE DELETES: $desc\n" if $debug2; test(!$vobj == !$r1->virtual_object($qroot), $desc) if $flag =~ /V/; die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 2) + 2 + 1 + (($flag =~ /V/) ? 1 : 0); nukevar($qroot, $mroot); delete $r1->{named_objects}{root}; print "#PROGRESS: POST DELETES, PRE COMMIT: $desc\n" if $debug2; $r1->commit; print "#PROGRESS: FINAL COMMIT DONE: $desc\n" if $debug2; undef $qroot; nocon; nodata unless $val =~ /root/; die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 2) + 2 + 1 + ($flag =~ /V/ ? 1 : 0) + (($val =~ /root/) ? 0 : 1); notied($desc); die if $debug && $okay != $preok + 3 + 2 + ($skippre ? 0 : 1) + ($skippost ? 0 : 2) + 2 + 1 + ($flag =~ /V/ ? 1 : 0) + (($val =~ /root/) ? 0 : 1) + 1; print "#PROGRESS: DONE WITH TEST: $desc\n" if $debug2; print "# okay: $okay expected6: $expected6\n"; die "bad prediction" if $debug && $okay != $expected6; } print "# okay: $okay expected5: $expected5\n"; die "bad prediction" if $debug && $okay != $expected5; } print "# okay: $okay expected4: $expected4\n"; die "bad prediction" if $debug && $okay != $expected4; } print "# okay: $okay expected3: $expected3\n"; die "bad prediction" if $debug && $okay != $expected3; } print "# okay: $okay expected2: $expected2\n"; die "bad prediction" if $debug && $okay != $expected2; } print "# okay: $okay expected: $expected\n"; die "bad prediction" if $debug && $okay != $expected; } $debug = $realdebug; } resetall; # -------------------------------------------------- print "# ---------------------------- done ---------------------------\n" if $debug; $okay--; print "# tests: $okay\n" if $debug; exit 0; # ---------------------------------------------------- sub count { return scalar(@_); } 1;
From: MSERGEANT [...] cpan.org
Download (untitled) / with headers
text/plain 673b
Perhaps it would help if I explained my dilemma in terms of how I read your bug report: "My application works fine if I use DBIx::Class for persistent storage, but fails if I use Class::DBI - download my application [here] and find the bug for me" It's HIGHLY likely given your report that this is a bug in how you're using DBI/DBD::SQLite rather than a bug in the library itself. But how would I know? I'd have to delve right into the guts of OOPS to figure it out - and that's not a fair use of my time. I do this for free you know. Figure out a small test case using just the DBI (no OOPS), and it'll get fixed. Meanwhile I'll once again mark this as rejected :-)
Subject: Re: [rt.cpan.org #20286] DBD::SQLite can't do what DBD::SQLite2 can
Date: Fri, 8 Sep 2006 13:43:50 -0700
To: bug-DBD-SQLite [...] rt.cpan.org
From: "David Muir Sharnoff" <muir [...] idiom.com>
Download (untitled) / with headers
text/plain 1.6k
No, it's not a bug in how I'm using DBI. OOPS works with DBI::SQLite2, DBD::Pg, and DBD::mysql. There is only a tiny bit of OOPS that is different between the databases. The test that is failing is essentialy: while(1..10000) { do static linear set of DBI calls } With DBI::SQLIte only, it fails after 6377 iterations. OOPS doesn't do anything fancy with DBI or SQL. OOPS is pure perl. OOPS stress the perl interpreter in various ways, so I have to admit there is a possibility of the problem being a bug in perl rather than DBD::SQLite. OOPS has a very extensive test suite and in the process of developing OOPS, I've found a bunch of bugs in perl, memory leaks in DBD::SQLite and DBD::SQLite2, and a deadlock detection bug in mysql. The particular problem that prompted this is a bug in DBD::SQLite or the underling sqlite library. -Dave On 9/8/06, via RT <bug-DBD-SQLite@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=20286 > > > Perhaps it would help if I explained my dilemma in terms of how I read your bug report: > > "My application works fine if I use DBIx::Class for persistent storage, but fails if I use Class::DBI > - download my application [here] and find the bug for me" > > It's HIGHLY likely given your report that this is a bug in how you're using DBI/DBD::SQLite rather > than a bug in the library itself. But how would I know? I'd have to delve right into the guts of > OOPS to figure it out - and that's not a fair use of my time. I do this for free you know. > > Figure out a small test case using just the DBI (no OOPS), and it'll get fixed. Meanwhile I'll once > again mark this as rejected :-) > >
Download (untitled) / with headers
text/plain 935b
On Fri Sep 08 16:44:02 2006, muir@idiom.com wrote: Show quoted text
> No, it's not a bug in how I'm using DBI. OOPS works with > DBI::SQLite2, DBD::Pg, and DBD::mysql. There is only a tiny bit of > OOPS that is different between the databases. > > The test that is failing is essentialy: > > while(1..10000) { > do static linear set of DBI calls > } >
If you reduced the test case to a small loop that did a few DBI calls Matt would probably be a lot more interested in looking at it. It looks like sqlite3pager_opentemp is leaking file handles (well maybe not leaking,my c is kinda weak but I asume if they were closed properly they wouldn't cause the system per process limit to be hit). If you increase your open files per process limit the tests will start passing. There are several hundred closing dbh with active statement handles at OOPS/TestCommon.pm line 600. warnings when I run t/scalarhash.t , maybe they are related.
Subject: Re: [rt.cpan.org #20286] DBD::SQLite can't do what DBD::SQLite2 can
Date: Fri, 8 Sep 2006 14:43:55 -0700
To: bug-DBD-SQLite [...] rt.cpan.org
From: "David Muir Sharnoff" <muir [...] idiom.com>
Download (untitled) / with headers
text/plain 1.3k
That's a great thought. I think you've found the bug. I lowered my descriptor limit from 1024 to 300 and the test failed much sooner. Thank you! Meanwhile, I'm part-way through writing an API tracing package so that it's easy to turn any program that uses DBI into a list of calls to DBI functions. -Dave On 9/8/06, Peter Sinnott via RT <bug-DBD-SQLite@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=20286 > > > On Fri Sep 08 16:44:02 2006, muir@idiom.com wrote:
> > No, it's not a bug in how I'm using DBI. OOPS works with > > DBI::SQLite2, DBD::Pg, and DBD::mysql. There is only a tiny bit of > > OOPS that is different between the databases. > > > > The test that is failing is essentialy: > > > > while(1..10000) { > > do static linear set of DBI calls > > } > >
> > If you reduced the test case to a small loop that did a few DBI calls > Matt would probably be a lot more interested in looking at it. > > > It looks like sqlite3pager_opentemp is leaking file handles (well maybe > not leaking,my c is kinda weak but I asume if they were closed properly > they wouldn't cause the system per process limit to be hit). If you > increase your open files per process limit the tests will start passing. > There are several hundred closing dbh with active statement handles at > OOPS/TestCommon.pm line 600. warnings when I run t/scalarhash.t , maybe > they are related. > >
Download (untitled) / with headers
text/plain 1.4k
If you add to OOPS::TestCommon inside check_refcount $actual = undef; $recorded = undef; just before you disconnect then the tests pass. It looks like calling finish does not correctly finish the statement. I presume if the statements went out of scope before the dbh disconnected things would get cleaned up properly. Looks like a new problem with 1.13 as my test program (below) works with 1.12. Show quoted text
sqlite> .schema
CREATE TABLE whatever ( id int, something int ); use strict; use warnings; use DBI; my %args = ( "dbi_dsn" => "DBI:SQLite:dbname=stuff.db", "user" => "fake", "password" => "fake", ); my $database = $args{"dbi_dsn"}; my $user = $args{"user"}; my $password = $args{"password"}; foreach ( 1 .. 1000 ) { foo(); } sub foo { my $dbh = DBI->connect( $database , $user , $password , { Taint => 0, PrintError => 0, RaiseError => 0, AutoCommit => 0} ); unless($dbh) { die $DBI::errstr; } my $sql = "select id,count(1) from whatever where something = 1"; my $actual = $dbh->prepare($sql) || die $dbh->errstr; $actual->execute() || die $actual->errstr; my (%actual, %recorded); my ($id, $count); while (($id, $count) = $actual->fetchrow_array()) { $id = "1" unless defined $id; $actual{$id} = $count; } $actual->finish; # $actual = undef; $dbh->disconnect; }
Subject: Re: [rt.cpan.org #20286] DBD::SQLite can't do what DBD::SQLite2 can
Date: Tue, 12 Sep 2006 11:10:00 -0700
To: bug-DBD-SQLite [...] rt.cpan.org
From: "David Muir Sharnoff" <muir [...] idiom.com>
I did as you suggested and undef'ed $actual and $recorded. I don't see any difference. I've set my file descriptor limit to just 100 and my tests fail in eactly the same place with the undef and without it. The problem is definately a file descriptor leak. DBD::SQLite2 doesn't leak file descriptors but DBD::SQLite does. -Dave On 9/11/06, Peter Sinnott via RT <bug-DBD-SQLite@rt.cpan.org> wrote: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=20286 > > > If you add to OOPS::TestCommon inside check_refcount > > $actual = undef; > $recorded = undef; > > just before you disconnect then the tests pass. It looks like > calling finish does not correctly finish the statement. I presume > if the statements went out of scope before the dbh disconnected things > would get cleaned up properly. > > Looks like a new problem with 1.13 as my test program (below) works with > 1.12. > >
> sqlite> .schema
> CREATE TABLE whatever ( id int, something int ); > > > use strict; > use warnings; > > use DBI; > > my %args = ( "dbi_dsn" => "DBI:SQLite:dbname=stuff.db", > "user" => "fake", > "password" => "fake", > ); > > my $database = $args{"dbi_dsn"}; > my $user = $args{"user"}; > my $password = $args{"password"}; > > foreach ( 1 .. 1000 ) > { > foo(); > } > > sub foo > { > my $dbh = DBI->connect( $database , $user , $password , { Taint > => 0, PrintError => 0, RaiseError => 0, AutoCommit => 0} ); > unless($dbh) > { > die $DBI::errstr; > } > > my $sql = "select id,count(1) from whatever where something = 1"; > my $actual = $dbh->prepare($sql) || die $dbh->errstr; > $actual->execute() || die $actual->errstr; > my (%actual, %recorded); > my ($id, $count); > > while (($id, $count) = $actual->fetchrow_array()) { > $id = "1" unless defined $id; > $actual{$id} = $count; > } > > $actual->finish; > > # $actual = undef; > $dbh->disconnect; > } > >
From: jmitchell [...] frb.gov
Download (untitled) / with headers
text/plain 558b
Here is some more debug info (using lsof) regarding this problem: 1) Even if you $sth->finish handle and $dbh->disconnect, you get an error message on the disconnect and the file remains open. 2) If you don't $sth->finish, you get an additional error message, and the file stays open. 3) If the statement handle goes out of scope before the database handle, everything seems to work as you'd expect. I doubt this is helpful, since I see from the correspondence that there is no problem in 1.12, but there is no problem in 1.09 for me. --Jeff Mitchell
Download output.txt
text/plain 2.3k
This is the output from test.pl as written: 1: before connect 1: after connect perl 28136 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 count: 5 closing dbh with active statement handles at /tmp/test.pl line 40. 1: after disconnect perl 28136 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 2: before connect perl 28136 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 2: after connect perl 28136 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 perl 28136 [xxxxx] 4r REG 8,5 2048 561 /tmp/test.sqlite3 count: 5 closing dbh with active statement handles at /tmp/test.pl line 40. 2: after disconnect perl 28136 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 perl 28136 [xxxxx] 4r REG 8,5 2048 561 /tmp/test.sqlite3 This is the output with the $sth->finish on line 36 commented out: 1: before connect 1: after connect perl 28267 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 count: 5 DBI::db=HASH(0x81b7490)->disconnect invalidates 1 active statement handle (either destroy statement handles or call finish on them before disconnecting) at /tmp/test.pl line 40. closing dbh with active statement handles at /tmp/test.pl line 40. 1: after disconnect perl 28267 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 2: before connect perl 28267 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 2: after connect perl 28267 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 perl 28267 [xxxxx] 4r REG 8,5 2048 561 /tmp/test.sqlite3 count: 5 DBI::db=HASH(0x81ba6fc)->disconnect invalidates 1 active statement handle (either destroy statement handles or call finish on them before disconnecting) at /tmp/test.pl line 40. closing dbh with active statement handles at /tmp/test.pl line 40. 2: after disconnect perl 28267 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 perl 28267 [xxxxx] 4r REG 8,5 2048 561 /tmp/test.sqlite3 This is the output with the braces on lines 30 and 38 uncommented: 1: before connect 1: after connect perl 28318 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 count: 5 1: after disconnect 2: before connect 2: after connect perl 28318 [xxxxx] 3r REG 8,5 2048 561 /tmp/test.sqlite3 count: 5 2: after disconnect
Download test.pl
text/x-perl 941b
#!/opt/local/bin/perl use strict; use warnings; use DBI; my $file = "/tmp/test.sqlite3"; $ENV{PATH} = "/bin:/usr/sbin"; my $dbh = DBI->connect("DBI:SQLite:dbname=$file", "", ""); { my $sth = $dbh->do(q{DROP TABLE IF EXISTS t1}); my $sth2 = $dbh->do(q{CREATE TABLE t1 (c1 text)}); my $sth3 = $dbh->prepare(q{INSERT INTO t1 VALUES (?)}); for my $i (1 .. 5) { $sth3->execute($i); } $sth3->finish(); } $dbh->disconnect(); my @dbh; for my $i (1 .. 2) { print "$i: before connect\n"; print `lsof -p $$|grep test`; $dbh[$i] = DBI->connect("DBI:SQLite:dbname=$file", "", ""); print "$i: after connect\n"; print `lsof -p $$|grep test`; # { my $sth = $dbh[$i]->prepare(q{SELECT count(1) from t1}); $sth->execute(); my ($count) = $sth->fetchrow_array; print "count: $count\n"; $sth->finish(); # } $dbh[$i]->disconnect(); print "$i: after disconnect\n"; print `lsof -p $$|grep test`, "\n"; }
Download (untitled) / with headers
text/plain 120b
I'm not sure what you mean about there being no problem in 1.09 and 1.13. I just tried 1.13 and it leaks like a seive.
Download (untitled) / with headers
text/plain 114b
I've checked the final attached test case. To the best of my knowledge, DBD::SQLite no longer leaks file handles.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.