Skip Menu |
 

This queue is for tickets about the Devel-Declare CPAN distribution.

Report information
The Basics
Id: 69552
Status: resolved
Priority: 0/
Queue: Devel-Declare

People
Owner: Nobody in particular
Requestors: zefram [...] fysh.org
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



From zefram [...] fysh.org Mon Jul 18 12: 02:07 2011
MIME-Version: 1.0
X-Spam-Status: No, score=-6.988 tagged_above=-99.9 required=10 tests=[AWL=-0.088, BAYES_00=-1.9, RCVD_IN_DNSWL_HI=-5] autolearn=ham
Content-Disposition: inline
X-Spam-Flag: NO
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
Content-Type: multipart/mixed; boundary="ew6BAiZeqk4r7MaW"
Message-ID: <20110718160150.GF4069 [...] lake.fysh.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
X-Spam-Score: -6.988
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id B8BE0240506 for <cpan-bug+Devel-Declare [...] hipster.bestpractical.com>; Mon, 18 Jul 2011 12:02:07 -0400 (EDT)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id CHBVoNLAAwMO for <cpan-bug+Devel-Declare [...] hipster.bestpractical.com>; Mon, 18 Jul 2011 12:02:04 -0400 (EDT)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id 2E4B22404BA for <bug-Devel-Declare [...] rt.cpan.org>; Mon, 18 Jul 2011 12:02:04 -0400 (EDT)
Received: (qmail 8864 invoked by uid 103); 18 Jul 2011 16:02:03 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 18 Jul 2011 16:02:03 -0000
Received: from lake.fysh.org (HELO lake.fysh.org) (81.94.195.195) by 16.mx.develooper.com (qpsmtpd/0.80/v0.80-19-gf52d165) with ESMTP; Mon, 18 Jul 2011 09:01:57 -0700
Received: from zefram by lake.fysh.org with local (Exim 4.69 #1 (Debian)) id 1QiqGU-0000n6-3c; Mon, 18 Jul 2011 17:01:50 +0100
Delivered-To: cpan-bug+Devel-Declare [...] hipster.bestpractical.com
Subject: linestr reallocation in large program
Return-Path: <zefram [...] fysh.org>
X-RT-Mail-Extension: devel-declare
X-Original-To: cpan-bug+Devel-Declare [...] hipster.bestpractical.com
X-Spam-Check-BY: 16.mx.develooper.com
Date: Mon, 18 Jul 2011 17:01:50 +0100
X-Spam-Level:
To: bug-Devel-Declare [...] rt.cpan.org
From: Zefram <zefram [...] fysh.org>
Content-Length: 0
content-type: text/plain; charset="utf-8"
Content-Disposition: inline
X-RT-Original-Encoding: us-ascii
Content-Length: 1676
Download (untitled) / with headers
text/plain 1.6k
When the tokeniser reads in a new line, if this requires the next block of input to be read from the file then PL_linestr will be reallocated if necessary, to accommodate its old contents plus the entire next block (even though it ends up only containing the old contents plus the next line). New line reading can be triggered through Devel::Declare via toke_skipspace(). Any reallocation of PL_linestr at this point breaks D:D, by making PL_bufptr invalid. It follows that D:D needs to ensure that PL_linestr is already large enough to accommodate the largest line expected to be seen *plus one I/O block*. Historically I/O blocks have been small enough that D:D's preferred PL_linestr size of 8192 was sufficient. But in 5.14 the buffer size has been increased, now defaulting to 8192. So if Devel::Declare is used on a large source file (tens of kilobytes) on 5.14, it will eventually have PL_linestr reallocated under it. I ran into this in work code, where we require D:D to parse multi-line custom syntax and we have a large number of large source files. The failure mode amounted to memory corruption, resulting from PL_bufptr being in freed space. Attached patch increases D:D's preferred PL_linestr size to 16384, which fixes the problem where the I/O block size is 8192. Possibly should be cleverer about that; look at PERLIOBUF_DEFAULT_BUFSIZ in the core if you want to work on that. The patch also makes toke_skipspace() check whether reallocation has actually occurred, and bomb out if it has. So if the problem recurs, it'll at least be reported cleanly. Patch also adds a test script, which is necessarily a bit large, but compresses well. -zefram
content-type: text/plain; charset="utf-8"
content-disposition: attachment; filename="d0"
X-RT-Original-Encoding: us-ascii
Content-Length: 35256
Download d0
text/plain 34.4k

Message body is not shown because sender requested not to inline it.



This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.