Skip Menu |
 
rt.cpan.org will be shut down on March 1st, 2021.

This queue is for tickets about the Encode CPAN distribution.

Report information
The Basics
Id: 99609
Status: resolved
Priority: 0/
Queue: Encode

People
Owner: Nobody in particular
Requestors: cpan [...] chmrr.net
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

Attachments


Subject: FB_CROAK may cause decode's argument to be modified
MIME-Version: 1.0
X-Mailer: MIME-tools 5.504 (Entity 5.504)
X-RT-Interface: Web
Message-ID: <rt-4.0.18-18968-1413868659-500.0-0-0 [...] rt.cpan.org>
X-RT-Original-Encoding: utf-8
Content-Type: multipart/mixed; boundary="----------=_1413868659-18968-9"
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 0
Content-Disposition: inline
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: binary
Content-Length: 133
Download (untitled) / with headers
text/plain 133b
Passing FB_CROAK to decode seems like it sometimes causes the argument to decode to be destructively modified; test attached. - Alex
Subject: encoding-argument-modified.pl
MIME-Version: 1.0
Content-Type: application/octet-stream; name="encoding-argument-modified.pl"
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline; filename="encoding-argument-modified.pl"
Content-Transfer-Encoding: base64
Content-Length: 1405
#!/usr/bin/env perl use strict; use warnings; use Encode; use Test::More; my $valid = "\x61\x00\x00\x00"; diag "This is perl $^V, Encode $Encode::VERSION"; { my $string = "$valid"; my $ret = decode( "UTF32-LE", $string, Encode::FB_DEFAULT ); is($string, $valid, "Decoding a valid string with FB_DEFAULT does not modify the provided string"); } { my $string = "$valid"; my $ret = decode( "UTF32-LE", $string, Encode::FB_CROAK ); is($string, $valid, "Decoding a valid string with FB_CROAK does not modify the provided string"); } { my $string = $valid; my $enc = find_encoding("UTF32-LE"); my $ret = $enc->decode( $string, Encode::FB_CROAK ); is($string, "\x61\x00\x00\x00", "Decoding a valid string directly via Encode::Unicode with FB_CROAK does not modify the provided string"); is($valid, "\x61\x00\x00\x00", "Decoding a valid string directly via Encode::Unicode with FB_CROAK does not modify the orginal string via COW"); } done_testing; __END__ # This is perl v5.20.0, Encode 2.63 ok 1 - Decoding a valid string with FB_DEFAULT does not modify the provided string not ok 2 - Decoding a valid string with FB_CROAK does not modify the provided string # Failed test 'Decoding a valid string with FB_CROAK does not modify the provided string' # at wat.pl line 24. # got: '' # expected: 'a' 1..2 # Looks like you failed 1 test of 2.
MIME-Version: 1.0
In-Reply-To: <rt-4.0.18-18968-1413868659-500.0-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <rt-4.0.18-18968-1413868659-500.0-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-2729-1414209045-1165.99609-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 374
Download (untitled) / with headers
text/plain 374b
It's not a bug. It's a feature. Replace Encode::FB_CROAK with (Encode::FB_CROAK | Encode::LEAVE_SRC) and all of your tests pass. See Encode's pod for details. Dan the Maintainer Thereof On Tue Oct 21 01:17:39 2014, ALEXMV wrote: Show quoted text
> Passing FB_CROAK to decode seems like it sometimes causes the argument > to decode to be destructively modified; test attached. > - Alex
MIME-Version: 1.0
X-Spam-Status: No, score=-4.442 tagged_above=-99.9 required=10 tests=[AWL=2.226, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FROM_OUR_RT=-4, RP_MATCHES_RCVD=-0.668] autolearn=ham
In-Reply-To: <rt-4.0.18-2729-1414209045-630.99609-6-0 [...] rt.cpan.org>
X-Spam-Flag: NO
X-RT-Interface: API
References: <RT-Ticket-99609 [...] rt.cpan.org> <rt-4.0.18-18968-1413868659-500.99609-6-0 [...] rt.cpan.org> <rt-4.0.18-2729-1414209045-630.99609-6-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
X-Virus-Scanned: Debian amavisd-new at chmrr.net
Message-ID: <544B297F.9000202 [...] chmrr.net>
content-type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
X-Spam-Score: -4.442
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] chmrr.net
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id 9E229240683 for <cpan-bug+Encode [...] hipster.bestpractical.com>; Sat, 25 Oct 2014 00:39:42 -0400 (EDT)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rfErWzN8fSzE for <cpan-bug+Encode [...] hipster.bestpractical.com>; Sat, 25 Oct 2014 00:39:41 -0400 (EDT)
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by hipster.bestpractical.com (Postfix) with SMTP id C19FE24067E for <bug-Encode [...] rt.cpan.org>; Sat, 25 Oct 2014 00:39:40 -0400 (EDT)
Received: (qmail 10434 invoked by alias); 25 Oct 2014 04:39:40 -0000
Received: from chmrr.net (HELO mycon.chmrr.net) (173.237.205.2) by la.mx.develooper.com (qpsmtpd/0.28) with ESMTP; Fri, 24 Oct 2014 21:39:38 -0700
Received: from localhost (localhost [127.0.0.1]) by mycon.chmrr.net (Postfix) with ESMTP id 4837C103F27 for <bug-Encode [...] rt.cpan.org>; Sat, 25 Oct 2014 00:39:32 -0400 (EDT)
Received: from mycon.chmrr.net ([127.0.0.1]) by localhost (mycon.chmrr.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id U5uEiljs43Ke for <bug-Encode [...] rt.cpan.org>; Sat, 25 Oct 2014 00:39:28 -0400 (EDT)
Received: from [192.168.1.145] (207-180-139-40.ma.subnet.cable.rcn.com [207.180.139.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mycon.chmrr.net (Postfix) with ESMTPSA id 13E3C100E78 for <bug-Encode [...] rt.cpan.org>; Sat, 25 Oct 2014 00:39:28 -0400 (EDT)
Delivered-To: cpan-bug+Encode [...] hipster.bestpractical.com
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0
Subject: Re: [rt.cpan.org #99609] FB_CROAK may cause decode's argument to be modified
Return-Path: <alex [...] chmrr.net>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=chmrr.net; s=mail; t=1414211968; bh=rh9gWSI4lO//K5LOnMd0RtX/tiapYO4KiiGSDDRUcYE=; h=Date:From:To:Subject:References:In-Reply-To:From; b=xK6feldGi2QMkpvNLQd3nNHMDyAbODJ9mokKdWahxaOJ4f2DqPUaVxkmMJ9DqNdkG q2haZhrVMDvjaZ/aEAKZwCUkAwNiN7JvrF5P833AO08sxMitw+SlZ+EReCRKKzW8pP O5BT9pR/v8MXaKpe0x6IdzqMfTJhVUl+B9sXPnLc=
X-Spam-Check-BY: la.mx.develooper.com
X-Original-To: cpan-bug+Encode [...] hipster.bestpractical.com
X-RT-Mail-Extension: encode
Date: Sat, 25 Oct 2014 00:39:27 -0400
X-Spam-Level:
To: bug-Encode [...] rt.cpan.org
Content-Transfer-Encoding: 8bit
From: Alex Vandiver <alex [...] chmrr.net>
RT-Message-ID: <rt-4.0.18-13398-1414211983-568.99609-0-0 [...] rt.cpan.org>
Content-Length: 936
Download (untitled) / with headers
text/plain 936b
On 10/24/2014 11:50 PM, Dan Kogai via RT wrote: Show quoted text
> It's not a bug. It's a feature. Replace Encode::FB_CROAK with > (Encode::FB_CROAK | Encode::LEAVE_SRC) and all of your tests pass. > See Encode's pod for details.
Hm -- I mis-parsed them, then. When they say: If the Encode::LEAVE_SRC bit is not set but CHECK is set, then the source string to encode() or decode() will be overwritten in place. ...I read "overwritten" to mean "replaced with the output of the encode() or decode() call." Which, upon reflection, would be rather unhelpful -- but I feel like that section can perhaps be clarified. Would it be correct to say that: If the Encode::LEAVE_SRC bit is not set but CHECK is set, then the successfully encoded or decoded parts of the source string will be set to the empty string. This is notably useful with Encode::FB_QUIET and Encode::FB_WARN. ...and would you accept a patch to that effect? - Alex


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.