Skip Menu |
 

This queue is for tickets about the DBD-mysql CPAN distribution.

Report information
The Basics
Id: 120141
Status: resolved
Priority: 0/
Queue: DBD-mysql

People
Owner: Nobody in particular
Requestors: tanabe [...] fa2.so-net.ne.jp
Cc: pali [...] cpan.org
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 4.041_02

Attachments
0001-Fix-decoding-UTF-8-field-names-and-tables-names-when.patch
0002-Fix-decoding-UTF-8-warning-and-error-messages-when-m.patch
utf8ColunnMsg.pl



MIME-Version: 1.0
X-Spam-Status: No, score=-1.335 tagged_above=-99.9 required=10 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, SPF_SOFTFAIL=0.665] autolearn=no
X-Cpan.org: This message routed through the cpan.org mail forwarding service. Please use PAUSE pause.perl.org to configure your delivery settings.
X-Spam-Flag: NO
Content-Type: multipart/mixed; boundary="------------98CFABD2A8DB0C29BA862D83"
Message-ID: <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
X-Spam-Score: -1.335
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id C9B272403AC for <cpan-bug+DBD-mysql [...] hipster.bestpractical.com>; Tue, 7 Feb 2017 23:07:50 -0500 (EST)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pGeqW+zBKg1u for <cpan-bug+DBD-mysql [...] hipster.bestpractical.com>; Tue, 7 Feb 2017 23:07:47 -0500 (EST)
Received: from xx1.develooper.com (xx1.develooper.com [207.171.7.115]) by hipster.bestpractical.com (Postfix) with ESMTPS id 6E70124037A for <bug-DBD-mysql [...] rt.cpan.org>; Tue, 7 Feb 2017 23:07:47 -0500 (EST)
Received: from localhost (xx1.develooper.com [127.0.0.1]) by localhost (Postfix) with ESMTP id AA05111DA00 for <bug-DBD-mysql [...] rt.cpan.org>; Tue, 7 Feb 2017 20:07:45 -0800 (PST)
Received: from xx1.develooper.com (xx1.develooper.com [127.0.0.1]) by localhost (Postfix) with SMTP id 4981711D897 for <bug-DBD-mysql [...] rt.cpan.org>; Tue, 7 Feb 2017 20:07:43 -0800 (PST)
Received: from ms-fb53.so-net.ne.jp (ms-fb53.so-net.ne.jp [202.238.84.157]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by xx1.develooper.com (Postfix) with ESMTPS id 5480C11D3FD for <bug-DBD-mysql [...] rt.cpan.org>; Tue, 7 Feb 2017 20:07:27 -0800 (PST)
Received: from ms-omx51.so-net.ne.jp (ms-omx51.plus.so-net.ne.jp [10.240.84.96]) by ms-fb53.so-net.ne.jp with ESMTP id v183gp6f008206 for <bug-DBD-mysql [...] rt.cpan.org>; Wed, 8 Feb 2017 12:42:51 +0900
Received: from ms-omx61.so-net.ne.jp (ms-omx61.plus.so-net.ne.jp [10.240.84.163]) by ms-omx51.plus.so-net.ne.jp with ESMTP id v183ggAV004444; Wed, 8 Feb 2017 12:42:42 +0900
Received: from [192.168.37.18] (p7bc67131.kngwnt01.ap.so-net.ne.jp [123.198.113.49]) (authenticated) by ms-omx61.plus.so-net.ne.jp with ESMTP id v183gfkg007119 (using TLSv1/SSLv3 with cipher DHE-RSA-AES128-SHA (128 bits)); Wed, 8 Feb 2017 12:42:41 +0900
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] fa2.so-net.ne.jp
Delivered-To: cpan-bug+DBD-mysql [...] hipster.bestpractical.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0
Subject: UTF-8 column names and error messages are octet-streams, not strings
Return-Path: <tanabe [...] fa2.so-net.ne.jp>
X-RT-Mail-Extension: dbd-mysql
X-Original-To: cpan-bug+DBD-mysql [...] hipster.bestpractical.com
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fa2.so-net.ne.jp; s=sn2017; t=1486525362; bh=JJRiJ2/rQPWccodvpZiAt0GoJ6JDOS7U2c1N1FflciU=; h=Subject:To:From:Date; b=FtLUcYXPaxmq25EvsPdpVzJYYOLaeiOU1AiLzRHvT9DX5VetWHVtRimH8WSV2l5/5 ExccU9EcxJmoQO+IxsK6eunikjibIpjl6V7O0i3HOvNIpaCkfXtq8gKfQJIydcySFq BPgQKqb5RWxlxM5RihOlcpN/7x+s7Bcz/mh8xJRWSfeuv1f2ugGhqOM8dHEmmFGvxx fdKwoAozL0fjiONMZX+0wFBHy3QvSlmhxKx4b2cZH6DRWstjVKHk21IosfP5k3PRsV t/t1tHMQfdJ+0IQSh+R3rSpHrisKMVQ4fO106bmwcxqk6k1tUK7avQ+Q+l3l9OQLL7 BTjj08dg/zuww==
X-PMX-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, MIME_TEXT_ONLY_MP_MIXED 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_4000_4999 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, DKIM_SIGNATURE 0, NO_CTA_URI_FOUND 0, NO_URI_FOUND 0, NO_URI_HTTPS 0, SPF_PASS 0, __BAT_BOUNDARY 0, __CT 0, __CTYPE_HAS_BOUNDARY 0, __CTYPE_MULTIPART 0, __CTYPE_MULTIPART_MIXED 0, __FRAUD_CONTACT_SEX 0, __FRAUD_MONEY 0, __FRAUD_MONEY_CURRENCY 0, __FRAUD_MONEY_CURRENCY_DOLLAR 0, __FRAUD_MONEY_DENOMINATION 0, __FRAUD_MONEY_VALUE 0, __HAS_ATTACHMENT 0, __HAS_ATTACHMENT1 0, __HAS_FROM 0, __HAS_MSGID 0, __MIME_CHARSET_FARAWAY 0, __MIME_TEXT_ONLY 0, __MIME_TEXT_P 0, __MIME_TEXT_P1 0, __MIME_TEXT_P2 0, __MIME_VERSION 0, __MOZILLA_USER_AGENT 0, __NO_HTML_TAG_RAW 0, __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0, __USER_AGENT 0, __blackholes.mail-abuse.org_TIMEOUT , __zen.spamhaus.org_ERROR '
Date: Wed, 8 Feb 2017 12:42:41 +0900
X-Spam-Level:
X-Greylist: delayed 1481 seconds by postgrey-1.34 at xx1.develooper.com; Tue, 07 Feb 2017 20:07:32 PST
X-PMX-Version: 5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2017.2.8.35717
To: bug-DBD-mysql [...] rt.cpan.org
From: Tanabe Yoshinori <tanabe [...] fa2.so-net.ne.jp>
X-RT-Interface: Email
Content-Length: 0
content-type: text/plain; charset="utf-8"; delsp="yes"; format="flowed"
Content-Transfer-Encoding: 7bit
X-RT-Original-Encoding: iso-2022-jp
Content-Length: 1345
Download (untitled) / with headers
text/plain 1.3k
Hello, Column names and error messages should be treated as strings, but they are octet-streams in DBD-mysql-4.041. The attached code creates a table with a column whose name contains a non ASCII character. After issueing a SELECT statement and fetchrow_hashref, it tries to get a value using the column name at (1), but the result is undef. If you use the octet stream for the column name as a key, you get the value, at (2). Also, when you use Japanese error messages by adding line lc_messages=ja_JP in [mysqld] section of my.ini, messages are not decoded in DBD::mysql. As a result, messages are unreadable in (3) and (4). We could explicitly decode them as in (5) for message caught, but this cannot be applied to (3). Of course, it can be avoided by not using automatic encoding for STDERR at (6), but then we need to manually encode all other strings, a nightmare. Finally, I noticed that when error messages are in Japanese, make test of DBD-mysql fails. It may be difficult to avoid (I do not know), but a warning message (lc_messages should not be changed) in make test would help. DBD::mysql version: 4.041 Strawberry perl 64bit, v5.22.1 MariaDB $dbh->{mysql_clientinfo, mysql_clientversion, mysql_serverversion} returns: 5.1.44, 50144, 50505, respectively. Windows 7 Pro Service Pack 1 Regards, Tanabe Yoshinori
content-type: text/plain; charset="utf-8"; name="utf8ColunnMsg.pl"
Content-Disposition: attachment; filename="utf8ColunnMsg.pl"
Content-Transfer-Encoding: base64
X-RT-Original-Encoding: utf-8
Content-Length: 1589
Download utf8ColunnMsg.pl
text/x-perl 1.5k

Message body is not shown because sender requested not to inline it.

MIME-Version: 1.0
In-Reply-To: <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-6744-1486549963-1303.120141-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 1609
Download (untitled) / with headers
text/plain 1.5k
On Tue Feb 07 23:07:52 2017, tanabe@fa2.so-net.ne.jp wrote: Show quoted text
> Hello, > > Column names and error messages should be treated as strings, but > they are octet-streams in DBD-mysql-4.041. > > The attached code creates a table with a column whose name > contains a non ASCII character. After issueing a SELECT statement > and fetchrow_hashref, it tries to get a value using the column name > at (1), but the result is undef. If you use the octet stream for > the column name as a key, you get the value, at (2). > > Also, when you use Japanese error messages by adding line > lc_messages=ja_JP > in [mysqld] section of my.ini, messages are not decoded in > DBD::mysql. As a result, messages are unreadable in (3) and (4). > We could explicitly decode them as in (5) for message caught, but > this cannot be applied to (3). Of course, it can be avoided by > not using automatic encoding for STDERR at (6), but then we need > to manually encode all other strings, a nightmare. > > Finally, I noticed that when error messages are in Japanese, make > test of DBD-mysql fails. It may be difficult to avoid (I do not > know), but a warning message (lc_messages should not be changed) > in make test would help. > > DBD::mysql version: 4.041 > Strawberry perl 64bit, v5.22.1 > MariaDB > $dbh->{mysql_clientinfo, mysql_clientversion, mysql_serverversion} > returns: > 5.1.44, 50144, 50505, respectively. > Windows 7 Pro Service Pack 1 > > Regards, > Tanabe Yoshinori >
Hello, please try development version 4.041_1 of DBD-mysql. That one has fixed UTF-8 support for passing statements and parameters.
CC: pali [...] cpan.org
MIME-Version: 1.0
X-Spam-Status: No, score=-5.335 tagged_above=-99.9 required=10 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FROM_OUR_RT=-4, SPF_SOFTFAIL=0.665] autolearn=ham
In-Reply-To: <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org>
X-Cpan.org: This message routed through the cpan.org mail forwarding service. Please use PAUSE pause.perl.org to configure your delivery settings.
X-Spam-Flag: NO
X-RT-Interface: API
References: <RT-Ticket-120141 [...] rt.cpan.org> <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp> <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <93de7163-415d-4004-9b27-b2ff3646e49d [...] fa2.so-net.ne.jp>
content-type: text/plain; charset="utf-8"; format="flowed"
X-RT-Original-Encoding: utf-8
X-Spam-Score: -5.335
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] fa2.so-net.ne.jp
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id 9A6982403B4 for <cpan-bug+DBD-mysql [...] hipster.bestpractical.com>; Wed, 8 Feb 2017 06:20:33 -0500 (EST)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ejs5mUE6lmqc for <cpan-bug+DBD-mysql [...] hipster.bestpractical.com>; Wed, 8 Feb 2017 06:20:29 -0500 (EST)
Received: from xx1.develooper.com (xx1.develooper.com [207.171.7.115]) by hipster.bestpractical.com (Postfix) with ESMTPS id 954D624020C for <bug-DBD-mysql [...] rt.cpan.org>; Wed, 8 Feb 2017 06:20:29 -0500 (EST)
Received: from localhost (xx1.develooper.com [127.0.0.1]) by localhost (Postfix) with ESMTP id E95C011E05D for <bug-DBD-mysql [...] rt.cpan.org>; Wed, 8 Feb 2017 03:20:00 -0800 (PST)
Received: from xx1.develooper.com (xx1.develooper.com [127.0.0.1]) by localhost (Postfix) with SMTP id 9082211D407 for <bug-DBD-mysql [...] rt.cpan.org>; Wed, 8 Feb 2017 03:19:56 -0800 (PST)
Received: from ms-omx53.so-net.ne.jp (ms-omx53.so-net.ne.jp [202.238.84.153]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by xx1.develooper.com (Postfix) with ESMTPS id EC1AE11EB05 for <bug-DBD-mysql [...] rt.cpan.org>; Wed, 8 Feb 2017 03:19:29 -0800 (PST)
Received: from ms-omx62.so-net.ne.jp (ms-omx62.plus.so-net.ne.jp [10.240.84.164]) by ms-omx53.plus.so-net.ne.jp with ESMTP id v18BJNYM031221; Wed, 8 Feb 2017 20:19:23 +0900
Received: from [192.168.225.55] (11.251.149.210.rev.vmobile.jp [210.149.251.11]) (authenticated) by ms-omx62.plus.so-net.ne.jp with ESMTP id v18BJMrY007730 (using TLSv1/SSLv3 with cipher DHE-RSA-AES128-SHA (128 bits)); Wed, 8 Feb 2017 20:19:23 +0900
Delivered-To: cpan-bug+DBD-mysql [...] hipster.bestpractical.com
Subject: Re: [rt.cpan.org #120141] UTF-8 column names and error messages are octet-streams, not strings
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0
Return-Path: <tanabe [...] fa2.so-net.ne.jp>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fa2.so-net.ne.jp; s=sn2017; t=1486552763; bh=eMmRIRLExbXwYM5J2TAw7POJ3BvU2Xatoy3wc1euHIc=; h=Subject:To:References:Cc:From:Date:In-Reply-To; b=0t996wQzelB9FvL4oPgayUBJO8QPB83tjWr7Meo2gVqmE2D8+NoCQk53VfNZF2TCJ WrZvfPSZnaq7MpcKwEz4by2w9zCHmZtU51hcHvVUK2T80aeVm2XLTRgU7V3qAI9Otx ZARAA2guSKiGT1S4W3uSsIDHqKgLaGAFFJyM7L96NNNAqjoJ68aQpn+vRXXf8MYrQp yKTYK4MMELsw2B4iDTYF7y67WMUGVNoOV9vVaHOLDnd08qeNyh3imCFh5dexiMdf7M dbYP/wqfwAeLzXA+zlNCl6UXmauCW3GvU+ZEDXAIVfZMjtU3EaYOYwoqKosed3eN/g 0+A5FsF5ZR3iQ==
X-Original-To: cpan-bug+DBD-mysql [...] hipster.bestpractical.com
X-RT-Mail-Extension: dbd-mysql
Date: Wed, 8 Feb 2017 20:19:20 +0900
X-PMX-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1900_1999 0, BODY_SIZE_2000_LESS 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, DKIM_SIGNATURE 0, IN_REP_TO 0, LEGITIMATE_SIGNS 0, MSG_THREAD 0, REFERENCES 0, SINGLE_URI_IN_BODY 0, SPF_PASS 0, URI_ENDS_IN_HTML 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CP_URI_IN_BODY 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __FORWARDED_MSG 0, __FRAUD_MONEY 0, __FRAUD_MONEY_CURRENCY 0, __FRAUD_MONEY_CURRENCY_DOLLAR 0, __FRAUD_MONEY_DENOMINATION 0, __FRAUD_MONEY_VALUE 0, __HAS_CC_HDR 0, __HAS_FROM 0, __HAS_MSGID 0, __HTTPS_URI 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_TEXT_P 0, __MIME_TEXT_P1 0, __MIME_VERSION 0, __MOZILLA_USER_AGENT 0, __NO_HTML_TAG_RAW 0, __REFERENCES 0, __SANE_MSGID 0, __SINGLE_URI_TEXT 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0, __URI_IN_BODY 0, __URI_NO_WWW 0, __URI_NS , __URI_WITH_PATH 0, __USER_AGENT 0, __blackholes.mail-abuse.org_TIMEOUT , __zen.spamhaus.org_ERROR '
X-Spam-Level:
X-PMX-Version: 5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2017.1.27.3017
To: bug-DBD-mysql [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: Tanabe Yoshinori <tanabe [...] fa2.so-net.ne.jp>
RT-Message-ID: <rt-4.0.18-10940-1486552834-673.120141-0-0 [...] rt.cpan.org>
Content-Length: 1929
Download (untitled) / with headers
text/plain 1.8k
On 2017/02/08 19:32, Pali via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > > > On Tue Feb 07 23:07:52 2017, tanabe@fa2.so-net.ne.jp wrote:
>> Hello, >> >> Column names and error messages should be treated as strings, but >> they are octet-streams in DBD-mysql-4.041. >> >> The attached code creates a table with a column whose name >> contains a non ASCII character. After issueing a SELECT statement >> and fetchrow_hashref, it tries to get a value using the column name >> at (1), but the result is undef. If you use the octet stream for >> the column name as a key, you get the value, at (2). >> >> Also, when you use Japanese error messages by adding line >> lc_messages=ja_JP >> in [mysqld] section of my.ini, messages are not decoded in >> DBD::mysql. As a result, messages are unreadable in (3) and (4). >> We could explicitly decode them as in (5) for message caught, but >> this cannot be applied to (3). Of course, it can be avoided by >> not using automatic encoding for STDERR at (6), but then we need >> to manually encode all other strings, a nightmare. >> >> Finally, I noticed that when error messages are in Japanese, make >> test of DBD-mysql fails. It may be difficult to avoid (I do not >> know), but a warning message (lc_messages should not be changed) >> in make test would help. >> >> DBD::mysql version: 4.041 >> Strawberry perl 64bit, v5.22.1 >> MariaDB >> $dbh->{mysql_clientinfo, mysql_clientversion, mysql_serverversion} >> returns: >> 5.1.44, 50144, 50505, respectively. >> Windows 7 Pro Service Pack 1 >> >> Regards, >> Tanabe Yoshinori >>
> > Hello, please try development version 4.041_1 of DBD-mysql. That one has fixed UTF-8 support for passing statements and parameters. >
Hello, I have just installed 4.041_01 ("print $DBD::mysql::VERSION" shows the number) and run the script again. The results are the same as in my first report. Thank you. Tanabe
MIME-Version: 1.0
In-Reply-To: <rt-4.0.18-10940-1486552834-673.120141-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <RT-Ticket-120141 [...] rt.cpan.org> <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp> <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org> <93de7163-415d-4004-9b27-b2ff3646e49d [...] fa2.so-net.ne.jp> <rt-4.0.18-10940-1486552834-673.120141-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-6907-1486903950-1054.120141-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 2417
Download (untitled) / with headers
text/plain 2.3k
On Str Feb 08 06:20:34 2017, tanabe@fa2.so-net.ne.jp wrote: Show quoted text
> On 2017/02/08 19:32, Pali via RT wrote:
> > <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > > > > > On Tue Feb 07 23:07:52 2017, tanabe@fa2.so-net.ne.jp wrote:
> >> Hello, > >> > >> Column names and error messages should be treated as strings, but > >> they are octet-streams in DBD-mysql-4.041. > >> > >> The attached code creates a table with a column whose name > >> contains a non ASCII character. After issueing a SELECT statement > >> and fetchrow_hashref, it tries to get a value using the column name > >> at (1), but the result is undef. If you use the octet stream for > >> the column name as a key, you get the value, at (2). > >> > >> Also, when you use Japanese error messages by adding line > >> lc_messages=ja_JP > >> in [mysqld] section of my.ini, messages are not decoded in > >> DBD::mysql. As a result, messages are unreadable in (3) and (4). > >> We could explicitly decode them as in (5) for message caught, but > >> this cannot be applied to (3). Of course, it can be avoided by > >> not using automatic encoding for STDERR at (6), but then we need > >> to manually encode all other strings, a nightmare. > >> > >> Finally, I noticed that when error messages are in Japanese, make > >> test of DBD-mysql fails. It may be difficult to avoid (I do not > >> know), but a warning message (lc_messages should not be changed) > >> in make test would help. > >> > >> DBD::mysql version: 4.041 > >> Strawberry perl 64bit, v5.22.1 > >> MariaDB > >> $dbh->{mysql_clientinfo, mysql_clientversion, > >> mysql_serverversion} > >> returns: > >> 5.1.44, 50144, 50505, respectively. > >> Windows 7 Pro Service Pack 1 > >> > >> Regards, > >> Tanabe Yoshinori > >>
> > > > Hello, please try development version 4.041_1 of DBD-mysql. That one > > has fixed UTF-8 support for passing statements and parameters. > >
> > Hello, > > I have just installed 4.041_01 ("print $DBD::mysql::VERSION" shows the > number) and run the script again. The results are the same as in my > first report. > > Thank you. > Tanabe
Hi! Can you try compile DBD::mysql (either 4.041_01 or from git master) with these two attached patches? It should fix wide Unicode characters in column names and error messages. Note that DBI itself has broken Unicode messages prior to version 1.635 (see https://rt.cpan.org/Public/Bug/Display.html?id=102404).
MIME-Version: 1.0
In-Reply-To: <rt-4.0.18-6907-1486903950-1054.120141-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
X-RT-Interface: Web
References: <RT-Ticket-120141 [...] rt.cpan.org> <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp> <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org> <93de7163-415d-4004-9b27-b2ff3646e49d [...] fa2.so-net.ne.jp> <rt-4.0.18-10940-1486552834-673.120141-0-0 [...] rt.cpan.org> <rt-4.0.18-6907-1486903950-1054.120141-0-0 [...] rt.cpan.org>
Content-Type: multipart/mixed; boundary="----------=_1486904043-6907-2"
Message-ID: <rt-4.0.18-6907-1486904043-1489.120141-0-0 [...] rt.cpan.org>
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 0
Content-Disposition: inline
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 2629
Download (untitled) / with headers
text/plain 2.5k
On Ned Feb 12 07:52:30 2017, PALI wrote: Show quoted text
> On Str Feb 08 06:20:34 2017, tanabe@fa2.so-net.ne.jp wrote:
> > On 2017/02/08 19:32, Pali via RT wrote:
> > > <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > > > > > > > On Tue Feb 07 23:07:52 2017, tanabe@fa2.so-net.ne.jp wrote:
> > >> Hello, > > >> > > >> Column names and error messages should be treated as strings, but > > >> they are octet-streams in DBD-mysql-4.041. > > >> > > >> The attached code creates a table with a column whose name > > >> contains a non ASCII character. After issueing a SELECT statement > > >> and fetchrow_hashref, it tries to get a value using the column > > >> name > > >> at (1), but the result is undef. If you use the octet stream for > > >> the column name as a key, you get the value, at (2). > > >> > > >> Also, when you use Japanese error messages by adding line > > >> lc_messages=ja_JP > > >> in [mysqld] section of my.ini, messages are not decoded in > > >> DBD::mysql. As a result, messages are unreadable in (3) and (4). > > >> We could explicitly decode them as in (5) for message caught, but > > >> this cannot be applied to (3). Of course, it can be avoided by > > >> not using automatic encoding for STDERR at (6), but then we need > > >> to manually encode all other strings, a nightmare. > > >> > > >> Finally, I noticed that when error messages are in Japanese, make > > >> test of DBD-mysql fails. It may be difficult to avoid (I do not > > >> know), but a warning message (lc_messages should not be changed) > > >> in make test would help. > > >> > > >> DBD::mysql version: 4.041 > > >> Strawberry perl 64bit, v5.22.1 > > >> MariaDB > > >> $dbh->{mysql_clientinfo, mysql_clientversion, > > >> mysql_serverversion} > > >> returns: > > >> 5.1.44, 50144, 50505, respectively. > > >> Windows 7 Pro Service Pack 1 > > >> > > >> Regards, > > >> Tanabe Yoshinori > > >>
> > > > > > Hello, please try development version 4.041_1 of DBD-mysql. That > > > one > > > has fixed UTF-8 support for passing statements and parameters. > > >
> > > > Hello, > > > > I have just installed 4.041_01 ("print $DBD::mysql::VERSION" shows > > the > > number) and run the script again. The results are the same as in my > > first report. > > > > Thank you. > > Tanabe
> > Hi! Can you try compile DBD::mysql (either 4.041_01 or from git > master) with these two attached patches? It should fix wide Unicode > characters in column names and error messages. Note that DBI itself > has broken Unicode messages prior to version 1.635 (see > https://rt.cpan.org/Public/Bug/Display.html?id=102404).
Trying to attach patches again...
MIME-Version: 1.0
Subject: 0001-Fix-decoding-UTF-8-field-names-and-tables-names-when.patch
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Type: application/octet-stream; name="0001-Fix-decoding-UTF-8-field-names-and-tables-names-when.patch"
Content-Disposition: inline; filename="0001-Fix-decoding-UTF-8-field-names-and-tables-names-when.patch"
Content-Transfer-Encoding: base64
Content-Length: 1400
From 10980f96fb33c73c9b50bbee3c52d4875a0cc3e5 Mon Sep 17 00:00:00 2001 From: Pali <pali@cpan.org> Date: Sun, 12 Feb 2017 13:02:08 +0100 Subject: [PATCH 1/2] Fix decoding UTF-8 field names and tables names when mysql_enable_utf8 is enabled Attributes $sth->{NAME} and $sth->{mysql_table} should be properly UTF-8 decoded. Otherwise column and table names stay in UTF-8 octets instead of correct wide Unicode strings. Partially fixes bug: https://rt.cpan.org/Public/Bug/Display.html?id=120141 --- dbdimp.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/dbdimp.c b/dbdimp.c index 6b1e268..9e128b4 100644 --- a/dbdimp.c +++ b/dbdimp.c @@ -4869,8 +4869,10 @@ dbd_st_FETCH_internal( { dTHX; D_imp_sth(sth); + D_imp_dbh_from_sth; AV *av= Nullav; MYSQL_FIELD *curField; + bool enable_utf8 = (imp_dbh->enable_utf8 || imp_dbh->enable_utf8mb4); /* Are we asking for a legal value? */ if (what < 0 || what >= AV_ATTRIB_LAST) @@ -4896,10 +4898,14 @@ dbd_st_FETCH_internal( switch(what) { case AV_ATTRIB_NAME: sv= newSVpvn(curField->name, strlen(curField->name)); + if (enable_utf8) + sv_utf8_decode(sv); break; case AV_ATTRIB_TABLE: sv= newSVpvn(curField->table, strlen(curField->table)); + if (enable_utf8) + sv_utf8_decode(sv); break; case AV_ATTRIB_TYPE: -- 1.7.9.5
Subject: 0002-Fix-decoding-UTF-8-warning-and-error-messages-when-m.patch
MIME-Version: 1.0
Content-Type: application/octet-stream; name="0002-Fix-decoding-UTF-8-warning-and-error-messages-when-m.patch"
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline; filename="0002-Fix-decoding-UTF-8-warning-and-error-messages-when-m.patch"
Content-Transfer-Encoding: base64
Content-Length: 3150
From 132dba0df6e426262dacfd5236f5d0842b2419f0 Mon Sep 17 00:00:00 2001 From: Pali <pali@cpan.org> Date: Sun, 12 Feb 2017 13:27:04 +0100 Subject: [PATCH 2/2] Fix decoding UTF-8 warning and error messages when mysql_enable_utf8 is enabled Warning and error messages can contains wide characters based on locale and encoding settings. If we do not properly decode them then messages have UTF-8 octets instead of correct wide Unicode strings. Note that at least DBI of version 1.635 is required. Otherwise this change has no effect. Partially fixes bug: https://rt.cpan.org/Public/Bug/Display.html?id=120141 --- dbdimp.c | 37 ++++++++++++++++++++++++++++++++++++- 1 file changed, 36 insertions(+), 1 deletion(-) diff --git a/dbdimp.c b/dbdimp.c index 9e128b4..91cc1a8 100644 --- a/dbdimp.c +++ b/dbdimp.c @@ -1594,14 +1594,30 @@ void do_error(SV* h, int rc, const char* what, const char* sqlstate) { dTHX; D_imp_xxh(h); + imp_dbh_t* dbh; SV *errstr; SV *errstate; + bool enable_utf8; + + if (DBIc_TYPE(imp_xxh) == DBIt_DB) { + D_imp_dbh(h); + dbh = imp_dbh; + } else { + D_imp_sth(h); + D_imp_dbh_from_sth; + dbh = imp_dbh; + } + + enable_utf8 = (dbh->enable_utf8 || dbh->enable_utf8mb4); if (DBIc_TRACE_LEVEL(imp_xxh) >= 2) PerlIO_printf(DBIc_LOGPIO(imp_xxh), "\t\t--> do_error\n"); errstr= DBIc_ERRSTR(imp_xxh); sv_setiv(DBIc_ERR(imp_xxh), (IV)rc); /* set err early */ + SvUTF8_off(errstr); sv_setpv(errstr, what); + if (enable_utf8) + sv_utf8_decode(errstr); #if MYSQL_VERSION_ID >= SQL_STATE_VERSION if (sqlstate) @@ -1626,10 +1642,26 @@ void do_warn(SV* h, int rc, char* what) { dTHX; D_imp_xxh(h); + imp_dbh_t* dbh; + bool enable_utf8; + + if (DBIc_TYPE(imp_xxh) == DBIt_DB) { + D_imp_dbh(h); + dbh = imp_dbh; + } else { + D_imp_sth(h); + D_imp_dbh_from_sth; + dbh = imp_dbh; + } + + enable_utf8 = (dbh->enable_utf8 || dbh->enable_utf8mb4); SV *errstr = DBIc_ERRSTR(imp_xxh); sv_setiv(DBIc_ERR(imp_xxh), (IV)rc); /* set err early */ + SvUTF8_off(errstr); sv_setpv(errstr, what); + if (enable_utf8) + sv_utf8_decode(errstr); /* NO EFFECT DBIh_EVENT2(h, WARN_event, DBIc_ERR(imp_xxh), errstr);*/ if (DBIc_TRACE_LEVEL(imp_xxh) >= 2) PerlIO_printf(DBIc_LOGPIO(imp_xxh), "%s warning %d recorded: %s\n", @@ -2748,7 +2780,8 @@ SV* dbd_db_FETCH_attrib(SV *dbh, imp_dbh_t *imp_dbh, SV *keysv) STRLEN kl; char *key = SvPV(keysv, kl); /* needs to process get magic */ SV* result = NULL; - dbh= dbh; + bool enable_utf8 = (imp_dbh->enable_utf8 || imp_dbh->enable_utf8mb4); + PERL_UNUSED_ARG(dbh); switch (*key) { case 'A': @@ -2804,6 +2837,8 @@ SV* dbd_db_FETCH_attrib(SV *dbh, imp_dbh_t *imp_dbh, SV *keysv) /* Note that errmsg is obsolete, as of 2.09! */ const char* msg = mysql_error(imp_dbh->pmysql); result= sv_2mortal(newSVpvn(msg, strlen(msg))); + if (enable_utf8) + sv_utf8_decode(result); } else if (kl == strlen("enable_utf8mb4") && strEQ(key, "enable_utf8mb4")) result = sv_2mortal(newSViv(imp_dbh->enable_utf8mb4)); -- 1.7.9.5
CC: pali [...] cpan.org
MIME-Version: 1.0
X-Spam-Status: No, score=-3.335 tagged_above=-99.9 required=10 tests=[AWL=2.000, BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FROM_OUR_RT=-4, SPF_SOFTFAIL=0.665] autolearn=ham
In-Reply-To: <rt-4.0.18-6907-1486903951-470.120141-6-0 [...] rt.cpan.org>
X-Cpan.org: This message routed through the cpan.org mail forwarding service. Please use PAUSE pause.perl.org to configure your delivery settings.
X-Spam-Flag: NO
X-RT-Interface: API
References: <RT-Ticket-120141 [...] rt.cpan.org> <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp> <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org> <93de7163-415d-4004-9b27-b2ff3646e49d [...] fa2.so-net.ne.jp> <rt-4.0.18-10940-1486552834-673.120141-6-0 [...] rt.cpan.org> <rt-4.0.18-6907-1486903951-470.120141-6-0 [...] rt.cpan.org>
X-Virus-Scanned: Debian amavisd-new at bestpractical.com
Message-ID: <486a1161-080b-3554-1f3d-d8c73c7346ad [...] fa2.so-net.ne.jp>
content-type: text/plain; charset="utf-8"; format="flowed"
X-RT-Original-Encoding: utf-8
X-Spam-Score: -3.335
Authentication-Results: hipster.bestpractical.com (amavisd-new); dkim=pass header.i= [...] fa2.so-net.ne.jp
Received: from localhost (localhost [127.0.0.1]) by hipster.bestpractical.com (Postfix) with ESMTP id ED3A0240101 for <cpan-bug+DBD-mysql [...] hipster.bestpractical.com>; Sun, 12 Feb 2017 21:34:35 -0500 (EST)
Received: from hipster.bestpractical.com ([127.0.0.1]) by localhost (hipster.bestpractical.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id IV3LDqdVuCfs for <cpan-bug+DBD-mysql [...] hipster.bestpractical.com>; Sun, 12 Feb 2017 21:34:32 -0500 (EST)
Received: from xx1.develooper.com (xx1.develooper.com [207.171.7.115]) by hipster.bestpractical.com (Postfix) with ESMTPS id 00B6A2400FD for <bug-DBD-mysql [...] rt.cpan.org>; Sun, 12 Feb 2017 21:34:31 -0500 (EST)
Received: from localhost (xx1.develooper.com [127.0.0.1]) by localhost (Postfix) with ESMTP id CC3C311EF85 for <bug-DBD-mysql [...] rt.cpan.org>; Sun, 12 Feb 2017 18:34:30 -0800 (PST)
Received: from xx1.develooper.com (xx1.develooper.com [127.0.0.1]) by localhost (Postfix) with SMTP id CD0F311EF51 for <bug-DBD-mysql [...] rt.cpan.org>; Sun, 12 Feb 2017 18:34:28 -0800 (PST)
Received: from ms-omx52.so-net.ne.jp (ms-omx52.so-net.ne.jp [202.238.84.152]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by xx1.develooper.com (Postfix) with ESMTPS id BEF5011EF8D for <bug-DBD-mysql [...] rt.cpan.org>; Sun, 12 Feb 2017 18:34:17 -0800 (PST)
Received: from ms-omx61.so-net.ne.jp (ms-omx61.plus.so-net.ne.jp [10.240.84.163]) by ms-omx52.plus.so-net.ne.jp with ESMTP id v1D2Y8xS028082; Mon, 13 Feb 2017 11:34:08 +0900
Received: from [192.168.37.18] (p7bc67131.kngwnt01.ap.so-net.ne.jp [123.198.113.49]) (authenticated) by ms-omx61.plus.so-net.ne.jp with ESMTP id v1D2Y86Q012692 (using TLSv1/SSLv3 with cipher DHE-RSA-AES128-SHA (128 bits)); Mon, 13 Feb 2017 11:34:08 +0900
Delivered-To: cpan-bug+DBD-mysql [...] hipster.bestpractical.com
Subject: Re: [rt.cpan.org #120141] UTF-8 column names and error messages are octet-streams, not strings
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.1
Return-Path: <tanabe [...] fa2.so-net.ne.jp>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fa2.so-net.ne.jp; s=sn2017; t=1486953248; bh=kRrnVSP0euvxj8uCdaju5jv0EPicw4teboC4RviRo1c=; h=Subject:To:References:From:Cc:Date:In-Reply-To; b=DzW2bIXBJOg/vaFOw/bJhSTIeglOjYxZDpc8tuZf0P4apNKfOdWfEb/cQ/ojdZt5h A83dXfXo0vv5VgyfETgChJ1T2xEZvPRJt6Z5nYYOsrt/U89snY51MOwduKdWPk/97f hfvhAA8xACWgUuFir5+ikCmW+9TNfx8DzZZtiPhiy20H2R2THLtl8O1U8iDUVsICCH UWbaWE6bV0Cgzzl4ciIfOFmIQviSX6vViOvDnqnms2q6oRINURTpu/j1hpgNGfpLPh J/59Leteg7ZZOz0GZVoI0Xl5lcUQfLD0Dm8hFpUkdGnqMYOTuNggIcFYUO630dBN5Z C1HphZjSp+a7g==
X-Original-To: cpan-bug+DBD-mysql [...] hipster.bestpractical.com
X-RT-Mail-Extension: dbd-mysql
Date: Mon, 13 Feb 2017 11:34:08 +0900
X-PMX-Spam: Gauge=IIIIIIII, Probability=8%, Report=' HTML_00_01 0.05, HTML_00_10 0.05, SUPERLONG_LINE 0.05, BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_2000_2999 0, BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, DKIM_SIGNATURE 0, IN_REP_TO 0, LEGITIMATE_SIGNS 0, MSG_THREAD 0, REFERENCES 0, SPF_PASS 0, URI_ENDS_IN_HTML 0, __ANY_URI 0, __BOUNCE_CHALLENGE_SUBJ 0, __BOUNCE_NDR_SUBJ_EXEMPT 0, __CP_URI_IN_BODY 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __FORWARDED_MSG 0, __FRAUD_MONEY 0, __FRAUD_MONEY_CURRENCY 0, __FRAUD_MONEY_CURRENCY_DOLLAR 0, __FRAUD_MONEY_DENOMINATION 0, __FRAUD_MONEY_VALUE 0, __HAS_CC_HDR 0, __HAS_FROM 0, __HAS_MSGID 0, __HTTPS_URI 0, __IN_REP_TO 0, __MIME_TEXT_ONLY 0, __MIME_TEXT_P 0, __MIME_TEXT_P1 0, __MIME_VERSION 0, __MOZILLA_USER_AGENT 0, __MULTIPLE_URI_TEXT 0, __NO_HTML_TAG_RAW 0, __REFERENCES 0, __SANE_MSGID 0, __SUBJ_ALPHA_END 0, __SUBJ_ALPHA_NEGATE 0, __TO_MALFORMED_2 0, __TO_NO_NAME 0, __URI_IN_BODY 0, __URI_NO_WWW 0, __URI_NS , __URI_WITH_PATH 0, __USER_AGENT 0, __blackholes.mail-abuse.org_TIMEOUT , __zen.spamhaus.org_ERROR '
X-Spam-Level:
X-PMX-Version: 5.6.1.2065439, Antispam-Engine: 2.7.2.376379, Antispam-Data: 2017.2.13.22415
To: bug-DBD-mysql [...] rt.cpan.org
Content-Transfer-Encoding: 7bit
From: Tanabe Yoshinori <tanabe [...] fa2.so-net.ne.jp>
RT-Message-ID: <rt-4.0.18-19510-1486953277-1279.120141-0-0 [...] rt.cpan.org>
Content-Length: 2777
Download (untitled) / with headers
text/plain 2.7k
On 2017/02/12 21:52, Pali via RT wrote: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > > > On Str Feb 08 06:20:34 2017, tanabe@fa2.so-net.ne.jp wrote:
>> On 2017/02/08 19:32, Pali via RT wrote:
>>> <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > >>> >>> On Tue Feb 07 23:07:52 2017, tanabe@fa2.so-net.ne.jp wrote:
>>>> Hello, >>>> >>>> Column names and error messages should be treated as strings, but >>>> they are octet-streams in DBD-mysql-4.041. >>>> >>>> The attached code creates a table with a column whose name >>>> contains a non ASCII character. After issueing a SELECT statement >>>> and fetchrow_hashref, it tries to get a value using the column name >>>> at (1), but the result is undef. If you use the octet stream for >>>> the column name as a key, you get the value, at (2). >>>> >>>> Also, when you use Japanese error messages by adding line >>>> lc_messages=ja_JP >>>> in [mysqld] section of my.ini, messages are not decoded in >>>> DBD::mysql. As a result, messages are unreadable in (3) and (4). >>>> We could explicitly decode them as in (5) for message caught, but >>>> this cannot be applied to (3). Of course, it can be avoided by >>>> not using automatic encoding for STDERR at (6), but then we need >>>> to manually encode all other strings, a nightmare. >>>> >>>> Finally, I noticed that when error messages are in Japanese, make >>>> test of DBD-mysql fails. It may be difficult to avoid (I do not >>>> know), but a warning message (lc_messages should not be changed) >>>> in make test would help. >>>> >>>> DBD::mysql version: 4.041 >>>> Strawberry perl 64bit, v5.22.1 >>>> MariaDB >>>> $dbh->{mysql_clientinfo, mysql_clientversion, >>>> mysql_serverversion} >>>> returns: >>>> 5.1.44, 50144, 50505, respectively. >>>> Windows 7 Pro Service Pack 1 >>>> >>>> Regards, >>>> Tanabe Yoshinori >>>>
>>> >>> Hello, please try development version 4.041_1 of DBD-mysql. That one >>> has fixed UTF-8 support for passing statements and parameters. >>>
>> >> Hello, >> >> I have just installed 4.041_01 ("print $DBD::mysql::VERSION" shows the >> number) and run the script again. The results are the same as in my >> first report. >> >> Thank you. >> Tanabe
> > Hi! Can you try compile DBD::mysql (either 4.041_01 or from git master) with these two attached patches? It should fix wide Unicode characters in column names and error messages. Note that DBI itself has broken Unicode messages prior to version 1.635 (see https://rt.cpan.org/Public/Bug/Display.html?id=102404). >
Hello, I have confirmed that the problems have gone by applying the patches (and upgrading DBI to a later version). Thank you very much for the quick fix. One concern is that the fix can break code currently running. Best regards, Tanabe
MIME-Version: 1.0
In-Reply-To: <rt-4.0.18-19510-1486953277-1279.120141-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <RT-Ticket-120141 [...] rt.cpan.org> <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp> <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org> <93de7163-415d-4004-9b27-b2ff3646e49d [...] fa2.so-net.ne.jp> <rt-4.0.18-10940-1486552834-673.120141-6-0 [...] rt.cpan.org> <rt-4.0.18-6907-1486903951-470.120141-6-0 [...] rt.cpan.org> <486a1161-080b-3554-1f3d-d8c73c7346ad [...] fa2.so-net.ne.jp> <rt-4.0.18-19510-1486953277-1279.120141-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-3088-1486974406-1475.120141-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 3193
Download (untitled) / with headers
text/plain 3.1k
On Sun Feb 12 21:34:37 2017, tanabe@fa2.so-net.ne.jp wrote: Show quoted text
> On 2017/02/12 21:52, Pali via RT wrote:
> > <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > > > > > On Str Feb 08 06:20:34 2017, tanabe@fa2.so-net.ne.jp wrote:
> >> On 2017/02/08 19:32, Pali via RT wrote:
> >>> <URL: https://rt.cpan.org/Ticket/Display.html?id=120141 > > >>> > >>> On Tue Feb 07 23:07:52 2017, tanabe@fa2.so-net.ne.jp wrote:
> >>>> Hello, > >>>> > >>>> Column names and error messages should be treated as strings, but > >>>> they are octet-streams in DBD-mysql-4.041. > >>>> > >>>> The attached code creates a table with a column whose name > >>>> contains a non ASCII character. After issueing a SELECT statement > >>>> and fetchrow_hashref, it tries to get a value using the column > >>>> name > >>>> at (1), but the result is undef. If you use the octet stream for > >>>> the column name as a key, you get the value, at (2). > >>>> > >>>> Also, when you use Japanese error messages by adding line > >>>> lc_messages=ja_JP > >>>> in [mysqld] section of my.ini, messages are not decoded in > >>>> DBD::mysql. As a result, messages are unreadable in (3) and (4). > >>>> We could explicitly decode them as in (5) for message caught, but > >>>> this cannot be applied to (3). Of course, it can be avoided by > >>>> not using automatic encoding for STDERR at (6), but then we need > >>>> to manually encode all other strings, a nightmare. > >>>> > >>>> Finally, I noticed that when error messages are in Japanese, make > >>>> test of DBD-mysql fails. It may be difficult to avoid (I do not > >>>> know), but a warning message (lc_messages should not be changed) > >>>> in make test would help. > >>>> > >>>> DBD::mysql version: 4.041 > >>>> Strawberry perl 64bit, v5.22.1 > >>>> MariaDB > >>>> $dbh->{mysql_clientinfo, mysql_clientversion, > >>>> mysql_serverversion} > >>>> returns: > >>>> 5.1.44, 50144, 50505, respectively. > >>>> Windows 7 Pro Service Pack 1 > >>>> > >>>> Regards, > >>>> Tanabe Yoshinori > >>>>
> >>> > >>> Hello, please try development version 4.041_1 of DBD-mysql. That > >>> one > >>> has fixed UTF-8 support for passing statements and parameters. > >>>
> >> > >> Hello, > >> > >> I have just installed 4.041_01 ("print $DBD::mysql::VERSION" shows > >> the > >> number) and run the script again. The results are the same as in my > >> first report. > >> > >> Thank you. > >> Tanabe
> > > > Hi! Can you try compile DBD::mysql (either 4.041_01 or from git > > master) with these two attached patches? It should fix wide Unicode > > characters in column names and error messages. Note that DBI itself > > has broken Unicode messages prior to version 1.635 (see > > https://rt.cpan.org/Public/Bug/Display.html?id=102404). > >
> > Hello, I have confirmed that the problems have gone by applying the > patches (and upgrading DBI to a later version). Thank you very much > for > the quick fix. > One concern is that the fix can break code currently running. > Best regards, > Tanabe
Thank you for testing. I will reuse your script to create tests for this issue. Currently Unicode support is broken for a long time in DBD::mysql and proper way is to fix current code.
MIME-Version: 1.0
In-Reply-To: <rt-4.0.18-3088-1486974406-1475.120141-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.504 (Entity 5.504)
Content-Disposition: inline
X-RT-Interface: Web
References: <RT-Ticket-120141 [...] rt.cpan.org> <cd073aa7-4d72-fba3-e4e6-e235751f26c1 [...] fa2.so-net.ne.jp> <rt-4.0.18-6744-1486549964-1951.120141-6-0 [...] rt.cpan.org> <93de7163-415d-4004-9b27-b2ff3646e49d [...] fa2.so-net.ne.jp> <rt-4.0.18-10940-1486552834-673.120141-6-0 [...] rt.cpan.org> <rt-4.0.18-6907-1486903951-470.120141-6-0 [...] rt.cpan.org> <486a1161-080b-3554-1f3d-d8c73c7346ad [...] fa2.so-net.ne.jp> <rt-4.0.18-19510-1486953277-1279.120141-0-0 [...] rt.cpan.org> <rt-4.0.18-3088-1486974406-1475.120141-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Message-ID: <rt-4.0.18-13339-1498900349-1413.120141-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
X-RT-Encrypt: 0
X-RT-Sign: 0
Content-Length: 37
Reopening, fix was reverted in 4.043.
X-RT-Interface: REST
MIME-Version: 1.0
X-Mailer: MIME-tools 5.504 (Entity 5.504)
RT-Message-ID: <rt-4.0.18-21852-1510732670-1726.120141-0-0 [...] rt.cpan.org>
Content-Type: text/plain; charset="utf-8"
Content-Disposition: inline
Content-Transfer-Encoding: binary
Content-Length: 78


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.