Skip Menu |
 

This queue is for tickets about the AnyEvent-Twitter CPAN distribution.

Report information
The Basics
Id: 53566
Status: resolved
Priority: 0/
Queue: AnyEvent-Twitter

People
Owner: Nobody in particular
Requestors: hideki.yamamura [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



MIME-Version: 1.0
X-Spam-Status: No, hits=-0.1 required=8.0 tests=DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS
content-type: text/plain; charset="utf-8"
Message-ID: <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com>
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id 811C34D8002 for <bug-anyevent-twitter [...] rt.cpan.org>; Sun, 10 Jan 2010 13:15:09 -0500 (EST)
Received: (qmail 26445 invoked by uid 103); 10 Jan 2010 18:15:08 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 10 Jan 2010 18:15:08 -0000
Received: from mail-pw0-f54.google.com (HELO mail-pw0-f54.google.com) (209.85.160.54) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Sun, 10 Jan 2010 10:15:07 -0800
Received: by pwi14 with SMTP id 14so1091943pwi.13 for <bug-anyevent-twitter [...] rt.cpan.org>; Sun, 10 Jan 2010 10:15:04 -0800 (PST)
Received: by 10.115.24.10 with SMTP id b10mr2962852waj.69.1263147304418; Sun, 10 Jan 2010 10:15:04 -0800 (PST)
Delivered-To: cpan-bug+anyevent-twitter [...] diesel.bestpractical.com
Subject: update_status does not support non-ASCII characters
Return-Path: <hideki.yamamura [...] gmail.com>
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=qL8AG40g6v/PIkFmVFivtZLNJyNzAHPY62nuTk3fOi7Rh5Vgb5v4A2oVfKNuDGtnQh WoQtV3ReF0AmBuK0K/CjEHtDOAQQlN7ilcKL0g7yyekcZeQxl0ldA/EvjLs9FPPAmsNJ t9s9H1Ymb4+CsqhroJqYFz1Mh7sGvNWGkbJ5M=
X-Original-To: bug-anyevent-twitter [...] rt.cpan.org
X-Spam-Check-BY: 16.mx.develooper.com
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=hXybxNhiLw0XVbrVb3Hvgkv0W/Wz1g8iu/JEZEE3hpk=; b=KyDBj+R6ImAbA0l13nlszt+di3XET1Aq7l+hdZDZcSEPWSBIQkKkhAhoNcM8yvb//p 8rFia1ooxZoDwQSvr/jb9lcUqlt4Gn4khPFpoRS/GGL0/+J2akB4TOaRCxBP8FNOo/Hr ujGExz2BhNdbvSj5yfWst4tJv3du+h3PJ/06U=
Date: Mon, 11 Jan 2010 03:15:04 +0900
X-Spam-Level: *
To: bug-anyevent-twitter [...] rt.cpan.org
From: 山村 英貴 <hideki.yamamura [...] gmail.com>
X-RT-Original-Encoding: ISO-8859-1
Content-Length: 1148
Download (untitled) / with headers
text/plain 1.1k
Thanks for this very useful module. I found a bug about handling utf8 strings. The bug is that update_status does not support non-ASCII characters. First, this module use common::sense, so it implies "use utf8". In update_status(), $url->query_form is called with two arguments: (status, $status_e) key:status is utf8-flagged, but val:$status_e is octets. So in URI::_query::query_form, $status_e will be converted to utf8 when those key & val are connected. But $status_e is already converted to utf-8, so posted update turns unreadable strings when $status contained non-ASCII characters (like CJK, etc). --- /usr/lib/perl5/site_perl/5.8.8/AnyEvent/Twitter.pm 2009-11-05 08:09:54.000000000 +0900 +++ ./lib/AnyEvent/Twitter.pm 2010-01-11 03:03:21.000000000 +0900 @@ -477,11 +477,9 @@ sub update_status { my ($self, $status, $done_cb) = @_; - my $status_e = _encode_status $status; - my $url = URI::URL->new ($self->{base_url}); $url->path_segments ('statuses', "update.json"); - $url->query_form (status => $status_e); + $url->query_form (status => decode_utf8($status)); my $hdrs = { $self->_get_basic_auth };
MIME-Version: 1.0
X-Spam-Status: No, hits=-0.7 required=8.0 tests=RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS
In-Reply-To: <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org>
Content-Disposition: inline
References: <RT-Ticket-53566 [...] rt.cpan.org> <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org>
Message-ID: <20100111155318.GA11363 [...] elmex2>
Content-Type: text/plain; charset=utf-8
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id 5DC1F4D8048 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Mon, 11 Jan 2010 10:53:29 -0500 (EST)
Received: (qmail 25799 invoked by uid 103); 11 Jan 2010 15:53:29 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 11 Jan 2010 15:53:29 -0000
Received: from munch.schmorp.de (HELO munch.schmorp.de) (62.116.167.251) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Mon, 11 Jan 2010 07:53:25 -0800
Received: from [10.0.0.28] (helo=elmex2) by munch.schmorp.de with esmtp (Exim 4.69) (envelope-from <elmex [...] ta-sa.org>) id 1NUMZz-0007bG-DC for bug-AnyEvent-Twitter [...] rt.cpan.org; Mon, 11 Jan 2010 15:53:19 +0000
Received: from root by elmex2 with local (Exim 4.69) (envelope-from <elmex [...] ta-sa.org>) id 1NUMZy-0002xN-UT for bug-AnyEvent-Twitter [...] rt.cpan.org; Mon, 11 Jan 2010 16:53:18 +0100
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
User-Agent: Mutt/1.5.18 (2008-05-17)
Return-Path: <elmex [...] ta-sa.org>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Mon, 11 Jan 2010 16:53:18 +0100
X-Spam-Level: *
To: 山村 英貴 via RT <bug-AnyEvent-Twitter [...] rt.cpan.org>
Content-Transfer-Encoding: 8bit
From: Robin Redeker <elmex [...] ta-sa.org>
RT-Message-ID: <rt-3.8.HEAD-10120-1263225219-1585.53566-0-0 [...] rt.cpan.org>
Content-Length: 2674
Download (untitled) / with headers
text/plain 2.6k
Hi! On Sun, Jan 10, 2010 at 01:15:39PM -0500, 山村 英貴 via RT wrote: Show quoted text
> Sun Jan 10 13:15:38 2010: Request 53566 was acted upon. > Transaction: Ticket created by hideki.yamamura@gmail.com > Queue: AnyEvent-Twitter > Subject: update_status does not support non-ASCII characters > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: hideki.yamamura@gmail.com > Status: new > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=53566 > > > > Thanks for this very useful module. > > I found a bug about handling utf8 strings. > The bug is that update_status does not support non-ASCII characters. > > First, this module use common::sense, so it implies "use utf8". > In update_status(), $url->query_form is called with two arguments: > (status, $status_e) > key:status is utf8-flagged, but val:$status_e is octets. > > So in URI::_query::query_form, $status_e will be converted to utf8 > when those key & val are connected. > But $status_e is already converted to utf-8, so posted update turns > unreadable strings > when $status contained non-ASCII characters (like CJK, etc). > > > --- /usr/lib/perl5/site_perl/5.8.8/AnyEvent/Twitter.pm 2009-11-05 > 08:09:54.000000000 +0900 > +++ ./lib/AnyEvent/Twitter.pm 2010-01-11 03:03:21.000000000 +0900 > @@ -477,11 +477,9 @@ > sub update_status { > my ($self, $status, $done_cb) = @_; > > - my $status_e = _encode_status $status; > - > my $url = URI::URL->new ($self->{base_url}); > $url->path_segments ('statuses', "update.json"); > - $url->query_form (status => $status_e); > + $url->query_form (status => decode_utf8($status)); > > my $hdrs = { $self->_get_basic_auth }; >
This patch will break the module. The problem is, that C<$status> should be a plain string containing un-encoded unicode characters. calling decode_utf8 on it does not make sense at all, as the input to the update_status function should get un-encoded strings already. As URIs can't represent unicode characters the string must be encoded, which should be utf8 (i think i read that in the twitter API). So the code as it is, is correct if $status is a unicode string. The UTF8 flag is just an internal flag, which' should not be exposed or handled specially on the Perl language level. So, to come back to your Problem: What is the Problem? Can you put together a small test case that exposes the problem? Which Perl version do you use? Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG elmex@ta-sa.org / r.redeker@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
MIME-Version: 1.0
X-Spam-Status: No, hits=-0.1 required=8.0 tests=DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS
In-Reply-To: <rt-3.8.HEAD-10120-1263225219-40.53566-6-0 [...] rt.cpan.org>
References: <RT-Ticket-53566 [...] rt.cpan.org> <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org> <20100111155318.GA11363 [...] elmex2> <rt-3.8.HEAD-10120-1263225219-40.53566-6-0 [...] rt.cpan.org>
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
Message-ID: <a72d13321001110837t9e62090yd79b0497aefcd8bd [...] mail.gmail.com>
Content-Type: multipart/mixed; boundary=00504502e6a6e8bc05047ce62701
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id 1E6A64D8048 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Mon, 11 Jan 2010 11:37:46 -0500 (EST)
Received: (qmail 3044 invoked by uid 103); 11 Jan 2010 16:37:46 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 11 Jan 2010 16:37:46 -0000
Received: from mail-pz0-f190.google.com (HELO mail-pz0-f190.google.com) (209.85.222.190) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Mon, 11 Jan 2010 08:37:44 -0800
Received: by pzk28 with SMTP id 28so1194250pzk.27 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Mon, 11 Jan 2010 08:37:40 -0800 (PST)
Received: by 10.114.49.11 with SMTP id w11mr4485790waw.183.1263227860728; Mon, 11 Jan 2010 08:37:40 -0800 (PST)
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=ePJPbqED2qCraSqKHrYAOc8uzkk1y2Qkwoz2HH6vJ/qdBPhWSJxT3A+v2z7SSVeWeG lmx/Ii99i9QhrlzyMYEKRNubndS2PzoM3JyS8MzP770YhjjyHOiF7WVsmWR1On2FqCu+ UN0EWKMzVlXOi3E4HW+SXtg4RH2epr3GkarYw=
Return-Path: <hideki.yamamura [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=HgBT/En0ipSIJ1erXLlpIxh3RxOWN7a6q/+EA7m+oB4=; b=Q4rlgpQAB/PX/fAxZnHoM8YWlSjY9yu+oPk7OsOmdJbIsojgkiOmh0pGOWaGyzZyIB UIuFpfDzz3zo8q7hNjE/8I8MF3ROnPVQe9s7ELiPIzVTtOm/Oyglx84gO5eXKZMSEpPu f9XD48JXWLg2bsfxWiJ7CjKkBFpcEzxNTwtOk=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Tue, 12 Jan 2010 01:37:40 +0900
X-Spam-Level: *
To: bug-AnyEvent-Twitter [...] rt.cpan.org
From: Hideki Yamamura <hideki.yamamura [...] gmail.com>
RT-Message-ID: <rt-3.8.HEAD-10123-1263227880-1108.53566-0-0 [...] rt.cpan.org>
Content-Length: 0
Content-Type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
Content-Length: 855
Download (untitled) / with headers
text/plain 855b
Thanks for replying. I'm using Perl version is 5.8.8 and 5.10.0 (on CentOS 5.3 and Debian lenny). I made a small script for URI::URL's strange behavior. Executing this script, you'll get the four encoded URLs. AnyEvent::Twitter 0.27 uses first result with these two factors: get_url_utf8() (because use common::sense will convert 'status' to utf8-flagged string) $jp_octets (because $status_e was encoded by encoded_utf8($status)) But this encoded url string is completely broken. Apparently URI::URL's query_form can handle utf8-flagged strings when all parameters are utf8-flagged, so I made a patch to make $status utf8-flagged with Encode::decode_utf8. This patch's intention is: When $status was plain-string (utf8 encoded), it would be decoded by decode_utf8. When $status was utf8-flagged strings, it would be done nothing by decode_utf8.
content-type: application/octet-stream; name="uri-utf8.pl"
X-Attachment-ID: f_g4bge31d1
content-disposition: attachment; filename="uri-utf8.pl"
Content-Transfer-Encoding: base64
Content-Length: 1914
Download uri-utf8.pl
text/x-perl 1.8k

Message body is not shown because sender requested not to inline it.

MIME-Version: 1.0
X-Spam-Status: No, hits=-0.7 required=8.0 tests=RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS
In-Reply-To: <rt-3.8.HEAD-10123-1263227880-788.53566-5-0 [...] rt.cpan.org>
Content-Disposition: inline
References: <RT-Ticket-53566 [...] rt.cpan.org> <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org> <20100111155318.GA11363 [...] elmex2> <rt-3.8.HEAD-10120-1263225219-40.53566-6-0 [...] rt.cpan.org> <a72d13321001110837t9e62090yd79b0497aefcd8bd [...] mail.gmail.com> <rt-3.8.HEAD-10123-1263227880-788.53566-5-0 [...] rt.cpan.org>
Message-ID: <20100112102005.GA12282 [...] elmex2>
Content-Type: text/plain; charset=utf-8
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id 6124B4D8048 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Tue, 12 Jan 2010 08:56:51 -0500 (EST)
Received: (qmail 1729 invoked by uid 103); 12 Jan 2010 10:20:14 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 12 Jan 2010 10:20:14 -0000
Received: from munch.schmorp.de (HELO munch.schmorp.de) (62.116.167.251) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Tue, 12 Jan 2010 02:20:10 -0800
Received: from [10.0.0.28] (helo=elmex2) by munch.schmorp.de with esmtp (Exim 4.69) (envelope-from <elmex [...] ta-sa.org>) id 1NUdr3-0005qq-NK for bug-AnyEvent-Twitter [...] rt.cpan.org; Tue, 12 Jan 2010 10:20:05 +0000
Received: from root by elmex2 with local (Exim 4.69) (envelope-from <elmex [...] ta-sa.org>) id 1NUdr3-0003CF-DD for bug-AnyEvent-Twitter [...] rt.cpan.org; Tue, 12 Jan 2010 11:20:05 +0100
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
User-Agent: Mutt/1.5.18 (2008-05-17)
Return-Path: <elmex [...] ta-sa.org>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Tue, 12 Jan 2010 11:20:05 +0100
X-Spam-Level: *
To: 山村 英貴 via RT <bug-AnyEvent-Twitter [...] rt.cpan.org>
Content-Transfer-Encoding: 8bit
From: Robin Redeker <elmex [...] ta-sa.org>
RT-Message-ID: <rt-3.8.HEAD-10121-1263304620-626.53566-0-0 [...] rt.cpan.org>
Content-Length: 1796
Download (untitled) / with headers
text/plain 1.7k
On Mon, Jan 11, 2010 at 11:38:01AM -0500, 山村 英貴 via RT wrote: Show quoted text
> Queue: AnyEvent-Twitter > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=53566 > > > Thanks for replying. > > I'm using Perl version is 5.8.8 and 5.10.0 (on CentOS 5.3 and Debian lenny). > > I made a small script for URI::URL's strange behavior. > Executing this script, you'll get the four encoded URLs. > > AnyEvent::Twitter 0.27 uses first result with these two factors: > get_url_utf8() (because use common::sense will convert 'status' to > utf8-flagged string) > $jp_octets (because $status_e was encoded by encoded_utf8($status)) > But this encoded url string is completely broken. > > Apparently URI::URL's query_form can handle utf8-flagged strings when > all parameters are utf8-flagged, > so I made a patch to make $status utf8-flagged with Encode::decode_utf8. > > This patch's intention is: > When $status was plain-string (utf8 encoded), it would be decoded by > decode_utf8. > When $status was utf8-flagged strings, it would be done nothing by decode_utf8. >
Best is, that you report this as a Bug against URI::URL. The bug seems to be, that URI::URL double-encodes the string $jp_octets in the first case, as far as I can see it. I don't know what you mess around with the utf8-flag. At Perl language level I should not have to think about that INTERNAL flag. If URI::URL does produce broken URLs if I pass in octets, then the bug is URI::URL. Even if URI::URL doesn't handle strings, which are internally flagged somehow, correctly, it should at least document that. Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG elmex@ta-sa.org / r.redeker@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
MIME-Version: 1.0
X-Spam-Status: No, hits=-0.1 required=8.0 tests=DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS
In-Reply-To: <rt-3.8.HEAD-10121-1263304621-1075.53566-6-0 [...] rt.cpan.org>
References: <RT-Ticket-53566 [...] rt.cpan.org> <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org> <20100111155318.GA11363 [...] elmex2> <rt-3.8.HEAD-10120-1263225219-40.53566-6-0 [...] rt.cpan.org> <a72d13321001110837t9e62090yd79b0497aefcd8bd [...] mail.gmail.com> <rt-3.8.HEAD-10123-1263227880-788.53566-5-0 [...] rt.cpan.org> <20100112102005.GA12282 [...] elmex2> <rt-3.8.HEAD-10121-1263304621-1075.53566-6-0 [...] rt.cpan.org>
X-Virus-Checked: Checked by ClamAV on 16.mx.develooper.com
Message-ID: <a72d13321001120734g65b3ff0fnd160dcd662521b6a [...] mail.gmail.com>
Content-Type: multipart/mixed; boundary=0016e64cce94a18c78047cf96276
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id B0A6F4D8021 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Tue, 12 Jan 2010 10:34:15 -0500 (EST)
Received: (qmail 27951 invoked by uid 103); 12 Jan 2010 15:34:14 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 12 Jan 2010 15:34:14 -0000
Received: from mail-px0-f174.google.com (HELO mail-px0-f174.google.com) (209.85.216.174) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Tue, 12 Jan 2010 07:34:13 -0800
Received: by pxi4 with SMTP id 4so4649240pxi.33 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Tue, 12 Jan 2010 07:34:10 -0800 (PST)
Received: by 10.114.188.31 with SMTP id l31mr5193222waf.201.1263310450297; Tue, 12 Jan 2010 07:34:10 -0800 (PST)
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=sn6JKtW4tSZfaUlHPrz75PUkSZPa40Qm9hdG1+nEeA5ipugBxz12b3fyYoAOnI+5hh D6Ynh8g6QBjpIrUdMI/0tKp31u7eljnN12cJCS1JVPtLdJ+rGj4vWZN62oWgRsmp23bO M3S1J8/dZCly13kfIPrsjGfb5kIlSG9YzL9e8=
Return-Path: <hideki.yamamura [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=BJyHVlq3UIBdqHK30H0BFAynkiVz45fe37JRE2yepDo=; b=TMVa3p+XWKD8YFunTzxwTBBaZs6b8X0b4YkRRKStm8gr6574utnl3fuAs/jpkxR052 AITYNyhzTGBt1JuK79vmWl8fbZS5sLY7XxjamIAGYIY7zlN+ylgWmX8TrwqkxnXQPO8m +DxkP23Cvfgq2RPXzWcheiJogTwxaa/JGEB60=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Wed, 13 Jan 2010 00:34:10 +0900
X-Spam-Level: *
To: bug-AnyEvent-Twitter [...] rt.cpan.org
From: Hideki Yamamura <hideki.yamamura [...] gmail.com>
RT-Message-ID: <rt-3.8.HEAD-10122-1263310484-611.53566-0-0 [...] rt.cpan.org>
Content-Length: 0
Content-Type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
Content-Length: 624
Download (untitled) / with headers
text/plain 624b
More simply, please refer this script. You are doing same thing in AnyEvent::Twitter line 484. It is very usual problem arond utf8-flag in Japan because we use multibyte languages. I think you should not use common::sense. And I find far more excellent solution. Please use this patch (and drop first patch). --- /usr/lib/perl5/site_perl/5.8.8/AnyEvent/Twitter.pm 2009-11-05 08:09:54.000000000 +0900 +++ ./lib/AnyEvent/Twitter.pm 2010-01-13 00:20:12.000000000 +0900 @@ -1,5 +1,6 @@ package AnyEvent::Twitter; -use common::sense; +use strict; +use warnings; use Carp qw/croak/; use AnyEvent; use AnyEvent::HTTP;
content-type: application/octet-stream; name="utf8-problem.pl"
X-Attachment-ID: f_g4ctmfhb0
content-disposition: attachment; filename="utf8-problem.pl"
Content-Transfer-Encoding: base64
Content-Length: 136
Download utf8-problem.pl
text/x-perl 136b

Message body is not shown because sender requested not to inline it.

MIME-Version: 1.0
X-Spam-Status: No, hits=-0.1 required=8.0 tests=DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS
In-Reply-To: <a72d13321001120734g65b3ff0fnd160dcd662521b6a [...] mail.gmail.com>
References: <RT-Ticket-53566 [...] rt.cpan.org> <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org> <20100111155318.GA11363 [...] elmex2> <rt-3.8.HEAD-10120-1263225219-40.53566-6-0 [...] rt.cpan.org> <a72d13321001110837t9e62090yd79b0497aefcd8bd [...] mail.gmail.com> <rt-3.8.HEAD-10123-1263227880-788.53566-5-0 [...] rt.cpan.org> <20100112102005.GA12282 [...] elmex2> <rt-3.8.HEAD-10121-1263304621-1075.53566-6-0 [...] rt.cpan.org> <a72d13321001120734g65b3ff0fnd160dcd662521b6a [...] mail.gmail.com>
Message-ID: <a72d13321001140410i117b2e92w30f90ad9db2d42ec [...] mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id E0EB04D8027 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Thu, 14 Jan 2010 07:11:04 -0500 (EST)
Received: (qmail 27272 invoked by uid 103); 14 Jan 2010 12:11:04 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 14 Jan 2010 12:11:04 -0000
Received: from mail-pz0-f193.google.com (HELO mail-pz0-f193.google.com) (209.85.222.193) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Thu, 14 Jan 2010 04:11:03 -0800
Received: by pzk31 with SMTP id 31so2705856pzk.28 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Thu, 14 Jan 2010 04:11:00 -0800 (PST)
Received: by 10.114.7.10 with SMTP id 10mr502400wag.90.1263471060103; Thu, 14 Jan 2010 04:11:00 -0800 (PST)
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=QDDJc3SRojiZbArwu7KSi6pnM/PPYGs+5cz6D4Ze5AfppNyPmLs0BlxOIyUtgTlzUD X9+mccyGBK2/AfmDLlvDDeEeGPJCNH1QcFPRgzdMomreuskxIxRa+kHxL0pcy2JEYfQH SROA69s91kdQ7o1WG/H5kwGDJsgjcbyTCdomc=
Return-Path: <hideki.yamamura [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=UlZ5wdZN36rjH3P3S9Hj89ziE092by8OQmjNSK7/2l0=; b=cNtjn/CoOiS04rnvqrIT/BZ89vgPkV0O41ho2Y9BS+wYrOl2FlhE7NViX+316fxbsy szKLF3ouJht8tmy4zj0Lvk64Vp4EEfTTccdkVY/1ZVi1KD70cagugXxU40lHbYuakW/G 8cx+oVCVMbLnSMTnraNlgHBChnd2AsfILDWDk=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Thu, 14 Jan 2010 21:10:59 +0900
X-Spam-Level: *
To: bug-AnyEvent-Twitter [...] rt.cpan.org
From: Hideki Yamamura <hideki.yamamura [...] gmail.com>
RT-Message-ID: <rt-3.8.HEAD-10122-1263471083-59.53566-0-0 [...] rt.cpan.org>
Content-Length: 602
Download (untitled) / with headers
text/plain 602b
I found another way to avoid utf8-related problem. This is because status is utf8-on but 'status' is utf8-off. Following your advice, I submit a bug report for URI module. Thanks. --- /usr/lib/perl5/site_perl/5.8.8/AnyEvent/Twitter.pm 2009-11-05 08:09:54.000000000 +0900 +++ ./lib/AnyEvent/Twitter.pm 2010-01-14 21:01:23.000000000 +0900 @@ -481,7 +481,7 @@ my $url = URI::URL->new ($self->{base_url}); $url->path_segments ('statuses', "update.json"); - $url->query_form (status => $status_e); + $url->query_form ('status' => $status_e); my $hdrs = { $self->_get_basic_auth };
MIME-Version: 1.0
X-Spam-Status: No, hits=-0.7 required=8.0 tests=RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS
In-Reply-To: <rt-3.8.HEAD-10122-1263471087-1800.53566-5-0 [...] rt.cpan.org>
Content-Disposition: inline
References: <a72d13321001101015x92a5eebr1be5d7e6b31c0385 [...] mail.gmail.com> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org> <20100111155318.GA11363 [...] elmex2> <a72d13321001110837t9e62090yd79b0497aefcd8bd [...] mail.gmail.com> <rt-3.8.HEAD-10123-1263227880-788.53566-5-0 [...] rt.cpan.org> <20100112102005.GA12282 [...] elmex2> <rt-3.8.HEAD-10121-1263304621-1075.53566-6-0 [...] rt.cpan.org> <a72d13321001120734g65b3ff0fnd160dcd662521b6a [...] mail.gmail.com> <a72d13321001140410i117b2e92w30f90ad9db2d42ec [...] mail.gmail.com> <rt-3.8.HEAD-10122-1263471087-1800.53566-5-0 [...] rt.cpan.org>
Message-ID: <20100114122006.GA7721 [...] elmex2>
Content-Type: text/plain; charset=utf-8
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id 60AF84D8014 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Thu, 14 Jan 2010 07:20:16 -0500 (EST)
Received: (qmail 23879 invoked by uid 103); 14 Jan 2010 12:20:15 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 14 Jan 2010 12:20:15 -0000
Received: from munch.schmorp.de (HELO munch.schmorp.de) (62.116.167.251) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Thu, 14 Jan 2010 04:20:12 -0800
Received: from [10.0.0.28] (helo=elmex2) by munch.schmorp.de with esmtp (Exim 4.69) (envelope-from <elmex [...] ta-sa.org>) id 1NVOgJ-0001ge-5V for bug-AnyEvent-Twitter [...] rt.cpan.org; Thu, 14 Jan 2010 12:20:07 +0000
Received: from root by elmex2 with local (Exim 4.69) (envelope-from <elmex [...] ta-sa.org>) id 1NVOgI-00020d-QF for bug-AnyEvent-Twitter [...] rt.cpan.org; Thu, 14 Jan 2010 13:20:06 +0100
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
User-Agent: Mutt/1.5.18 (2008-05-17)
Return-Path: <elmex [...] ta-sa.org>
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Thu, 14 Jan 2010 13:20:06 +0100
X-Spam-Level: *
To: 山村 英貴 via RT <bug-AnyEvent-Twitter [...] rt.cpan.org>
Content-Transfer-Encoding: 8bit
From: Robin Redeker <elmex [...] ta-sa.org>
RT-Message-ID: <rt-3.8.HEAD-10120-1263471628-1636.53566-0-0 [...] rt.cpan.org>
Content-Length: 1227
Download (untitled) / with headers
text/plain 1.1k
On Thu, Jan 14, 2010 at 07:11:29AM -0500, 山村 英貴 via RT wrote: Show quoted text
> Queue: AnyEvent-Twitter > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=53566 > > > I found another way to avoid utf8-related problem. > This is because status is utf8-on but 'status' is utf8-off. > Following your advice, I submit a bug report for URI module. Thanks.
Thats great. Thanks! Btw. the "status => ..." utf-8 flag on bareword bug is known in 5.10 and is fixed in the development version and the next release of Perl5 I think. I stumbled across that myself once. Show quoted text
> > > --- /usr/lib/perl5/site_perl/5.8.8/AnyEvent/Twitter.pm 2009-11-05 > 08:09:54.000000000 +0900 > +++ ./lib/AnyEvent/Twitter.pm 2010-01-14 21:01:23.000000000 +0900 > @@ -481,7 +481,7 @@ > > my $url = URI::URL->new ($self->{base_url}); > $url->path_segments ('statuses', "update.json"); > - $url->query_form (status => $status_e); > + $url->query_form ('status' => $status_e); > > my $hdrs = { $self->_get_basic_auth }; >
Greetings, Robin -- Robin Redeker | Deliantra, the free code+content MORPG elmex@ta-sa.org / r.redeker@gmail.com | http://www.deliantra.net http://www.ta-sa.org/ |
MIME-Version: 1.0
X-Spam-Status: No, hits=-0.1 required=8.0 tests=DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,SPF_PASS
In-Reply-To: <rt-3.8.HEAD-10120-1263471629-217.53566-6-0 [...] rt.cpan.org>
References: <RT-Ticket-53566 [...] rt.cpan.org> <rt-3.8.HEAD-10119-1263147339-1559.53566-4-0 [...] rt.cpan.org> <20100111155318.GA11363 [...] elmex2> <20100112102005.GA12282 [...] elmex2> <rt-3.8.HEAD-10121-1263304621-1075.53566-6-0 [...] rt.cpan.org> <a72d13321001120734g65b3ff0fnd160dcd662521b6a [...] mail.gmail.com> <a72d13321001140410i117b2e92w30f90ad9db2d42ec [...] mail.gmail.com> <rt-3.8.HEAD-10122-1263471087-1800.53566-5-0 [...] rt.cpan.org> <20100114122006.GA7721 [...] elmex2> <rt-3.8.HEAD-10120-1263471629-217.53566-6-0 [...] rt.cpan.org>
Message-ID: <a72d13321001292231u7a7db0a4wea7845aa2d14452b [...] mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
X-RT-Original-Encoding: utf-8
Received: from la.mx.develooper.com (x1.develooper.com [207.171.7.70]) by diesel.bestpractical.com (Postfix) with SMTP id 168134D8014 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Sat, 30 Jan 2010 01:32:33 -0500 (EST)
Received: (qmail 1190 invoked by uid 103); 30 Jan 2010 06:32:33 -0000
Received: from x16.dev (10.0.100.26) by x1.dev with QMQP; 30 Jan 2010 06:32:33 -0000
Received: from mail-px0-f182.google.com (HELO mail-px0-f182.google.com) (209.85.216.182) by 16.mx.develooper.com (qpsmtpd/0.80) with ESMTP; Fri, 29 Jan 2010 22:31:58 -0800
Received: by pxi12 with SMTP id 12so2366117pxi.33 for <bug-AnyEvent-Twitter [...] rt.cpan.org>; Fri, 29 Jan 2010 22:31:46 -0800 (PST)
Received: by 10.114.251.10 with SMTP id y10mr1194398wah.149.1264833103615; Fri, 29 Jan 2010 22:31:43 -0800 (PST)
Delivered-To: cpan-bug+AnyEvent-Twitter [...] diesel.bestpractical.com
Subject: Re: [rt.cpan.org #53566] update_status does not support non-ASCII characters
Domainkey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=L0AtSEYpQ4sETsRz9s//vMyLpIy4/8/ECRY5FQAaeuWZIXq93pQ+y+yibwjS5ehsIz 3mjKZKEGz1HBnpkrHK6M93g+llRpsJ6qfkHClzPjEIqEesk5rJeBsKcQD+qGk/2T+MXd LT/TZxcdtQLZEQkF6XPHLXsxq18tEZI4r6eRs=
Return-Path: <hideki.yamamura [...] gmail.com>
Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=PTwiAB4QmTy95vzjJ22gfMw04QBCCYIKbJ8KCYq/Hrs=; b=SfxWgFW5dc/uOetTqa35IMesKu0ESxaJHTp3ymZm2Oy46Gf13ZngMVIw/qDhJ8cCWw WLulYfOmPY5U94O8U2nfWSbk1PgCckZMQ0G0lShu0qBBz2EC6vXb2lC5QObsQerRYNNw zfrPMn0qKlrPwjdoZasY+0QR1wutAR8S5vx1Y=
X-Spam-Check-BY: 16.mx.develooper.com
X-Original-To: bug-AnyEvent-Twitter [...] rt.cpan.org
Date: Sat, 30 Jan 2010 15:31:43 +0900
X-Spam-Level: *
To: bug-AnyEvent-Twitter [...] rt.cpan.org
Content-Transfer-Encoding: quoted-printable
From: Hideki Yamamura <hideki.yamamura [...] gmail.com>
RT-Message-ID: <rt-3.8.HEAD-18077-1264833163-1058.53566-0-0 [...] rt.cpan.org>
Content-Length: 730
Download (untitled) / with headers
text/plain 730b
Hi. I have difficulty that URI.pm's maintainer does not respond my ticket. We have truly no way to post Japanese text to Twitter with AnyEvent::Twitter(0.27) on Perl 5.8 and 5.10. Would you apply this patch? Thanks. Show quoted text
>> --- /usr/lib/perl5/site_perl/5.8.8/AnyEvent/Twitter.pm  2009-11-05 >> 08:09:54.000000000 +0900 >> +++ ./lib/AnyEvent/Twitter.pm      2010-01-14 21:01:23.000000000 +0900 >> @@ -481,7 +481,7 @@ >> >>     my $url = URI::URL->new ($self->{base_url}); >>     $url->path_segments ('statuses', "update.json"); >> -   $url->query_form (status => $status_e); >> +   $url->query_form ('status' => $status_e); >> >>     my $hdrs = { $self->_get_basic_auth };
-- Hideki YAMAMURA <hideki.yamamura@gmail.com>
MIME-Version: 1.0
In-Reply-To: <rt-3.8.HEAD-18077-1264833163-1058.53566-0-0 [...] rt.cpan.org>
X-Mailer: MIME-tools 5.427 (Entity 5.427)
Content-Disposition: inline
References: <rt-3.8.HEAD-10121-1263304621-1075.53566-6-0 [...] rt.cpan.org> <a72d13321001120734g65b3ff0fnd160dcd662521b6a [...] mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Message-ID: <rt-3.8.HEAD-18078-1265015425-1452.53566-0-0 [...] rt.cpan.org>
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: utf-8
Content-Length: 519
Download (untitled) / with headers
text/plain 519b
On Sat Jan 30 01:32:43 2010, hideki.yamamura@gmail.com wrote: Show quoted text
> Hi. > I have difficulty that URI.pm's maintainer does not respond my ticket. > We have truly no way to post Japanese text to Twitter with > AnyEvent::Twitter(0.27) > on Perl 5.8 and 5.10. > Would you apply this patch? Thanks. >
Oh, thats sad @ URI maintainer. I've applied your workaround. You can fetch the updated version from my git repository: http://git.ta-sa.org/AnyEvent-Twitter.git I will release it probably this week. Greetings, Robin


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.