Skip Menu |
 

This queue is for tickets about the Encode-Detect CPAN distribution.

Report information
The Basics
Id: 43548
Status: new
Priority: 0/
Queue: Encode-Detect

People
Owner: Nobody in particular
Requestors: joungkyun [...] gmail.com
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.01
Fixed in: 1.01



Subject: 1.01 don't detect Latin2 Hungarian / Win1250 Hungarian / TIS620 Thai charset
Download (untitled) / with headers
text/plain 366b
I write libchardet for C/C++ with Encode-Detect C++ binding source. And It will find http://cvs.oops.org/?cvsroot=OOPS- Project&module=libchardet. On porting time, I found some bug of Encode-Detect 1.01 that can't detect Latin2 Hungarian / Win1250 Hungarian / TIS620 Thai charset, And these charset missing on SBCMGroupProber. So, I report fixed patch. Thanks.
Subject: 1.01.patch
Download 1.01.patch
text/x-diff 2.8k
--- nsSBCSGroupProber.cpp 2009/02/21 20:08:02 1.1.1.1 +++ nsSBCSGroupProber.cpp 2009/02/23 13:47:40 1.2 @@ -56,21 +56,24 @@ mProbers[7] = new nsSingleByteCharSetProber(&Win1253Model); mProbers[8] = new nsSingleByteCharSetProber(&Latin5BulgarianModel); mProbers[9] = new nsSingleByteCharSetProber(&Win1251BulgarianModel); + mProbers[10] = new nsSingleByteCharSetProber(&Latin2HungarianModel); + mProbers[11] = new nsSingleByteCharSetProber(&Win1250HungarianModel); + mProbers[12] = new nsSingleByteCharSetProber(&TIS620ThaiModel); nsHebrewProber *hebprober = new nsHebrewProber(); // Notice: Any change in these indexes - 10,11,12 must be reflected // in the code below as well. - mProbers[10] = hebprober; - mProbers[11] = new nsSingleByteCharSetProber(&Win1255Model, PR_FALSE, hebprober); // Logical Hebrew - mProbers[12] = new nsSingleByteCharSetProber(&Win1255Model, PR_TRUE, hebprober); // Visual Hebrew + mProbers[13] = hebprober; + mProbers[14] = new nsSingleByteCharSetProber(&Win1255Model, PR_FALSE, hebprober); // Logical Hebrew + mProbers[15] = new nsSingleByteCharSetProber(&Win1255Model, PR_TRUE, hebprober); // Visual Hebrew // Tell the Hebrew prober about the logical and visual probers - if (mProbers[10] && mProbers[11] && mProbers[12]) // all are not null + if (mProbers[13] && mProbers[14] && mProbers[15]) // all are not null { - hebprober->SetModelProbers(mProbers[11], mProbers[12]); + hebprober->SetModelProbers(mProbers[14], mProbers[15]); } else // One or more is null. avoid any Hebrew probing, null them all { - for (PRUint32 i = 10; i <= 12; ++i) + for (PRUint32 i = 13; i <= 15; ++i) { delete mProbers[i]; mProbers[i] = 0; @@ -79,8 +82,8 @@ // disable latin2 before latin1 is available, otherwise all latin1 // will be detected as latin2 because of their similarity. - //mProbers[10] = new nsSingleByteCharSetProber(&Latin2HungarianModel); - //mProbers[11] = new nsSingleByteCharSetProber(&Win1250HungarianModel); + //mProbers[13] = new nsSingleByteCharSetProber(&Latin2HungarianModel); + //mProbers[14] = new nsSingleByteCharSetProber(&Win1250HungarianModel); Reset(); } --- nsSBCSGroupProber.h 2009/02/21 20:08:02 1.1.1.1 +++ nsSBCSGroupProber.h 2009/02/23 13:47:40 1.2 @@ -40,7 +40,7 @@ #define nsSBCSGroupProber_h__ -#define NUM_OF_SBCS_PROBERS 13 +#define NUM_OF_SBCS_PROBERS 16 class nsCharSetProber; class nsSBCSGroupProber: public nsCharSetProber { --- nsSBCharSetProber.h 2009/02/21 20:08:02 1.1.1.1 +++ nsSBCharSetProber.h 2009/02/23 13:47:40 1.2 @@ -118,6 +118,7 @@ extern SequenceModel Win1251BulgarianModel; extern SequenceModel Latin2HungarianModel; extern SequenceModel Win1250HungarianModel; +extern SequenceModel TIS620ThaiModel; extern SequenceModel Win1255Model; #endif /* nsSingleByteCharSetProber_h__ */


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.