Skip Menu |
 

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Locale-Maketext CPAN distribution.

Report information
The Basics
Id: 46738
Status: resolved
Priority: 0/
Queue: Locale-Maketext

People
Owner: ferreira [...] shoo.cpan.org
Requestors: dmuey [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: (no value)
Fixed in: 1.15_01



Subject: Support readonly lexicon hashes (patch included)
Download (untitled) / with headers
text/plain 753b
If you tie() a lexicon to, say, a GDBM_File tied via & GDBM_READER; file the first call to maketext() is fatal. That is because once it compiles a value it re-stores it in the Lexicon hash. It is along the lines of this pseudo code: $gdbm_reader{$phrase} = $compile_value; # die()s The way to support this sort of hash is to provide a mechanism to not store the compiled strings back in the Lexicon The patched maktext() is in Locale::Maketext::Utils version 0.16 (w/ tests: t/04.use_external_lex_cache.t) but it'd be better suited for Locale::Maketext The way it works is this: You's init $lh->{'use_external_lex_cache'} = 1; Anytime something it is compiled it is stored in $lh- Show quoted text
>{_external_lex_cache} instead of the lexicon hash.
Subject: external_lex_cache.patch
--- Maketext.pm.orig 2009-06-05 11:42:02.000000000 -0500 +++ Maketext.pm 2009-06-08 10:03:26.000000000 -0500 @@ -129,10 +129,9 @@ my $phrase = shift; $handle->{'failure_lex'} ||= {}; - my $lex = $handle->{'failure_lex'}; - my $value; - $lex->{$phrase} ||= ($value = $handle->_compile($phrase)); + my $value = $handle->_compile($phrase); + $handle->{'failure_lex'}{$phrase} ||= $value; # ? respect 'use_external_lex_cache' || document not to set 'failure_lex' as a readonly type hash || ... ? # Dumbly copied from sub maketext: return ${$value} if ref($value) eq 'SCALAR'; @@ -179,34 +178,47 @@ my($handle, $phrase) = splice(@_,0,2); Carp::confess('No handle/phrase') unless (defined($handle) && defined($phrase)); - # Don't interefere with $@ in case that's being interpolated into the msg. local $@; # Look up the value: - my $value; - foreach my $h_r ( - @{ $isa_scan{ref($handle) || $handle} || $handle->_lex_refs } - ) { - DEBUG and warn "* Looking up \"$phrase\" in $h_r\n"; - if(exists $h_r->{$phrase}) { - DEBUG and warn " Found \"$phrase\" in $h_r\n"; - unless(ref($value = $h_r->{$phrase})) { - # Nonref means it's not yet compiled. Compile and replace. - $value = $h_r->{$phrase} = $handle->_compile($value); + if (exists $handle->{'_external_lex_cache'}{$phrase}) { + DEBUG and warn "* Using external lex cache version of \"$phrase\"\n"; + $value = $handle->{'_external_lex_cache'}{$phrase}; + } + else { + foreach my $h_r ( + @{ $isa_scan{ref($handle) || $handle} || $handle->_lex_refs } + ) { + DEBUG and warn "* Looking up \"$phrase\" in $h_r\n"; + if(exists $h_r->{$phrase}) { + DEBUG and warn " Found \"$phrase\" in $h_r\n"; + unless(ref($value = $h_r->{$phrase})) { + # Nonref means it's not yet compiled. Compile and replace. + if ($handle->{'use_external_lex_cache'}) { + $value = $handle->{'_external_lex_cache'}{$phrase} = $handle->_compile($value); + } + else { + $value = $h_r->{$phrase} = $handle->_compile($value); + } + } + last; } - last; - } - elsif($phrase !~ m/^_/s and $h_r->{'_AUTO'}) { - # it's an auto lex, and this is an autoable key! - DEBUG and warn " Automaking \"$phrase\" into $h_r\n"; - - $value = $h_r->{$phrase} = $handle->_compile($phrase); - last; + elsif($phrase !~ m/^_/s and $h_r->{'_AUTO'}) { + # it's an auto lex, and this is an autoable key! + DEBUG and warn " Automaking \"$phrase\" into $h_r\n"; + if ($handle->{'use_external_lex_cache'}) { + $value = $handle->{'_external_lex_cache'}{$phrase} = $handle->_compile($phrase); + } + else { + $value = $h_r->{$phrase} = $handle->_compile($phrase); + } + last; + } + DEBUG>1 and print " Not found in $h_r, nor automakable\n"; + # else keep looking } - DEBUG>1 and print " Not found in $h_r, nor automakable\n"; - # else keep looking } unless(defined($value)) {
Download (untitled) / with headers
text/plain 112b
this behavior could be implemented with an object lexicon. See https://rt.cpan.org/Ticket/Display.html?id=47315
Download (untitled) / with headers
text/plain 296b
On Thu Jun 25 09:45:58 2009, DMUEY wrote: Show quoted text
> this behavior could be implemented with an object lexicon. > > See https://rt.cpan.org/Ticket/Display.html?id=47315
On second thought, while that is true, this way is much less overhead if all you are wanting an object for is to gain this behavior
Download (untitled) / with headers
text/plain 1.1k
Locale/Maketext.pm would also need this: @@ -234,7 +234,7 @@ } last; } - elsif($phrase !~ m/^_/s and $h_r->{'_AUTO'}) { + elsif($phrase !~ m/^_/s and ($handle- Show quoted text
>{'use_external_lex_cache'} ? $handle->{'_external_lex_cache'}{'_AUTO'}
: $h_r->{'_AUTO'})) { # it's an auto lex, and this is an autoable key! DEBUG and warn " Automaking \"$phrase\" into $h_r\n"; Note to self: Locale/Maketext/Utils.pm @@ -62,9 +62,15 @@ } } - no strict 'refs'; - local ${ $lh->get_base_class() . '::Lexicon' }{'_AUTO'} = 1; - return $lh->maketext( $key, @args ); + if ($lh->{'use_external_lex_cache'}) { + local $lh->{'_external_lex_cache'}{'_AUTO'} = 1; + return $lh->maketext( $key, @args ); + } + else { + no strict 'refs'; + local ${ $lh->get_base_class() . '::Lexicon' }{'_AUTO'} = 1; + return $lh->maketext( $key, @args ); + } } ); }
Subject: [1.14 target - item #1] Support readonly lexicon hashes (patch included)
Changing title to reflect preference over obj rt
Download (untitled) / with headers
text/plain 223b
To make it easier to review and implement I've put all changes into a single small patch. This is a complete w/ Changelog, POD, tests, and version 1.13_82 -> 1.13_83 The POD describes when this feature would be necessary.
Download rt46738.patch
text/x-diff 7.3k
diff -ruN Locale-Maketext-1.13_82.orig/ChangeLog Locale-Maketext-1.13_82/ChangeLog --- Locale-Maketext-1.13_82.orig/ChangeLog 2009-06-23 21:29:36.000000000 -0500 +++ Locale-Maketext-1.13_82/ChangeLog 2009-07-30 12:26:19.000000000 -0500 @@ -1,5 +1,10 @@ Revision history for Perl suite Locale::Maketext +2009-08-01 Adriano Ferreira + * Development release 1.13_83 + + "Readonly" lexicon support (thanks Dan Muey) + 2009-06-23 Adriano Ferreira * Development release 1.13_82 diff -ruN Locale-Maketext-1.13_82.orig/MANIFEST Locale-Maketext-1.13_82/MANIFEST --- Locale-Maketext-1.13_82.orig/MANIFEST 2009-06-23 19:16:50.000000000 -0500 +++ Locale-Maketext-1.13_82/MANIFEST 2009-07-30 13:12:26.000000000 -0500 @@ -12,6 +12,7 @@ README t/00_load.t t/01_about_verbose.t +t/04.use_external_lex_cache.t t/10_make.t t/20_get.t t/30_local.t diff -ruN Locale-Maketext-1.13_82.orig/lib/Locale/Maketext.pm Locale-Maketext-1.13_82/lib/Locale/Maketext.pm --- Locale-Maketext-1.13_82.orig/lib/Locale/Maketext.pm 2009-06-23 21:23:47.000000000 -0500 +++ Locale-Maketext-1.13_82/lib/Locale/Maketext.pm 2009-07-30 13:07:35.000000000 -0500 @@ -10,7 +10,7 @@ BEGIN { unless(defined &DEBUG) { *DEBUG = sub () {0} } } # define the constant 'DEBUG' at compile-time -$VERSION = '1.13_82'; +$VERSION = '1.13_83'; $VERSION = eval $VERSION; @ISA = (); @@ -186,27 +186,44 @@ # Look up the value: my $value; - foreach my $h_r ( - @{ $isa_scan{ref($handle) || $handle} || $handle->_lex_refs } - ) { - DEBUG and warn "* Looking up \"$phrase\" in $h_r\n"; - if(exists $h_r->{$phrase}) { - DEBUG and warn " Found \"$phrase\" in $h_r\n"; - unless(ref($value = $h_r->{$phrase})) { - # Nonref means it's not yet compiled. Compile and replace. - $value = $h_r->{$phrase} = $handle->_compile($value); + if (exists $handle->{'_external_lex_cache'}{$phrase}) { + DEBUG and warn "* Using external lex cache version of \"$phrase\"\n"; + $value = $handle->{'_external_lex_cache'}{$phrase}; + } + else { + foreach my $h_r ( + @{ $isa_scan{ref($handle) || $handle} || $handle->_lex_refs } + ) { + DEBUG and warn "* Looking up \"$phrase\" in $h_r\n"; + if(exists $h_r->{$phrase}) { + DEBUG and warn " Found \"$phrase\" in $h_r\n"; + unless(ref($value = $h_r->{$phrase})) { + # Nonref means it's not yet compiled. Compile and replace. + if ($handle->{'use_external_lex_cache'}) { + $value = $handle->{'_external_lex_cache'}{$phrase} = $handle->_compile($value); + } + else { + $value = $h_r->{$phrase} = $handle->_compile($value); + } + } + last; } - last; - } - elsif($phrase !~ m/^_/s and $h_r->{'_AUTO'}) { - # it's an auto lex, and this is an autoable key! - DEBUG and warn " Automaking \"$phrase\" into $h_r\n"; - - $value = $h_r->{$phrase} = $handle->_compile($phrase); - last; + # extending packages need to be able to localize _AUTO and if readonly can't "local $h_r->{'_AUTO'} = 1;" + # but they can "local $handle->{'_external_lex_cache'}{'_AUTO'} = 1;" + elsif($phrase !~ m/^_/s and ($handle->{'use_external_lex_cache'} ? ( exists $handle->{'_external_lex_cache'}{'_AUTO'} ? $handle->{'_external_lex_cache'}{'_AUTO'} : $h_r->{'_AUTO'} ) : $h_r->{'_AUTO'})) { + # it's an auto lex, and this is an autoable key! + DEBUG and warn " Automaking \"$phrase\" into $h_r\n"; + if ($handle->{'use_external_lex_cache'}) { + $value = $handle->{'_external_lex_cache'}{$phrase} = $handle->_compile($phrase); + } + else { + $value = $h_r->{$phrase} = $handle->_compile($phrase); + } + last; + } + DEBUG>1 and print " Not found in $h_r, nor automakable\n"; + # else keep looking } - DEBUG>1 and warn " Not found in $h_r, nor automakable\n"; - # else keep looking } unless(defined($value)) { diff -ruN Locale-Maketext-1.13_82.orig/lib/Locale/Maketext.pod Locale-Maketext-1.13_82/lib/Locale/Maketext.pod --- Locale-Maketext-1.13_82.orig/lib/Locale/Maketext.pod 2009-06-20 14:49:46.000000000 -0500 +++ Locale-Maketext-1.13_82/lib/Locale/Maketext.pod 2009-07-30 12:40:31.000000000 -0500 @@ -937,6 +937,25 @@ arbitrarily decided to use a leading underscore as a signal to distinguish those few. +=head1 READONLY LEXICONS + +If your lexicon is a tied hash the simple act of caching the compiled value can be fatal. + +For example a L<GDBM_File> GDBM_READER tied hash will die with something like: + + gdbm store returned -1, errno 2, key "..." at ... + +All you need to do is turn on caching outside of the lexicon hash itself like so: + + sub init { + my ($lh) = @_; + ... + $lh->{'use_external_lex_cache'} = 1; + ... + } + +And then instead of storing the compiled value in the lexicon hash it will store it in $lh->{'_external_lex_cache'} + =head1 CONTROLLING LOOKUP FAILURE If you call $lh->maketext(I<key>, ...parameters...), diff -ruN Locale-Maketext-1.13_82.orig/t/04.use_external_lex_cache.t Locale-Maketext-1.13_82/t/04.use_external_lex_cache.t --- Locale-Maketext-1.13_82.orig/t/04.use_external_lex_cache.t 1969-12-31 18:00:00.000000000 -0600 +++ Locale-Maketext-1.13_82/t/04.use_external_lex_cache.t 2009-07-30 13:04:30.000000000 -0500 @@ -0,0 +1,40 @@ +use Test::More tests => 11; + +BEGIN { + chdir 't'; + unshift @INC, qw(lib ../lib); + use_ok('Locale::Maketext'); +}; + +package MyTestLocale; + +@MyTestLocale::ISA = qw(Locale::Maketext); +%MyTestLocale::Lexicon = (); +%MyTestLocale::Lexicon = (); # to avoid warnings + +package MyTestLocale::fr; + +@MyTestLocale::fr::ISA = qw(MyTestLocale); + +%MyTestLocale::fr::Lexicon = ( + '_AUTO' => 1, + 'Hello World' => 'Bonjour Monde', +); + +package main; + +my $lh = MyTestLocale->get_handle('fr'); +$lh->{'use_external_lex_cache'} = 1; +ok(exists $MyTestLocale::fr::Lexicon{'Hello World'} && !ref $MyTestLocale::fr::Lexicon{'Hello World'}, 'lex value not a ref'); + +ok($lh->maketext('Hello World') eq 'Bonjour Monde', 'renders correctly first time'); +ok(exists $lh->{'_external_lex_cache'}{'Hello World'} && ref $lh->{'_external_lex_cache'}{'Hello World'}, 'compiled into lex_cache'); +ok(exists $MyTestLocale::fr::Lexicon{'Hello World'} && !ref $MyTestLocale::fr::Lexicon{'Hello World'}, 'lex value still not a ref'); + +ok($lh->maketext('Hello World') eq 'Bonjour Monde', 'renders correctly second time time'); +ok(exists $lh->{'_external_lex_cache'}{'Hello World'} && ref $lh->{'_external_lex_cache'}{'Hello World'}, 'still compiled into lex_cache'); +ok(exists $MyTestLocale::fr::Lexicon{'Hello World'} && !ref $MyTestLocale::fr::Lexicon{'Hello World'}, 'lex value still not a ref'); + +ok($lh->maketext('This is not a key') eq 'This is not a key', '_AUTO renders correctly first time'); +ok(exists $lh->{'_external_lex_cache'}{'This is not a key'} && ref $lh->{'_external_lex_cache'}{'This is not a key'}, '_AUTO compiled into lex_cache'); +ok(!exists $MyTestLocale::fr::Lexicon{'This is not a key'}, '_AUTO lex value not added to lex');
Download (untitled) / with headers
text/plain 328b
On Thu Jul 30 14:20:05 2009, DMUEY wrote: Show quoted text
> To make it easier to review and implement I've put all changes into a > single small patch. > > This is a complete w/ Changelog, POD, tests, and version 1.13_82 ->
1.13_83 Show quoted text
> > The POD describes when this feature would be necessary.
A ready-to-upload-via-pause tarball of said patch
Download Locale-Maketext-1.13_83.tar.gz
application/x-gzip 48.3k

Message body not shown because it is not plain text.

From: toddr [...] null.net
Download (untitled) / with headers
text/plain 514b
On Thu Jul 30 14:20:05 2009, DMUEY wrote: Show quoted text
> To make it easier to review and implement I've put all changes into a > single small patch. > > This is a complete w/ Changelog, POD, tests, and version 1.13_82 ->
1.13_83 Show quoted text
> > The POD describes when this feature would be necessary.
forked perl blead on github - http://github.com/toddr/perl/tree/toddr/maketext-locale will be submitting to P5P post 5.12.0 freeze per advice from jessee http://github.com/toddr/perl/commit/52527e24a19e76942f10372430 dc63c2a7df4fd0
Download (untitled) / with headers
text/plain 1.1k
Accepted into blead by p5p commit ace47d680c1383b41a705467fadb2c64e7f39c71 Author: Todd Rinaldo <toddr@cpanel.net> Date: Tue Jul 6 01:28:00 2010 -0400 Locale::Maketext external cache support This patch with tests provides RO support for lexicon hashes in Locale::Maketext. This allows you to have GDBM language files owned by root which can be accessed by non-root, but not altered. If your lexicon is a tied hash the simple act of caching the compiled value can be fatal. For example a GDBM_File GDBM_READER tied hash will die with something like: gdbm store returned -1, errno 2, key "..." at ... All you need to do is turn on caching outside of the lexicon hash itself like so: sub init { my ($lh) = @_; ... $lh->{'use_external_lex_cache'} = 1; ... } And then instead of storing the compiled value in the lexicon hash it will store it in $lh->{'_external_lex_cache'} I've verified that blead is the authoritative location for Locale::Maketext source. Signed-off-by: David Golden <dagolden@cpan.org>
1.15_02 is the RC for 1.16. No issues so far. closing ticket.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.