This queue is for tickets about the Params-Check CPAN distribution.

Report information
The Basics
Id:
27161
Status:
open
Priority:
Low/Low
Queue:

People
Owner:
Nobody in particular
Requestors:
DAMS [...] cpan.org
Cc:
AdminCc:



Subject: Enhancement proposal
Hi, I've worked on a reimplementation of this module for use in my company. Here is the version I ended with, that is backward compatible, contains the existing features, add 1 new feature, has the same code size, but is (sometimes really) faster. Attached are : - My implementation (file Check.pm) - The patch agains 0.26 version (file faster_reimpl.patch) - A benchmark file (file benchmark.pl) - The benchmark result, that I have commented (file result.txt) The new feature is that if you ask the template to store the value into a reference on a hash or array, then it's stored directyl in the array or hash. Thus you can write : my @array; check({ foo => { default => [], store => \@array} }, { foo => [ 42 ] } # @array is ( 42 ); Enjoy ! :) dams
Subject: result.txt
*** store2 Rate orig new orig 28169/s -- -18% new 34483/s 22% -- *** subs1 Rate orig new orig 25316/s -- -37% new 40000/s 58% -- # no difference, same code is run *** no_override Rate orig new orig 16129/s -- -8% new 17544/s 9% -- # first run is valid arg, second is broken arg # new code is optimized for valid args *** strict2 Rate orig new orig 27397/s -- -40% new 45455/s 66% -- Rate orig new orig 19231/s -- -13% new 21978/s 14% -- *** edge_case Rate orig new orig 37975/s -- -35% new 58824/s 55% -- *** preserver_case2 Rate orig new orig 15385/s -- -11% new 17241/s 12% -- # first run is valid arg, second is broken arg # new code is optimized for valid args *** required Rate orig new orig 29851/s -- -40% new 50000/s 67% -- Rate orig new orig 21739/s -- -4% new 22727/s 5% -- # first run is valid arg, second is broken arg # new code is optimized for valid args *** strict1 Rate orig new orig 26316/s -- -38% new 42553/s 62% -- Rate orig new orig 19048/s -- -12% new 21739/s 14% -- *** subs3 Rate orig new orig 25641/s -- -40% new 42553/s 66% -- *** unknown1 Rate orig new orig 18018/s -- -19% new 22222/s 23% -- *** r1 Rate orig new orig 6061/s -- -36% new 9434/s 56% -- *** real_life2 Rate orig new orig 5848/s -- -37% new 9259/s 58% -- # first run is valid arg, second is broken arg # new code is optimized for valid args *** define1 Rate orig new orig 27778/s -- -39% new 45455/s 64% -- Rate orig new orig 21053/s -- -14% new 24390/s 16% -- *** real_life Rate orig new orig 6993/s -- -14% new 8130/s 16% -- *** invalid_store Rate orig new orig 43103/s -- -34% new 64935/s 51% -- # they are all failing test, so not that much diff *** invalid_key_tests Rate orig new orig 14493/s -- -16% new 17241/s 19% -- Rate orig new orig 14493/s -- -12% new 16393/s 13% -- Rate orig new orig 14085/s -- -13% new 16129/s 15% -- Rate orig new orig 13699/s -- -12% new 15625/s 14% -- # last case is broken arg # new code is optimized for valid args *** default Rate orig new orig 38961/s -- -36% new 61224/s 57% -- Rate orig new orig 29412/s -- -40% new 49180/s 67% -- Rate orig new orig 28302/s -- -24% new 37037/s 31% -- Rate orig new orig 15873/s -- -14% new 18519/s 17% -- *** big_template Rate orig new orig 5348/s -- -22% new 6897/s 29% -- # Huge gain here, with ALLOW_UNKNOWN = 1 *** unknown2 Rate orig new orig 35088/s -- -47% new 66667/s 90% -- *** store1 Rate orig new orig 28571/s -- -11% new 32258/s 13% -- *** simple Rate orig new orig 30612/s -- -38% new 49180/s 61% -- *** r2 Rate orig new orig 5814/s -- -38% new 9346/s 61% -- *** preserver_case1 Rate orig new orig 32258/s -- -31% new 46512/s 44% -- # first run is valid arg, second is broken arg # new code is optimized for valid args *** define2 Rate orig new orig 28571/s -- -39% new 46512/s 63% -- Rate orig new orig 21739/s -- -14% new 25316/s 16% -- *** subs2 Rate orig new orig 25316/s -- -38% new 40816/s 61% --
Subject: benchmark.pl
#!/usr/bin/perl use strict; use warnings; use Benchmark qw(cmpthese); use lib qw(/home/dams/scratchpad/Params-Check-0.26/lib); use Params::Check; use Params::Check2; my %tests = ( real_life => { count => 10000, code_in_func => 'my $foo1; my $foo2; my $foo3', template => q({ arg1 => { defined => 1, strict_type => 1, default => '' }, arg2 => { default => 'bar', store => \$foo1 }, arg3 => { default => 'mod', store => \$foo2 }, arg4 => { defined => 1, strict_type => 1, default => [], store => \$foo3 }, arg5 => { default => {}, strict_type => 1 }, arg6 => { default => 'a', allow => [ qw(b a c d) ], strict_type => 1}, }), params => [q({ arg1 => 'foo', arg2 => 'moz', arg4 => [ 'answer', 42 ], arg5 => { answer => 42 }, arg6 => 'c', })], }, real_life2 => { count => 10000, code => '$Params::Check::PRESERVE_CASE= 0; $Params::Check::ALLOW_UNKNOWN = 1; $Params::Check::SANITY_CHECK_TEMPLATE = 0; ', template => q({ arg1 => { defined => 1, strict_type => 1, default => ''}, arg2 => { default => 'bar'}, arg3 => { default => 'mod'}, arg4 => { defined => 1, strict_type => 1, default => ''}, arg5 => { default => 'bar'}, arg6 => { default => 'mod'}, arg7 => { defined => 1, strict_type => 1, default => ''}, arg8 => { default => 'bar'}, arg9 => { default => 'mod'}, }), params => [q({ arg1 => 'foo', arg2 => 'moz', arg3 => 'answer', arg4 => 'foo', arg5 => 'moz', arg6 => 'answer', arg7 => 'foo', arg8 => 'moz', arg9 => 'answer', })], }, r1 => { count => 10000, template => q({ arg1 => { defined => 1, strict_type => 1, default => ''}, arg2 => { default => 'bar'}, arg3 => { default => 'mod'}, arg4 => { defined => 1, strict_type => 1, default => ''}, arg5 => { default => 'bar'}, arg6 => { default => 'mod'}, arg7 => { defined => 1, strict_type => 1, default => ''}, arg8 => { default => 'bar'}, arg9 => { default => 'mod'}, }), params => [q({ arg1 => 'foo', arg2 => 'moz', arg3 => 'answer', arg4 => 'foo', arg5 => 'moz', arg6 => 'answer', arg7 => 'foo', arg8 => 'moz', arg9 => 'answer', })], }, r2 => { count => 10000, code => '$Params::Check::SANITY_CHECK_TEMPLATE = 0; ', template => q({ arg1 => { defined => 1, strict_type => 1, default => ''}, arg2 => { default => 'bar'}, arg3 => { default => 'mod'}, arg4 => { defined => 1, strict_type => 1, default => ''}, arg5 => { default => 'bar'}, arg6 => { default => 'mod'}, arg7 => { defined => 1, strict_type => 1, default => ''}, arg8 => { default => 'bar'}, arg9 => { default => 'mod'}, }), params => [q({ arg1 => 'foo', arg2 => 'moz', arg3 => 'answer', arg4 => 'foo', arg5 => 'moz', arg6 => 'answer', arg7 => 'foo', arg8 => 'moz', arg9 => 'answer', })], }, simple => { count => 30000, template => '{ arg1 => {}, }', params => ['{ arg1 => 1, }'], }, default => { count => 30000, template => '{ foo => { default => 1 } }', params => [ '{}', '{ foo => 2}', '{ FOO => 2}', '{ -foo => 2}', ], }, no_override => { count => 10000, template => '{ foo => { no_override => 1, default => 42 } }', params => [ '{ foo => 13 }', ], }, required => { count => 20000, template => '{ foo => { required => 1 } }', params => [ '{ foo => 42 }', '{ }', ], }, invalid_key_tests => { count => 10000, template => '{ foo => { allow => sub { 0 } } }', params => [ '{ foo => 1 }', '{ foo => "foo" }', '{ foo => [] }', '{ foo => bless({},__PACKAGE__) }', ], }, invalid_store => { count => 50000, template => q({ foo => { store => '' } }), params => [ '{ }', ], }, edge_case => { count => 30000, template => q({ foo => { default => '' } }), params => [ '{ }', ], }, big_template => { count => 10000, template => q({ firstname => { required => 1, defined => 1 }, lastname => { required => 1, store => \$lastname }, gender => { required => 1, allow => [qr/M/i, qr/F/i], }, married => { allow => [0,1] }, age => { default => 21, allow => qr/^\d+$/, }, id_list => { default => [], strict_type => 1 }, phone => { allow => sub { 1 if +shift } }, bureau => { default => 'NSA', no_override => 1 }, }), params => [ q({ firstname => 'joe', lastname => 'jackson', gender => 'M', married => 1, age => 21, id_list => [1..3], phone => '555-8844', }), ], }, preserver_case1 => { count => 20000, code => '$Params::Check::PRESERVE_CASE = 1', template => q({ Foo => { default => 1 } }), params => [ '{ Foo => 42 }', ], }, preserver_case2 => { count => 20000, code => '$Params::Check::PRESERVE_CASE = 0', template => q({ Foo => { default => 1 } }), params => [ '{ Foo => 42 }', ], }, unknown1 => { count => 20000, template => q({ }), params => [ '{ foo => 42 }', ], }, unknown2 => { count => 20000, code => '$Params::Check::ALLOW_UNKNOWN = 1', template => q({ }), params => [ '{ foo => 42 }', ], }, store1 => { count => 20000, code => '$Params::Check::NO_DUPLICATES = 1; my $foo;', template => q({ foo => { store => \$foo } }), params => [ '{ foo => 42 }', ], }, store2 => { count => 20000, code => '$Params::Check::NO_DUPLICATES = 0; my $foo;', template => q({ foo => { store => \$foo } }), params => [ '{ foo => 42 }', ], }, strict1 => { count => 20000, code => '$Params::Check::STRICT_TYPE = 0', template => q({ foo => { strict_type => 1, default => [] } }), params => [ '{ foo => [] }', '{ foo => {} }', ], }, strict2 => { count => 20000, code => '$Params::Check::STRICT_TYPE = 1', template => q({ foo => { default => [] } }), params => [ '{ foo => [] }', '{ foo => {} }', ], }, define1 => { count => 20000, code => '$Params::Check::ONLY_ALLOW_DEFINED = 0', template => q({ foo => { defined => 1, default => 1 } }), params => [ '{ foo => 42 }', '{ foo => undef }', ], }, define2 => { count => 20000, code => '$Params::Check::ONLY_ALLOW_DEFINED = 1', template => q({ foo => { default => 1 } }), params => [ '{ foo => 42 }', '{ foo => undef }', ], }, subs1 => { count => 20000, template => q({ foo => { allow => sub { [] } } }), params => [ '{ foo => [] }', ], }, subs2 => { count => 20000, template => q({ foo => { allow => sub { {} } } }), params => [ '{ foo => {} }', ], }, subs3 => { count => 20000, template => q({ foo => { allow => sub { 1 } } }), params => [ '{ foo => 1 }', ], }, ); #my @keep = qw(store1 store2); #my @keep = qw(r1 r2); #my @keep = qw(real_life real_life2); #if (@keep) { # my %f; # @f{@keep} = @tests{@keep}; # %tests = %f; #} while (my ($name, $data) = each(%tests)) { my ($count, $template, $params, $code, $code_in_func) = @{$data}{qw(count template params code code_in_func)}; local $Params::Check::VERBOSE; local $Params::Check::NO_DUPLICATES; local $Params::Check::STRIP_LEADING_DASHES; local $Params::Check::STRICT_TYPE; local $Params::Check::ALLOW_UNKNOWN; local $Params::Check::PRESERVE_CASE; local $Params::Check::ONLY_ALLOW_DEFINED; local $Params::Check::SANITY_CHECK_TEMPLATE; local $Params::Check::WARNINGS_FATAL; local $Params::Check::CALLER_DEPTH; local $Params::Check2::VERBOSE; local $Params::Check2::NO_DUPLICATES; local $Params::Check2::STRIP_LEADING_DASHES; local $Params::Check2::STRICT_TYPE; local $Params::Check2::ALLOW_UNKNOWN; local $Params::Check2::PRESERVE_CASE; local $Params::Check2::ONLY_ALLOW_DEFINED; local $Params::Check2::SANITY_CHECK_TEMPLATE; local $Params::Check2::WARNINGS_FATAL; local $Params::Check2::CALLER_DEPTH; $code_in_func ||= ''; $code ||= ''; my $code_original = $code; my $code_new = $code; my $code_new3 = $code; $code_new =~ s/Check/Check2/g; $code_new3 =~ s/Check/Check3/g; print "*** $name\n"; my $i = 0; foreach (@$params) { local $SIG{__WARN__} = sub {}; cmpthese($count, { orig => qq( $code_original; foo($_); sub foo { $code_in_func; Params::Check::check($template, \$_[0]) } ), new => qq( $code_new; foo($_); sub foo { $code_in_func; Params::Check2::check($template, \$_[0]) } ), }); } print "\n"; }
Subject: faster_reimpl.patch
--- Check__.pm 2007-05-14 18:58:41.000000000 +0100 +++ Check.pm 2007-05-16 20:16:55.000000000 +0100 @@ -5,8 +5,6 @@ use Carp qw[carp croak]; use Locale::Maketext::Simple Style => 'gettext'; -use Data::Dumper; - BEGIN { use Exporter (); use vars qw[ @ISA $VERSION @EXPORT_OK $VERBOSE $ALLOW_UNKNOWN @@ -179,9 +177,20 @@ This is basically shorthand for saying: - my $args = check( { foo => { default => 1 }, $input ); + my $args = check( { foo => { default => 1 } }, $input ); my $x = $args->{foo}; +It works for arrays or hash reference too. You can write : + + my @array; + my %hash; + my $args = check(foo => { default => [ 1 ], store => \@array }, + bar => { default => { answer => 42 }, store => \%hash }, + $input); + +And @array and %hash contains directly the corresponding array or hash dereferenced. + + You can alter the global variable $Params::Check::NO_DUPLICATES to control whether the C<store>'d key will still be present in your result set. See the L<Global Variables> section below. @@ -244,135 +253,253 @@ =cut -sub check { - my ($utmpl, $href, $verbose) = @_; - ### did we get the arguments we need? ### - return if !$utmpl or !$href; +{ + +my @to_store; +my @to_delete; - ### sensible defaults ### - $verbose ||= $VERBOSE || 0; +sub check { + # for speed purpose we don't copy @_; check if we have anything to work on + if (!$_[0] || !$_[1]) { + return; + } - ### clear the current error string ### + my %template = %{$_[0]}; + my %args = %{$_[1]}; + my $verbose = $_[2] || $VERBOSE || 0; + + # clear current error _clear_error(); - ### XXX what type of template is it? ### - ### { key => { } } ? - #if (ref $args eq 'HASH') { - # 1; - #} - - ### clean up the template ### - my $args = _clean_up_args( $href ) or return; - - ### sanity check + defaults + required keys set? ### - my $defs = _sanity_check_and_defaults( $utmpl, $args, $verbose ) - or return; - - ### deref only once ### - my %utmpl = %$utmpl; - my %args = %$args; - my %defs = %$defs; + # flag to see if we warned for anything, needed for warnings_fatal + my $warned; - ### flag to see if anything went wrong ### + # flag to see if anything went wrong my $wrong; - - ### flag to see if we warned for anything, needed for warnings_fatal - my $warned; - for my $key (keys %args) { + # key to remove from the args, if unauthorised + my @keys_to_remove = (); - ### you gave us this key, but it's not in the template ### - unless( $utmpl{$key} ) { + # keys to rename : [ old_key_name, new_key_name] + my @keys_to_rename = (); - ### but we'll allow it anyway ### - if( $ALLOW_UNKNOWN ) { - $defs{$key} = $args{$key}; + # list of values to store into ref : [ $type, $ref, $value ] + @to_store = (); - ### warn about the error ### - } else { + # list of keys to delete from args + @to_delete = (); + + # loop on the arguments + while (my ($arg_key, $arg_value) = each %args) { + + # handle key name + if (!$PRESERVE_CASE || $STRIP_LEADING_DASHES) { + my $orig_arg_key = $arg_key; + $arg_key = lc($arg_key) unless $PRESERVE_CASE; + $arg_key =~ s/^-// if $STRIP_LEADING_DASHES; + if ($arg_key ne $orig_arg_key) { + push @keys_to_rename, [ $arg_key, $orig_arg_key ]; + } + } + + # the argument doesn't exist in the template + if ( !exists $template{$arg_key} ) { + if (!$ALLOW_UNKNOWN) { _store_error( - loc("Key '%1' is not a valid key for %2 provided by %3", - $key, _who_was_it(), _who_was_it(1)), $verbose); - $warned ||= 1; + loc(q(Key '%1' is not a valid key for %2 provided by %3), + $arg_key, _who_was_it(), _who_was_it(1)), $verbose); + $warned = 1; + push @keys_to_remove, $arg_key; } next; + }; + + # copy of this keys template instructions, to save derefs + my %arg_template = %{delete $template{$arg_key} }; + + if ($SANITY_CHECK_TEMPLATE) { + foreach(grep { ! $known_keys{$_} } keys %arg_template) { + _store_error(loc(q(Template type '%1' not supported [at key '%2']), + $_, $arg_key), $verbose) + } } - ### check if you're even allowed to override this key ### - if( $utmpl{$key}->{'no_override'} ) { + # the argument cannot be overridden + if ($arg_template{no_override}) { _store_error( - loc(q[You are not allowed to override key '%1']. - q[for %2 from %3], $key, _who_was_it(), _who_was_it(1)), + loc(q(You are not allowed to override key '%1' for %2 from %3), + $arg_key, _who_was_it(), _who_was_it(1)), $verbose ); - $warned ||= 1; + $warned = 1; + push @keys_to_remove, $arg_key; + $template{$arg_key} = \%arg_template; next; } - ### copy of this keys template instructions, to save derefs ### - my %tmpl = %{$utmpl{$key}}; + # check if you were supposed to provide defined() values + if ( ($arg_template{defined} || $ONLY_ALLOW_DEFINED) && !defined $arg_value ) { + _store_error(loc(q(Key '%1' must be defined when passed), $arg_key), + $verbose ); + $wrong = 1; + push @keys_to_remove, $arg_key; + $template{$arg_key} = \%arg_template; + next; + } - ### check if you were supposed to provide defined() values ### - if( ($tmpl{'defined'} || $ONLY_ALLOW_DEFINED) and - not defined $args{$key} - ) { - _store_error(loc(q|Key '%1' must be defined when passed|, $key), - $verbose ); - $wrong ||= 1; + # check if they should be of a strict type, and if it is + if ( ($arg_template{strict_type} || $STRICT_TYPE) && ref $arg_value ne ref $arg_template{default}) { + _store_error(loc(q(Key '%1' needs to be of type '%2'), + $arg_key, ref($arg_template{default}) || 'SCALAR'), + $verbose ); + $wrong = 1; + push @keys_to_remove, $arg_key; + $template{$arg_key} = \%arg_template; next; } - ### check if they should be of a strict type, and if it is ### - if( ($tmpl{'strict_type'} || $STRICT_TYPE) and - (ref $args{$key} ne ref $tmpl{'default'}) - ) { - _store_error(loc(q|Key '%1' needs to be of type '%2'|, - $key, ref $tmpl{'default'} || 'SCALAR'), $verbose ); - $wrong ||= 1; + # check if we have an allow handler, to validate against + # allow() will report its own errors + if (exists $arg_template{allow} && !do { + local $_ERROR_STRING; + allow($arg_value, $arg_template{allow}) + }) { + # stringify the value in the error report -- we don't want dumps + # of objects, but we do want to see *roughly* what we passed + _store_error(loc(q(Key '%1' (%2) is of invalid type for '%3' provided by %4), + $arg_key, $arg_value, _who_was_it(), + _who_was_it(1)), $verbose); + $wrong = 1; + push @keys_to_remove, $arg_key; + $template{$arg_key} = \%arg_template; next; } - ### check if we have an allow handler, to validate against ### - ### allow() will report its own errors ### - if( exists $tmpl{'allow'} and not do { - local $_ERROR_STRING; - allow( $args{$key}, $tmpl{'allow'} ) - } - ) { - ### stringify the value in the error report -- we don't want dumps - ### of objects, but we do want to see *roughly* what we passed - _store_error(loc(q|Key '%1' (%2) is of invalid type for '%3' |. - q|provided by %4|, - $key, "$args{$key}", _who_was_it(), - _who_was_it(1)), $verbose); - $wrong ||= 1; + # check if we need to store the argument value to a provided ref + if (my $ref = $arg_template{store}) { + if ( !_store_var($arg_key, $ref, $arg_value, $verbose)) { + $wrong = 1; + next; + } + } + } + + + # if we needed to rename keys + foreach (@keys_to_rename) { + $args{$_->[0]} = delete $args{$_->[1]}; + } + + # if we needed to remove unknown keys, so that default applies + if (@keys_to_remove) { + delete @args{@keys_to_remove}; + } + + # now check if there is any key left in the template + while (my ($t_key, $t_value) = each %template) { + + # check if required key is missing + if ($t_value->{required}) { + _store_error( + loc(q(Required option '%1' is not provided for %2 by %3), + $t_key, _who_was_it(), _who_was_it(1)), $verbose ); + $wrong = 1; next; } - ### we got here, then all must be OK ### - $defs{$key} = $args{$key}; + # set default argument omitted + if (exists $t_value->{default}) { + $args{$t_key} = $t_value->{default}; + # check if we need to store the default value to a provided ref + if (my $ref = $t_value->{store}) { + if (!_store_var($t_key, $ref, $t_value->{default}, $verbose)) { + $wrong = 1; + next; + } + } + } + # special case to be backward compatible + if ($SANITY_CHECK_TEMPLATE && exists $t_value->{store} && !ref $t_value->{store} ) { + _store_error( loc( + q(Store variable for '%1' is not a reference!), $t_key + ), $verbose); + } } - ### croak with the collected errors if there were errors and - ### we have the fatal flag toggled. - croak(__PACKAGE__->last_error) if ($wrong || $warned) && $WARNINGS_FATAL; + # croak with the collected errors if there were errors and we have the + # fatal flag toggled. + if ( ($wrong || $warned) && $WARNINGS_FATAL) { + croak(__PACKAGE__->last_error()); + } - ### done with our loop... if $wrong is set, somethign went wrong - ### and the user is already informed, just return... + # if $wrong is set, somethign went wrong and the user is already informed, + # just return... return if $wrong; - ### check if we need to store any of the keys ### - ### can't do it before, because something may go wrong later, - ### leaving the user with a few set variables - for my $key (keys %defs) { - if( my $ref = $utmpl{$key}->{'store'} ) { - $$ref = $NO_DUPLICATES ? delete $defs{$key} : $defs{$key}; + # check if we need to store any of the keys. can't do it before, because + # something may go wrong later, leaving the user with a few set variables + + foreach(@to_store) { + my ($type, $ref, $value) = @$_; + if ($type == 0) { + $$ref = $value; + } + elsif ($type == 1) { + @{$ref} = @{$value}; } + elsif ($type == 2) { + %{$ref} = %{$value}; + } + } + $NO_DUPLICATES and delete @args{@to_delete}; + + # now, everything is fine, we can return the arguments + return(\%args); +} + +sub _store_var { + my ($key, $ref, $value, $verbose) = @_; + + if ($SANITY_CHECK_TEMPLATE && !ref($ref)) { + _store_error( loc( + q(Store variable for '%1' is not a reference!), $key + ), $verbose, 1 ); + return; #error } + + if (ref($ref) eq 'ARRAY') { + if (ref($value) ne 'ARRAY') { + _store_error( + loc(q(Key '%1' (value %2) is not a ARRAYREF. For %3 by %4), + $key, $value, _who_was_it(1), _who_was_it(2)), $verbose, 1); + return; # error + } + # push the refs/values to execute later + push @to_store, [ 1, $ref, $value]; # 1 = array + $NO_DUPLICATES and push @to_delete, $key; + } + elsif (ref($ref) eq 'HASH') { + if (ref($value) ne 'HASH') { + _store_error( + loc(q(Key '%1' (value %2) is not a HASHREF. For %3 by %4), + $key, $value, _who_was_it(1), _who_was_it(2)), $verbose, 1); + return; # error + } + # push the refs/values to execute later + push @to_store, [ 2, $ref, $value]; # 2 = hash + $NO_DUPLICATES and push @to_delete, $key; + } + else { + # push the refs/values to execute later + push @to_store, [ 0, $ref, $value]; # 0 = scalar ref + $NO_DUPLICATES and push(@to_delete, $key); + } + + return 1; # success +} - return \%defs; } =head2 allow( $test_me, \@criteria ); @@ -418,121 +545,40 @@ =cut sub allow { - ### use $_[0] and $_[1] since this is hot code... ### - #my ($val, $ref) = @_; - ### it's a regexp ### - if( ref $_[1] eq 'Regexp' ) { - local $^W; # silence warnings if $val is undef # - return if $_[0] !~ /$_[1]/; - - ### it's a sub ### - } elsif ( ref $_[1] eq 'CODE' ) { - return unless $_[1]->( $_[0] ); - - ### it's an array ### - } elsif ( ref $_[1] eq 'ARRAY' ) { - - ### loop over the elements, see if one of them says the - ### value is OK - ### also, short-cicruit when possible - for ( @{$_[1]} ) { - return 1 if allow( $_[0], $_ ); - } - - return; - - ### fall back to a simple, but safe 'eq' ### - } else { - return unless _safe_eq( $_[0], $_[1] ); + # it's a regexp + if (ref($_[1]) eq 'Regexp') { + no warnings; + return(scalar $_[0] =~ /$_[1]/); ## no critic (Regular expression) } - ### we got here, no failures ### - return 1; -} - -### helper functions ### - -### clean up the template ### -sub _clean_up_args { - ### don't even bother to loop, if there's nothing to clean up ### - return $_[0] if $PRESERVE_CASE and !$STRIP_LEADING_DASHES; - - my %args = %{$_[0]}; - - ### keys are note aliased ### - for my $key (keys %args) { - my $org = $key; - $key = lc $key unless $PRESERVE_CASE; - $key =~ s/^-// if $STRIP_LEADING_DASHES; - $args{$key} = delete $args{$org} if $key ne $org; - } - - ### return references so we always return 'true', even on empty - ### arguments - return \%args; -} - -sub _sanity_check_and_defaults { - my %utmpl = %{$_[0]}; - my %args = %{$_[1]}; - my $verbose = $_[2]; - - my %defs; my $fail; - for my $key (keys %utmpl) { - - ### check if required keys are provided - ### keys are now lower cased, unless preserve case was enabled - ### at which point, the utmpl keys must match, but that's the users - ### problem. - if( $utmpl{$key}->{'required'} and not exists $args{$key} ) { - _store_error( - loc(q|Required option '%1' is not provided for %2 by %3|, - $key, _who_was_it(1), _who_was_it(2)), $verbose ); - - ### mark the error ### - $fail++; - next; - } + # it's a sub + if (ref($_[1]) eq 'CODE') { + return $_[1]->($_[0]); + } + + # it's an array + if (ref($_[1])eq 'ARRAY') { - ### next, set the default, make sure the key exists in %defs ### - $defs{$key} = $utmpl{$key}->{'default'} - if exists $utmpl{$key}->{'default'}; - - if( $SANITY_CHECK_TEMPLATE ) { - ### last, check if they provided any weird template keys - ### -- do this last so we don't always execute this code. - ### just a small optimization. - map { _store_error( - loc(q|Template type '%1' not supported [at key '%2']|, - $_, $key), 1, 1 ); - } grep { - not $known_keys{$_} - } keys %{$utmpl{$key}}; - - ### make sure you passed a ref, otherwise, complain about it! - if ( exists $utmpl{$key}->{'store'} ) { - _store_error( loc( - q|Store variable for '%1' is not a reference!|, $key - ), 1, 1 ) unless ref $utmpl{$key}->{'store'}; + # loop over the elements, see if one of them says the + # value is OK + # also, short-cicruit when possible + foreach (@{$_[1]}) { + if (allow($_[0], $_)) { + return 1; } } + return; } - ### errors found ### - return if $fail; - - ### return references so we always return 'true', even on empty - ### defaults - return \%defs; + # fall back to a simple, but safe 'eq' + return (defined $_[0] && defined $_[1] + ? $_[0] eq $_[1] + : defined $_[0] eq defined $_[1] + ); } -sub _safe_eq { - ### only do a straight 'eq' if they're both defined ### - return defined($_[0]) && defined($_[1]) - ? $_[0] eq $_[1] - : defined($_[0]) eq defined($_[1]); -} +# helper functions sub _who_was_it { my $level = $_[0] || 0;
Subject: Check.pm
package Params::Check; use strict; use Carp qw[carp croak]; use Locale::Maketext::Simple Style => 'gettext'; BEGIN { use Exporter (); use vars qw[ @ISA $VERSION @EXPORT_OK $VERBOSE $ALLOW_UNKNOWN $STRICT_TYPE $STRIP_LEADING_DASHES $NO_DUPLICATES $PRESERVE_CASE $ONLY_ALLOW_DEFINED $WARNINGS_FATAL $SANITY_CHECK_TEMPLATE $CALLER_DEPTH $_ERROR_STRING ]; @ISA = qw[ Exporter ]; @EXPORT_OK = qw[check allow last_error]; $VERSION = '0.26'; $VERBOSE = $^W ? 1 : 0; $NO_DUPLICATES = 0; $STRIP_LEADING_DASHES = 0; $STRICT_TYPE = 0; $ALLOW_UNKNOWN = 0; $PRESERVE_CASE = 0; $ONLY_ALLOW_DEFINED = 0; $SANITY_CHECK_TEMPLATE = 1; $WARNINGS_FATAL = 0; $CALLER_DEPTH = 0; } my %known_keys = map { $_ => 1 } qw| required allow default strict_type no_override store defined |; =pod =head1 NAME Params::Check - A generic input parsing/checking mechanism. =head1 SYNOPSIS use Params::Check qw[check allow last_error]; sub fill_personal_info { my %hash = @_; my $x; my $tmpl = { firstname => { required => 1, defined => 1 }, lastname => { required => 1, store => \$x }, gender => { required => 1, allow => [qr/M/i, qr/F/i], }, married => { allow => [0,1] }, age => { default => 21, allow => qr/^\d+$/, }, phone => { allow => [ sub { return 1 if /$valid_re/ }, '1-800-PERL' ] }, id_list => { default => [], strict_type => 1 }, employer => { default => 'NSA', no_override => 1 }, }; ### check() returns a hashref of parsed args on success ### my $parsed_args = check( $tmpl, \%hash, $VERBOSE ) or die qw[Could not parse arguments!]; ... other code here ... } my $ok = allow( $colour, [qw|blue green yellow|] ); my $error = Params::Check::last_error(); =head1 DESCRIPTION Params::Check is a generic input parsing/checking mechanism. It allows you to validate input via a template. The only requirement is that the arguments must be named. Params::Check can do the following things for you: =over 4 =item * Convert all keys to lowercase =item * Check if all required arguments have been provided =item * Set arguments that have not been provided to the default =item * Weed out arguments that are not supported and warn about them to the user =item * Validate the arguments given by the user based on strings, regexes, lists or even subroutines =item * Enforce type integrity if required =back Most of Params::Check's power comes from its template, which we'll discuss below: =head1 Template As you can see in the synopsis, based on your template, the arguments provided will be validated. The template can take a different set of rules per key that is used. The following rules are available: =over 4 =item default This is the default value if none was provided by the user. This is also the type C<strict_type> will look at when checking type integrity (see below). =item required A boolean flag that indicates if this argument was a required argument. If marked as required and not provided, check() will fail. =item strict_type This does a C<ref()> check on the argument provided. The C<ref> of the argument must be the same as the C<ref> of the default value for this check to pass. This is very useful if you insist on taking an array reference as argument for example. =item defined If this template key is true, enforces that if this key is provided by user input, its value is C<defined>. This just means that the user is not allowed to pass C<undef> as a value for this key and is equivalent to: allow => sub { defined $_[0] && OTHER TESTS } =item no_override This allows you to specify C<constants> in your template. ie, they keys that are not allowed to be altered by the user. It pretty much allows you to keep all your C<configurable> data in one place; the C<Params::Check> template. =item store This allows you to pass a reference to a scalar, in which the data will be stored: my $x; my $args = check(foo => { default => 1, store => \$x }, $input); This is basically shorthand for saying: my $args = check( { foo => { default => 1 } }, $input ); my $x = $args->{foo}; It works for arrays or hash reference too. You can write : my @array; my %hash; my $args = check(foo => { default => [ 1 ], store => \@array }, bar => { default => { answer => 42 }, store => \%hash }, $input); And @array and %hash contains directly the corresponding array or hash dereferenced. You can alter the global variable $Params::Check::NO_DUPLICATES to control whether the C<store>'d key will still be present in your result set. See the L<Global Variables> section below. =item allow A set of criteria used to validate a particular piece of data if it has to adhere to particular rules. See the C<allow()> function for details. =back =head1 Functions =head2 check( \%tmpl, \%args, [$verbose] ); This function is not exported by default, so you'll have to ask for it via: use Params::Check qw[check]; or use its fully qualified name instead. C<check> takes a list of arguments, as follows: =over 4 =item Template This is a hashreference which contains a template as explained in the C<SYNOPSIS> and C<Template> section. =item Arguments This is a reference to a hash of named arguments which need checking. =item Verbose A boolean to indicate whether C<check> should be verbose and warn about what went wrong in a check or not. You can enable this program wide by setting the package variable C<$Params::Check::VERBOSE> to a true value. For details, see the section on C<Global Variables> below. =back C<check> will return when it fails, or a hashref with lowercase keys of parsed arguments when it succeeds. So a typical call to check would look like this: my $parsed = check( \%template, \%arguments, $VERBOSE ) or warn q[Arguments could not be parsed!]; A lot of the behaviour of C<check()> can be altered by setting package variables. See the section on C<Global Variables> for details on this. =cut { my @to_store; my @to_delete; sub check { # for speed purpose we don't copy @_; check if we have anything to work on if (!$_[0] || !$_[1]) { return; } my %template = %{$_[0]}; my %args = %{$_[1]}; my $verbose = $_[2] || $VERBOSE || 0; # clear current error _clear_error(); # flag to see if we warned for anything, needed for warnings_fatal my $warned; # flag to see if anything went wrong my $wrong; # key to remove from the args, if unauthorised my @keys_to_remove = (); # keys to rename : [ old_key_name, new_key_name] my @keys_to_rename = (); # list of values to store into ref : [ $type, $ref, $value ] @to_store = (); # list of keys to delete from args @to_delete = (); # loop on the arguments while (my ($arg_key, $arg_value) = each %args) { # handle key name if (!$PRESERVE_CASE || $STRIP_LEADING_DASHES) { my $orig_arg_key = $arg_key; $arg_key = lc($arg_key) unless $PRESERVE_CASE; $arg_key =~ s/^-// if $STRIP_LEADING_DASHES; if ($arg_key ne $orig_arg_key) { push @keys_to_rename, [ $arg_key, $orig_arg_key ]; } } # the argument doesn't exist in the template if ( !exists $template{$arg_key} ) { if (!$ALLOW_UNKNOWN) { _store_error( loc(q(Key '%1' is not a valid key for %2 provided by %3), $arg_key, _who_was_it(), _who_was_it(1)), $verbose); $warned = 1; push @keys_to_remove, $arg_key; } next; }; # copy of this keys template instructions, to save derefs my %arg_template = %{delete $template{$arg_key} }; if ($SANITY_CHECK_TEMPLATE) { foreach(grep { ! $known_keys{$_} } keys %arg_template) { _store_error(loc(q(Template type '%1' not supported [at key '%2']), $_, $arg_key), $verbose) } } # the argument cannot be overridden if ($arg_template{no_override}) { _store_error( loc(q(You are not allowed to override key '%1' for %2 from %3), $arg_key, _who_was_it(), _who_was_it(1)), $verbose ); $warned = 1; push @keys_to_remove, $arg_key; $template{$arg_key} = \%arg_template; next; } # check if you were supposed to provide defined() values if ( ($arg_template{defined} || $ONLY_ALLOW_DEFINED) && !defined $arg_value ) { _store_error(loc(q(Key '%1' must be defined when passed), $arg_key), $verbose ); $wrong = 1; push @keys_to_remove, $arg_key; $template{$arg_key} = \%arg_template; next; } # check if they should be of a strict type, and if it is if ( ($arg_template{strict_type} || $STRICT_TYPE) && ref $arg_value ne ref $arg_template{default}) { _store_error(loc(q(Key '%1' needs to be of type '%2'), $arg_key, ref($arg_template{default}) || 'SCALAR'), $verbose ); $wrong = 1; push @keys_to_remove, $arg_key; $template{$arg_key} = \%arg_template; next; } # check if we have an allow handler, to validate against # allow() will report its own errors if (exists $arg_template{allow} && !do { local $_ERROR_STRING; allow($arg_value, $arg_template{allow}) }) { # stringify the value in the error report -- we don't want dumps # of objects, but we do want to see *roughly* what we passed _store_error(loc(q(Key '%1' (%2) is of invalid type for '%3' provided by %4), $arg_key, $arg_value, _who_was_it(), _who_was_it(1)), $verbose); $wrong = 1; push @keys_to_remove, $arg_key; $template{$arg_key} = \%arg_template; next; } # check if we need to store the argument value to a provided ref if (my $ref = $arg_template{store}) { if ( !_store_var($arg_key, $ref, $arg_value, $verbose)) { $wrong = 1; next; } } } # if we needed to rename keys foreach (@keys_to_rename) { $args{$_->[0]} = delete $args{$_->[1]}; } # if we needed to remove unknown keys, so that default applies if (@keys_to_remove) { delete @args{@keys_to_remove}; } # now check if there is any key left in the template while (my ($t_key, $t_value) = each %template) { # check if required key is missing if ($t_value->{required}) { _store_error( loc(q(Required option '%1' is not provided for %2 by %3), $t_key, _who_was_it(), _who_was_it(1)), $verbose ); $wrong = 1; next; } # set default argument omitted if (exists $t_value->{default}) { $args{$t_key} = $t_value->{default}; # check if we need to store the default value to a provided ref if (my $ref = $t_value->{store}) { if (!_store_var($t_key, $ref, $t_value->{default}, $verbose)) { $wrong = 1; next; } } } # special case to be backward compatible if ($SANITY_CHECK_TEMPLATE && exists $t_value->{store} && !ref $t_value->{store} ) { _store_error( loc( q(Store variable for '%1' is not a reference!), $t_key ), $verbose); } } # croak with the collected errors if there were errors and we have the # fatal flag toggled. if ( ($wrong || $warned) && $WARNINGS_FATAL) { croak(__PACKAGE__->last_error()); } # if $wrong is set, somethign went wrong and the user is already informed, # just return... return if $wrong; # check if we need to store any of the keys. can't do it before, because # something may go wrong later, leaving the user with a few set variables foreach(@to_store) { my ($type, $ref, $value) = @$_; if ($type == 0) { $$ref = $value; } elsif ($type == 1) { @{$ref} = @{$value}; } elsif ($type == 2) { %{$ref} = %{$value}; } } $NO_DUPLICATES and delete @args{@to_delete}; # now, everything is fine, we can return the arguments return(\%args); } sub _store_var { my ($key, $ref, $value, $verbose) = @_; if ($SANITY_CHECK_TEMPLATE && !ref($ref)) { _store_error( loc( q(Store variable for '%1' is not a reference!), $key ), $verbose, 1 ); return; #error } if (ref($ref) eq 'ARRAY') { if (ref($value) ne 'ARRAY') { _store_error( loc(q(Key '%1' (value %2) is not a ARRAYREF. For %3 by %4), $key, $value, _who_was_it(1), _who_was_it(2)), $verbose, 1); return; # error } # push the refs/values to execute later push @to_store, [ 1, $ref, $value]; # 1 = array $NO_DUPLICATES and push @to_delete, $key; } elsif (ref($ref) eq 'HASH') { if (ref($value) ne 'HASH') { _store_error( loc(q(Key '%1' (value %2) is not a HASHREF. For %3 by %4), $key, $value, _who_was_it(1), _who_was_it(2)), $verbose, 1); return; # error } # push the refs/values to execute later push @to_store, [ 2, $ref, $value]; # 2 = hash $NO_DUPLICATES and push @to_delete, $key; } else { # push the refs/values to execute later push @to_store, [ 0, $ref, $value]; # 0 = scalar ref $NO_DUPLICATES and push(@to_delete, $key); } return 1; # success } } =head2 allow( $test_me, \@criteria ); The function that handles the C<allow> key in the template is also available for independent use. The function takes as first argument a key to test against, and as second argument any form of criteria that are also allowed by the C<allow> key in the template. You can use the following types of values for allow: =over 4 =item string The provided argument MUST be equal to the string for the validation to pass. =item regexp The provided argument MUST match the regular expression for the validation to pass. =item subroutine The provided subroutine MUST return true in order for the validation to pass and the argument accepted. (This is particularly useful for more complicated data). =item array ref The provided argument MUST equal one of the elements of the array ref for the validation to pass. An array ref can hold all the above values. =back It returns true if the key matched the criteria, or false otherwise. =cut sub allow { # it's a regexp if (ref($_[1]) eq 'Regexp') { no warnings; return(scalar $_[0] =~ /$_[1]/); ## no critic (Regular expression) } # it's a sub if (ref($_[1]) eq 'CODE') { return $_[1]->($_[0]); } # it's an array if (ref($_[1])eq 'ARRAY') { # loop over the elements, see if one of them says the # value is OK # also, short-cicruit when possible foreach (@{$_[1]}) { if (allow($_[0], $_)) { return 1; } } return; } # fall back to a simple, but safe 'eq' return (defined $_[0] && defined $_[1] ? $_[0] eq $_[1] : defined $_[0] eq defined $_[1] ); } # helper functions sub _who_was_it { my $level = $_[0] || 0; return (caller(2 + $CALLER_DEPTH + $level))[3] || 'ANON' } =head2 last_error() Returns a string containing all warnings and errors reported during the last time C<check> was called. This is useful if you want to report then some other way than C<carp>'ing when the verbose flag is on. It is exported upon request. =cut { $_ERROR_STRING = ''; sub _store_error { my($err, $verbose, $offset) = @_[0..2]; $verbose ||= 0; $offset ||= 0; my $level = 1 + $offset; local $Carp::CarpLevel = $level; carp $err if $verbose; $_ERROR_STRING .= $err . "\n"; } sub _clear_error { $_ERROR_STRING = ''; } sub last_error { $_ERROR_STRING } } 1; =head1 Global Variables The behaviour of Params::Check can be altered by changing the following global variables: =head2 $Params::Check::VERBOSE This controls whether Params::Check will issue warnings and explanations as to why certain things may have failed. If you set it to 0, Params::Check will not output any warnings. The default is 1 when L<warnings> are enabled, 0 otherwise; =head2 $Params::Check::STRICT_TYPE This works like the C<strict_type> option you can pass to C<check>, which will turn on C<strict_type> globally for all calls to C<check>. The default is 0; =head2 $Params::Check::ALLOW_UNKNOWN If you set this flag, unknown options will still be present in the return value, rather than filtered out. This is useful if your subroutine is only interested in a few arguments, and wants to pass the rest on blindly to perhaps another subroutine. The default is 0; =head2 $Params::Check::STRIP_LEADING_DASHES If you set this flag, all keys passed in the following manner: function( -key => 'val' ); will have their leading dashes stripped. =head2 $Params::Check::NO_DUPLICATES If set to true, all keys in the template that are marked as to be stored in a scalar, will also be removed from the result set. Default is false, meaning that when you use C<store> as a template key, C<check> will put it both in the scalar you supplied, as well as in the hashref it returns. =head2 $Params::Check::PRESERVE_CASE If set to true, L<Params::Check> will no longer convert all keys from the user input to lowercase, but instead expect them to be in the case the template provided. This is useful when you want to use similar keys with different casing in your templates. Understand that this removes the case-insensitivy feature of this module. Default is 0; =head2 $Params::Check::ONLY_ALLOW_DEFINED If set to true, L<Params::Check> will require all values passed to be C<defined>. If you wish to enable this on a 'per key' basis, use the template option C<defined> instead. Default is 0; =head2 $Params::Check::SANITY_CHECK_TEMPLATE If set to true, L<Params::Check> will sanity check templates, validating for errors and unknown keys. Although very useful for debugging, this can be somewhat slow in hot-code and large loops. To disable this check, set this variable to C<false>. Default is 1; =head2 $Params::Check::WARNINGS_FATAL If set to true, L<Params::Check> will C<croak> when an error during template validation occurs, rather than return C<false>. Default is 0; =head2 $Params::Check::CALLER_DEPTH This global modifies the argument given to C<caller()> by C<Params::Check::check()> and is useful if you have a custom wrapper function around C<Params::Check::check()>. The value must be an integer, indicating the number of wrapper functions inserted between the real function call and C<Params::Check::check()>. Example wrapper function, using a custom stacktrace: sub check { my ($template, $args_in) = @_; local $Params::Check::WARNINGS_FATAL = 1; local $Params::Check::CALLER_DEPTH = $Params::Check::CALLER_DEPTH + 1; my $args_out = Params::Check::check($template, $args_in); my_stacktrace(Params::Check::last_error) unless $args_out; return $args_out; } Default is 0; =head1 AUTHOR This module by Jos Boumans E<lt>kane@cpan.orgE<gt>. =head1 Acknowledgements Thanks to Richard Soderberg for his performance improvements. =head1 COPYRIGHT This module is copyright (c) 2003,2004 Jos Boumans E<lt>kane@cpan.orgE<gt>. All rights reserved. This library is free software; you may redistribute and/or modify it under the same terms as Perl itself. =cut # Local variables: # c-indentation-style: bsd # c-basic-offset: 4 # indent-tabs-mode: nil # End: # vim: expandtab shiftwidth=4:
On Wed May 16 15:22:21 2007, DAMS wrote:
Show quoted text
> Hi, > > I've worked on a reimplementation of this module for use in my company. > Here is the version I ended with, that is backward compatible, contains > the existing features, add 1 new feature, has the same code size, but is > (sometimes really) faster.
Thanks for the patch -- allow me some time to take a look at it and i'll get back to you. Cheers, Jos


This service runs on Request Tracker, is sponsored by The Perl Foundation, and maintained by Best Practical Solutions.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.