Skip Menu |
 

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the PPI CPAN distribution.

Report information
The Basics
Id: 68176
Status: resolved
Priority: 0/
Queue: PPI

People
Owner: Nobody in particular
Requestors: MOREGAN [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.215
Fixed in: (no value)



Subject: Some augmented assignment operators parse as two operators
Download (untitled) / with headers
text/plain 3.3k
Some augmented assignment operators like .= parse as a single PPI::Token::Operator: perl -MPPI::Document -MPPI::Dumper -e 'my $d = PPI::Document->new(\"$x .= 1;"); PPI::Dumper->new($d)->print();' PPI::Document PPI::Token::Whitespace ' ' PPI::Statement PPI::Token::Operator '.=' PPI::Token::Whitespace ' ' PPI::Token::Number '1' PPI::Token::Structure ';' and some parse as two: perl -MPPI::Document -MPPI::Dumper -e 'my $d = PPI::Document->new(\"$x <<= 1;"); PPI::Dumper->new($d)->print();' PPI::Document PPI::Token::Whitespace ' ' PPI::Statement PPI::Token::Operator '<<' PPI::Token::Operator '=' PPI::Token::Whitespace ' ' PPI::Token::Number '1' PPI::Token::Structure ';' I assume that they all should parse as a single operator. For all 16 augmented operators in Perl 5.10: perl -MPPI::Document -M5.10.0 -e 'print " += ops: "; my $d = PPI::Document->new(\"my $x; $x += 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " -= ops: "; my $d = PPI::Document->new(\"my $x; $x -= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " .= ops: "; my $d = PPI::Document->new(\"my $x; $x .= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " *= ops: "; my $d = PPI::Document->new(\"my $x; $x *= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " /= ops: "; my $d = PPI::Document->new(\"my $x; $x /= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " %= ops: "; my $d = PPI::Document->new(\"my $x; $x %= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " x= ops: "; my $d = PPI::Document->new(\"my $x; $x x= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " &= ops: "; my $d = PPI::Document->new(\"my $x; $x &= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " |= ops: "; my $d = PPI::Document->new(\"my $x; $x |= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print " ^= ops: "; my $d = PPI::Document->new(\"my $x; $x ^= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "**= ops: "; my $d = PPI::Document->new(\"my $x; $x **= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "<<= ops: "; my $d = PPI::Document->new(\"my $x; $x <<= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print ">>= ops: "; my $d = PPI::Document->new(\"my $x; $x >>= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "&&= ops: "; my $d = PPI::Document->new(\"my $x; $x &&= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "||= ops: "; my $d = PPI::Document->new(\"my $x; $x ||= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' perl -MPPI::Document -M5.10.0 -e 'print "//= ops: "; my $d = PPI::Document->new(\"my $x; $x //= 1;"); say scalar @{ $d->find("PPI::Token::Operator") };' += ops: 1 -= ops: 1 .= ops: 1 *= ops: 1 /= ops: 0 %= ops: 2 x= ops: 2 &= ops: 2 |= ops: 2 ^= ops: 2 **= ops: 2 <<= ops: 2 Show quoted text
>>= ops: 2
&&= ops: 2 ||= ops: 2 //= ops: 1
Download (untitled) / with headers
text/plain 367b
Attached patch adds basic testing of PPI::Token::Operator, looping through all the operators in %PPI::Token::Operator::OPERATOR. It also updates the list of operators in the documentation to match the list of operators actually supported. With the new tests in place, adding the augmented assignment operators not already in OPERATOR passes all tests except for 'x='
Subject: operator-tests.patch
Index: t/ppi_token_operator.t =================================================================== --- t/ppi_token_operator.t (revision 0) +++ t/ppi_token_operator.t (revision 0) @@ -0,0 +1,60 @@ +#!/usr/bin/perl + +# Unit testing for PPI, generated by Test::Inline + +use strict; +use File::Spec::Functions ':ALL'; +BEGIN { + $| = 1; + $^W = 1; + no warnings 'once'; + $PPI::XS_DISABLE = 1; + $PPI::Lexer::X_TOKENIZER ||= $ENV{X_TOKENIZER}; +} +use PPI; + +# Execute the tests +use Test::More 'no_plan'; + +# =begin testing +{ +foreach my $op ( sort keys %PPI::Token::Operator::OPERATOR ) { + my $source = $op eq '<>' ? '<>;' : "1 $op 2;"; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "operator $op parsed '$source'" ); + my $ops = $doc->find( $op eq '<>' ? 'Token::QuoteLike::Readline' : 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator $op found operators" ); + is( @$ops, 1, "operator $op found exactly once" ); + is( $ops->[0]->content(), $op, "operator $op operator text matches" ); +} + +{ + my $source = '$a = .987;'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::Number::Float' ); + is( ref $ops, 'ARRAY', "found number" ); + is( @$ops, 1, "number found exactly once" ); + is( $ops->[0]->content(), '.987', "text matches" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in number test" ); + is( @$ops, 1, "operator = found exactly once in number test" ); +} + +{ + my $source = '$a = <<PERL_END;' . "\n" . 'PERL_END'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::HereDoc' ); + is( ref $ops, 'ARRAY', "found heredoc" ); + is( @$ops, 1, "heredoc found exactly once" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in heredoc test" ); + is( @$ops, 1, "operator = found exactly once in heredoc test" ); +} +} + + +1; Index: lib/PPI/Token/Operator.pm =================================================================== --- lib/PPI/Token/Operator.pm (revision 14482) +++ lib/PPI/Token/Operator.pm (working copy) @@ -21,8 +21,8 @@ == != <=> . .. ... , & | ^ && || // ? : = += -= *= .= //= - < > <= >= <> => -> - and or dor not eq ne + /= < > <= >= <> => -> + and or xor not eq ne =head1 DESCRIPTION @@ -72,6 +72,51 @@ ##################################################################### # Tokenizer Methods +=pod + +=begin testing + +foreach my $op ( sort keys %PPI::Token::Operator::OPERATOR ) { + my $source = $op eq '<>' ? '<>;' : "1 $op 2;"; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "operator $op parsed '$source'" ); + my $ops = $doc->find( $op eq '<>' ? 'Token::QuoteLike::Readline' : 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator $op found operators" ); + is( @$ops, 1, "operator $op found exactly once" ); + is( $ops->[0]->content(), $op, "operator $op operator text matches" ); +} + +{ + my $source = '$a = .987;'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::Number::Float' ); + is( ref $ops, 'ARRAY', "found number" ); + is( @$ops, 1, "number found exactly once" ); + is( $ops->[0]->content(), '.987', "text matches" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in number test" ); + is( @$ops, 1, "operator = found exactly once in number test" ); +} + +{ + my $source = '$a = <<PERL_END;' . "\n" . 'PERL_END'; + my $doc = PPI::Document->new( \$source ); + isa_ok( $doc, 'PPI::Document', "parsed '$source'" ); + my $ops = $doc->find( 'Token::HereDoc' ); + is( ref $ops, 'ARRAY', "found heredoc" ); + is( @$ops, 1, "heredoc found exactly once" ); + + $ops = $doc->find( 'Token::Operator' ); + is( ref $ops, 'ARRAY', "operator = found operators in heredoc test" ); + is( @$ops, 1, "operator = found exactly once in heredoc test" ); +} + +=end testing + +=cut + sub __TOKENIZER__on_char { my $t = $_[1]; my $char = substr( $t->{line}, $t->{line_cursor}, 1 );
Download (untitled) / with headers
text/plain 404b
Forgot to say that adding '||=' to OPERATOR causes a failure in the larger test suite: [16:40:51] t/ppi_lexer.t .................... 1/33 # Failed test 'The curly in $foo ||= { One => 1 }; isa PPI::Structure::Constructor' # at t/ppi_lexer.t line 92. # The curly in $foo ||= { One => 1 }; isn't a 'PPI::Structure::Constructor' it's a 'PPI::Token::Structure' # Looks like you failed 1 test of 33.
The full version of the fix for this is now released to CPAN as PPI 1.218.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.