Skip Menu |
 

This queue is for tickets about the SQL-Tokenizer CPAN distribution.

Report information
The Basics
Id: 44918
Status: open
Priority: 0/
Queue: SQL-Tokenizer

People
Owner: Nobody in particular
Requestors: JKRAMER [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 0.18
Fixed in: (no value)

Attachments
0001-Name-quote-and-separator-fix.patch



Subject: Dot gets lost when combined with field quotations
Download (untitled) / with headers
text/plain 640b
I'm creating a new bug report for this because the other dot-bug is slightly different from this one and seems to be already fixed. When quoting table/column names, the seperating dot seems to get lost. For example, in SELECT "foo"."bar" FROM "foo" "foo"."bar" becomes [ '"foo"', '"bar"' ] instead of [ '"foo"', '.', '"bar"']. When using MySQL quotation, the quotes get lost too: SELECT `foo`.`bar` FROM `foo`. Becomes: [ ... 'foo', 'bar, ... ] instead of [ ..., '`foo`', '.', '`bar`', ...] The bug appears in 0.18, but older versions might be affected too (didn't test them). Best regards and thanks for the great module, Jonas
Download (untitled) / with headers
text/plain 267b
Hi again, I made a quick patch which seems to fix the problem. I didn't test it too much, but the SQL-Tokenizer tests run perfectly. What I did: 1. Allow `` quoted strings (similar to "" and '') 2. Add dot to punctuation characters Hope it doesn't break anything.
From dc9639b640fe536e75e372d05a7af35b86cb87f2 Mon Sep 17 00:00:00 2001 From: Jonas Kramer <jkramer@nex.scrapping.cc> Date: Fri, 10 Apr 2009 12:11:06 +0200 Subject: [PATCH] Name quote and separator fix. --- lib/SQL/Tokenizer.pm | 5 ++++- t/11-name-sep.t | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletions(-) create mode 100644 t/11-name-sep.t diff --git a/lib/SQL/Tokenizer.pm b/lib/SQL/Tokenizer.pm index 7d5b1c8..6e8b158 100644 --- a/lib/SQL/Tokenizer.pm +++ b/lib/SQL/Tokenizer.pm @@ -20,7 +20,7 @@ my $re= qr{ (?:<>|<=>|>=|<=|==|=|!=|!|<<|>>|<|>|\|\||\||&&|&|-|\+|\*(?!/)|/(?!\*)|\%|~|\^|\?) # operators and tests | - [\[\]\(\),;] # punctuation (parenthesis, comma) + [\[\]\(\),;.] # punctuation (parenthesis, comma) | \'\'(?!\') # empty single quoted string | @@ -28,6 +28,9 @@ my $re= qr{ | ".*?(?:(?:""){1,}"|(?<!["\\])"(?!")|\\"{2}) # anything inside double quotes, ungreedy + | + `.*?(?:(?:``){1,}`|(?<![`\\])`(?!`)|\\`{2}) + # anything inside backticks quotes, ungreedy | '.*?(?:(?:''){1,}'|(?<!['\\])'(?!')|\\'{2}) # anything inside single quotes, ungreedy. diff --git a/t/11-name-sep.t b/t/11-name-sep.t new file mode 100644 index 0000000..9964f8f --- /dev/null +++ b/t/11-name-sep.t @@ -0,0 +1,37 @@ +use strict; +use warnings; + +use Test::More; + +use SQL::Tokenizer; + +use constant SPACE => ' '; +use constant COMMA => ','; +use constant NL => "\n"; + +my $query; +my @query; +my @tokenized; + +my @tests = ( + { + description => q{PostgreSQL style}, + query => qq{SELECT "foo"."bar"}, + wanted => [ 'SELECT', SPACE, '"foo"', '.', '"bar"' ], + }, { + description => q{MySQL style}, + query => qq{SELECT `foo`.`bar`}, + wanted => [ 'SELECT', SPACE, '`foo`', '.', '`bar`' ], + } + +); + +plan tests => scalar @tests; + +foreach my $test (@tests) { + my @tokenized= SQL::Tokenizer->tokenize( $test->{query} ); + is_deeply( \@tokenized, $test->{wanted}, $test->{description} ); +} + +__END__ + -- 1.6.2.1
Hi Jonas, Your patch was applied and I'm uploading 0.19 right now. Thanks! -Igor


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.