Skip Menu |
 

This queue is for tickets about the SQL-Tokenizer CPAN distribution.

Report information
The Basics
Id: 34889
Status: open
Priority: 0/
Queue: SQL-Tokenizer

People
Owner: izut [...] cpan.org
Requestors: hillsc [...] ncsoft.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: "dot" token being lost in versions 0.10 and above
Date: Fri, 11 Apr 2008 15:40:18 -0500
To: <bug-SQL-Tokenizer [...] rt.cpan.org>
From: "Charlie Hills" <hillsc [...] ncsoft.com>
Download (untitled) / with headers
text/plain 1.3k
I've been using version 0.09 for about a month or more. I had some unit tests written that suddenly failed when going from 0.09 to 0.10 (and 0.11). Apparently when using table.* notation in the select statement, the "." token is getting eaten, which is bad. For example: "select * from table" Discounting whitespace: This is four tokens in 0.09: "select", "*", "from", "table" In version 0.10 and above: "select", "*", "from", "table" So far, so good. :) Now this example: "select t.* from table t" This is five tokens in 0.09: "select", "t.*", "from", "table", "t" In version 0.10 and above: "select", "t", "*", "from", "table", "t" In my code, when I reassemble the tokens after processing, I end up with: "select t* from table t" It's a wonderfully simple perl module and I'd like to keep up with the current version. Here's a test script I put together to test the modules side by side: #!/usr/bin/perl -w use strict; use Tokenizer09; use Tokenizer11; use Data::Dumper; my $query = "select * from table"; my @tokens09 = Tokenizer09->tokenize($query); my @tokens11 = Tokenizer11->tokenize($query); print Dumper(\@tokens09); print Dumper(\@tokens11); $query = "select t.* from table t"; @tokens09 = Tokenizer09->tokenize($query); @tokens11 = Tokenizer11->tokenize($query); print Dumper(\@tokens09); print Dumper(\@tokens11); Thanks for your time and effort, Charlie Hills
From: izut [...] cpan.org
Download (untitled) / with headers
text/plain 1.9k
Hi, In order to SQL::Tokenizer support SQL operators (<>, <=>, <<, >>, etc), and less structured queries (`SELECT * FROM table WHERE 1<0'), the regex changed significantly. Before, I was treating everything not matching certain requirements like `();' and others as a single token, but this now is changed. It is fixed now in version 0.12, and it will be available in CPAN soon. Please let me know if it solves your problem :-) On Fri Apr 11 16:40:44 2008, hillsc@ncsoft.com wrote: Show quoted text
> I've been using version 0.09 for about a month or more. I had some unit > tests written that suddenly failed when going from 0.09 to 0.10 (and > 0.11). Apparently when using table.* notation in the select statement, > the "." token is getting eaten, which is bad. > > For example: > > "select * from table" > > Discounting whitespace: > > This is four tokens in 0.09: "select", "*", "from", "table" > In version 0.10 and above: "select", "*", "from", "table" > > So far, so good. :) Now this example: > > "select t.* from table t" > > This is five tokens in 0.09: "select", "t.*", "from", "table", "t" > In version 0.10 and above: "select", "t", "*", "from", "table", "t" > > > In my code, when I reassemble the tokens after processing, I end up > with: > > "select t* from table t" > > > It's a wonderfully simple perl module and I'd like to keep up with the > current version. Here's a test script I put together to test the modules > side by side: > > > #!/usr/bin/perl -w > > use strict; > use Tokenizer09; > use Tokenizer11; > use Data::Dumper; > > my $query = "select * from table"; > my @tokens09 = Tokenizer09->tokenize($query); > my @tokens11 = Tokenizer11->tokenize($query); > > print Dumper(\@tokens09); > print Dumper(\@tokens11); > > > $query = "select t.* from table t"; > @tokens09 = Tokenizer09->tokenize($query); > @tokens11 = Tokenizer11->tokenize($query); > > print Dumper(\@tokens09); > print Dumper(\@tokens11); > > > Thanks for your time and effort, > > Charlie Hills
Subject: RE: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above
Date: Tue, 15 Apr 2008 16:37:27 -0500
To: <bug-SQL-Tokenizer [...] rt.cpan.org>
From: "Charlie Hills" <hillsc [...] ncsoft.com>
Download (untitled) / with headers
text/plain 236b
Found a new bug with 0.14. This time "@" signs are getting eaten. Example: DECLARE @id INT; SELECT @id = column FROM TABLE; These are coming out: DECLARE id INT; SELECT id = column FROM TABLE; This worked okay in 0.09.
Subject: Re: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above
Date: Wed, 16 Apr 2008 23:12:00 +0200
To: bug-SQL-Tokenizer [...] rt.cpan.org
From: Igor Sutton <igorsutton [...] mac.com>
Download (untitled) / with headers
text/plain 539b
Hi Charlie, Please check version 0.15, this issue must be solved now. Let me know if you have any other issue! – Igor On Apr 15, 2008, at 11:37 PM, Charlie Hills via RT wrote: Show quoted text
> > > Queue: SQL-Tokenizer > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > > > Found a new bug with 0.14. This time "@" signs are getting eaten. > > Example: > > DECLARE @id INT; > SELECT @id = column FROM TABLE; > > These are coming out: > > DECLARE id INT; > SELECT id = column FROM TABLE; > > This worked okay in 0.09. >
Subject: RE: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above
Date: Thu, 24 Apr 2008 18:09:29 -0500
To: <bug-SQL-Tokenizer [...] rt.cpan.org>
From: "Charlie Hills" <hillsc [...] ncsoft.com>
Download (untitled) / with headers
text/plain 1.1k
Sorry I didn't reply. 0.15 did fix the dropped dot. But a few minutes ago, I found this in our codebase: SELECT SUM(column) FROM database..table; It's SQL server's full notation for DATABASE.SCHEMA.TABLE and if the default schema is used, the result becomes DATABASE..TABLE. Sorry! Charlie Show quoted text
-----Original Message----- From: Igor Sutton via RT [mailto:bug-SQL-Tokenizer@rt.cpan.org] Sent: Wednesday, April 16, 2008 4:12 PM To: Charlie Hills Subject: Re: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > Hi Charlie, Please check version 0.15, this issue must be solved now. Let me know if you have any other issue! - Igor On Apr 15, 2008, at 11:37 PM, Charlie Hills via RT wrote:
> > > Queue: SQL-Tokenizer > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > > > Found a new bug with 0.14. This time "@" signs are getting eaten. > > Example: > > DECLARE @id INT; > SELECT @id = column FROM TABLE; > > These are coming out: > > DECLARE id INT; > SELECT id = column FROM TABLE; > > This worked okay in 0.09. >
Subject: Re: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above
Date: Fri, 25 Apr 2008 12:12:30 +0200
To: bug-SQL-Tokenizer [...] rt.cpan.org
From: Igor Sutton <igorsutton [...] mac.com>
Download (untitled) / with headers
text/plain 1.4k
Hi Charlie, I uploaded 0.16 which fixes this problem. Please take a look. Thanks! On Apr 25, 2008, at 1:09 AM, Charlie Hills via RT wrote: Show quoted text
> > Queue: SQL-Tokenizer > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > > > Sorry I didn't reply. 0.15 did fix the dropped dot. But a few minutes > ago, I found this in our codebase: > > SELECT SUM(column) FROM database..table; > > It's SQL server's full notation for DATABASE.SCHEMA.TABLE and if the > default schema is used, the result becomes DATABASE..TABLE. > > Sorry! > > Charlie > > > -----Original Message----- > From: Igor Sutton via RT [mailto:bug-SQL-Tokenizer@rt.cpan.org] > Sent: Wednesday, April 16, 2008 4:12 PM > To: Charlie Hills > Subject: Re: [rt.cpan.org #34889] "dot" token being lost in versions > 0.10 and above > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > > > > Hi Charlie, > > Please check version 0.15, this issue must be solved now. Let me know > if you have any other issue! > > - Igor > > On Apr 15, 2008, at 11:37 PM, Charlie Hills via RT wrote:
>> >> >> Queue: SQL-Tokenizer >> Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > >> >> Found a new bug with 0.14. This time "@" signs are getting eaten. >> >> Example: >> >> DECLARE @id INT; >> SELECT @id = column FROM TABLE; >> >> These are coming out: >> >> DECLARE id INT; >> SELECT id = column FROM TABLE; >> >> This worked okay in 0.09. >>
> >
Subject: RE: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above
Date: Mon, 28 Apr 2008 12:25:03 -0500
To: <bug-SQL-Tokenizer [...] rt.cpan.org>
From: "Charlie Hills" <hillsc [...] ncsoft.com>
Download (untitled) / with headers
text/plain 1.7k
It works! Sorry for the delay in replying... but thanks again for the quick turnaround on all these fixes. Show quoted text
-----Original Message----- From: Igor Sutton via RT [mailto:bug-SQL-Tokenizer@rt.cpan.org] Sent: Friday, April 25, 2008 5:13 AM To: Charlie Hills Subject: Re: [rt.cpan.org #34889] "dot" token being lost in versions 0.10 and above <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > Hi Charlie, I uploaded 0.16 which fixes this problem. Please take a look. Thanks! On Apr 25, 2008, at 1:09 AM, Charlie Hills via RT wrote:
> > Queue: SQL-Tokenizer > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > > > Sorry I didn't reply. 0.15 did fix the dropped dot. But a few minutes > ago, I found this in our codebase: > > SELECT SUM(column) FROM database..table; > > It's SQL server's full notation for DATABASE.SCHEMA.TABLE and if the > default schema is used, the result becomes DATABASE..TABLE. > > Sorry! > > Charlie > > > -----Original Message----- > From: Igor Sutton via RT [mailto:bug-SQL-Tokenizer@rt.cpan.org] > Sent: Wednesday, April 16, 2008 4:12 PM > To: Charlie Hills > Subject: Re: [rt.cpan.org #34889] "dot" token being lost in versions > 0.10 and above > > > <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > > > > Hi Charlie, > > Please check version 0.15, this issue must be solved now. Let me know > if you have any other issue! > > - Igor > > On Apr 15, 2008, at 11:37 PM, Charlie Hills via RT wrote:
>> >> >> Queue: SQL-Tokenizer >> Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=34889 > >> >> Found a new bug with 0.14. This time "@" signs are getting eaten. >> >> Example: >> >> DECLARE @id INT; >> SELECT @id = column FROM TABLE; >> >> These are coming out: >> >> DECLARE id INT; >> SELECT id = column FROM TABLE; >> >> This worked okay in 0.09. >>
> >


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.