Skip Menu |
 

This queue is for tickets about the JSON-PP CPAN distribution.

Report information
The Basics
Id: 90071
Status: resolved
Priority: 0/
Queue: JSON-PP

People
Owner: Nobody in particular
Requestors: SJM [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 2.27203
Fixed in: (no value)



Subject: Incremental Parsing of Large Objects
Download (untitled) / with headers
text/plain 603b
When creating a non-blocking client/server I read data off the wire in chunks of 4096 bytes and pass them off to the incremental parser. Since I don't know how many, if any, JSON objects are contained in that 4096 bytes, I then attempt to read from the incremental parser. When the object is larger than 4096, and the end of the parsing occurs in the middle of a string, it croaks with the error: unexpected end of string while parsing JSON string, at character offset 4096 (before "(end of string)") Attached is a simple program demonstrating the issue. This problem does not appear with JSON::XS.
Subject: json.pl
Download json.pl
text/x-perl 870b
use strict; use Test::More; use IO::String; use JSON::PP; my $json = JSON::PP->new; my $kb = 'a' x 1024; my $hash = { map { $_ => $kb } (1..40) }; my $data = join ( '', ($json->encode($hash), $json->encode($hash) ) ); my $size = length($data); print "Total size: [$size]\n"; my $fh = IO::String->new($data) or die "Failed to opne FH"; my $count = 0; my $offset = 0; while ($size) { print "Bytes left [$size]\n"; my $bytes = sysread($fh, my $incr, 4096); if ($bytes) { $offset += $bytes; $size -= $bytes; $json->incr_parse($incr); $count++; while ( my $obj = $json->incr_parse ) { print "Got JSON object\n"; } } elsif (defined $bytes) { print "End of file\n"; close $fh; last; } else { die "Error occurred reading handle: $!\n"; } }
Download (untitled) / with headers
text/plain 796b
On Wed Nov 06 07:10:51 2013, SJM wrote: Show quoted text
> When creating a non-blocking client/server I read data off the wire in > chunks of 4096 bytes and pass them off to the incremental parser. > Since I don't know how many, if any, JSON objects are contained in > that 4096 bytes, I then attempt to read from the incremental parser. > When the object is larger than 4096, and the end of the parsing occurs > in the middle of a string, it croaks with the error: > > unexpected end of string while parsing JSON string, at character > offset 4096 (before "(end of string)") > > Attached is a simple program demonstrating the issue. This problem > does not appear with JSON::XS.
Thanks for the report. Fixed in the master. https://github.com/makamaka/JSON-PP/commit/06a86ce0e65c2297c8fdce5153d3a380652748d4
Download (untitled) / with headers
text/plain 922b
On Wed Oct 26 16:43:56 2016, ISHIGAKI wrote: Show quoted text
> On Wed Nov 06 07:10:51 2013, SJM wrote:
> > When creating a non-blocking client/server I read data off the wire > > in > > chunks of 4096 bytes and pass them off to the incremental parser. > > Since I don't know how many, if any, JSON objects are contained in > > that 4096 bytes, I then attempt to read from the incremental parser. > > When the object is larger than 4096, and the end of the parsing > > occurs > > in the middle of a string, it croaks with the error: > > > > unexpected end of string while parsing JSON string, at character > > offset 4096 (before "(end of string)") > > > > Attached is a simple program demonstrating the issue. This problem > > does not appear with JSON::XS.
> > Thanks for the report. Fixed in the master. > > https://github.com/makamaka/JSON- > PP/commit/06a86ce0e65c2297c8fdce5153d3a380652748d4
Closed as 2.92 was released. Thanks.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.