Skip Menu |
 

This queue is for tickets about the Compress-Raw-Zlib CPAN distribution.

Report information
The Basics
Id: 122695
Status: open
Priority: 0/
Queue: Compress-Raw-Zlib

People
Owner: Nobody in particular
Requestors: felipe [...] felipegasper.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: sync flush appending extra empty uncompressed block
Date: Thu, 3 Aug 2017 01:48:09 +0200
To: bug-Compress-Raw-Zlib [...] rt.cpan.org
From: Felipe Gasper <felipe [...] felipegasper.com>
Download (untitled) / with headers
text/plain 795b
#!/usr/bin/env perl use strict; use warnings; use Compress::Raw::Zlib; my $deflate = Compress::Raw::Zlib::Deflate->new( -WindowBits => -15, -AppendOutput => 1 ); my $str = 'Hello'; for ( 1 .. 2 ) { $deflate->deflate( $str, my $out ); $deflate->flush( $out, Compress::Raw::Zlib::Z_SYNC_FLUSH() ); printf "sync: %v.02x\n", $out; $out = q<>; $deflate->deflate( $str, $out ); $deflate->flush( $out, Compress::Raw::Zlib::Z_FULL_FLUSH() ); printf "full: %v.02x\n", $out; } =================== sync: f2.48.cd.c9.c9.07.00.00.00.ff.ff.00.00.00.ff.ff full: f2.00.11.00.00.00.00.ff.ff sync: f2.48.cd.c9.c9.07.00.00.00.ff.ff full: f2.00.11.00.00.00.00.ff.ff =================== That first sync doesn’t seem like it should be appending the extra 5 bytes? -FG
Download (untitled) / with headers
text/plain 846b
Hey Filipe, my reading of the zlib docs (https://zlib.net/manual.html) says that is expected behaviour for Z_SYNC_FLUSH. Below is the relevant paragraph from the "deflate" section that talks about the extra block appended by Z_SYNC_FLUSH. If the parameter flush is set to Z_SYNC_FLUSH, all pending output is flushed to the output buffer and the output is aligned on a byte boundary, so that the decompressor can get all input data available so far. (In particular avail_in is zero after the call if enough output space has been provided before the call.) Flushing may degrade compression for some compression algorithms and so it should be used only when necessary. This completes the current deflate block and follows it with an empty stored block that is three bits plus filler bits to the next byte, followed by four bytes (00 00 ff ff).
Subject: Re: [rt.cpan.org #122695] sync flush appending extra empty uncompressed block
Date: Sun, 6 Aug 2017 12:30:26 +0200
To: bug-Compress-Raw-Zlib [...] rt.cpan.org
From: Felipe Gasper <felipe [...] felipegasper.com>
Download (untitled) / with headers
text/plain 1.2k
Show quoted text
> On Aug 6, 2017, at 10:47 AM, Paul Marquess via RT <bug-Compress-Raw-Zlib@rt.cpan.org> wrote: > > <URL: https://rt.cpan.org/Ticket/Display.html?id=122695 > > > Hey Filipe, > > my reading of the zlib docs (https://zlib.net/manual.html) says that is expected behaviour for Z_SYNC_FLUSH. Below is the relevant paragraph from the "deflate" section that talks about the extra block appended by Z_SYNC_FLUSH. > > > If the parameter flush is set to Z_SYNC_FLUSH, all pending output is flushed to the output buffer and the output is aligned on a byte boundary, so that the decompressor can get all input data available so far. (In particular avail_in is zero after the call if enough output space has been provided before the call.) Flushing may degrade compression for some compression algorithms and so it should be used only when necessary. This completes the current deflate block and follows it with an empty stored block that is three bits plus filler bits to the next byte, followed by four bytes (00 00 ff ff). >
Hi Paul, C::R::Z, though, is appending *two* empty blocks with 00.00.ff.ff -- and the behaviour is different the first time from the second time. Should the two Z_SYNC_FLUSH calls not produce the same output given that there is a Z_FULL_FLUSH between them? Thank you! -FG
Download (untitled) / with headers
text/plain 1.8k
On Sun Aug 06 06:31:16 2017, felipe@felipegasper.com wrote: Show quoted text
>
> > On Aug 6, 2017, at 10:47 AM, Paul Marquess via RT <bug-Compress-Raw- > > Zlib@rt.cpan.org> wrote: > > > > <URL: https://rt.cpan.org/Ticket/Display.html?id=122695 > > > > > Hey Filipe, > > > > my reading of the zlib docs (https://zlib.net/manual.html) says that > > is expected behaviour for Z_SYNC_FLUSH. Below is the relevant > > paragraph from the "deflate" section that talks about the extra block > > appended by Z_SYNC_FLUSH. > > > > > > If the parameter flush is set to Z_SYNC_FLUSH, all pending output is > > flushed to the output buffer and the output is aligned on a byte > > boundary, so that the decompressor can get all input data available > > so far. (In particular avail_in is zero after the call if enough > > output space has been provided before the call.) Flushing may degrade > > compression for some compression algorithms and so it should be used > > only when necessary. This completes the current deflate block and > > follows it with an empty stored block that is three bits plus filler > > bits to the next byte, followed by four bytes (00 00 ff ff). > >
> > Hi Paul, > > C::R::Z, though, is appending *two* empty blocks with 00.00.ff.ff -- > and the behaviour is different the first time from the second time. > > Should the two Z_SYNC_FLUSH calls not produce the same output given > that there is a Z_FULL_FLUSH between them? > > Thank you! > > -FG
Thanks for coming back to me on this Filipe - that does indeed look like an extra block sneaking in. Apologies for not spotting it first time. The thing is, the code I use to interface to zlib should be doing *exactly* the same thing both time the call with Z_SYNC_FLUSH is made. Unless this is a bug in zlib (which is very unlikely), there is something slightly sub-optimal with how I'm interfacing to zlib. Need to dig into the code & see what is triggering the extra block. cheers Paul


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.