Skip Menu |
 

This queue is for tickets about the IO-Compress CPAN distribution.

Report information
The Basics
Id: 110644
Status: open
Priority: 0/
Queue: IO-Compress

People
Owner: Nobody in particular
Requestors: marius [...] ieval.ro
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: (no value)



Subject: Tutorial about writing a new IO::Compress module
Download (untitled) / with headers
text/plain 434b
Hello, I'd like to write an IO::Compress module. However, there seems to be no documentation about the steps that should be followed. I figure I should subclass IO::Compress::Base and IO::Uncompress::Base, but neither of the modules have documentation about what methods must/should be implemented. I guess I can find this out by reading the source, but it would be much better if a tutorial or some documentation would be available.
Download (untitled) / with headers
text/plain 891b
On Sun Dec 27 04:22:15 2015, MGV wrote: Show quoted text
> Hello, > > I'd like to write an IO::Compress module. However, there seems to be > no documentation about the steps that should be followed. I figure I > should subclass IO::Compress::Base and IO::Uncompress::Base, but > neither of the modules have documentation about what methods > must/should be implemented. I guess I can find this out by reading the > source, but it would be much better if a tutorial or some > documentation would be available.
Hey Marius, writing a tutorial for this would be difficult - the code isn't well enough structured to allow an easy tutorial. The source is probably your best bet. Also - not all compression libraries are suitable for the IO:Compress treatment. Which were you planning to implement? Please shout if you get stuck & need a hand. sorry for the delay in replying. Christmas got in the way. Paul
Subject: Re: [rt.cpan.org #110644] Tutorial about writing a new IO::Compress module
Date: Thu, 31 Dec 2015 14:00:31 +0200
To: "Paul Marquess via RT" <bug-IO-Compress [...] rt.cpan.org>
From: Marius Gavrilescu <marius [...] ieval.ro>
Download (untitled) / with headers
text/plain 1.4k
"Paul Marquess via RT" <bug-IO-Compress@rt.cpan.org> writes: Show quoted text
> writing a tutorial for this would be difficult - the code isn't well > enough structured to allow an easy tutorial. The source is probably > your best bet. Also - not all compression libraries are suitable for > the IO:Compress treatment. Which were you planning to implement?
When I said "tutorial" I was thinking of something of the form "Here's a list of methods you must/should implement and what each of them should do". I am implementing a module for the Brotli compression format, which is the format originally used in web fonts and more recently as a HTTP Content-Encoding, thus replacing gzip and deflate. At the moment it's only implemented in Firefox, but moreimplementations are coming. The official repository is https://github.com/google/brotli/ I will release an initial version today. It will only handle decompression (I still have to figure out how to use XS with C++, as the official compression code is C++ while the decompression code is C), and it will be a simple wrapper around the official API. In the following weeks, after I read the source of IO::{Unc,C}ompress::Base, I'll release updated versions that support compression and follow the IO::{Unc,C}ompress API. Show quoted text
> Please shout if you get stuck & need a hand.
Alright. Show quoted text
> sorry for the delay in replying. Christmas got in the way.
No problem, thanks for your reply! -- Marius Gavrilescu
Download signature.asc
application/pgp-signature 818b

Message body not shown because it is not plain text.

Download (untitled) / with headers
text/plain 2.3k
On Thu Dec 31 07:01:34 2015, MGV wrote: Show quoted text
> "Paul Marquess via RT" <bug-IO-Compress@rt.cpan.org> writes: >
> > writing a tutorial for this would be difficult - the code isn't well > > enough structured to allow an easy tutorial. The source is probably > > your best bet. Also - not all compression libraries are suitable for > > the IO:Compress treatment. Which were you planning to implement?
> > When I said "tutorial" I was thinking of something of the form "Here's a > list of methods you must/should implement and what each of them should do".
That would be good to do. Will add it to my (long) list of things to deal with. Show quoted text
> I am implementing a module for the Brotli compression format, which is > the format originally used in web fonts and more recently as a HTTP > Content-Encoding, thus replacing gzip and deflate. At the moment it's > only implemented in Firefox, but moreimplementations are coming. > > The official repository is https://github.com/google/brotli/
Yep - looked at that one a while ago. It's on my list of compression formats to keep an eye on. My main problem is getting the time to devote to my perl modules at the moment. If you want this to work with IO::Compress it has to use the Brolti streaming interface that allows compression/uncompression to be done a bit at a time. I *think* Brolti also has an interface that takes the complete compressed input and uncompresses (and the same for compression) -- that won't work with IO::Compress. Show quoted text
> I will release an initial version today. It will only handle > decompression (I still have to figure out how to use XS with C++, as the > official compression code is C++ while the decompression code is C), and > it will be a simple wrapper around the official API.
That makes sense. The IO::Compress code assumes there is a module that handles the API to the compresion library. For example all the zlib based IO::Compress modules (gzip, deflate, zip) use Compress::Raw::Zlib to access the library. Show quoted text
> In the following > weeks, after I read the source of IO::{Unc,C}ompress::Base, I'll release > updated versions that support compression and follow the > IO::{Unc,C}ompress API. >
> > Please shout if you get stuck & need a hand.
> > Alright. >
> > sorry for the delay in replying. Christmas got in the way.
> > No problem, thanks for your reply!
cheers Paul ps - I see Perl6 has a Brolti module. See https://github.com/sylvarant/Compress-Brotli
Subject: Re: [rt.cpan.org #110644] Tutorial about writing a new IO::Compress module
Date: Thu, 31 Dec 2015 21:14:16 +0200
To: "Paul Marquess via RT" <bug-IO-Compress [...] rt.cpan.org>
From: Marius Gavrilescu <marius [...] ieval.ro>
Download (untitled) / with headers
text/plain 1.3k
"Paul Marquess via RT" <bug-IO-Compress@rt.cpan.org> writes: Show quoted text
>> I will release an initial version today. It will only handle >> decompression (I still have to figure out how to use XS with C++, as the >> official compression code is C++ while the decompression code is C), and >> it will be a simple wrapper around the official API.
> > That makes sense. The IO::Compress code assumes there is a module that > handles the API to the compresion library. For example all the zlib > based IO::Compress modules (gzip, deflate, zip) use > Compress::Raw::Zlib to access the library.
And IO-Compress-Brotli-0.001 (which can't compress yet) is now on the CPAN: https://metacpan.org/release/MGV/IO-Compress-Brotli-0.001 It wraps both the one-shot (whole-buffer) and the streaming interfaces of the Brotli decompression library. It also passes the official test suite (with the streaming interface, as the one-shot interface can't reasonably handle 3 of the test cases). I'll read up on Compress::Raw and IO::{Unc,C}ompress::Base in the following weeks and try to implement conforming to their APIs. Show quoted text
> ps - I see Perl6 has a Brolti module. See https://github.com/sylvarant/Compress-Brotli
Ooh, that's nice. But it does not seem to offer a streaming interface... Also, I didn't know libbrotli existed! I just shipped the relevant part of the Brotli source code and linked against it. Happy new year! -- Marius Gavrilescu
Download signature.asc
application/pgp-signature 818b

Message body not shown because it is not plain text.

Download (untitled) / with headers
text/plain 613b
... Show quoted text
> > And IO-Compress-Brotli-0.001 (which can't compress yet) is now on the > CPAN: > https://metacpan.org/release/MGV/IO-Compress-Brotli-0.001
Excellent! Show quoted text
> It wraps both the one-shot (whole-buffer) and the streaming interfaces > of the Brotli decompression library. It also passes the official test > suite (with the streaming interface, as the one-shot interface can't > reasonably handle 3 of the test cases). > > I'll read up on Compress::Raw and IO::{Unc,C}ompress::Base in the > following weeks and try to implement conforming to their APIs.
OK. Drop me a line if you get stuck on anything. Paul


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.