Skip Menu |
 

This queue is for tickets about the IO-Compress CPAN distribution.

Report information
The Basics
Id: 126005
Status: open
Priority: 0/
Queue: IO-Compress

People
Owner: Nobody in particular
Requestors:
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: (no value)

Attachments


Subject: Please add IO-Compress-Zstandard
Download (untitled) / with headers
text/plain 203b
Please add IO-Compress-Zstandard so that Any(Un)Compress can work on such files. If there's a description of what needs to be done to create such a module let me know and I see if I can help to do that.
Download (untitled) / with headers
text/plain 491b
On Sun Aug 05 08:58:17 2018, https://me.yahoo.com/howdidwegetherereally#f714d wrote: Show quoted text
> Please add IO-Compress-Zstandard so that Any(Un)Compress can work on > such files. If there's a description of what needs to be done to > create such a module let me know and I see if I can help to do that.
Zstandard is on my list to keep an eye on. Haven't seen it used in the wild, or had anyone expressing interest in it (until now). Are you aware of anytihing that makes use of this format? Paul
On Sun Aug 05 16:44:44 2018, PMQS wrote: Show quoted text
> Zstandard is on my list to keep an eye on. Haven't seen it used in the > wild, or had anyone expressing interest in it (until now). > > Are you aware of anything that makes use of this format?
Btrfs uses it for on-disk compression (they even changed the licence to get it into the kernel) and it's moving quite fast into similar application areas (good compression at high speed and _really_ fast uncompression): http://facebook.github.io/zstd/#references I am mostly interested in getting something that compresses almost as good as LZMA (if a tad slower), but decompresses faster into memory than you could read it from external filers. That way, you essentially get the compression ratio as a speedup on read. The existing Perl binding is here: https://metacpan.org/release/Compress-Zstd It uses the streaming API and looks OK to me in general, but I really want to have it available via AnyUncompress in order to keep supporting all other formats without adding extra code.
Download (untitled) / with headers
text/plain 723b
On Mon Aug 06 15:13:31 2018, https://me.yahoo.com/howdidwegetherereally#f714d wrote: ... Show quoted text
> > The existing Perl binding is here: > > https://metacpan.org/release/Compress-Zstd > > It uses the streaming API and looks OK to me in general, but I really > want to have it available via AnyUncompress in order to keep > supporting all other formats without adding extra code.
Unfortunately the streaming support in Compress-Zstd is incomplete. It cannot integrate with AnyUncompress in its current form. Created a proof-of-concept pull request at https://github.com/spiritloose/Compress-Zstd/pull/18 to add the missing features. Have a tentative implementation of IO-Compress-Zstandard created that uses the pull request.
Download (untitled) / with headers
text/plain 692b
On Tue Feb 19 08:56:27 2019, PMQS wrote: Show quoted text
> Unfortunately the streaming support in Compress-Zstd is incomplete. It > cannot integrate with AnyUncompress in its current form.
I see. Hopefully that can be rectified soon. Show quoted text
> Created a proof-of-concept pull request at > https://github.com/spiritloose/Compress-Zstd/pull/18 to add the > missing features. > > Have a tentative implementation of IO-Compress-Zstandard created that > uses the pull request.
Thanks for working on this. BTW, the CI tests only succeeded for 5.26 and fails for all other Perl versions. Is your POC iplementation available somewhere? I'd like to give it a good test (Cygwin's Perl is 5.26.3, so that should work).
Download (untitled) / with headers
text/plain 567b
... Show quoted text
> > Thanks for working on this. BTW, the CI tests only succeeded for 5.26 > and fails for all other Perl versions.
My excuse is this is just a proof-of-concept. :-) Show quoted text
> Is your POC implementation available somewhere? I'd like to give it a > good test (Cygwin's Perl is 5.26.3, so that should work).
See enclosed. It (obviously) needs the version of Compress-ZStd at https://github.com/spiritloose/Compress-Zstd/pull/18 to work. That pull request is very much a proof-of-concept. I don't expect the change to be merged into Compress-ZStd mainline as is. Paul
Subject: IO-Compress-Zstd-master.zip
Download IO-Compress-Zstd-master.zip
application/x-zip-compressed 88.9k

Message body not shown because it is not plain text.

Download (untitled) / with headers
text/plain 847b
Show quoted text
> See enclosed. It (obviously) needs the version of Compress-ZStd at > https://github.com/spiritloose/Compress-Zstd/pull/18 to work. That > pull request is very much a proof-of-concept. I don't expect the > change to be merged into Compress-ZStd mainline as is.
I finally got around to test this. The MANIFEST is missing and the private/MakeUtil.pm is a dangling symlink, but I replaced it with the one in the release of IO-Compress and that worked. I also patched Compress-Zstd to use the system zstd library instead of the bundled one and removed the legacy MT interface (which I didn't configure in). After all this, things actually work and all tests pass except one that is missing a file: t/25interop-io-string.t .. Can't locate interop-io-string.pl in @INC I haven't yet used it in any actual code, but I hope to get to that soon-ish.
Download (untitled) / with headers
text/plain 1.2k
On Sun Sep 08 15:44:57 2019, https://me.yahoo.com/howdidwegetherereally#f714d wrote: Show quoted text
> > See enclosed. It (obviously) needs the version of Compress-ZStd at > > https://github.com/spiritloose/Compress-Zstd/pull/18 to work. That > > pull request is very much a proof-of-concept. I don't expect the > > change to be merged into Compress-ZStd mainline as is.
> > I finally got around to test this. The MANIFEST is missing and the > private/MakeUtil.pm is a dangling symlink, but I replaced it with the > one in the release of IO-Compress and that worked. I also patched > Compress-Zstd to use the system zstd library instead of the bundled > one and removed the legacy MT interface (which I didn't configure in). > After all this, things actually work and all tests pass except one > that is missing a file: > > t/25interop-io-string.t .. Can't locate interop-io-string.pl in @INC > > I haven't yet used it in any actual code, but I hope to get to that > soon-ish.
Good to hear that it works for you. Bear in mind that this is unreleased code that is likely to change (both in Compress-Zstd and my code). I can't move forward with this until Compress-Zstd is updated. My pull request for that is still pending - see https://github.com/spiritloose/Compress-Zstd/pull/18.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.