This queue is for tickets about the Cache-Memcached-Fast CPAN distribution.

Report information
The Basics
Id:
35588
Status:
resolved
Worked:
3 hours (180 minutes)
Users:
KROKI: 3 hours (180 minutes)
Priority:
Low/Low

People
Owner:
Nobody in particular
Requestors:
KAPPA [...] cpan.org
Cc:
AdminCc:

BugTracker
Severity:
Wishlist
Broken in:
0.10
Fixed in:
(no value)



Subject: Do not try to store in cache too big values
memcached does not accept values bigger than 1mb. Moreover, it's commonly not very useful to cache big values -- they are probably rare. We here at Rambler have an option "too_big_threshold" in Cache::Memcached which sets the size in bytes after which no storing is even attempted. It would be nice to have the same option in Fast. The change is very simple and I thought it would be more practical for the author to implement it :) And I missed my XS-fu classes, too :( Such logic cannot be elegantly implemented on the application side because serialization is done inside Cache::Memcached. I attach my Perl patch to the latest BRADFITZ's Cache::Memcached, it could be useful.
Subject: toobig.patch
--- Memcached.pm.orig 2007-07-17 22:40:23.000000000 +0400 +++ Memcached.pm 2008-05-02 21:43:22.000000000 +0400 @@ -25,6 +25,7 @@ bucketcount _single_sock _stime connect_timeout cb_connect_fail parser_class + too_big_threshold }; # flag definitions @@ -76,6 +77,7 @@ $self->{'stat_callback'} = $args->{'stat_callback'} || undef; $self->{'readonly'} = $args->{'readonly'}; $self->{'parser_class'} = $args->{'parser_class'} || $parser_class; + $self->{'too_big_threshold'} = $args->{'too_big_threshold'}; # TODO: undocumented $self->{'connect_timeout'} = $args->{'connect_timeout'} || 0.25; @@ -460,6 +462,10 @@ my $len = length($val); + if ($self->{'too_big_threshold'} && $len >= $self->{'too_big_threshold'}) { + return 1; # positive NOP + } + if ($self->{'compress_threshold'} && $HAVE_ZLIB && $self->{'compress_enable'} && $len >= $self->{'compress_threshold'}) { @@ -959,6 +965,10 @@ Values larger than this threshold will be compressed by C<set> and decompressed by C<get>. +Use C<too_big_threshold> to set the upper limit on size of cached data +items. Values larger than this threshold won't be sent over the +network. It's no use to send >1MB values anyway, they are not stored. + Use C<no_rehash> to disable finding a new memcached server when one goes down. Your application may or may not need this, depending on your expirations and key usage. @@ -1002,6 +1012,10 @@ Sets the compression threshold. See C<new> constructor for more information. +=item C<set_too_big_threshold> + +Sets the overall size threshold. See C<new> constructor for more information. + =item C<enable_compress> Temporarily enable or disable compression. Has no effect if C<compress_threshold>
On Fri May 02 13:44:58 2008, KAPPA wrote:
Show quoted text
> We here at Rambler have an option "too_big_threshold" in > Cache::Memcached which sets the size in bytes after which no storing is > even attempted. It would be nice to have the same option in Fast.
The request makes sense. Indeed you can't implement such threshold with serialization and/or compression hooks in C::M::F. However, a good question is _when_ to check the size? I'd say that it would be more logical to test it after the compression, because that's the point when we know what exactly is going to be stored in memcached. However in your patch you test the size before the compression, and one may say that this is more optimal when we know in advance that values do not compress well. So, what you suggest? Will one test after the compression be enough? Or should we have two thresholds?
From: KAPPA@cpan.org
Show quoted text
> serialization and/or compression hooks in C::M::F. However, a good > question is _when_ to check the size? I'd say that it would be more > logical to test it after the compression, because that's the point when > we know what exactly is going to be stored in memcached. However in > your patch you test the size before the compression, and one may say > that this is more optimal when we know in advance that values do not > compress well. So, what you suggest? Will one test after the > compression be enough? Or should we have two thresholds?
I think it's more practical to have one threshold and test it right after compression. If one knows in advance that his values do not compress well, he should switch the compression off at all. We don't use compression, and I just forgot about it, sorry. Updated patch attached.
--- Memcached.pm.orig 2007-07-17 22:40:23.000000000 +0400 +++ Memcached.pm 2008-05-03 04:09:26.000000000 +0400 @@ -25,6 +25,7 @@ bucketcount _single_sock _stime connect_timeout cb_connect_fail parser_class + too_big_threshold }; # flag definitions @@ -76,6 +77,7 @@ $self->{'stat_callback'} = $args->{'stat_callback'} || undef; $self->{'readonly'} = $args->{'readonly'}; $self->{'parser_class'} = $args->{'parser_class'} || $parser_class; + $self->{'too_big_threshold'} = $args->{'too_big_threshold'}; # TODO: undocumented $self->{'connect_timeout'} = $args->{'connect_timeout'} || 0.25; @@ -474,6 +476,10 @@ } } + if ($self->{'too_big_threshold'} && $len >= $self->{'too_big_threshold'}) { + return 1; # positive NOP + } + $exptime = int($exptime || 0); local $SIG{'PIPE'} = "IGNORE" unless $FLAG_NOSIGNAL; @@ -959,6 +965,10 @@ Values larger than this threshold will be compressed by C<set> and decompressed by C<get>. +Use C<too_big_threshold> to set the upper limit on size of cached data +items. Values larger than this threshold won't be sent over the +network. It's no use to send >1MB values anyway, they are not stored. + Use C<no_rehash> to disable finding a new memcached server when one goes down. Your application may or may not need this, depending on your expirations and key usage. @@ -1002,6 +1012,10 @@ Sets the compression threshold. See C<new> constructor for more information. +=item C<set_too_big_threshold> + +Sets the overall size threshold. See C<new> constructor for more information. + =item C<enable_compress> Temporarily enable or disable compression. Has no effect if C<compress_threshold>


This service runs on Request Tracker, is sponsored by The Perl Foundation, and maintained by Best Practical Solutions.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.