Skip Menu |
 

This queue is for tickets about the Crypt-SSLeay CPAN distribution.

Report information
The Basics
Id: 95663
Status: open
Priority: 0/
Queue: Crypt-SSLeay

People
Owner: nanis [...] runu.moc.invalid
Requestors: Steffen_Ullrich [...] genua.de
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Question: officially deprecate Crypt::SSLeay and remove from LWP?
Download (untitled) / with headers
text/plain 1.2k
Hi, I'm the maintainer of IO::Socket::SSL and a bit involved in the https support for LWP too. We currently see usability, security and maintainability problems with the https support in LWP, because it uses IO::Socket::SSL but tries a fallback to the less capable Crypt::SSLeay if IO::Socket::SSL/Net::SSLeay is not available. See the discussion at https://github.com/libwww-perl/lwp-protocol-https/pull/14. With the recent fix of proxy support for https in LWP I see no feature, why Crypt::SSLeay is still needed for LWP. Do you see any problems if we would remove support for Crypt::SSLeay from LWP? I for myself would only see a problem if there are platforms supported by Crypt::SSLeay which are not supported by Net::SSLeay (and thus not by IO::Socket::SSL). Are you aware of such platforms? In the long run I would think it would be better to remove Crypt::SSLeay completely or replace it with a (more secure) compatibility layer using IO::Socket::SSL/Net::SSLeay. I see no need for two OpenSSL stacks in Perl. And, while I appreciate your work on Crypt::SSLeay, I assume you would be glad if you would not need to care about bugs which are solved in Net::SSLeay already (like threading). And from my contacts with the Net::SSLeay developers I feel that they welcome any help they can get. Regards, Steffen
Download (untitled) / with headers
text/plain 2.6k
Hello Steffen: Apologies for the late response. I personally have no objection to removing Crypt::SSLeay from LWP. You may have noticed that the build process and the documentation now asks users explicitly if they really need or even use Crypt::SSLeay. The reason for that question is the simple fact that unless one takes explicit steps to force its use, Crypt::SSLeay is simply not used by current versions of LWP. And, upgrading Crypt::SSLeay pulls in LWP::Protocol::https which in turn pulls in IO::Socket::SSL etc. Again, unless one takes extra steps, one's programs continue to work, but they simply do not use Crypt::SSLeay at all. However, I have the impression from my informal interactions with people that there are quite a few projects that explicitly specify a dependency on Crypt::SSLeay. The current situation enable their builds etc to continue working as normal, while transparently switching them off of using Crypt::SSLeay (even though it is still installed). See also https://metacpan.org/pod/Crypt::SSLeay#DO-YOU-NEED-Crypt::SSLeay Therefore, I think your goals are already being achieved, maybe somewhat stealthily ;-) I am not sure if anything more radical is called for, especially if it might disturb projects/codebases where people may not be paying close attention. Hope this makes sense, -- Sinan On Thu May 15 03:27:09 2014, SULLR wrote: Show quoted text
> Hi, > I'm the maintainer of IO::Socket::SSL and a bit involved in the https > support for LWP too. We currently see usability, security and > maintainability problems with the https support in LWP, because it > uses IO::Socket::SSL but tries a fallback to the less capable > Crypt::SSLeay if IO::Socket::SSL/Net::SSLeay is not available. See the > discussion at https://github.com/libwww-perl/lwp-protocol- > https/pull/14. > > With the recent fix of proxy support for https in LWP I see no > feature, why Crypt::SSLeay is still needed for LWP. Do you see any > problems if we would remove support for Crypt::SSLeay from LWP? I for > myself would only see a problem if there are platforms supported by > Crypt::SSLeay which are not supported by Net::SSLeay (and thus not by > IO::Socket::SSL). Are you aware of such platforms? > > In the long run I would think it would be better to remove > Crypt::SSLeay completely or replace it with a (more secure) > compatibility layer using IO::Socket::SSL/Net::SSLeay. I see no need > for two OpenSSL stacks in Perl. And, while I appreciate your work on > Crypt::SSLeay, I assume you would be glad if you would not need to > care about bugs which are solved in Net::SSLeay already (like > threading). And from my contacts with the Net::SSLeay developers I > feel that they welcome any help they can get. > > Regards, > Steffen
Very good idea, Steffen! -- Reini Urban
Download (untitled) / with headers
text/plain 450b
Show quoted text
> I see no need > for two OpenSSL stacks in Perl.
As it stands, i don't see Net::SSLeay having much of a long term future. At some point we will want to secure the CPAN toolchain and ship an OpenSSL binding (not OpenSSL itself!) with Perl. And due to its incompatible license that's not going to be Net::SSLeay. So, i believe there's a good chance Crype::SSLeay may be coming back at some point simply because of it's Artistic license. -- sebastian
Download (untitled) / with headers
text/plain 2.6k
Am Mi 06. Aug 2014, 17:26:53, SRI schrieb: Show quoted text
> > I see no need > > for two OpenSSL stacks in Perl.
> > As it stands, i don't see Net::SSLeay having much of a long term > future. At some point we will want to secure the CPAN toolchain and > ship an OpenSSL binding (not OpenSSL itself!) with Perl. And due to > its incompatible license that's not going to be Net::SSLeay. So, i > believe there's a good chance Crype::SSLeay may be coming back at some > point simply because of it's Artistic license.
I agree partially: Perl needs to have proper (Open)SSL bindings in the core and I don't think Net::SSLeay is the right thing for this. While you see the license as the problem I see the the code as the main thing. Currently Net::SSLeay is just a collection of parts from OpenSSL which were needed by the authors or contributors. There is no real design in it, it just collected more functions over time. But the worst thing for me is that you have to work from Perl with pointers into OpenSSL internals all the time and you have to manually care about OpenSSLs reference counters. It is too easy to get either memory leaks or segmentation faults this way. Crypt::SSLeay is better in this regard, because it treats some OpenSSL structures as objects and automatically calls the relevant _free functions on DESTROY. But it lacks a lot of functionality like usable non-blocking sockets or security essentials like hostname verification. I envision that in the long run we should have a clean and easy to use interface to OpenSSL, not only for SSL but also to the X509 and crypto functionality. The interfaces ruby or python provide are much cleaner than what we have in Perl today. And it should not have anything with SSLeay anymore in the name :) I don't see it based on Crypt::SSLeay because this is just too far away from providing enough functionality. I think a rewrite would be easier and we could use the experience we got when writing the Net::SSLeay code. I'm willing to help and we also have some useful code at work which we probably could share. IO::Socket::SSL then would be re-implemented on top of this new module(s). But this will probably take a long time, maybe as many years as adding basic IPv6 support in the core took. That's why we need to think of the present too. I think, that it would be better to remove Crypt::SSLeay from LWP as soon as possible, because Crypt::SSLeay is too insecure. Especially the missing hostname verification makes man-in-the-middle attacks easy. We should also officially deprecate the module now. If there are users outside of LWP we should help them to migrate to IO::Socket::SSL (usually this simplifies the code). Again, I'm willing to help. Regards, Steffen
Download (untitled) / with headers
text/plain 2.3k
On Sat Aug 16 17:22:44 2014, SULLR wrote: Show quoted text
> I think, that it would be better to remove > Crypt::SSLeay from LWP as soon as possible, because Crypt::SSLeay is > too insecure. Especially the missing hostname verification makes man- > in-the-middle attacks easy. We should also officially deprecate the > module now. If there are users outside of LWP we should help them to > migrate to IO::Socket::SSL (usually this simplifies the code). > > Again, I'm willing to help.
Hello Steffen: I am wondering if I am missing something here. 1) Crypt::SSLeay is not part of LWP. 2) Installing a recent version of Crypt::SSLeay pulls in LWP::Protocol::https which in turn pulls in IO::Socket::SSL. LWP uses Net::HTTPS which defaults to using IO::Socket::SSL. 3) Therefore, users of Crypt::SSLeay end up opting in to use IO::Socket::SSL unless THEY TAKE SPECIFIC ACTION to force LWP to use Crypt::SSLeay, and use it without hostname verification. That is, if you install Crypt::SSLeay, then force LWP to use it like this: use Net::SSL; use LWP::UserAgent; my $ua = LWP::UserAgent->new; my $response = $ua->get('https://www.example.com/'); print $response->content, "\n"; Then, you will get: Show quoted text
> Can't connect to www.example.com:443 (Crypt-SSLeay can't verify hostnames)
Show quoted text
> Net::SSL from Crypt-SSLeay can't verify hostnames; > either install IO::Socket::SSL or turn off verification > by setting the PERL_LWP_SSL_VERIFY_HOSTNAME environment > variable to 0
That is, it is not possible to silently disable host verification by installing Crypt::SSLeay. One must TAKE SPECIFIC ACTION to do that. While that may not be desirable from your perspective, I do not see the concern if people knowingly do something. What is the action item with which you are offering to help? Right now, things are set up such that people who choose not to verify hostnames can, but people who take no specific action opt in to using hostname verification. As in the following program: use LWP::UserAgent; my $ua = LWP::UserAgent->new; my $response = $ua->get('https://www.example.com/'); print $response->content, "\n"; will fail: Show quoted text
> Can't connect to www.example.com:443 (certificate verify failed)
Show quoted text
> LWP::Protocol::https::Socket: SSL connect attempt failed error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
HTH, -- Sinan
Download (untitled) / with headers
text/plain 4.3k
Show quoted text
> 3) Therefore, users of Crypt::SSLeay end up opting in to use > IO::Socket::SSL unless THEY TAKE SPECIFIC ACTION to force LWP to use > Crypt::SSLeay, and use it without hostname verification.
You are correct, but unfortunately you find the recommendation to do exactly this on lots of places. If you search for "lwp ssl" or "lwp https" you find Crypt::SSLeay and examples which enforce use of it in the top search hits and if you monitor stackoverflow you see people referring with https problems in LWP to old posts describing the use of Crypt::SSLeay or even actively recommend to enforce use of Crypt::SSLeay over IO::Socket::SSL. And because this often solves the problem, users will do it. I try to find and refute this posts and give the correct recommendations, but it's a hard fight because often it needs the help from the user to pin down the real problem instead of just ignoring it. And, of course, usually none of the posters which recommend Crypt::SSLeay point out where you decrease the security if you use it instead of IO::Socket::SSL, because the poster usually does not understand the problem either. And while I appreciate that Crypt::SSLeay explicitly recommends to not use it and to use IO::Socket::SSL instead, probably most users install Crypt::SSLeay not from the source. From http://www.activestate.com/activeperl/ppm-perl-modules I see that Crypt::SSLeay is packaged with ActiveState Perl while IO::Socket::SSL is not, which means that LWP (with the help of Net::HTTPS) will probably silently fall back to use Crypt::SSLeay unless the user specifically installs IO::Socket::SSL. And while Debian has the dependencies correctly you can also install Crypt::SSLeay there easily without getting any warnings that it should better not be used. Show quoted text
> Then, you will get: >
> > Can't connect to www.example.com:443 (Crypt-SSLeay can't verify > > hostnames)
>
> > Net::SSL from Crypt-SSLeay can't verify hostnames; > > either install IO::Socket::SSL or turn off verification > > by setting the PERL_LWP_SSL_VERIFY_HOSTNAME environment > > variable to 0
> > That is, it is not possible to silently disable host verification by > installing Crypt::SSLeay. One must TAKE SPECIFIC ACTION to do that. > > While that may not be desirable from your perspective, I do not see > the concern if people knowingly do something.
If the user has environment settings for HTTPS_CA_FILE or HTTPS_CA_DIR it will not need to set PERL_LWP_SSL_VERIFY_HOSTNAME because LWP::UserAgent assumes that the user upgraded from Crypt::SSLeay and will *silently* disable hostname verification (see source of LWP::UserAgent::new). Of course you find the recommendations to set HTTPS_CA_* this way because older LWP versions did not depend on Mozilla::CA and thus needed a way to specify the place for the trusted CA. You also find these environment variables documented in LWP::UserAgent without the information that it will disable hostname verification if you use them. Also, lots of users don't understand why the hostname verification is an important part of SSL and that just checking for a proper CA is not sufficient. So they just follow the recommendation to disable it. IO::Socket::SSL provides a better way to deal with this problem: If the hostname in the certificate is not the name inside the URL you can knowingly provide another hostname it should use instead to verify the certificate. Thus you keep the check against the expected hostname and this is much more secure than completely disabling hostname verification. Show quoted text
> What is the action item with which you are offering to help? Right > now, things are set up such that people who choose not to verify > hostnames can, but people who take no specific action opt in to using > hostname verification. As in the following program:
One goal I have is that LWP should do the right thing out of the box and that it should be very hard to unknowingly do the bad thing. Right now it is just too easy to do the wrong thing accidentally. Removing support for Crypt::SSLeay would help here, but one has to be sure that there are no important use cases anymore (like platforms unsupported by IO::Socket::SSL but supported by Crypt::SSLeay). I'm not sure if Crypt::SSLeay has other users than LWP. If this is the case one could enhance Crypt::SSLeay by making it a wrapper around IO::Socket::SSL and that is where I offered help. But, if LWP is the only important user then this would not be necessary. Regards, Steffen
Download (untitled) / with headers
text/plain 350b
On Sat Sep 06 03:22:49 2014, SULLR wrote: Show quoted text
> I'm not sure if Crypt::SSLeay has other users than LWP.
I understand your other concerns, but I need to think about those. As for this specific point, the module specifically states "Crypt::SSLeay - OpenSSL support for LWP". I am not aware of any other use cases, but that is not saying much. -- Sinan
Download (untitled) / with headers
text/plain 1.1k
On Sat Sep 06 03:22:49 2014, SULLR wrote: Show quoted text
> http://www.activestate.com/activeperl/ppm-perl-modules I see that > Crypt::SSLeay is packaged with ActiveState Perl while IO::Socket::SSL > is not, which means that LWP (with the help of Net::HTTPS) will > probably silently fall back to use Crypt::SSLeay unless the user > specifically installs IO::Socket::SSL.
Here's where I start getting confused. LWP does not fall back silently to using Crypt::SSLeay without user action to ensure that. Also, installing a recent version of LWP::Protocol::https requires installing IO::Socket::SSL. In any case, I looked at ActiveState's build log http://ppm4.activestate.com/MSWin32-x86/5.16/1600/S/SU/SULLR/IO-Socket-SSL-1.997.d/log-20140715T055247.txt for IO::Socket::SSL. Show quoted text
> Net::SSLeay could not find a random number generator on > your system. This will likely cause most of the tests > to fail. Please see the README file for more information. > the message from Net::SSLeay was: Random number generator not seeded!!!
Maybe it would be a good idea to ensure IO::Socket::SSL actually builds on AS's machines so everyone who installs Perl on Windows can get the benefit. -- Sinan
Download (untitled) / with headers
text/plain 1.3k
On Sun Sep 07 09:07:51 2014, NANIS wrote: Show quoted text
> In any case, I looked at ActiveState's build log > > http://ppm4.activestate.com/MSWin32-x86/5.16/1600/S/SU/SULLR/IO- > Socket-SSL-1.997.d/log-20140715T055247.txt > > for IO::Socket::SSL. >
> > Net::SSLeay could not find a random number generator on > > your system. This will likely cause most of the tests > > to fail. Please see the README file for more information. > > the message from Net::SSLeay was: Random number generator not > > seeded!!!
Using https://metacpan.org/pod/Crypt::Random::Source::Strong::Win32 in addition to /dev/random and /dev/urandom in Net::SSLeay might help. Also, I looked at the README.Win32 file in Net::SSLeay, and I would never consider following those instructions. I've always built my own OpenSSL on Windows, following the instructions that come with the library, and using the VC Express tools. So, Crypt::SSLeay is insecure because it does not do host-verification ... Although you might be totally fine if your program is communicating with a server that is also under your control, where you installed your certificate, and told your LWP program what certificates to trust. If you get Net::SSLeay and IO::Socket::SSL built on Windows (as I did), you are stuck without a good source of randomness. I must admit, this is just from a cursory look, but is that really so? -- Sinan
Download (untitled) / with headers
text/plain 997b
Show quoted text
> In any case, I looked at ActiveState's build log > > http://ppm4.activestate.com/MSWin32-x86/5.16/1600/S/SU/SULLR/IO- > Socket-SSL-1.997.d/log-20140715T055247.txt > > for IO::Socket::SSL.
I doubt that these reports are correct. If you look at the same place you find also failed build reports for Net::SSLeay and Crypt::SSLeay, where they just failed to set up OpenSSL properly. Show quoted text
> Maybe it would be a good idea to ensure IO::Socket::SSL actually > builds on AS's machines so everyone who installs Perl on Windows can > get the benefit.
I don't know if the PPM build reports are just junk or if people outside of ActiveState manage to do what ActiveState does not. At least I get successful reports with ActiveState Perl on CPAN, like this one: http://www.cpantesters.org/cpan/report/b9bd916f-6bf4-1014-ad44-60d3b85c3ac1 So it does not seem to be a problem of the modules itself. And I'm also not getting any bug reports from the ActiveState guys so that I doubt that are serious problems.
Download (untitled) / with headers
text/plain 977b
Show quoted text
> > So, Crypt::SSLeay is insecure because it does not do host-verification > ... Although you might be totally fine if your program is > communicating with a server that is also under your control, where you > installed your certificate, and told your LWP program what > certificates to trust.
Yes, within the limited environment where you have the PKI under your control and each of yours CA only issued only a single certificate (otherwise you would need to check hostname) it is safe. Unfortunately that is not the environment most people use. Show quoted text
> If you get Net::SSLeay and IO::Socket::SSL built on Windows (as I > did), you are stuck without a good source of randomness. > > I must admit, this is just from a cursory look, but is that really so?
I doubt it. At least I don't get any problem reports and from the look into the output from cpantesters I cannot find any problems too. So this are hopefully only problems which were solved in OpenSSL on Windows long ago.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.