Skip Menu |
 

This queue is for tickets about the HTTP-Proxy CPAN distribution.

Report information
The Basics
Id: 90414
Status: resolved
Priority: 0/
Queue: HTTP-Proxy

People
Owner: Nobody in particular
Requestors: vincenzo.buttazzo [...] altervista.it
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: 0.303



Subject: HTTPS data transfer fails
Date: Fri, 15 Nov 2013 18:02:55 +0100
To: bug-HTTP-Proxy [...] rt.cpan.org
From: "Vincenzo Buttazzo" <vincenzo.buttazzo [...] altervista.it>
Download (untitled) / with headers
text/plain 1.3k
Hi, we're using HTTP::Proxy to connect PHP scripts on our shared hosting with the internet. In this context a customized version of HTTP::Proxy is used to block connections to undesired domains. Things work good except for HTTPS connections because with some web servers we don't got any data back after the protocol handshake and sending POST data. After some investigation we found out that the size of POST data was related to the problem and finally we found a solution adding this line after the opening of the $upstream socket in the _handle_CONNECT function (around row 600), just after the $upstream check block: $upstream->setsockopt(SOL_SOCKET, SO_SNDBUF, $conn->getsockopt(SOL_SOCKET, SO_RCVBUF)); In fact in our context the $upstream's "send buffer" was smaller than the packets incoming from $conn, so data packets sent from PHP to HTTP::Proxy where splitted when relayed to the external website. Our supposition is that this behavior breaks someway the HTTP protocol, but the fix works good and now HTTPS works for all websites. The problem was not related to our customization because we had the same problem using a vanilla HTTP::Proxy. So, thanks for your great module, and I hope my hint can be useful for improving the module. Bye! -- Vincenzo Buttazzo Responsabile sviluppo web http://www.altervista.org/
Download (untitled) / with headers
text/plain 1.8k
On Fri Nov 15 11:58:44 2013, vincenzo.buttazzo@altervista.it wrote: Show quoted text
> Hi, > we're using HTTP::Proxy to connect PHP scripts on our shared hosting with > the internet. In this context a customized version of HTTP::Proxy is used > to block connections to undesired domains. > > Things work good except for HTTPS connections because with some web > servers we don't got any data back after the protocol handshake and > sending POST data. > > After some investigation we found out that the size of POST data was > related to the problem and finally we found a solution adding this line > after the opening of the $upstream socket in the _handle_CONNECT function > (around row 600), just after the $upstream check block: > > $upstream->setsockopt(SOL_SOCKET, SO_SNDBUF, $conn->getsockopt(SOL_SOCKET, > SO_RCVBUF)); > > In fact in our context the $upstream's "send buffer" was smaller than the > packets incoming from $conn, so data packets sent from PHP to HTTP::Proxy > where splitted when relayed to the external website. Our supposition is > that this behavior breaks someway the HTTP protocol, but the fix works > good and now HTTPS works for all websites. > > The problem was not related to our customization because we had the same > problem using a vanilla HTTP::Proxy. > > > So, thanks for your great module, and I hope my hint can be useful for > improving the module. > > Bye! > >
I'm not sure if this is the best module to use for HTTPS proxying, but also you could be shrinking the size of the send buffer, making it truncate portions of the transmission. I would think that the data you send to the proxy would end up being larger than that which you received since you're adding proxy headers as well. The module `IO::Stream::Proxy::HTTPS` seems like it might be a better option for proxying HTTPS. Does this sound like gibberish?
Subject: Re: [rt.cpan.org #90414] HTTPS data transfer fails
Date: Mon, 13 Apr 2015 11:53:17 +0200
To: bug-HTTP-Proxy [...] rt.cpan.org
From: Vincenzo Buttazzo <vincenzo.buttazzo [...] altervista.it>
Download (untitled) / with headers
text/plain 1002b
Il 12/04/2015 16:27, Michael D. Stemle, Jr. via RT ha scritto: Show quoted text
> I'm not sure if this is the best module to use for HTTPS proxying,
Maybe, but it supports HTTP and HTTPS at the same time and that's our case. Show quoted text
> but also you could be shrinking the size of the send buffer, making it > truncate portions of the transmission.
Yes, that's the other way. The important thing is that both socket's buffers have to be of the same size. Show quoted text
> I would think that the data you send to the proxy would end up being > larger than that which you received since you're adding proxy headers > as well.
Right, but the proxy first receives the CONNECT header and then opens a socket to the destination. From that point the data sent to the proxy will be passed unchanged from the downstream socket to the upstream socket and viceversa, so the buffer sizes of this sockets have to be the same to not break the encrypted stream into unusable chunks. -- Vincenzo Buttazzo Sviluppo Altervista + GialloZafferano
Download (untitled) / with headers
text/plain 943b
On Fri Nov 15 11:58:44 2013, vincenzo.buttazzo@altervista.it wrote: Show quoted text
> > Things work good except for HTTPS connections because with some web > servers we don't got any data back after the protocol handshake and > sending POST data. > > After some investigation we found out that the size of POST data was > related to the problem and finally we found a solution adding this line > after the opening of the $upstream socket in the _handle_CONNECT function > (around row 600), just after the $upstream check block: > > $upstream->setsockopt(SOL_SOCKET, SO_SNDBUF, $conn->getsockopt(SOL_SOCKET, > SO_RCVBUF)); >
I've applied this change, based on my understanding of the above paragraph (a patch would have been less ambiguous ;-): https://github.com/book/HTTP-Proxy/commit/a3c2514d779d3335d947a52a614d5649bcd31bd0 Please let me know if it works for you, so that I can cut a new release. Thanks for your feedback, -- BooK
Subject: Re: [rt.cpan.org #90414] HTTPS data transfer fails
Date: Tue, 21 Apr 2015 13:50:26 +0200
To: bug-HTTP-Proxy [...] rt.cpan.org
From: Vincenzo Buttazzo <vincenzo.buttazzo [...] altervista.it>
Download (untitled) / with headers
text/plain 1.2k
Sorry for the late reply and the not so precise indication. Anyway that's exactly the same patch we have used in the last 2 years! Il 14/04/2015 18:52, Philippe Bruhat (BooK) via RT ha scritto: Show quoted text
> <URL: https://rt.cpan.org/Ticket/Display.html?id=90414 > > > On Fri Nov 15 11:58:44 2013, vincenzo.buttazzo@altervista.it wrote:
>> Things work good except for HTTPS connections because with some web >> servers we don't got any data back after the protocol handshake and >> sending POST data. >> >> After some investigation we found out that the size of POST data was >> related to the problem and finally we found a solution adding this line >> after the opening of the $upstream socket in the _handle_CONNECT function >> (around row 600), just after the $upstream check block: >> >> $upstream->setsockopt(SOL_SOCKET, SO_SNDBUF, $conn->getsockopt(SOL_SOCKET, >> SO_RCVBUF)); >>
> I've applied this change, based on my understanding of the above paragraph > (a patch would have been less ambiguous ;-): > > https://github.com/book/HTTP-Proxy/commit/a3c2514d779d3335d947a52a614d5649bcd31bd0 > > Please let me know if it works for you, so that I can cut a new release. > > Thanks for your feedback, > > -- BooK >
-- Vincenzo Buttazzo Sviluppo Altervista + GialloZafferano
Download (untitled) / with headers
text/plain 264b
On Tue Apr 21 07:50:41 2015, vincenzo.buttazzo@altervista.it wrote: Show quoted text
> Sorry for the late reply and the not so precise indication. > > Anyway that's exactly the same patch we have used in the last 2 years! >
Thanks! It will be in the next release, then. -- BooK
From: ppisar [...] redhat.com
Download (untitled) / with headers
text/plain 393b
Dne Út 21.dub.2015 16:30:26, BOOK napsal(a): Show quoted text
> On Tue Apr 21 07:50:41 2015, vincenzo.buttazzo@altervista.it wrote:
> > Sorry for the late reply and the not so precise indication. > > > > Anyway that's exactly the same patch we have used in the last 2 years! > >
> > Thanks! It will be in the next release, then. >
This is in released 0.303 version. I think this bug report can be closed.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.