Skip Menu |
 

This queue is for tickets about the HTML-Parser CPAN distribution.

Report information
The Basics
Id: 83570
Status: open
Priority: 0/
Queue: HTML-Parser

People
Owner: Nobody in particular
Requestors: ceklof [...] thanxmedia.com
Cc:
AdminCc:

Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)



Subject: Incorrect tokenization in HTML::Parser
Date: Sat, 23 Feb 2013 17:42:30 +0000
To: "bug-HTML-Parser [...] rt.cpan.org" <bug-HTML-Parser [...] rt.cpan.org>
From: Carl Eklof <ceklof [...] thanxmedia.com>
Download (untitled) / with headers
text/plain 2.5k
Hi Gisle, First, thank you for all of your huge contributions to Perl over the years! I've discovered a site (http://www.scotts.com/) that has HTML that HTML-Parser does not tokenize correctly. Envs (tried on two machines, same results): * HTML::Parser (3.65 and 3.69) * Perl 5.14.2, and 5.10.1 * 'full_uname' => 'Linux 449876-app3.blosm.com 2.6.18-238.37.1.el5 #1 SMP Fri Apr 6 13:47:10 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux', * 'os_distro' => 'Red Hat Enterprise Linux Server release 5.9 (Tikanga) Kernel \\r on an \\m<file:///\\m>', * 'full_uname' => 'Linux idx02 2.6.43.5-2.fc15.x86_64 #1 SMP Tue May 8 11:09:22 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux', * 'os_distro' => 'Fedora release 15', I'm attaching a representative page. The page came from: http://www.scotts.com/smg/templates/index.jsp?pageUrl=orthoLanding The problem seems to occur around the HTML: <noscript> <iframe height="0" width="0" style="display:none; visibility:hidden;" src="//www.googletagmanager.com/ns.html?id=GTM-PVLS" /> </noscript> <script> I've added some debugging to the HTML::TokeParser::get_tag sub so it looks like: use Data::Dumper; sub get_tag { my $self = shift; my $token; while (1) { $token = $self->get_token || return undef; warn "Checking token: [".Dumper($token)."]"; my $type = shift @$token; next unless $type eq "S" || $type eq "E"; substr($token->[0], 0, 0) = "/" if $type eq "E"; return $token unless @_; for (@_) { return $token if $token->[0] eq $_; } } } I've tried both version 3.65 and 3.69 of HTML::Parser, which both produce the same results. They produce output in the "output" attachment. You can see on like 290 of the output that it is tokenizing almost the entire page after the iframe as one big text blob. Thanks again, -Carl Carl Eklof CTO @ Blosm Inc. blosm.com<http://blosm.com/> 424.888.4BEE Confidentiality Note: This e-mail message and any attachments to it are intended only for the named recipients and may contain confidential information. If you are not one of the intended recipients, please do not duplicate or forward this e-mail message and immediately delete it from your computer. By accepting and opening this email, recipient agrees to keep all information confidential and is not allowed to distribute to anyone outside their organization.
Download (untitled) / with headers
text/html 18.4k

Message body is not shown because it is too large.

Message body is not shown because sender requested not to inline it.

Message body is not shown because sender requested not to inline it.

Download (untitled) / with headers
text/plain 322b
I've been seeing this with some code I'm working on soon. To summarize this very simply, it seems like HTML::TokeParser does something weird when a tag contains a self-closing slash. If the tag is written as "<hr/>" then the parser things the tag is "hr/". If it's written as "<hr />" then we end up with a "/" attribute.
Download (untitled) / with headers
text/plain 244b
I cloned the repo with the intention of fixing this, but when I looked through the test cases I realized that this behavior is actually tested for. Gisle, what's up with this? It's not documented, AFAICT, and it really doesn't make much sense.
On Sun Jan 04 11:00:15 2015, DROLSKY wrote:
Show quoted text
> I cloned the repo with the intention of fixing this, but when I looked
> through the test cases I realized that this behavior is actually
> tested for.
>
> Gisle, what's up with this? It's not documented, AFAICT, and it really
> doesn't make much sense.

Perhaps just based on my understanding of what status this had based on this advice from the XHTML spec.

C.2. Empty Elements

Include a space before the trailing / and > of empty elements, e.g. <br />, <hr /> and <img src="karen.jpg" alt="Karen" />. Also, use the minimized tag syntax for empty elements, e.g. <br />, as the alternative syntax <br></br> allowed by XML gives uncertain results in many existing user agents.

http://www.w3.org/TR/html5/syntax.html#tag-name-state seems clear on allowing this, so feel free to change the tests
Just turning on the "empty_element_tags" option might make the parser behave the way you expect.  It might be that we should just switch the default for this option.


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.