Bug #118433 for Apache-Tika: CommonsDigester calculates wrong hashes on large files

# Wed Oct 19 07:35:51 2016 yahavamsi [...] gmail.com - Ticket created

Subject:	CommonsDigester calculates wrong hashes on large files
Date:	Wed, 19 Oct 2016 14:35:37 +0300
To:	bug-Apache-Tika [...] rt.cpan.org
From:	Yahav Amsalem <yahavamsi [...] gmail.com>

Download (untitled) / with headers
text/plain 1.1k

Hi, I would like to report the next bug description: When passing more than one algorithm to CommonsDigester constructor and then trying to digest a file which is larger than 7.5 MB, results wrong hashe calculation for all the algorithms except the first. The next code will reproduce the bug: *// The file that was used was a simple plain text file with size > 7.5 MB* *File file = new File("c:\\testLargeFile.txt");* *BufferedInputStream bufferedInputStream = new BufferedInputStream(new FileInputStream(file));* *Metadata metadata = new Metadata();* *CommonsDigester digester = new CommonsDigester(20000000,* * CommonsDigester.DigestAlgorithm.MD5,* * CommonsDigester.DigestAlgorithm.SHA1,* * CommonsDigester.DigestAlgorithm.SHA256);* *digester.digest(bufferedInputStream, metadata, null);* *// Will print correct MD5 but wrong SHA1 and wrong SHA256* *System.out.println(metadata);* Initial direction: from a little research it seems that the inner buffered stream that is being used doesn't reset to 0 position after the first algorithm. If there are any further questions I would be happy to deliver more details. Thanks, Yahav Amsalem

Download (untitled) / with headers
text/html 1.7k

# Thu Oct 20 06:59:22 2016 ribugent [...] gmail.com - Correspondence added

Download (untitled) / with headers
text/plain 1.5k

Hi, this bugtracker is for Apache::Tika cpan perl module, you sent the bugreport to the wrong bugt racker, may be https://issues.apache.org/jira/browse/DIGESTER/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel is the right place. Gerard Ribugent Sobre Dmc Oct 19 07:35:51 2016, yahavamsi@gmail.com ha escrit: Show quoted text

> Hi, > > I would like to report the next bug description: > > When passing more than one algorithm to CommonsDigester constructor and > then trying to digest a file which is larger than 7.5 MB, results wrong > hashe calculation for all the algorithms except the first. > > The next code will reproduce the bug: > > *// The file that was used was a simple plain text file with size > 7.5 MB* > *File file = new File("c:\\testLargeFile.txt");* > > *BufferedInputStream bufferedInputStream = new BufferedInputStream(new > FileInputStream(file));* > > *Metadata metadata = new Metadata();* > > *CommonsDigester digester = new CommonsDigester(20000000,* > * CommonsDigester.DigestAlgorithm.MD5,* > * CommonsDigester.DigestAlgorithm.SHA1,* > * CommonsDigester.DigestAlgorithm.SHA256);* > > *digester.digest(bufferedInputStream, metadata, null);* > > *// Will print correct MD5 but wrong SHA1 and wrong SHA256* > *System.out.println(metadata);* > > Initial direction: from a little research it seems that the inner buffered > stream that is being used doesn't reset to 0 position after the first > algorithm. > > > If there are any further questions I would be happy to deliver more details. > > > Thanks, > > Yahav Amsalem

# Thu Oct 20 06:59:23 2016 The RT System itself - Status changed from 'new' to 'open'

# Thu Oct 20 06:59:33 2016 ribugent [...] gmail.com - Status changed from 'open' to 'rejected'

# Thu Oct 20 07:54:42 2016 yahavamsi [...] gmail.com - Correspondence added

Subject:	Re: [rt.cpan.org #118433] CommonsDigester calculates wrong hashes on large files
Date:	Thu, 20 Oct 2016 14:54:30 +0300
To:	bug-Apache-Tika [...] rt.cpan.org
From:	Yahav Amsalem <yahavamsi [...] gmail.com>

Download (untitled) / with headers
text/plain 1.9k

Hi, I noticed that after I sent the mail but thought it might be of help to you too. Anyway, thanks for the kind reply, Yahav Amsalem בתאריך 20 באוק׳ 2016 2:00 אחה״צ,‏ "Gerard via RT" < bug-Apache-Tika@rt.cpan.org> כתב: Show quoted text

> <URL: https://rt.cpan.org/Ticket/Display.html?id=118433 > > > Hi, > > this bugtracker is for Apache::Tika cpan perl module, you sent the > bugreport to the wrong bugt racker, may be https://issues.apache.org/ > jira/browse/DIGESTER/?selectedTab=com.atlassian.jira.jira-projects-plugin: > summary-panel is the right place. > > Gerard Ribugent > > Sobre Dmc Oct 19 07:35:51 2016, yahavamsi@gmail.com ha escrit:

> > Hi, > > > > I would like to report the next bug description: > > > > When passing more than one algorithm to CommonsDigester constructor and > > then trying to digest a file which is larger than 7.5 MB, results wrong > > hashe calculation for all the algorithms except the first. > > > > The next code will reproduce the bug: > > > > *// The file that was used was a simple plain text file with size > 7.5

> MB*

> > *File file = new File("c:\\testLargeFile.txt");* > > > > *BufferedInputStream bufferedInputStream = new BufferedInputStream(new > > FileInputStream(file));* > > > > *Metadata metadata = new Metadata();* > > > > *CommonsDigester digester = new CommonsDigester(20000000,* > > * CommonsDigester.DigestAlgorithm.MD5,* > > * CommonsDigester.DigestAlgorithm.SHA1,* > > * CommonsDigester.DigestAlgorithm.SHA256);* > > > > *digester.digest(bufferedInputStream, metadata, null);* > > > > *// Will print correct MD5 but wrong SHA1 and wrong SHA256* > > *System.out.println(metadata);* > > > > Initial direction: from a little research it seems that the inner

> buffered

> > stream that is being used doesn't reset to 0 position after the first > > algorithm. > > > > > > If there are any further questions I would be happy to deliver more

> details.

> > > > > > Thanks, > > > > Yahav Amsalem

> > > >

Download (untitled) / with headers
text/html 2.8k

Bug #118433 for Apache-Tika: CommonsDigester calculates wrong hashes on large files

Preferred bug tracker