Skip Menu |
 

This queue is for tickets about the CGI-Uploader CPAN distribution.

Report information
The Basics
Id: 12058
Status: open
Priority: 0/
Queue: CGI-Uploader

People
Owner: MARKSTOS [...] cpan.org
Requestors: william [...] knowmad.com
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: 1.00
Fixed in: (no value)

Attachments


Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
Subject: Preserve name of uploaded files / Directory mapping
X-RT-Original-Encoding: iso-8859-1
Content-Length: 460
Download (untitled) / with headers
text/plain 460b
Hello again Mark, It'd be really nice if the name of the file could be preserved when saving to the filesystem as well as stored into the uploads table. Some error checking would need to added to ensure that duplicates are not overwritten (or add a flag to allow overwrite). Along these lines, it'd be useful to add the feature of storing files into directory hierarchies like CPAN.pm does for authors. (I don't know what this technique is called). William
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
From: william [...] knowmad.com
Content-Type: multipart/mixed; boundary="----------=_1112129813-31900-0"
Content-Length: 0
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: iso-8859-1
Content-Length: 509
Download (untitled) / with headers
text/plain 509b
Actually preserving the file_name turned out to be easier than I expected. I only had to add a line to extract_meta, tweak the call to store_file in store_upload and remove an assertion in store_file. I now have a working example. I've attached a patch so that you can review my modifications and consider adding this feature for future releases. Note that this version will overwrite any existing files with the same name. This patch includes the mods that I added to support Pg via ODBC. Enjoy! William
Content-Type: application/octet-stream; name="Uploader.pm.diff2"
Content-Disposition: inline; filename="Uploader.pm.diff2"
Content-Transfer-Encoding: base64
Content-Length: 1700
Download Uploader.pm.diff2
application/octet-stream 1.6k

Message body not shown because it is not plain text.

Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
X-RT-Original-Encoding: iso-8859-1
Content-Length: 272
Download (untitled) / with headers
text/plain 272b
Oh, and you need to include the file_name field in your uploads table: file_name character varying(255), Also, if you use up_table_map in your spec, you should add the following line: file_name => 'file_name', I think that's everything. William
Return-Path: <mark [...] summersault.com>
X-Original-To: bug-CGI-Uploader [...] rt.cpan.org
Delivered-To: cpan-bug+cgi-uploader [...] diesel.bestpractical.com
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id D14A94D809E for <bug-CGI-Uploader [...] rt.cpan.org>; Tue, 29 Mar 2005 16:00:07 -0500 (EST)
Received: (qmail 3380 invoked by alias); 29 Mar 2005 21:00:05 -0000
X-Spam-Check-BY: la.mx.develooper.com
Received-SPF: pass (x1.develooper.com: local policy)
Received: from tanagra.summersault.com (HELO tanagra.summersault.com) (12.161.105.149) by la.mx.develooper.com (qpsmtpd/0.28) with SMTP; Tue, 29 Mar 2005 13:00:02 -0800
Received: (qmail 69879 invoked from network); 29 Mar 2005 20:59:58 -0000
Received: from simba.summersault.com (12.161.105.182) by tanagra.summersault.com with SMTP; 29 Mar 2005 20:59:58 -0000
Received: by simba.summersault.com (Postfix, from userid 501) id 603E148956; Tue, 29 Mar 2005 16:00:45 -0500 (EST)
Date: Tue, 29 Mar 2005 16:00:45 -0500
From: Mark Stosberg <mark [...] summersault.com>
To: Guest via RT <bug-CGI-Uploader [...] rt.cpan.org>
Subject: Re: [cpan #12058] Preserve name of uploaded files / Directory mapping
Message-ID: <20050329210045.GY32231 [...] summersault.com>
References: <rt-12058 [...] cpan> <rt-12058-35064.8.33264806023124 [...] cpan.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <rt-12058-35064.8.33264806023124 [...] cpan.org>
User-Agent: Mutt/1.5.6i
RT-Send-Cc:
X-RT-Original-Encoding: us-ascii
Content-Length: 1083
On Tue, Mar 29, 2005 at 03:21:40PM -0500, Guest via RT wrote: Show quoted text
> > It'd be really nice if the name of the file could be preserved when > saving to the filesystem as well as stored into the uploads table. > Some error checking would need to added to ensure that duplicates are > not overwritten (or add a flag to allow overwrite).
I'm not interested in preserving the file name on the file system (although I would consider a patch). What's an example of when you would use the file name if it was stored in the database? I never use them. I suppose rt.cpan.org uses these to some affect-- to display the name of attachments to download. Some care would need to be taken that the file name was "safe"-- didn't include some evil XSS for example. :) Show quoted text
> Along these lines, it'd be useful to add the feature of storing files > into directory hierarchies like CPAN.pm does for authors. (I don't > know what this technique is called).
Did you find the option named 'file_scheme'? It does something like that, and I use it to store what I suspect is about 1,000,000 images. Mark
Return-Path: <mark [...] summersault.com>
X-Original-To: bug-CGI-Uploader [...] rt.cpan.org
Delivered-To: cpan-bug+cgi-uploader [...] diesel.bestpractical.com
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id B28344D80A2 for <bug-CGI-Uploader [...] rt.cpan.org>; Tue, 29 Mar 2005 16:02:03 -0500 (EST)
Received: (qmail 3843 invoked by alias); 29 Mar 2005 21:02:03 -0000
X-Spam-Check-BY: la.mx.develooper.com
Received-SPF: pass (x1.develooper.com: local policy)
Received: from tanagra.summersault.com (HELO tanagra.summersault.com) (12.161.105.149) by la.mx.develooper.com (qpsmtpd/0.28) with SMTP; Tue, 29 Mar 2005 13:02:01 -0800
Received: (qmail 70235 invoked from network); 29 Mar 2005 21:01:58 -0000
Received: from simba.summersault.com (12.161.105.182) by tanagra.summersault.com with SMTP; 29 Mar 2005 21:01:58 -0000
Received: by simba.summersault.com (Postfix, from userid 501) id 5409048956; Tue, 29 Mar 2005 16:02:45 -0500 (EST)
Date: Tue, 29 Mar 2005 16:02:45 -0500
From: Mark Stosberg <mark [...] summersault.com>
To: Guest via RT <bug-CGI-Uploader [...] rt.cpan.org>
Subject: Re: [cpan #12058] Preserve name of uploaded files / Directory mapping
Message-ID: <20050329210245.GZ32231 [...] summersault.com>
References: <rt-12058 [...] cpan> <rt-12058-35066.5.2038193136287 [...] cpan.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <rt-12058-35066.5.2038193136287 [...] cpan.org>
User-Agent: Mutt/1.5.6i
RT-Send-Cc:
X-RT-Original-Encoding: us-ascii
Content-Length: 741
Download (untitled) / with headers
text/plain 741b
On Tue, Mar 29, 2005 at 03:59:27PM -0500, Guest via RT wrote: Show quoted text
> > This message about CGI-Uploader was sent to you by guest <> via rt.cpan.org > > Full context and any attached attachments can be found at: > <URL: https://rt.cpan.org/Ticket/Display.html?id=12058 > > > Oh, and you need to include the file_name field in your uploads table: > > file_name character varying(255), > > Also, if you use up_table_map in your spec, you should add the following > line: > > file_name => 'file_name', > > I think that's everything.
If you work out a few more details, this could turn into a real patch. :) ( Secretly, I'm hard at work on finishing a DFV 4.0 release with some nice interface enhancements. ) Mark
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
X-RT-Original-Encoding: iso-8859-1
Content-Length: 1525
Download (untitled) / with headers
text/plain 1.4k
Hi Mark, Glad you're warming up to the idea even if you don't need it personally. Here's the use case (which also has a placeholder in your cookbook): I need to create an anonymous uploader which can accept up to 5 files which are stored onto a web server. An email will be sent to a selected recipient. This message will have a hyperlink(s) to the uploaded file(s). I need to preserve file name so that any communication between the uploader and the recipient will make sense. BTW, I'd be glad to include the current project I'm building in the examples. Now for the gruesome details. Since you are more familiar with the API, can you suggest a way of indicating when to preserve filenames? I'd also need a flag to indicate whether to overwrite existing files. Perhaps a new spec option such as file_name which would default to "id" but could also be "filename" to support my proposal as well as future growth. I'll also need to fix fk_meta to return a proper path. Also, what happens if a field is in up_table_map but isn't available in the uploads database? Will that generate an error? I can probably delete the file_name field from the map if we are using id's as filenames. Thanks for the note about the file_scheme option. I suppose one option is to add a new file_scheme to support formats such as I had suggested: w/william.jpg (I've only built a one-level hierarchy; if you have code to support multiple levels, I'd be glad to include it). For now, I'll stick with the one directory. Cheers! William
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
From: william [...] knowwmad.com
Content-Type: multipart/mixed; boundary="----------=_1112189892-23148-0"
Content-Length: 0
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: iso-8859-1
Content-Length: 1245
Download (untitled) / with headers
text/plain 1.2k
OK, here it is. I was able to get all the tests to pass the original Uploader.pm then applied my changes. I've also included a new test file to ensure that my code is working as advertised. The attached archive contains patches for Upload.pm and create_uploader_table.Pg.sql (sorry, I don't have MySQL installed) as well as the new file. I think that should be enough to patch your version. Let me know how it goes. One issue that I'm not sure how to handle is not including the file_name field in the uploads database. I suspect there's a way to handle this but it would probably take some monkeying with the shared_meta. I noticed that you had an entry, bytes, commented out in the default up_table_map. Perhaps you already have a way of supporting extra fields that I can tap into. The problem I see with not putting file_name into up_table_map is that the field name cannot be easily changed by the user. If you incoporate these changes, I'll also send over my uploader cgi program that supports anonymous uploads of up to 5 files at a time and sends an email notification to any interested parties. Thanks, William PS: In the store_upload method, you are running extract_meta twice. The first one as it doesn't appear to be necessary.
Content-Type: application/x-bzip2; name="CGI-Uploader.diff.tar.bz2"
Content-Disposition: inline; filename="CGI-Uploader.diff.tar.bz2"
Content-Transfer-Encoding: base64
Content-Length: 3672
Download CGI-Uploader.diff.tar.bz2
application/x-bzip2 3.5k

Message body not shown because it is not plain text.

Return-Path: <mark [...] summersault.com>
X-Original-To: bug-CGI-Uploader [...] rt.cpan.org
Delivered-To: cpan-bug+cgi-uploader [...] diesel.bestpractical.com
Received: from la.mx.develooper.com (x1.develooper.com [63.251.223.170]) by diesel.bestpractical.com (Postfix) with SMTP id 467D84D80D5 for <bug-CGI-Uploader [...] rt.cpan.org>; Wed, 30 Mar 2005 09:00:24 -0500 (EST)
Received: (qmail 21901 invoked by alias); 30 Mar 2005 14:00:23 -0000
X-Spam-Check-BY: la.mx.develooper.com
Received-SPF: pass (x1.develooper.com: local policy)
Received: from tanagra.summersault.com (HELO tanagra.summersault.com) (12.161.105.149) by la.mx.develooper.com (qpsmtpd/0.28) with SMTP; Wed, 30 Mar 2005 06:00:22 -0800
Received: (qmail 10529 invoked from network); 30 Mar 2005 14:00:18 -0000
Received: from simba.summersault.com (12.161.105.182) by tanagra.summersault.com with SMTP; 30 Mar 2005 14:00:18 -0000
Received: by simba.summersault.com (Postfix, from userid 501) id ACC1B4898E; Wed, 30 Mar 2005 09:01:04 -0500 (EST)
Date: Wed, 30 Mar 2005 09:01:04 -0500
From: Mark Stosberg <mark [...] summersault.com>
To: Guest via RT <bug-CGI-Uploader [...] rt.cpan.org>
Subject: Re: [cpan #12058] Preserve name of uploaded files / Directory mapping
Message-ID: <20050330140104.GF11302 [...] summersault.com>
References: <rt-12058 [...] cpan> <rt-12058-35097.10.7725621914324 [...] cpan.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <rt-12058-35097.10.7725621914324 [...] cpan.org>
User-Agent: Mutt/1.5.6i
RT-Send-Cc:
X-RT-Original-Encoding: us-ascii
Content-Length: 362
Download (untitled) / with headers
text/plain 362b
On Wed, Mar 30, 2005 at 08:38:12AM -0500, Guest via RT wrote: Show quoted text
> > This message about CGI-Uploader was sent to you by guest <> via rt.cpan.org > > Full context and any attached attachments can be found at: > <URL: https://rt.cpan.org/Ticket/Display.html?id=12058 > > > OK, here it is.
Thanks a bunch. I hope to look at this in detail this weekend. Mark
MIME-Version: 1.0
X-Mailer: MIME-tools 5.415 (Entity 5.415)
From: william [...] knowwmad.com
Content-Type: multipart/mixed; boundary="----------=_1112196490-26791-0"
Content-Length: 0
Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
X-RT-Original-Encoding: iso-8859-1
Content-Length: 875
Download (untitled) / with headers
text/plain 875b
[mark@summersault.com - Wed Mar 30 09:00:26 2005]: Show quoted text
> Thanks a bunch. I hope to look at this in detail this weekend.
Well let me toss out another version for you to review. This one does not automatically add the file_name to up_table_map. This means that I'm using a different sql file for generating the table when including the file_name. Also while working on this modification, I had Image::Magick go wacko on me. While debugging, I added some tests to the ImageMagick.pm module. I've included a patch in case you want to add them. It would have saved me a few minutes while tracking down the bug. All tests are passing for me using Perl 5.8.4 under Debian Sarge. I promise to stop thinking about this module now!! Thanks, William PS: I figured out why you called extract_meta twice in store_upload; it's being called with slightly different data the second time.
Content-Type: application/x-bzip2; name="CGI-Uploader-v2.diff.tar.bz2"
Content-Disposition: inline; filename="CGI-Uploader-v2.diff.tar.bz2"
Content-Transfer-Encoding: base64
Content-Length: 4253
Download CGI-Uploader-v2.diff.tar.bz2
application/x-bzip2 4.1k

Message body not shown because it is not plain text.

Content-Type: text/plain
Content-Disposition: inline
Content-Transfer-Encoding: binary
MIME-Version: 1.0
X-Mailer: MIME-tools 5.418 (Entity 5.418)
X-RT-Original-Encoding: iso-8859-1
Content-Length: 1331
Download (untitled) / with headers
text/plain 1.2k
Show quoted text
> > I need to create an anonymous uploader which can accept up to 5 files > which are stored onto a web server. An email will be sent to a selected > recipient. This message will have a hyperlink(s) to the uploaded > file(s). I need to preserve file name so that any communication between > the uploader and the recipient will make sense.
William, What about this idea to address this case. I think it solves your desire for clean URLs and my desire for unique IDs on the file system. Now we are storing the file name in the database. So a URL could be constructed that looks like this: images/2/PICT0001.jpg images/3/PICT0002.jpg images/4/PICT0001.jpg # a different one than the first! Using mod_rewrite or our own dispatcher (like CGI::Application::Dispatch) the file actually being accessed would be images/2.jpg. (The actual name would be just for show ). Avoiding file system conflicts seems important, especially for the common digital camera case (illustrated above), where the default names get reset after every roll, generating lots of conflicting file names. Within some photo editting programs, it's easy not to even notice a photo has a bad name, because the interface always shows you a thumbnail of the photo to identify it by. I have some other thoughts to share I'll follow up seperately. Mark


This service is sponsored and maintained by Best Practical Solutions and runs on Perl.org infrastructure.

Please report any issues with rt.cpan.org to rt-cpan-admin@bestpractical.com.