JavaScript apps with read/write access to S3
Update: I've started experiments on S3Ajax.
Notes for my future reference:
<ul>
<li>
<span>There's a <a href="http://pajhome.org.uk/crypt/md5/sha1src.html">SHA1</a> implementation for JavaScript.</span>
</li>
<li>
<span>The <a href="http://developer.amazonwebservices.com/connect/entry.jspa?categoryID=47&externalID=128">authentication scheme for Amazon's S3</a> requires SHA1.</span>
</li>
<li>
<span>JavaScript-based apps can be served up from the S3 domain as files stored via S3.</span>
</li>
<li>
<span>All of this adds up to JavaScript apps hosted on S3 with AJAX-based read/write access to S3 itself.</span>
</li>
</ul>
Also: Imagine GreaseMonkey scripts using S3 as a universal configuration pool and long-term data store. (The configuration pool idea, by the way, came from Dave Winer.)
Archived Comments
I would love to see Greasemonkey apps that use S3 for storage. I've written down some ideas at http://aws.typepad.com/aws/2006/03/firefoxs3some.html .
Hmph. Looks like my Markdown comments mangled your link to this blog entry:
Firefox + S3: Some Ideas for Developers
I had the same thought about GM value store as soon as I heard about S3.
I hope to add a GM_setCloudValue API real soon now.
Naming suggestions are welcome. ;-)
GM_(get|set)CloudValue sounds as good as any name to me - but I do have one wishlist request:
Make the S3 base URL configurable with maybe a GM_setCloud. I'm scheming to write an S3 server clone in PHP, if I can, and it would be very cool if all it took to use it was to swap out the base URL.
You can also use a little Flash helper object to allow cross-domain calls from javascript, such as the one I wrote at: Cross-domain AJAX using Flash.
This way, you can host a service on top of S3 in my own domain and have many users, but most of the user traffic would go directly to S3 without going thru your servers. All you'd need is to "prime" the bucket for a given user with the proper crossdomain.xml file...
Please don't call it "cloud". That has a specific meaning to some developers, and unless you're replicating the cloud API (which doesn't sound like what we need in Greasemonkey), just save yourself the trouble and name it something else.
I like the idea though. Definitely needs to be pushed upstream into GM itself.
Hmm, yeah, that's a good point. Seems to me that the term "cloud" has been used in RSS and OPML to refer specifically to Dave Winer's xmlStorageSystem API.
I made a php interface (don't bother - it's not pretty, though it works) to s3 and only then realized that I'd have to move the entire thing client-side (and port it) because uploading one file at a time (or picking one upload file at a time) was untenable. So I decided to make an extension instead.
So I came back to look in this wiki and I have to say, that code in S3Ajax.js is terse and elegant. I think I'm gonna swipe some of it ;-)
chris: Terse and elegant? Oh that makes my day. :) And steal away - that's what it's there for! One problem, though, is that I've yet to find a way to access/upload files from a local hard drive purely via AJAX. I can do everything else on S3 though.
"I’ve yet to find a way to access/upload files from a local hard drive purely via AJAX"
Yes, javascript of course doesn't have the necessary privileges. By "purely with AJAX," I take it you mean to exclude even an HTML file input, which needs a submit, and which limits you to one file per input anyway. I coincidentally just ran across this example of javascript-driven asynchronous file pick/upload, but it requires flash8 on the client. Made me think of this thread.
On further investigation, in addition to flash8 on the client and a pretty fat library on the server, aflex (which is what the link in my last comment points to) is also limited to a single file at a time of less than 100Mb. So that's a non-starter, as they say. Which is kind of a relief because I was uncomfortable with the fact that it involved flash in any manner.
In general though the idea of making a web-based s3 interface is plagued by the problem of that last mile - from the browser to the filesystem.
Chris: You're right, Flash 8 is a hassle, but frankly it's a worthy one. I'm speaking from the vantage point of filicio.us (http://filicio.us), which is basically a file storage system based on ajax, tags and s3 in a pretty mashup. We use Flash 8 with an auto-install thingy to get the job done, and when it's been interfaced with JS and limited only to uploading, it actually gives us the best of all worlds: Multiple file selection and cross-browser support. (For us, the main problem is the mile from our server to Amazon's -- we'd like to have a Flash app do the upload directly, but currently we're forced to have the app upload to filicio.us and then we upload the files onwards. A gigantic waste of bandwidth...)
Oh, I forgot one thing that could also address some of your problems. Apparerently, Mozilla includes some extra modules for handling local files if given the necessary permissions manually from the user. This is an example: http://www.captain.at/ajax-file-upload.php
I haven't tried this, and obviously it's lacking scalability. But it may address very specific needs in environment where Moz is always on hand. And did I dream that VBScript includes some extra function for browsing, reading and writing local file, even in IE? I may be drifting back to 1998, but I recall that it was a selling point back then...
Steffen, Interesting. Did you write the file-uploader yourselves? Any pointers on the applicable actionscript (for a non-actionscripter)? The methods on the fileRef object I was looking at only included a single-file upload.
We're not Flash coders ourselves, but we've basically reused a piece of code which was written for our flagship, 23 (http://www.23hq.com). The code was a retailoring by the author of the Flash widget: http://blog.oinam.com/archives/2005/08/flash-8-file-upload-download/
Hope this helps...
mhh, your code doesnt work - or I dont get it the xmlhttp request cant be made to s3? or what I am doing wrong?
also, is there another way to download the files from your page - i must download each file by clicking right save as? a zip would be great
This is a very interesting discussion. I'd like to second Steve's points about your examples, but more pressingly I'd like to ask about how you can upload directly to s3 (not via your own server - as Steffen says, a colossal waste of bandwidth). Is there a way of doing this via any browser-based technologies? In fact, is there a way of doing this via a small app (not in your browser)? Many thanks.
(Dislaimer: I'm not a backend developer, but am aware of many of the issues involved, and I'm investigating the feasibility of using s3 in a product)
Dojo can do file uploading:
http://alex.dojotoolkit.org/?p=528
Best, Brad Neuberg