0xDECAFBAD - Tag: mozilla

Putting clouds in boxes for webdevs at Mozilla

2011-10-02T22:45:37+00:00

TL;DR: At Mozilla, we're redoubling efforts to attract contributors. And, in the Webdev team, we're using virtual machines to make it even easier to get started hacking. It makes no sense to demand that a JavaScript ninja or a CSS artisan also be a Python-wielding sysadmin, so we're trying to build ready-baked VMs for our projects.

Lieferung 3 by erix on Flickr

Firefox is more than just a desktop browser, and Mozilla is more than just Firefox. An ever-growing mass of our code is going toward building web sites and services that build, ship, update, support, promote, augment, document, sync, share, identify, and report. (And I'm sure I'm missing a further cast of hundreds, here.)

Sighting storm clouds on the horizon

In other words, more and more of Mozilla is in the cloud. Unfortunately, many of these projects are really hard to get into as a contributor. It's often hard to find what needs doing in Bugzilla, and it's often hard to get the code up and running in a meaningful way without being both a bit of a sysadmin and a Mozilla insider.

This is a huge problem. Although Mozilla has a core group of employees, we're not nearly enough to build all of these projects and pursue the mission on our own. We've gotten to where we are thanks to the volunteer efforts of thousands of contributors—and the only way we'll get where we want to go is through the efforts of thousands (or millions) more. The good news is that is feels like we're waking up to this problem and starting to get really serious about it.

Sharing recipes for fluffy clouds

For my part, I've spent idle time over the past few years trying to work out how to make webdev projects more accessible to new contributors. We've come a long way from the days of just uploading WordPress to an FTP server; these things come with a lot of moving parts nowadays, and the INSTALL.txt files run long and consume weekends. And so, they filter out all but the most dedicated (or masochistic) volunteers.

For months, I spun my wheels playing with virtual machines and exporting appliances, doing a lot of things by hand. But then, earlier this year, I discovered both Vagrant and Puppet and had my mind blown. With those tools, I could share recipes for VMs right alongside my code.

I would never have to set up a webdev environment for myself by hand again—all the fiddly bits of installation and configuration could be codified, checked into github, and flushed from my brain. But, the real magic is that I could share this stuff. So, future coworkers and collaborators could benefit from the same reproducible dev enviroments—never having wasted brainpower and weekends on an INSTALL.txt to begin with.

This is a really big deal. I presented on it this past June at Open Source Bridge in Portland (slides). I've been talking myself blue about this stuff for months, in opposition to my usual serially enthuastic habits. But, it looks like I've infected my coworkers at Mozilla, because now we're talking about developing with Vagrant, Puppet, and Playdoh.

No assembly required

So, with Vagrant, Puppet, and Playdoh, we'll have projects that you can fire up with just a git clone and a vagrant up in a command line shell. Forget about MacPorts or Homebrew on Mac OS X, or whatever it is that people using Windows have to install. Take a bike ride or go to lunch, and you'll have a fully configured server running and source code ready for hacking.

I think we can do better, though. For one, spinning up a fairly complex virtual machine from scratch with Vagrant and Puppet takes a long time and a lot of bandwidth. This can be sped up by packaging pre-built boxes. So, one improvement we can pursue is to spin up VMs with Vagrant as a part of our continual integration and deployment processes and package boxes for public download. These are also handy on a thumb drive for sharing at hack events and meetups.

This still requires Vagrant and working with the command-line, though. Not everyone who has something valuable to contribute is a command-line jockey. And, a few of us webdevs have run into issues with Vagrant—it's still early days for that project, after all. Additionally, not everyone who could be a Mozilla contributor owns the kind of hardware necessary to support running a substantial virtual machine.

So, I've also started playing with Puppet by itself, to build plain-vanilla VM appliances and to spin up cloud servers on Amazon EC2 and Rackspace Cloud Servers. It's going pretty well, and it's conceivable we could do without Vagrant to script the build of downloadable VM appliances and to spin up disposable cloud servers.

So, now what?

There are still some issues to be worked out, so I've started a wiki page. And, we need to spread the word that we're doing this to see what people think—and more importantly, see if this helps attract more help. From my years at Mozilla so far and bouncing between projects, VM appliances are things I wish I'd had when faced with a new project. So, having been there—albeit with insider advantages—I'm pretty optimistic that this is a good step forward.

Of course, it's not the only step: The other thing I mentioned—and have conveniently avoided addressing again throughout the rest of this post—is the issue of finding things to do in Bugzilla. Well, we're working on that too. It's a bit beyond the scope of this blog post, but we have a Mozilla-wide effort to start classifying good first bugs for new contributors and identifying mentors to contact for help.

So... what do you think?

Badger 2: Return of the Mustelidae

2011-07-18T15:03:13+00:00

TL;DR: A year ago, I said I wanted to build a badging service for the Mozilla community, so we could all celebrate how awesome we all are. Well, I putzed around and didn't get far, so I'm starting over with something more immediately useful.

(This is an awesome illustration by Sean Martell, used entirely without permission. I'll take it down if anyone gets mad.)

Almost a year ago, I wrote about building a badging service for all of Mozilla.

In a nutshell, I left the 2010 Mozilla Summit wanting to build a tool that empowered people to craft social objects representing achievement and gratitude, and give them to each other.

Since then, I've played around with a prototype in Django with Pinax. But, I abandoned that thing due to getting super busy with the Firefox 4 release. It also didn't help that I was a Django newb and that I got caught up in screwing around with shiny things like Pinax and Lettuce and neglected the actual feature set.

So now, finally, I'm picking the thread back up again. This time, with a little less ambition and a few more practical aims.

Make something more useful right away

While I have spent some dangerous time screwing around with shinies like Vagrant, Puppet, and Playdoh, I think I've gotten that mostly out of my system and have gotten down to work as of last night.

Rather than build a Grand Central Station for badges, I've reset my goals to a progression of practical results:

Build a reusable Django app that supports some basic use cases of creating and awarding badges to users.
Add Activity Streams narrating badge claims
Add support for PubSubHubbub with Badger activity streams
Add a REST API, maybe with OAuth
Accept badges for local users, via PuSH from external sites
See what the Open Badges Project is up to, and how Badger can fit in with their goals.

The idea here is to make Badger a practical, reusable, drop-in app for existing sites—preferably (but not necessarily) the Playdoh-flavored sites we're building at Mozilla. Then, incrementally and optionally empower a Badger-enabled site to become a badge hub by accepting reports of badge claims from other Badger-enabled sites.

Take baby steps to federation

So, for example, let's say we dropped Badger into a few sites like these:

At first, Badger would just augment those sites with badging facilities for their respective user bases. That could fill some needs right away.

Then, say we built a site that offered personal profiles to members of the Mozilla community. With the Activity Stream and PuSH augmentations, users could connect their profiles with each of the other Mozilla sites they use. That way, whenever they claim a badge out there, it gets delivered to their community profile.

Why bother federating at all?

If the only cross-site goal to build an all-in-one display case for badges on a community profile, I could just build JSON feeds and wrap them in a JS-based widget. In fact, a JSON-formatted Activity Stream will work nicely for that. Drop that into the profile page, and we're done.

There are lots of reasons to federate. But, at least one thing not supported by this solution are meta-badges. That is, badges awarded for badges.

Let's say you close 100 bugs on Bugzilla, answer 100 questions on SUMO, edit 100 pages of documentation on MDN, and author 100 add-ons. And, let's say you get badges on the respective sites for all of those activities.

Well, now, I'd like to give you a badge on your community profile for being a Mozilla Superhero. I could just do that by hand, having observed all your greatness. But, I'd rather the system just do that automatically.

If your community profile treated all your badges from your chosen affiliated sites as first-class citizens, then I could build the meta-badge trigger right there.

Of course, I could also write a program that monitors badge feeds from various sites. Then, use the community profile site's Badger API to issue a badge when the conditions are met. But, as maintainer of the monitoring program, I'd have to keep track of what feeds where count for which people, which could be a mess.

There's still some thinking to do here.

What about identity and co-opting badges?

I'm still thinking through how someone connects a profile to other sites. As in, how can Badger ensure that the owner of the community profile is also the owner of a badge claim feed on MDN? Maybe some kind of OAuth dance that results in a PuSH subscription between sites?

Consider the least-capable federation case: a Badger-compatible feed hosted as a static file on Amazon S3. And, by Badger-compatible, I mean an Activity Stream that contains badge claims. You shouldn't need my specific implementation of Badger-in-Django to produce one of those.

Although PuSH is still the ideal case (I think), occasional polling of an Activity Stream feed should still be supported. If a user connects that up to their community profile, we just have to take their word for it. We'd rely on community members to flag stinkers trying to claim badge feeds that aren't theirs.

Beyond that, there's probably some crypto-voodoo we can invoke here, for cases where badge claim validity really matters—as opposed to the fun and games I have in mind for Mozilla.

What about the Open Badges Project?

From various conversations over the past year, it sounds to me like the Open Badges Project is getting the parts in place to do even more interesting federation of badges that gives ownership of the badge hub to the user.

In that scenario, you have a backpack in your browser and can carry your badge claims with you between sites with no prior coordination between the sites themselves.

This sounds like something in which Badger should be able to participate, eventually. If not as a hub, Badger should at least offer the tools to create and award badges that work with the user-owned hub scheme.

So, what next?

Now that I've written all this out, I need to hack on it. If you're interested in this stuff, let me know. That way, it'll look like someone might actually find this stuff handy when I get something working!

Archived Comments

David Boswell
2011-07-19T01:18:24

Very cool to see an update on Badger. I'm definitely interested in this and would love to see something like this show up in the Mozillians phonebook that will be ready soonish :) David
Brian King
2011-07-19T15:12:01

Over at the Mozilla Reps program, we are discussing the possible use of badges - https://wiki.mozilla.org/ReMo
Gen Kanai
2011-07-19T15:48:22

Agreed with DBos. Would love to see Badger as an official service, especially with the new Campus Reps program that Jason Haas is rebooting as well as with the Mozilla Representatives program. Also you might get a badge for contributing to the Join Mozilla program? So many great uses for us.
interstar
2011-08-05T03:15:39

Hi Les, Very very interesting. I was just thinking that there needed to be something like the Open Badges project today. (I'd even got as far as knocking up a data-model in web2py.) And then I saw this. I quickly cloned and tried out your app with an ordinary Django project but it's blowing up without jingo and I guess other Playdoh related modules. Are you committed to Playdoh for Badger2 ?
- Les Orchard
  2011-08-05T18:05:31
  
  I'm committed to Playdoh insofar as my initial users will probably be my co-workers at Mozilla, who are all building Playdoh-based sites. But, there's no reason why it shouldn't eventually work with vanilla Django. It's using core template loaders and shortcuts, so switching over to standard stuff shouldn't be an issue for the views and models. I'm hoping at some point to put together some more Django-conventional template tags and templates toward that end, unless someone gets to it before me in a pull request (patches welcome!). Really, I think the template tags are the more important thing, since any templates out of the box will likely get overridden on a real site.
Dr. Nellie Deutsch
2011-09-02T14:07:45

I love the idea, Leslie. I've been heading a self-funded organization that provides free online professional development workshops for educators around the world since 2006. We need badges for our graduates. Let me know how I can be of help.

Pay phones and Firefox features

2011-02-07T04:32:31+00:00

TL;DR: Pay phones have been disappearing, and so might some Firefox features. But, that's not necessarily a bad thing: Some features steal attention from others that are more important. Some could be spun off into add-ons, which are getting easier to make.

Update: Feel free to comment on this blog post. But, if you feel strongly about these features, you might get a better discussion with more people on this thread from the mozilla.dev.apps.firefox newsgroup (in Google Groups).

Phone Don't Work No More by Thomas Hawk

When's the last time you used a pay phone? For me, it's been over a decade—which is about how long I've had a cell phone.

As it turns out, pay phones have been disappearing for awhile now. Why? Because pay phones are expensive to keep in working order. If more and more people have wireless phones, it's less likely that any particular pay phone will result in value for the phone company.

So what?

The reason I'm thinking about this is that I'm troubled by a few bugs I ran into in Bugzilla:

Are these like pay phones?

Each one requires engineering effort to keep working from release to release in Firefox. Some of them have issues in performance and functionality that could use some attention. And, the introduction of Firefox Sync, for example, introduces some new complexities in tracking changes.

But, do any of these features provide enough value to justify continued attention?

Microsummaries

Ever heard of microsummaries? I have, and I thought they'd be neat. I installed a plugin on my blog to offer a microsummary, and I tried hacking them into Delicious at one point during a Hack Day.

That was about 4 years ago, and I haven't thought about them much since. Sounds like a pay phone to me.

Smart Bookmarks

I always wondered how those out-of-box "Recently Bookmarked", "Recent Tags", and "Most Visited" folders worked in Firefox. Turns out that you can bookmark queries against the internal Places database as a persistent search folder. Seems pretty keen.

But, I've been using Firefox for years and only just learned about this in the past month. The feature's not very exposed, and it's not obvious how to use it. In fact, the bug to remove it might list the clearest hints on using it.

Seems like it might be a good feature to jettison into an add-on, like a pay phone to play with in your basement.

Tagging Bookmarks

This is a no-brainer for me: I like tagging my bookmarks. The problem is that I've exceeded 16,000 bookmarks. It's been years since any browser coped well with my corpus of bookmarks—that's one of the reasons I started really liking Delicious.

However, tagging bookmarks in Firefox is kind of rough. The UI isn't awesome, and the database supporting it could use some work. Rather than improve things, though, bug 622047 proposes removing the functionality altogether.

Here, I'm conflicted: I don't use tags in Firefox, because I don't use bookmarks in Firefox. If they worked better, I might use them. Instead, I use pinboard.in or a private Delicious clone of my own that copes with my scale—in fact, I got to this scale because those services handle it. So, usage data from me would support the feature's removal despite what I think I'd like to see.

But, does my suspicion that I would use a feature justify someone else's time in making it better? I don't think I can ask for that with a straight face.

Put another way, I could see how that pay phone they removed from the corner gas station could someday come in handy, but should I expect the phone company to keep it clear of graffiti until I come along with quarters?

Livemarks

Live Bookmarks (or Livemarks) are bookmark folders whose contents are generated from RSS and Atom feeds periodically polled by Firefox. They're feed subscriptions, built into the browser.

So, when I saw bug 622049, my first thought was "Hey! I'm using those!"

I just spent a week building an add-on powered by Livemarks. I've been using Fireriver on a daily basis, and I've been thinking about trying more experiments. But, if livemarks are going away, then I'm working toward a dead end.

Like tagging bookmarks, the UI and feed polling code behind livemarks could use some work. My add-on was an attempt to improve the interface, but I did report a bug on performance issues raised by my import of around 700 subscriptions from Google Reader. That's just one more strike against livemarks for bug 622049.

In the end, if livemarks go away I'd still have Google Reader. I'll have blown a week or so on this add-on, though I did learn some interesting things. And, if Google Reader goes away, it's not like I've never written a feed reader before.

Still, this one bothers me the most, in terms of what I want to see for the web itself. This is better explored in another blog post, but I want my browser to do more of the surfing for me. I think of bookmarks as long-term relationships to web resources—and livemarks turn those static links into live updates. I can think of a lot of uses for this functionality built into the browser, rather than delegated to lobster traps and content silos.

But, livemarks do complicate things for Firefox engineers. They don't interact well with Sync, and the feed polling code has cases where it can really drag down the whole browser. And, of course, livemarks aren't good for much when a laptop's closed or if Firefox isn't running. Granted, the former happens more often than the latter for me, but we're not talking about an always-on server here in any case.

Are livemarks a pay phone feature? Personally, I don't think so. I think Firefox is diminished if it loses this feature, rather than taking it further.

Although I'm reticent to do so, I might just rebuild livemarks in some form with an add-on if they go away. The good news is that add-ons are getting easier to write, which also deserves its own blog post. The bad news is, I might not have time or motivation to do the work myself. But, either, I'm not all that comfortable clamoring for other people's time for this beyond registering my disagreement and backing off.

Conclusion

Most of these things do indeed look like pay phones to me. Most of them seem not to offer enough value to pay for upkeep.

Livemarks are the exception, in my mind, but I don't have answers as to how to fix their flaws. Maybe they're inherently doomed in the face of ubiquitous computing, maybe they're just in need of a better execution. But, I won't expect other busy engineers to drop what they're doing to come up with those answers in the meantime.

Archived Comments

Bo
2011-02-07T05:38:07

I don't care about microsummaries or smart bookmarks, but Firefox without tags would totally break how I interact with my several hundred bookmarks.
Manoj Mehta
2011-02-07T06:20:40

Agree with you 100% of the way.
Anon
2011-02-07T06:24:22

Interesting read, but it hurts, after spending a lot of time searching for barely-existent documentation, to configure a great system with smart bookmarks keeping track of my tags and bookmark organizations. I'd really miss the feature and I don't think a bookmark can provide the same level of functionality. On the other hand, I loathe live bookmarks and I use google reader and minor desktop applets to fetch site updates.

Notice that what I consider useful and productive you consider a pay phone. Firefox's strength over its competition is the customization power (because no matter how you put it, Chrome's extensions are laughable in comparison). I think that, out of the 4 features, only the first is actually useless to most users.

Smart bookmarks were neglected after getting out of the 3.0 heat, while pursuit for other things such as Ubiquity, Jetpacks and Panorama took place. I hope Panorama doesn't get abandoned this fast, as I really see myself using it (although can't upgrade until imglikeopera does).
- l.m.orchard
  2011-02-07T07:37:13
  
  Would Smart Bookmarks still be useful to you, if they were enabled with an add-on? I suspect it could be done, which might be win-win for core Firefox devs vs people interested in carrying the feature foward in an add-on
  
  I never knew what Smart Bookmarks were until this month, and I've been using Firefox since the beginning. So, yeah, they look pay-phone-y to me.
  
  Nonetheless, it might be interesting to see what kind of system you have working with them. They seem like they could be pretty powerful
  - blufive
    2011-02-07T08:35:39
    
    (pardon me for butting in)
    
    yes, an add-on would work for me.
    
    My system is pretty much that suggested by David Regev below: use the smart bookmarks to create a "folder" system based on tags. This is way easier to maintain/update than manually sorted folders, and allows bookmarks to exist in multiple locations.
  - Anon
    2011-02-07T12:07:12
    
    If the full capabilities of both tagging and places queries are available, then yes, as long as it's not one of those "can't update firefox until it updates 2 months later" extensions...But from the sounds of it the whole backend is going to be removed or simplified. Will it be possible for an addon to get there?
David Regev
2011-02-07T07:09:59

I’ll add some input from my own experience with bookmark tagging. I have bookmarks going back over a decade. Until about a year ago, they were a huge mess and nearly impossible to deal with. The big problem was that the folders were too restrictive and, so, the placement of many bookmarks was pretty arbitrary. So, I finally decided to go through every bookmark, delete all the useless stuff (the vast majority of bookmarks), and attach one or more tag to each bookmark. It took about a week. The result: the organization system now makes sense, and I have few enough bookmarks that I actually use them quite a lot. The problem is that the interface is designed for folders and not tags. I cannot drill down tags like I can with folders. Tags don’t appear in the bookmarks menu like folders. I had to create smart bookmarks to work around this limitation. Even with these artificial limitations in the interface, tags are much more humane than folders.

My suggestion: deprecate folders altogether. In a future version of Firefox, migrate everyone to tags by converting them to tags. Moreover, fix the interface so as to make sense for tags instead of folders. Do that, and you’ll find that people start using tags. I agree that it’s kind of overkill to have both folders and tags. But the solution is to get rid of the archaic system of forced hierarchy (folders) and give tags the chance they deserve (by greatly improving the tagging interface).
- l.m.orchard
  2011-02-07T07:40:18
  
  FWIW, if I used bookmarks in Firefox, this is exactly what I'd do. I really prefer using tags over folders, for escaping restrictions of hierarchy and for organizing on the fly
  - Asa Dotzler
    2011-02-07T09:01:44
    
    Folders currently have the advantage of "open all in tabs" which is kind of nice for batches of sites you commonly access at the same time. I'm not saying that the same feature couldn't be implemented for groups of tags, but currently folders have it and tags don't.
    - David Regev
      2011-02-07T18:55:10
      
      Asa: That’s actually already possible for tags (such as when exposed as smart bookmarks). The command doesn’t appear in the submenu, like it does for folders, but it’s there in the context menu. You can also middle-click on them instead.
- Matěj Cepl
  2011-02-07T09:22:12
  
  You don't need to look through tags ... just throwing bunch of them in the AwesomeBar will put the appropriate links to the top of the list. From all mentioned payphones tags would really hurt me, and I am on the fence with LiveBookmarks ... I don't use them (rss2email works better for me for reading feeds), but I distrust The Cloud and I don't like the trend to save all our personal data with an advertising agency for their free use. Certainly better UI would help.
Felix Pleșoianu
2011-02-07T07:39:30

"I’ve been using Firefox for years and only just learned about this in the past month. The feature’s not very exposed, and it’s not obvious how to use it."

That's the problem right there. It's like putting a payphone into a basement, inside a broom closet with a sign on the door labelled "beware the leopard" and then complaining nobody uses it.

"And, of course, livemarks aren’t good for much when a laptop’s closed or if Firefox isn’t running."

Neither is a dedicated RSS client by that logic. I agree that live bookmarks are largely useless (too fussy about the feed format is what bothers me most about them), but you're using the wrong argument.

More generally, I agree that these features are rather obscure and may not be missed by most people, but do they complicate and slow down Firefox more than, say, XUL? Seriously, Opera manages to have a proper RSS reader, mail client and lots of other features built in while being much smaller and lighter than Firefox. And it's a proprietary browser. Aren't you picking on small issues while ignoring the large ones?
- l.m.orchard
  2011-02-07T07:59:28
  
  "It’s like putting a payphone into a basement, inside a broom closet with a sign on the door labelled “beware the leopard” and then complaining nobody uses it."
  
  That's true, and part of what bothers me. Seems like some features can get axed with usage data as the rationale, but they could be made better.
  
  But, whether any particular feature got a fair shake or not, dev priorities change.
  
  "Neither is a dedicated RSS client by that logic."
  
  Actually, the argument I left out is ubiquitous computing. That is desktop to laptop to mobile to TV to car.
  
  I stopped using a dedicated RSS client around 4 years ago when my phone got capable of skimming feeds and I couldn't get to NetNewsWire on my Mac from it.
  
  "More generally, I agree that these features are rather obscure and may not be missed by most people, but do they complicate and slow down Firefox more than, say, XUL?"
  
  I can't really speak to that... I'm more webdev than core Firefox dev. You can read the bugs, though, to see the arguments made. Though, from what I hear, XUL isn't really such a big problem, especially not since the JS code behind the scenes enjoys the same performance boosts as JS in page content.
Mardeg
2011-02-07T08:06:49

It seems most of these revolve around the hassle they've become for Firefox Sync to support. Here's a thought: Sync was an extension to begin with, let's move it back there after Firefox 4 is released and it fails to cope with the massive influx of people using it, instead of killing the other features in an attempt to prevent Sync dying.
FP
2011-02-07T09:00:15

Do the developers not realise that driving away power users doesn't just lose you a few users it loses you a whole network of users since many power users maintain their friends and families PCs too? It also loses you popular support from tech blogs and sites etc.

Anyway, I hope they don't remove tagging and livemarks that would be painful, like Mardeg, I'd much rather lose sync than these features.
Nathaniel Tucker
2011-02-07T10:02:06

Please don't remove bookmark tagging! I like being able to find webpages using the awesomebar, and I couldn't do that without control over labeling. I don't see what's so horrible about the UI - you just enter what you want.
Michael Lefevre
2011-02-07T10:50:16

I can't say I make a lot of use of any of those features. I do use tags, but I could do the same thing with bookmark folders.

The trouble is that Firefox is still introducing features which aren't very polished or discoverable. If Firefox 6 is going to drop support for, say, sync or panorama, then it has been a bit of a waste of core development time implementing those features, hasn't it? Is there a way of getting development to focus on improving the existing stuff, rather than focusing on new features?
Andy
2011-02-07T11:38:11

Mardeg has a point. Throughout FF4's development it's felt like any new team or project can make drastic changes to the browser with seemingly flimsy justification (see the UI team's constant fiddling with a stable interface to save miniscule amounts of vertical space and emulate Chrome's aesthetic). Sync is a nice feature but it needs time to bake just like everything else; we shouldn't rip out other features that have existed for years to prop up something unproven.

In some ways I think the Test Pilot program has been dangerous; the statistics gathered tend to result in design decisions that favour the tyranny of the majority while breaking long-term usage patterns employed by smaller groups ("power users" as they're affectionately called, also known as Firefox's Chief Proponents). This popularist-majority mindset can be seen in the design decisions regarding the status bar, tab positioning, the hiding of the bookmark bar on first launch after upgrade and the new awful addons manager.

It's also lame to see it argued on the bugtracker that tags should be stored in the cloud but that we shouldn't use the new feature that does exactly that to put them there. Who knows which direction Sync will go a year after release? Maybe we'll gain the ability to set bookmarks public and browse them on a website like Delicious... surely we'll need the tag info then.

One last thing: this may be a little snarky, but if a major argument for removing tagging support is an underexposed and unpolished UI then Sync is doomed. A menu entry that shows a dialog with no description of the feature isn't the best way to sell it. I still don't have a clue whether it's working or not.
Andy
2011-02-07T12:16:15

Mardeg has a point. Throughout FF4's development it's felt like any new team or project can make drastic changes to the browser with seemingly flimsy justification (see the UI team's constant fiddling with a stable interface to save miniscule amounts of vertical space and emulate Chrome's aesthetic). Sync is a nice feature but it needs time to bake just like everything else; we shouldn't rip out other features that have existed for years to prop up something unproven.

In some ways I think the Test Pilot program has been dangerous; the statistics gathered tend to result in design decisions that favour the tyranny of the majority while breaking long-term usage patterns employed by smaller groups ("power users" as they're affectionately called, also known as Firefox's Chief Proponents). This popularist-majority mindset can be seen in the design decisions regarding the status bar, tab positioning, the hiding of the bookmark bar on first launch after upgrade and the new awful addons manager.

It's also lame to see it argued on the bugtracker that tags should be stored in the cloud but that we shouldn't use the new feature that does exactly that to put them there. Who knows which direction Sync will go in a year after release? Maybe we'll gain the ability to set bookmarks public and browse them on a website like Delicious... surely we'll need the tag info then.

One last thing: this may be a little snarky, but if a major argument for removing tagging support is an underexposed and unpolished UI then Sync is doomed. A menu entry that shows a dialog with no description of the feature isn't the best way to sell it. I still don't have a clue whether it's working or not.
Benedikt P.
2011-02-07T12:55:49

My bookmarks are organized in a rather flat manner with little to no sub-folders.Being able to tag a bookmark and search and save the searchs (Smartbookmarks?) are much more useful to me. (I have hundreds of bookmarks not thousands, though)

Microsummaries sound useful in theory but I think I never found one in the wild.

Livebookmarks are unnecessary for me. Firefox is just not a real feed reader.
Danny Moules
2011-02-07T13:01:25

Are Livemarks really a niche thing? I started using them after seeing a large number of users using them and wondering what I was missing. Now my entire bookmarks toolbar is full of them.

We don't even have metrics to back this up, do we? I would have never anticipated that this would be a feature worth dropping.

Anyway, I've replied to the bug. I wonder what we'll get from test pilot if it goes that way..
Adil Allawi
2011-02-07T13:59:29

"The payphone in the basement" is a good idea for (to use a horrible phrase from Yahoo) sunsetting little-used browser features into add-ons. But the problems happen when new browser versions come out that are incompatible with these add-ons and no one is around to update them. I will have this issue moving to Thunderbird 3.3 and a number of add-ons that i depend on will no longer work.

My suggestion is to make sure such features are reimplemented in new browser APIs like JetPack (and update JetPack to support the hooks needed by these features). Then make sure the JetPack API is stable enough not to break in future browsers.
Gruber
2011-02-07T14:14:35

Tags are the best feature introduced that was introduced with Fx 3.0! I don't need the other stuff, I don't need folders but tags (together with the Awesome-Bar) are one of the two reasons I love Firefox (the other is the extensibility through add-ons, and I am thinking about a particular that ATM can't be ported to other browsers).

BTW, I always thought pay phones were heavily subsidised.
alanjstr
2011-02-07T14:21:22

Of those features, I see more potential for Smart Bookmarks/Tagging than I do for the other two.

If a website has a microsummary, I have no idea. I don't know where it would show up in my web browser. I had to Google to find The MozillaWiki describing them as "Microsummaries are better for labeling bookmarks than static page titles, because they give users quicker access to the most interesting information behind a bookmark, and they give web sites a way to notify users of updates and entice them to revisit the site." If you have your bookmarks open constantly, and are monitoring something, then sure that would be helpful. But usually if we want real-time monitoring of something, there are other methods that we use. We have moved to increasing screen real-estate, making tabs into just icons and hiding the bookmark toolbar.

I never got the hang of LiveMarks in the browser. That definitely has a lot to do with UI. But trying to turn a web browser into a full-fledged feed-reader, as part of the core technology, isn't the right direction. There are addons, websites, and stand-alone readers that do a much better job. The more RSS feeds you subscribe to, it quickly overwhelms the basics. Personally, I do like having the little orange button in my URL bar to subscribe (I have it auto-routing to Google Reader), but that's mostly because it provides a fast way to subscribe instead of me hunting for a link.

Lastly, Smart Bookmarks and Tags have a lot of potential, especially as the volume of bookmarks grows. I don't use either of them, but I wouldn't want to see them gone.
Gruber
2011-02-07T14:35:08

One more thing, I wonder if these arguments against these "pay phones" could/will also be made for Panorama and Tab Groups, I personally wouldn't shed a tear for it.
augustm
2011-02-07T19:16:39

Firstly SQL has been broken in Firefox 3.6.x and 4.0b since the summer. It makes searching on tags in bookmarks almost impossible. So tags are mostly broken. Any data on hot areas is unusable- the interface does not work.

Sync has deleted, duplicated, re-ordered my bookmarks multiple times. Disaster! Worse than anything else, multiple data-loss bugs, requires restore from backup (Mac's have time machine)

There is not document that says what Sync does. How does it manage the use of multiple machines? I am on a campus where I read, mark and delete in five departments. How do I manage asynchronous marking and deleting? I often turn on/off machines quickly. What is my mental model of the final state?

I should have stayed with rync!

Bookmarks needs a project leader
James Napolitano
2011-02-08T08:49:27

About the idea of moving features into addons: dbaron had a nice post a while back about the drawbacks of this approach: http://dbaron.org/log/2006-01#e20060122a The situation with addons.mozilla.org may have changed since then, but I think a lot of what he says still holds.
Mook
2011-02-08T08:54:22

Seems like for all of these features, the problem is that the UI wasn't really that good. It's somewhat difficult to create microsummaries or smart bookmarks (as a user, without the web site supporting it). Tags are easy to create, but are presented everywhere as a single list, which makes large numbers of them infeasible (rather than something closer to a tag cloud). Livemarks... well, they basically have no useful way of showing the summary, just... half a headline :p Only if it actually opened the feed preview page rather than a useless dropdown...

As I understand it, it's mostly things getting cut to make the release, then... well, just getting dropped on the floor once the release is out, really. There's never a focus on polishing things. But that's pretty much true of all the projects I've looked close enough at, sadly.

I wonder which ones would be next? Devtools, with the backend sticking around for Firebug? More changes to the URL display on hovering links? Firefox menu-button?
csb
2011-02-08T20:09:45

Bookmark tagging is one of the best features of Firefox. If anything the problem is that as currently implemented it has not enabled more great addons like tagsieve [1].

[1] https://addons.mozilla.org/en-US/firefox/addon/tagsieve/
Stuart Woodward
2011-05-18T14:45:46

Several years ago, my wife laughed when I accidentally bought a prepaid payphone card instead of train card. It sat in my wallet for years until the day my cellphone battery died and all I had was my WIFI iPad. I looked up her phone number in my contacts and called her from the nearest payphone though it was a struggle to find the payphone in one of the most modern building complexes here in Tokyo.

Another payphone story that came to mind was shortly after the recent Earthquake. Tens, if not hundreds of thousands of people walked home. Some walking several hours. In my short walk to camp out a nearby friend's house, I saw people queuing up for payphones after their mobile phones had become unusable due to the mobile network congestion. The line to use the payphone was hundreds of people long. I'm sure that the maximum queue length at that phone in the previous 10 years was 2 people. This experience changed my mind.

There are much fewer payphones than their heyday 15 years ago when we had payphones with ISDN jacks to plug in your laptop. It is difficult to find one now. Also, payphones are clunky and in Tokyo at least, very reliable. They are going to work, if nothing else works.

What should be done about feeds in browsers?

2011-01-27T06:11:00+00:00

TL;DR: Browsers need help doing a better job using feeds for discovery, aggregation, and publishing on the web to keep us from swimming into self-destructing lobster traps.

Last week, I wrote some blog posts about Bug 578967, wherein the feed auto-discovery icon in Firefox 4 has been moved off the URL bar and hidden by default. I support that decision of the UX team, because that feed button is a slacker. And even if it does get used, I'm not a fan of what it does anyway.

But, I've got lots more rattling around in my head about this stuff.

What's the point of feeds?

Feeds—that's RSS and Atom and maybe even JSON—let robots pull lots of stuff from a bunch of places and pile it all together in one spot for your perusal, analysis, and remixing pleasure. Feed readers and news aggregators do the surfing for you, so you don't have to.

But, Twitter and Facebook and others have managed to make the people themselves all gather in one spot. So, to see what those people are saying and doing, you just have to go there—kind of like what feeds promised, right? Only, you go there rather than all that stuff coming here.

The problem is, even if the logo says it loves you, the people who made it say that won't always stay in charge. And, someday, when whomever's left rides off into the sunset, you'll learn how much they didn't care about you in the end.

For some people and some places, that's no big tragedy. It's just a party; you go home when it's over. You made memories, you hope to find some of those people again elsewhere. Pownce croaked, and so did Vox, but who cares? There's always Twitter next door for the afterparty.

But, what if a world you lived in evaporated while you were on vacation elsewhere for the holidays? I guess you could say that was just a game and, again, who cares?

What if Flickr went away just like that? People are not only partying there—they're leaving their visual memories at the coat check. Things are a little more real there, by some definitions thereof.

What if you never swam into that lobster trap, though? What if your words and images and creations were borrowed and copied, but never wholly kept in someone else's bucket? What if Flickr were just like Google Search? That is, a collector and an index—but never the host—of photos and conversations?

Feeds can help make that happen, but not by themselves.

Do feeds belong in the browser?

Well, yeah. Would sites like Twitter and Facebook have been as successful if feeds worked better in the browser? I think so. But, by working better, I don't just mean improvements to that funky blue thing that showed up in the location bar every now and then. No, I mean all of the following:

Discovery — help me find and follow interesting people and things
Aggregation — give me one spot where what I find and follow is delivered to me
Publishing — enable me to post things where people (and their agents) can find and follow

To me, this is where Twitter and Facebook are totally nailing it. But, much of this can happen in the browser, and I think more of it should. Projects like RockMelt and Flock have seemed promising to me in this context, but have yet to really thrill me. I don't think they go far enough.

There is a point where the rubber (client) meets the road (server)—but that point can live so much farther down the stack than it does now. Mind you, I don't want a revival of Netscape Gold as such—but recall that "the first browser was an editor, it was a writer as well as a reader".

We need some services and hosts out there as rendezvous points for our browsers and agents, but we really don't need any of those services to collapse into the next all-encompassing AOL-style singularity. There's money to be made there, but most of us are the product.

Whose job is it anyway?

I'd like to see my employer, Mozilla, do more with feeds in the browser—and I suspect we will. Mozilla Labs has had some interesting Concept Series topics, and I look forward to seeing some of those things get pursued further. If I can carve out the time, I wouldn't mind helping.

Something I find interesting about feed auto-discovery—the thing that started this latest round of buzz—is that it didn't originate with browser makers. RSS auto-discovery was itself discovered by bloggers, way back in 2002. It wasn't until we sussed out the details of that HTML link tag and shoved it into our pages that browsers like Firefox started looking for it.

Hell, even though RSS started with Netscape, it took Dave Winer and UserLand and friends running with it to keep it alive and kicking. Now, Dave's suggested forking an open source browser to show Mozilla how it's done. I support the sentiment, but there are easier ways to go about this.

Making it your job

Christopher Blizzard:

We created add-ons with the original Firefox as a way to be able to say “no” in a constructive manner. If you want something that you think is important to you, you can make an add-on.

That could be taken as a cop-out, but it's not. With the efforts behind Jetpack—now just the Add-on SDK—building add-ons for Firefox is being made easier. Rather than dealing with a morass of XUL and obscure APIs that I know has scared me personally away all these years, you can now do most everything you'd like though much-simplified JavaScript APIs and HTML/CSS.

In other words, if you are a web developer—albeit a fairly advanced one—you can make an add-on for Firefox. Hopefully, as the Add-on SDK advances, the bar to participation will continue to drop.

To be continued

I'm running out of steam and it's time for bed, so I think I'll just trail off for now. I do have a bit more in me, though. The next thing is about taking my own advice in that last section and making some of this my job to fix.

Archived Comments

Majken "Lucy" Connor
2011-01-27T06:38:04

Yay!

Feeds are soo soo soooo great, but not as live bookmarks. Google Reader is the place to be in terms of functionality, but there's no reason I have to put everything through google.

When I find a site I like, why should I have to go to it to check for new content? A feed reader brings it to me AND it remembers where I was. I've been trying to get my kids hooked on it, though they seem to forget about it.

Speaking of the kids, their teachers are getting all technical and they post the homework online, but I've been pulling my hair out for 2 years trying to get them to do it in a way that enables feeds. Several nights it happened that they didn't write down what they had and then we had to remember to check periodically because the homework wasn't updated right away.

Right now they're sent home with books with it all written down and I'm supposed to sign that I saw it, but who remembers to look at a book all the time? It's such a complicated process - kids have to write it down in the first place, then one of us has to remember to show it to me. FEEDS!!! No one has to remember, it just shows up where I'm already looking!

So glad you're on board.
- l.m.orchard
  2011-01-27T20:54:22
  
  Not sure if the homework lists online are behind a login or not, but if it's in HTML we can hack a feed out of it :)
Bee
2011-01-27T08:13:46

"Discovery — help me find and follow interesting people and things Aggregation — give me one spot where what I find and follow is delivered to me Publishing — enable me to post things where people (and their agents) can find and follow" .. Erm google reader?
- l.m.orchard
  2011-01-27T15:02:20
  
  Yup. I use Google Reader every day. Google owns it, though. It's not Facebook or Twitter, but it's not mine.
  - fluffy
    2011-01-27T17:01:25
    
    Whenever I'm going through my occasional "Google scares me" phases I go back to FeedOnFeeds. It has a few rough edges, but it has the nice advantage that it's software you install on your own server somewhere, and it works pretty much just like Google Reader in terms of aggregation. It also has some rudimentary publishing stuff, although IIRC it's in the form of a public RSS feed that other people subscribe to. Which actually isn't a bad model, really.
    - l.m.orchard
      2011-01-27T17:34:03
      
      I think I've used FeedOnFeeds before, though it looks like it's not moved in awhile. There's also Tiny Tiny RSS, which looks a little more kept-up.
CiaranG
2011-01-27T10:45:08

Do feeds belong in the browser? Well, to the extent that while looking at a web page, there may be related feeds that the browser can auto-discover. Do I want to read them in the browser? No.

All I ever wanted was for the feed button in Mozilla product A, Firefox, to be able to add a subscription to that feed in Mozilla product B, Thunderbird. You'd think that would be a simple and obvious thing that would just work, but even by hooking the two together with a selection of Heath Robinson-style scripts, it's still never been possible to get them to play nicely.
- l.m.orchard
  2011-01-27T15:06:57
  
  That's a good point - a lot of this falls under the bailiwick of Mozilla Messaging, who work on Thunderbird. Personally, I never liked feeds in an email client, though.
  
  But, that reminds me they've got this thing called Raindrop that I haven't looked at in awhile. It might be interesting.
Robert Kaiser
2011-01-27T14:42:53

Fun that you say that XUL scared you away from doing add-ons, as doing everything that I think should be markup in JS scares me away from doing restartless add-ons or using the new SDK. I guess there's different people thinking differently. ;-)

For the actual topic, I think you're dead-on. I have subscribed to feeds from twitter, statusnet and identica, but as there's no interface to write messages from there, I just don't post anything there (and I intentionally don't have accounts on monolithic services like Facebook or twitter).

We need to get a better integration for pulling together info and communication in the browser. My "RMD" concept - http://wiki.kairo.at/wiki/RMD - could be one possible basis, but it might be overdoing it, given that I very much come from traditional messaging...
- l.m.orchard
  2011-01-28T18:32:07
  
  Yeah, XUL has been one of those things on my to-research list for years and years, but I've never really dove in. Then, Jetpack comes along, and at least the entry point is similar to what I've been doing for the last decade or so.
  
  But even that is sneaky, because I have access now to those "obscure Mozilla APIs" and I'm finding myself learning them now anyway since Jetpack got me in the door. :)
  
  Your RMD concept is interesting, though. Might be more to the point than the piggy-backing I'm doing on Live Bookmarks. I've also been wondering how things like pubsubhubub might feed into things like this
@sull
2011-01-27T19:50:36

Question. Whats your opinion on FF (and other browsers) handling of the direct feed url? I've long wanted browsers to back off from forcing their own baked-in handling (stylesheets, functionality etc). I've just wanted the simple alert dialog (now standardized for all alerts like geolocation, passwords etc) to appear when necessary before the browser takes over (Like you include with Fireriver auto-discovery).

The obvious example is when a feed specifies its own XSL stylesheet but FF ignores this instruction and the sniffer only cares about the presence of the <rss or <feed element before treating this 'document' differently than any other XML document. This can be overcome with some hacks but that is a moot point.

Ideally, the browser will adhere to the content publisher's (or feed generator's) instructions on how to handle the presentation of the data. That plus XSLT is a W3C standard.

Definitely interested in your thoughts on this angle of the "feeds in browser" discussion.

Thanks,

Sull
- l.m.orchard
  2011-01-27T20:52:52
  
  I'm not a fan of the built-in stylesheet for feeds. That change actually cost me a bunch of hours at work back then, trying to figure out why my XSL stopped working.
  
  IMO, that broke something I was using, but didn't go far enough to make it worth breaking. And, it's not really gone much farther since then.
  - @sull
    2011-01-28T03:23:25
    
    thanks for pointing out your post from yesteryear. the issue needs some new noise. FF4 is unchanged.
skierpage
2011-01-28T11:09:07

"We need some services and hosts out there as rendezvous points for our browsers and agents,"

Why? I can e-mail directly to another user at a computer; BitTorrent transfers file chunks directly to other computers. So why can't my computer transfer my pictures, status updates, favorite links, snarky comments, etc. DIRECTLY to the computers of lists of people (misnamed as "friends" by Facebook)?

The correct response to the dangers of centralization is running your own server. That needs to become the norm rather than some scary IT project that only geeks ever attempt! Your "server" could just be Firefox accepting connections from lists of people.
- l.m.orchard
  2011-01-28T18:04:52
  
  No, generally, you can't e-mail directly to another user at a computer. You send a message to a server, which then sends it to their server, and eventually their mail client picks it up. There are intermediaries.
  
  BitTorrent relies on trackers (ie. rendezvous points) before you can start transferring chunks, though the distributed hash stuff helps get away from that. But, if all that were easy (including discovery), it would have taken off long, long ago.
  
  You can't go P2P for everything, at least not easily. Not every P is online at all times, so we need store-and-forward services. Some P's have multiple devices and agents, and most of those are behind firewalls and NAT. So, we need servers out there as accessible fixed points that are (mostly) always available. We can try to build P2P meshes that do the same thing, but servers are simpler.
  
  And, yeah, I do believe that running your own server is the ultimate solution. Or, at least, finding someone to run it for you who cares about keeping it up, whether through direct payment or other direct incentive (ie. they're family or friends). Otherwise, you still have the problem where what they care about doesn't match what you care about (eg. ad supported services).
  But, with the way the user agent picture is splitting up across devices (eg. the cloud, ambient computing), I don't think Firefox will ever be the server. Even if you connect user agents together with something like Jabber/XMPP, you still need a server out there—though at least that's pushing things in the right direction, since an XMPP server can be agnostic about the messages that go through it
  
  I do have another blog post rattling around in my head about hosting. It's too hard right now; charging money smothers a lot of use; etc.
  - @sull
    2011-01-28T21:17:32
    
    Opera Unite is out there but it too leverages a centralized service as a proxy. But it is a great experiment and give them credit for moving it forward and into their browser product. The problem that will creep up is of course bandwidth limitations, especially on the UP. And this is only going to get worse as providers tighten up the pipes.

How to use feed auto-discovery in Firefox 4

2011-01-15T17:20:22+00:00

TL;DR: The feed button is not dead; it's just been sent to sing backup in Firefox 4 because it's not pulling its weight up front. This post discusses how you can still use feed auto-discovery, even restoring the icon to the toolbar with a few clicks and a drag.

There's a brouhaha about Bug 578967, wherein the feed auto-discover icon in the Firefox 4 Beta has been hidden by default. Being a feed nerd, the author of a book on the stuff, and a Mozilla employee—I've got at least a few opinions.

This post is one of several on this subject.

Feeds are too ubiquitous to need an indicator

Feeds are so successful and ubiquitous that it's simpler to assume every site has one, rather than keeping an un-lit indicator around to tell you when one's missing. So, Firefox 4 has an option in the bookmark menu to subscribe to the current page:

The only indicator of a missing feed is that the menu item greys out. Since feeds are everywhere, you should feel comfortable reaching for that subscription menu whenever you like. Of course, this assumes that subscribing to a page falls into the same thought process as bookmarking it—but I don't think that's an unreasonable notion.

On the other hand, if you didn't know that subscribing to a page was possible—or if you worry there might be people (ie. the potential readers of your feeds) who are in that position—this change may seem problematic.

I don't think that's an unreasonable notion, either. But, I'll get back to that in a future post.

The feed icon isn't dead, it's just hiding

Even with the new bookmark menu item, the feed subscription button is still available and ready to return to your toolbar in the Firefox 4 Beta.

First, right-click somewhere in the empty space of your Firefox 4 toolbar. Try somewhere after the tabs, or somewhere between the toolbar buttons. This is how it works on Windows, and this is what it looks like on my Mac:

Click Customize, and you'll get a panel like this:

This lets you customize which buttons and controls appear on the toolbar. If you scroll down in the panel, you'll see a "**Subscribe**" button. Drag that from the panel to a position in the toolbar, and you'll get a result like this: Click "**Done**", and your browser should end up like this: I took my browser screenshots with [Dave Winer's screenshot of the old Firefox feed icon](http://scripting.com/stories/2011/01/15/mozillaPleaseKeepTheRssIco.html) for comparison. The result is different, but not radically so. It even enables and darkens when you visit a site with a feed: The main difference is that Dave's screenshot is the default in Firefox 3.6, whereas mine is the result of the last few screenshots of toolbar customization in Firefox 4 beta 9. That customization is easy, if you know it's possible—but the worry is, as they say: out of sight, out of mind.

Archived Comments

Felix Pleșoianu
2011-01-16T08:00:44

I have to take issue with some of your points, based on my personal experience. :)

"Feeds are so successful and ubiquitous that it’s simpler to assume every site has one,"

Um, no. I follow a hundred webcomics, and while the situation has gotten better recently, many of them still don't have a feed. That's not unique to webcomics, either. Lack of a newsfeed is common enough that being able to tell at first glace is very useful. Heck, many people, even website owners, are ignorant of RSS. More about that in a moment.

Besides, the feed button serves a second purpose, which is to allow easy access to the feed. Just as I don't want to hunt for an "RSS" or "XML" link inside the website, I don't want to hunt for it in the browser's interface.

Last but not least, many people still don't know about RSS. That big prominent orange icon is a big auto-discovery feature, not just in the technical sense, but also because a newbie could see it and click it out of curiosity, and thus learn about feeds. I had to explain what RSS is to someone whose browser didn't have one, and that proved to be quite difficult.

Of course, all this talk is for nothing as the decision has already been made but, you know. Just my two cents.
Bruce Wayne
2011-01-16T11:34:59

Nice explaination But by removing the rss indicator Firefox has shifted content aggregation control from an open browser to closed siloed companies. There is no valid excuse for this.... Even the one you have eloquently made.... I'm quiet certain that with this decision Firfox will speed up it's demise...Just to let you know... we are close to launching several application platforms for consumers that use RSS as as a content transport protocol
Meepmeepmeep
2011-01-16T11:38:58

Feeds are too ubiquitous to need an indicator

So why do i need to resort to Yahoo Pipes so often on sites i want to see updates on?
- l.m.orchard
  2011-01-16T21:36:00
  
  I don't know why you need to resort to Yahoo Pipes so often—what kinds of sites are you trying to subscribe to? I haven't touched Yahoo Pipes in years, and I have 1000 feeds in Google Reader, having amassed them over the course of 8 years or so.
  That said, I know there are exceptions—I'm being optimistic in my explanation here. In my experience, most sites really do have feeds.
  - Meepmeepmeep
    2011-01-17T09:14:50
    
    Web Comics, Web 1.0 blogs, software download listings, just to mention a few.
    - l.m.orchard
      2011-01-17T16:13:18
      
      Those sound like exceptions to me—that, or sites whose owners have intentionally chosen not to offer feeds. Either way, the feed icon doesn't really help you there anyway.
      - Meepmeepmeep
        2011-01-17T18:42:08
        
        Of course it does. Its existence leads to more people using it, which translates into pressure on webmasters to implement it.
        
        THAT is the whole damn point of why everyone is upset. Its existence may not be a strong pressure, but it is a constant one. It going away means less webmasters will bother to implement rss feeds. This is a plain fact.
Meepmeepmeep
2011-01-16T11:48:09

out of sight, out of mind.

Wow, you seriously have not managed to understand that that is the ENTIRE crux of the issue?

Dead or hidden does not matter, because it does the exact same thing: It means less people will realize RSS feeds exist and are useful.
- l.m.orchard
  2011-01-16T23:31:21
  
  No, I understand that's the crux of the worry. But barely anyone's using that button anyway—so if the whole existence and usefulness of RSS feeds rest on that one icon, feeds are in big trouble.
  - Meepmeepmeep
    2011-01-17T09:12:38
    
    barely anyone’s using that button anyway
    
    Complete fallacy. 3% out of the group participating in a time-limited beta test clicked on the button during that timeframe to subscribe to new feeds.
    
    Now ponder how often you discover new feeds in comparison to how often you use the reload button.
    
    It just plain has a different use model. A more realistic measurement would've been to compare how often an rss preview was opened via the button versus a link on the page itself.
    - l.m.orchard
      2011-01-17T16:36:15
      
      If you think that'd be an interesting study, go ahead and propose it to the Test Pilot team: https://testpilot.mozillalabs.com/
      - Meepmeepmeep
        2011-01-17T18:45:07
        
        I've tweeted at them, since their site does not provide an email i can send something at. If you know of an email address of theirs or better, an actual chat contact, please let me know.
      - l.m.orchard
        2011-01-17T20:34:39
        
        I don't think there's a direct email contact, but the Test Pilot / Labs site lists all of these things:
        
        Forums - http://groups.google.com/group/mozilla-labs-testpilot/topics
        
        Proposals - https://wiki.mozilla.org/Labs/Test_Pilot#Propose_and_develop_Test_Pilot_studies
        
        IRC - irc://irc.mozilla.org/labs
Craig Overend
2011-01-16T13:06:07

Feed discovery is already a pain in the arse, now it's made even harder. I'm going to miss the bright orange dropdown icon (until someone brings it back as an add-on). Does the new "space wasting" icon drop down with the list of feeds still?
- l.m.orchard
  2011-01-16T23:32:29
  
  Both the icon and the subscribe menu list the available feeds. Have you tried the new beta?
Meepmeepmeep
2011-01-16T16:13:16

More flat out wrong stuff:

subscribing to a page falls into the same thought process as bookmarking it—but I don’t think that’s an unreasonable notion.

It is completely and utterly unreasonable.

One bookmarks a page to be able to quickly visit it later and directly.

One subscribes to its feed in order to avoid having to actually visit it at all.

I sincerely hope i do not need to explain how these concepts are completely and utterly opposed to each other.
- l.m.orchard
  2011-01-16T21:33:47
  
  I disagree. To me, the bookmark menu is the place where I manage a long-term relationship with a page. That's where I'm coming from.
  That menu includes all my bookmarklets for sharing, making new bookmarks—and now, subscribing to a page. Makes sense to me. Maybe it would make more sense if the menu was renamed.
  - Meepmeepmeep
    2011-01-17T09:13:47
    
    a long-term relationship with a page
    
    There are different kinds of relationships.
SilverWave
2011-01-16T17:19:01

I think Mozilla missed fully exploiting RSS in Firefox after a great start with LiveBookMarks...

You can fix it like I did with some small add-ons...

I use Boox with Places Full Tiles and Stay-Open Menu. Then all your feeds are on the BookMark Toolbar and easily accessible, bolded if unread.

RSS nirvana.

Oh yeah - sorry but put the RSS icon back !

Heh ;-)
- l.m.orchard
  2011-01-16T21:31:49
  
  I actually agree that Mozilla (and Microsoft, and Apple, and Google) have missed out on exploiting RSS by not going beyond Live Bookmarks—I'm probably going to write about that in the near future. And, yes, the feed icon can be put back in its former place by way of add-on. But, it won't be going back in Firefox 4
opi
2011-01-16T18:36:51

But if it's not being there by default most of users won't reactivate it so we're going to lose readership. What percent of Firefox users do changes to its chrome? I know I do, but I can dig RSS URI straight from website code.
- l.m.orchard
  2011-01-16T23:35:02
  
  The reason it'll no longer be there by default is because it's rarely used. I doubt you got much readership from it in the first place.
@JoeHobot
2011-01-16T18:48:17

Great article my friend. I liked the fact you pointed out "The feed icon isn’t dead, it’s just hiding"

Anyway, hope that FF4 is coming out soon, out of beta release.

Wish you good day now..

Founder of MWD
Andy Gongea
2011-01-16T20:45:36

I understand that the idea is to make the user interface lighter and without any distractions. On the other hand I also understand that a lot of blogs depend on this feature to increase their readers base.

Let's try a scenario somehow similar to the RSS problem. What if, Firefox would be launched with the Navigation Toolbar hidden? How many will enable it?

I have another one, less scary. What if, Tabs on Top would be unchecked by default? How many people will change this setting?

Similar to these scenarios, RSS usage will drop on FF.
- dude
  2011-01-17T03:33:25
  
  Great point. I have an even scarier proposition, what if tabs were a hidden feature?
Felix Pleșoianu
2011-01-17T06:30:09

I forgot to mention something.

Saying that the feed icon is rarely used misses the point big time. There are websites I've been visiting three times a week for years, but I only needed to click the feed icon once. That's what feeds are for: to spare you from manually clicking a button to check for updates.

Statistics are great, but they don't substitute for common sense.
- l.m.orchard
  2011-01-17T20:38:43
  
  So, if you only use it once per site, then why should it be there in the toolbar all the time for every site? My common sense tells me that's a good candidate to put somewhere less obtrusive. Beyond that, the feature still does everything else you mention.
  - Felix Pleșoianu
    2011-01-18T07:00:57
    
    "So, if you only use it once per site, then why should it be there in the toolbar all the time for every site?"
    
    For the same reason you have fire hydrants everywhere in a city, even if they are very rarely used. Heck, ideally you should never need them; but would you advocate taking them out?
    
    I gave a bunch of reasons why the RSS feed icon is good in my first comment. And I hope all of them were already known, and have been weighted, when the decision was made.
Michael K Pate
2011-01-17T17:53:19

In Chrome, I have the RSS Subscription Extension (by Google). Now in Firefox, I guess I will be installing RSS Icon every time (or at least as soon as it is updated).

I still regard the decision to break functionality as disappointing, though.
- l.m.orchard
  2011-01-17T20:40:45
  
  This is one of those things I file under not pleasing everyone. I'm guessing there are lots of people who are just fine with it, and so aren't a part of these discussions. How much outrage has there been over Chrome lacking a feed icon by default, versus people praising Chrome's UI simplicity over Firefox?
  - dude
    2011-01-18T02:43:52
    
    I read more about Firefox starting slow, Chrome being responsive, than I do praise for "Chrome's UI simplicity", and yes, people do harp on Chrome for the lack of a feed icon as well.
Danny
2011-01-18T19:23:44

If my own experience is anything to go on, the feed icon's benefit in spreading the idea of feeds is being highly exaggerated in some of the comments here. It was a very long time before I noticed it, for pages without an icon in them I would do View Source to find the alternate links.

On the other hand, I reckon having something in the bookmarks menu would give more exposure to people who want to do more with a given resource (bookmarking and subscribing to feeds are both rather unidirectional relationships!).

In fact it would be really useful to have a fully integrated, user-accessible system for handling all such things, ideally one that could be transparently shifted to the Web (with appropriate access control). A built-in HTTP server would be a good start (is there one yet?), plus support for (say) Atom protocol for shoving data out. This could make it a lot easier to work with semi-social sites like del.icio.us and Bloglines. It's notable that these kind of tools have traditionally been very server-driven, with browsers playing (limited) catch up with plugins. Why can't the client set the agenda for a while?
- l.m.orchard
  2011-01-19T03:16:41
  
  FWIW: Firefox Sync does some shifting of things to the web. It's a pretty dumb store, providing just enough structure for the client to push encrypted hunks of data into a storage cloud shared between devices. But, it is a bespoke API for sync and not composed from standards like Atom.
  
  As for a built-in HTTP server... I've been toying with the idea of playing with something like that since I started poking at the new add-on API and seeing a lot in common with node.js. It seems like that's getting easier soon
Dexter
2011-05-03T13:14:26

Thanks for your explanation. Even though it is helpful, FF's decision is still a step back. Fortunatelly, there are addon developers that help fixing FF's stepabacks: https://addons.mozilla.org/en-US/firefox/addon/rss-icon/

What happened to feed auto-discovery in Firefox 4?

2011-01-15T16:18:56+00:00

TL;DR: The feed button is not dead; it's just been sent to sing backup in Firefox 4 because it's not pulling its weight. This post talks about why things have changed.

This post is one of several on this subject.

Defaults are hard to pare down

The feed icon still lives in the Firefox 4 Beta; it's just not there by default any more. But, if you look at the controls available in the toolbar customization panel, you'll notice a lot of things that don't show up by default—there are a lot of things in there, period.

This is the dilemma for the Firefox User Experience team at Mozilla: Like it or not, one of the main themes of this next generation of browsers is minimalism—faster, smaller, less browser to get in the way of what you're browsing. Yet, at the same time, Firefox 4 has the features of Firefox 3.6 and more.

You can't just cram it all in there, so what gets prime real estate by default? People say it's just a few pixels, that feed button—but is it so much more important than anything else that could go there? And before you answer, consider that not just for your personal use, but for the 100's of millions of people using Firefox. How do you check your own biases and make a decision on that scale? You could make an educated guess, make a gut check. A lot of brilliant design happens that way.

You can also gather some telemetry from beta installs to see what people really use. Looking at a heatmap of clicks, the feed button is an absolute stinker. This isn't a random whim of the UX team—seriously, it's an order of magnitude less used than anything else in the toolbar (notice the one black spot):

There have been comments dismissing the validity of that heatmap study. But, as far as I can tell, none of them really stick. So, for the sake of argument and a shorter blog post, let's assume barely anyone is using the feed icon and that it's not pulling its weight in this new age of browser minimalism.

Keep in mind that Firefox isn't the only browser to deprioritize the feed button: Google Chrome doesn't even have a feed button at all, and for many people that's the gold standard for minimal browser UI:

Serving the users

Let's hammer on the point of disuse some more—what's the payoff for clicking that thing, anyway?

+1 — You get an option to create a Live Bookmark, which is really handy for things like Bugzilla searches or light headline reading. (I like those so much that I tried building them on the server at Delicious once, but we never quite worked it out at scale.)
0 — You get a list of sites to which Firefox will delegate subscription, if you happen to have installed a content handler. Useful if you know what it's about; a mystery if you don't—not much more useful than a bookmarklet, to me.
-1 — You get a plainly-styled version of what you were probably already looking at on a site, something I criticized back in 2006, like some are criticizing the feed icon change now. (At the time, I was working for a marketing company, and the change cost me days trying to incorporate branding in feeds on a client's site. I think we gave up after a bit, but I suspect that FeedBurner built a nice business routing around that change.)

That's been status quo for years, and it's less than compelling—I've even had people ask me if they broke the page when the button was clicked on accident.

So, given little reward for clicking that feed icon, pushing it into the background in its current state is a service to the users of Firefox. In other words, that feed button has to get a lot more interesting if it's going to serve alongside UI all-stars like Back, Reload, and Location.

Hell, even with Firebug installed, I use View Source more than the feed button these days, and I've never stuck that on the toolbar.

Leading the web

The most compelling response I've seen to the "removal" of the feed auto-discovery button is the challenge that Mozilla and other browser makers should be improving feed-related features, rather than pushing them into the background. With this, I wholeheartedly agree—but I'm not entirely sure what to do about it.

So, there's a balance here: Make the experience of the web better by improving how Firefox handles what's already out there, and make the web itself better by building more powerful and enabling technologies into Firefox and other products. In the best of times, Mozilla can do and has done both at once. WebGL and technologies associated with HTML5 represent great examples of this.

But, alas, syndication feeds in the browser have stagnated. The feed button has been a weak feature for years. In the meantime, other things have taken priority in the Firefox project. The rise of Twitter and Facebook and more complex applications on the web have reduced the need for most people to interact directly with feeds, so demands for attention to the feed button haven't exactly topped the charts in comparison to things like making sure Flash doesn't crash the browser and getting super fast for everything else.

Ship it!

In the end, Mozilla has a browser to ship. The decision on the feed button has been made and, believe it or not, with a great deal of thought behind it.

I know that this is the first time many people have heard of this change, but many already have heard and have given the UX team a number of earsful. Coming in at this late stage and requesting—nay demanding—a reversal of that decision will do nothing but piss off all the people who've been banging away at this and many other things up to now. They're people who've worked hard, stayed up late—and the last thing anyone wants to do is rehash every conversation ever about the topic.

Archived Comments

Jacques
2011-01-16T07:22:31

I understand that we always try to enhance UI of any software and that it takes a lot of time and discussions to make decisions. And some decisions may not please everyone.

As for the Feed button, the one in the toolbar customization panel does not highlight when there are feeds available or change colour when there are no feeds. You need to click on it to see if Firefox 4 discovered feeds of not.

In Firefox 3.6, you don't need to guess or click on a button to find if a feed is available.

So instead of a feed button that is always shown, feed available or not, it's much more usable to show a button that highlight properly (takes more space), or is shown only when feeds are available (optimal space use).

Using a heatmap of clicks does not do justice to the feeds button, since it's of no use to click on it in Firefox 3.6 to instantly see if or not there is a feeds available.

There are other information that have been removed in Firefox 4 that are open to discussion.

For example, a heatmap does not show loading icons, throbbers, urls loading in the status bar, "done" message in the status bar, etc. In fact, any status information won't show up in a clicks heatmap. So if that information does not show up in a clicks heatmap, it does not mean these statuses must be removed from the UI.
- l.m.orchard
  2011-01-16T21:25:30
  
  Actually, the Feed button in the toolbar does change state depending on the presence or absence of feeds: It's enabled and clickable with a feed, and greys out without a feed. I mentioned that in the post. If that's not noticeable enough for you, then that's a separate bug specific to that button.
  
  As for the validity of the heatmap—what would you suggest get done as an alternative? There's no way to fit a large number of beta users with eye-tracking gear. Also, I personally don't buy that people just like to have the feed indicator around because it's nice to see it light up. It's a call to action (eg. subscribe to this site)—and if barely anyone takes that action, how valuable can it be?
Giuliano
2011-01-16T08:40:45

And if you really miss the feed-autodiscovery feature, just use RSS Icon In Awesombar. ;)
- dude
  2011-01-17T03:17:05
  
  RSS/Atom are standardized web features, it's a shame we have to rely on 3rd parties to enable support. At least Google has an official RSS extension so I can trust the security of the addon, but still, both Mozilla and Google are going backwards on this.
voracity
2011-01-16T11:07:30

Well argued. I am a little disappointed with its removal because RSS is a distributed status update mechanism, and we're currently being swamped by centralised status update mechanisms (twitter and whatnot). But the experience has always been seriously crap (to put it bluntly), so better to remove it until it improves I suppose.

Sadly, my guess is that once its removed, it will never reappear.
antistress
2011-01-16T11:27:15

i agree with the new UI concerning feed subscription (along with bookmarks).

But there is indeed a concern with Firefox 4 concerning feeds. Since Firefox doesn't really handle feeds, Sync doesn't care about them either. Therefore if you have installed an add-on to deal with feeds, they will not be synchronized and that's not a good experience for the user. Bookmarks are synchronized but feeds are not, whereas they supposed to belong to the same scheme
- l.m.orchard
  2011-01-17T00:04:24
  
  Good points. I'd say there are lots of things Firefox could do better with feeds in general.
Pavel Cvrcek (Mozilla.cz)
2011-01-16T11:29:40

I think this is right decision even I'm one of the people who use RSS button which is inside location bar. But we have extensions right? Because this function is important for me I created extension RSS Icon In Awesombar which brings this button back to the location bar.

https://addons.mozilla.org/firefox/addon/rss-icon-in-awesombar/
Meepmeepmeep
2011-01-16T11:34:59

The only reason i'll even get to see your follow-up entries to this is because my browser includes an rss reader and a button that tells me you have an rss feed for your blog.
- l.m.orchard
  2011-01-16T21:38:06
  
  So, you mean to tell me that you didn't already think my blog had a feed, before you looked for the feed icon?
  - dude
    2011-01-17T03:22:53
    
    Are you surprised he would be more likely to find an RSS button on his toolbar than somewhere on the side of the page? Or that someone wouldn't assume that all blogs have RSS, especially many older ones? Or that asking Google Reader to test if a URL had a feed is a whole lot more cumbersome than using the browser to get Google Reader to subscribe to it?
  - Meepmeepmeep
    2011-01-17T09:08:24
    
    Yes.
    
    I stumble upon too many blogs of various kinds that have no feeds to make the assumption that every blog has one. As i mentioned on your other post, Yahoo Pipes is a tool i often need to resort to.
    
    Maybe you need to step outside your Web 2.0 bubble sometime?
Janusz Lewandowski
2011-01-16T12:12:53

I suggest removing the Scroll Left button, it's used only by 1% of Fx users.
- l.m.orchard
  2011-01-16T21:28:22
  
  I'm pretty sure removing the Scroll Left button was discussed—if only for a minute—and a call was made that basic spatial navigation was more important than feeds.
dude
2011-01-16T21:25:26

Janusz, I totally agree, let's remove the site identity, remove bookmark, more info, scroll right, print menu item, save page as, bookmarks all tab, etc., they are obviously never used.
- l.m.orchard
  2011-01-17T00:07:20
  
  Pretty much all of those you list are used more than twice as much as the feed icon, or don't appear on the toolbar at all.
Jim L.
2011-01-16T21:32:14

I can put "View Source" in the toolbar? How?!
- l.m.orchard
  2011-01-16T21:40:34
  
  Actually, for "View Source", I installed the Web Developer add-on. So, that's not built-in like I implied.
Tulapi
2011-01-16T22:23:44

Why do you care about this "Back" button, "Location bar" and so on ? Just put one big "facebook" button and one big "google" button, it's enough...

With this kind of decisions, I think you are not "building a better Internet" as Mozilla Foundation is supposed to do.

RSS and twitter are very different ways to access information, with their "+" and "-". Leaving RSS dying seems to me a regression. I am sorry, but when I read "The rise of Twitter and Facebook and more complex applications on the web have reduced the need for most people to interact directly with feeds", I am a bit frightened, because I understand it a bit like : "Hey guy, why don't you go on Facebook like everybody to surf the web. It's cool and your friends tell you what is interesting to read".

I don'k know about the usability of the feed concept, it could be certainly improved, but I really felt it was going towards "give people tools to take control of their online lives" (Mozilla Mission)
- l.m.orchard
  2011-01-17T00:14:38
  
  So, I did say that Twitter and Facebook has left users with less need to interact with feeds—but I didn't say that I liked it. I'm the last person who'd want closed silos to own the web, and I want to see RSS and Atom and other open web technologies grow.
  - dude
    2011-01-17T03:25:24
    
    I'm currently a Chrome user, wishing that 1 day Firefox will go back to why I liked it in the first place. I've been subsribing to Twitter feeds and Facebook pages more than I have with feeds with Chrome, not only because the prior are established, but also because Chrome doesn't encourage RSS with its exclusion. I don't bother with addons either, I don't think most users do either (yet Mozilla decides addons need a whole bar).
Andrew
2011-01-17T01:39:38

I can't be the only person who, when I decide I want to subscribe to a site, looks for the RSS Icon on the page first before I ever even glance at the auto-discovery icon. In many cases, especially on larger sites, I rarely want what they would consider the "main" feed anyway and on the sites where I do want that, their feed is usually in a place that's easily clicked on.

So, in my case, it was needless clutter that I'm rather happy to see getting removed by default. Of course, I remove a lot more than that from Firefox's chrome, but at least this is one less thing I have to do manually.
fflover
2011-01-17T16:26:58

Sir, looks like you know a lot about FF. Is it a conscious decision to not have Awesombar as the default search bar .i hate it when it gives me google search results. if i type bbc news and hit enter, it should take me to bbc news site, right? not a google search of it? If there is any way i can get my awesombar back, please let me know.

Sincerely, fflover :)
- l.m.orchard
  2011-01-17T17:33:17
  
  Sounds like you might be interested in this add-on:
  
  https://addons.mozilla.org/en-US/firefox/addon/luckybar/
ffuser
2011-01-17T19:40:00

beta users is not equal to representative user population. how much percent of all Firefox users does your study cover, is it enough to make statistical conclusions?
- l.m.orchard
  2011-01-17T20:46:23
  
  Well, it's not my study, it's the Test Pilot team's study. I'd say it's a decent chunk of users, with some interesting segmentation. But, that's my opinion. If you have an idea of how to execute a study that covers more, feel free to let the Test Pilot team know.
Doug Bromley
2011-01-18T08:46:09

Wow. There's a lot of strong feelings flying around. Well as one of the silent majority I'd just like to offer my support to the removal. I never used the feed button and whenever I wanted a sites feed I normally scanned the page I was on for an icon to click.
Olivier
2011-01-18T18:03:16

Thank you for your post. I, too, would rather have the button clutter up a little than risk RSS being less discovered. But I understand your point and respect your decisions.
rektide
2011-01-22T02:17:52

your metric for evaluating the utility of the feed button -- click rate -- is about as incorrect and contrary as possibly can be conceived.

the entire point of the feed button is that you only need to click it once for a site. after a user clicks it, the user shouldn't have to click it again on that site, and, going further, shouldn't even have to visit the site. those are the signals for success.

if you want a useful metric, dont look at how often the button is clicked. that's entirely the purpose of what it's trying to avoid. look at return rates for sites where a user has clicked the feed button. if you see a precipitous drop in return rates, you know the button is functioning completely as desired. the bigger the drop, the less users return, the better the button works.

baby HCI jesus is weeping in his crib.
- l.m.orchard
  2011-01-22T08:45:28
  
  So... if it only gets used once per site, then it doesn't belong in the same toolbar with navigational controls that get used constantly. It's a special action, used rarely. Thus, it's in the bookmark menu now - you know, bookmarks, another thing you ideally do once per site if at all.
  
  But as I've said in other comments, if you feel strongly enough, come up with a study to propose to the Test Pilot team and spread your HCI jesus gospel

Firefox Sync server on Google App Engine

2010-07-05T23:00:56+00:00

TL;DR: I built an implementation of the Firefox Sync server API for Google App Engine.

To celebrate Independence Day, I figured I might take a shot at liberating Firefox Sync from the tyranny of Mozilla's servers.

Thus, over the past few days, I've built a sync server using the 1.0 Sync API, hosted on Google App Engine.

I lied about the tyranny thing, though—I just wanted to say something clever about the holiday. In reality, with respect to Firefox Sync, Mozilla has done all of the following:

Published the Sync API spec;
Released the source code for the server used in-house;
Explicitly included the option to use a custom server when setting up sync in the browser.

This means that, although Mozilla offers servers to go along with Firefox Sync, you're totally free to take your data elsewhere. Since your sync data is encrypted and practically opaque to the server, there's no direct profit for Mozilla in offering free sync hosting—not even through any clandestine data mining for devious purposes. It's just that sync makes Firefox a better browser, and somebody has to run some servers to make it work.

So, there's every incentive to make it easy for you to switch sync providers and stop freeloading on Mozilla's servers. Building a server on Google App Engine means I can freeload on Google's servers!

I kid, of course. No one's really complaining about freeloaders, and App Engine has quotas in place to head off any serious mooching—which is why I'm not telling you where to find my sync server deployed on Google App Engine, by the way.

No, I did this because:

Firefox Sync and Google App Engine are interesting and important technologies;
I've already done a bit of work on the PHP-based Firefox Sync server at Mozilla;
I really wanted to take a break from PHP and spend some time with my old friend Python.

There are, of course, a number of bugs in this server. But, it seems to be working between a number of machines and browser profiles I have at home. Things are really in need of optimization, it suffers from my inexperience with App Engine, and I keep running into those aforementioned App Engine resource limits—especially when updating or deleting large numbers of items (ie. 1000's to 10000's of items).

Pull requests and issue reports on GitHub are welcome!

A next step I'd like to take with this thing is to revisit another old friend, the desktop web app server. (Also known as the desktop website.) It seems to me that it would be interesting to scale this server down to a household appliance—say, just for use by my wife and I.

I'd be especially happy if the work I'm doing for a Google-hosted app could be self-hosted at home. Seeing as the development environment for App Engine runs on my laptop, I'm willing to bet I can hack the whole shebang into a simple, special-purpose app to download and double-click on a home desktop PC for use as your sync hub.

Anyway, check it out and let me know what you think.

Archived Comments

James Socol
2010-07-06T01:28:02

Assuming you already had a network and a Mac Mini or something at home, it would be pretty interesting to run your own small Sync server just to keep your home computers together. Maybe even use localtunnel for when you're away.
Mike
2010-07-06T01:36:37

You are a real-life hero!
l.m.orchard
2010-07-06T01:52:00

@James: Yup, exactly that. Maybe even roll in some SSL and UPnP port forwarding for easier setup behind a home router. And, if there were a Windows version, it could run on that cruddy desktop back in the spare room.

@Mike: Naw... this guy is the real hero, even John Lilly agrees!
Crash
2010-07-06T09:20:54

Is there already a App Id for this tool?
l.m.orchard
2010-07-06T10:18:57

@Crash: Sure, there's an app ID for my instance of this sync server. But, as I said in the blog post, I'm not sharing it. At least, not until or unless I get the quota usage down to a point that I wouldn't exhaust the free hosting limits.

It's pretty easy to deploy your own sync server on AppEngine with the source, though.
Peter Petrov
2010-07-06T13:18:11

@Leslie: The app ID of your instance is visible in app.yaml, so you've shared it anyway :)
l.m.orchard
2010-07-06T17:18:47

@Peter hah! Right you are! But, at least you had to look at the source of the app first to figure that out :) That is mostly the point of this blog entry after all. Anyone who does that and then uses my installation anyway will probably be sad when I regularly blow away data and eventually make it invite only
Tobias
2010-07-07T00:24:41

Neat! TyphoonAE http://typhoonae.googlecode.com might help you to build your household appliance.
Kumar McMillan
2010-07-07T20:59:25

Hey Les, this is super cool! Google App Engine's Datastore API is still very shaky though. In fact, it's been so bad lately that I've been considering porting one of my heavily used apps over to something else. Thankfully, this post suggests that fixing the Datastore is their top priority: http://googleappengine.blogspot.com/2010/06/datastore-performance-growing-pains.html
Robert Kaiser
2010-07-07T21:21:51

Well, I actually feel more comfortable with my data being on Mozilla servers than on Google servers - even with the decreased trust I have in the Mozilla organization, I'd trust it more than Google any day! ;-)
Chris Arndt
2010-09-28T11:06:55

Instead of porting GAE to your desktop, why don't you just port your app to plain Django? Django should run off your desktop with no problems.

Case Study: Building a Bookmark Management UI for Mozilla's BYOB

2010-06-22T22:16:24+00:00

TL;DR: I just wrote a long case-study about my process in building a bookmark management feature for Mozilla's Build Your Own Browser web application.

So, remember that tutorial I wrote about writing an article in Tinderbox? I'd mentioned that it was a digression from another, different article.

Well, I just finished a first draft of that article:

It's a case study about building a bookmark management feature for Mozilla's BYOB.

Light on code but heavy on narration, it's about what I did and what I was thinking while I did it. There are links to the end product for reference, but it isn't a demonstration of some new trick. Rather, it's a look at my process as a whole.

I wrote this, in part, for myself: The writing has helped me review things. But, I'm hoping someone else reads it, and then picks up something new or can offer some interesting critique in return.

And, because I'm curious and like looking behind the scenes, I'd love to see this kind of write-up from more webdevs. There don't seem to be a lot of detailed case studies versus quick one-shot tutorials on isolated techniques. It's probably because the writing is time-consuming, as is the reading. It's not in the genre of bite-sized attractions optimized for promoting blog traffic, but I'd like to see more of them all the same.

HTML5 drag and drop in Firefox 3.5

2009-07-15T15:26:40+00:00

Oh hey, look! It's another blog post—and this one is cross-posted on hacks.mozilla.com. I won't say this is the start of a renewed blogging habit, but let's see what happens.

Drag and drop is one of the most fundamental interactions afforded by graphical user interfaces. In one gesture, it allows users to pair the selection of an object with the execution of an action, often including a second object in the operation. It's a simple yet powerful UI concept used to support copying, list reordering, deletion (ala the Trash / Recycle Bin), and even the creation of link relationships.

Since it's so fundamental, offering drag and drop in web applications has been a no-brainer ever since browsers first offered mouse events in DHTML. But, although mousedown, mousemove, and mouseup made it possible, the implementation has been limited to the bounds of the browser window. Additionally, since these events refer only to the object being dragged, there's a challenge to find the subject of the drop when the interaction is completed.

Of course, that doesn't prevent most modern JavaScript frameworks from abstracting away most of the problems and throwing in some flourishes while they're at it. But, wouldn't it be nice if browsers offered first-class support for drag and drop, and maybe even extended it beyond the window sandbox?

As it turns out, this very wish is answered by the HTML 5 specification section on new drag-and-drop events, and Firefox 3.5 includes an implementation of those events.

If you want to jump straight to the code, I've put together some simple demos of the new events.

I've even scratched an itch of my own and built the beginnings of an outline editor, where every draggable element is also a drop target—of which there could be dozens to hundreds in a complex document, something that gave me some minor hair-tearing moments in the past while trying to make do with plain old mouse events.

And, all the above can be downloaded or cloned from a GitHub repository I've created especially for this article—which continues after the jump.

The New Drag and Drop Events

So, with no further ado, here are the new drag and drop events, in roughly the order you might expect to see them fired:

dragstart: A drag has been initiated, with the dragged element as the event target.
drag: The mouse has moved, with the dragged element as the event target.
dragenter: The dragged element has been moved into a drop listener, with the drop listener element as the event target.
dragover: The dragged element has been moved over a drop listener, with the drop listener element as the event target. Since the default behavior is to cancel drops, returning false or calling preventDefault() in the event handler indicates that a drop is allowed here.
dragleave: The dragged element has been moved out of a drop listener, with the drop listener element as the event target.
drop: The dragged element has been successfully dropped on a drop listener, with the drop listener element as the event target.
dragend: A drag has been ended, successfully or not, with the dragged element as the event target.

Like the mouse events of yore, listeners can be attached to elements using addEventListener() directly or by way of your favorite JS library.

Consider the following example using jQuery, also available as a live demo:


    Drag me!
    Drop here!

Thanks to the new events and jQuery, this example is both short and simple—but it packs in a lot of functionality, as the rest of this article will explain.

Before moving on, there are at least three things about the above code that are worth mentioning:

Drop targets are enabled by virtue of having listeners for drop events. But, per the HTML 5 spec, draggable elements need an attribute of draggable="true", set either in markup or in JavaScript.

Thus, $('#newschool .dragme').attr('draggable', 'true').
The original DOM event (as opposed to jQuery's event wrapper) offers a property called dataTransfer. Beyond just manipulating elements, the new drag and drop events accomodate the transmission of user-definable data during the course of the interaction.
Since these are first-class events, you can apply the technique of Event Delegation.
What's that? Well, imagine you have a list of 1000 list items—as part of a deeply-nested outline document, for instance. Rather than needing to attach listeners or otherwise fiddle with all 1000 items, simply attach a listener to the parent node (eg. the

Using dataTransfer

As mentioned in the last section, the new drag and drop events let you send data along with a dragged element. But, it's even better than that: Your drop targets can receive data transferred by content objects dragged into the window from other browser windows, and even other applications.

Since the example is a bit longer, check out the live demo and associated code to get an idea of what's possible with dataTransfer.

In a nutshell, the stars of this show are the setData() and getData() methods of the dataTransfer property exposed by the Event object.

The setData() method is typically called in the dragstart listener, loading dataTransfer up with one or more strings of content with associated recommended content types.

For illustration, here's a quick snippet from the example code:

var dt = ev.originalEvent.dataTransfer;    
dt.setData('text/plain', $('#logo').parent().text());
dt.setData('text/html', $('#logo').parent().html());
dt.setData('text/uri-list', $('#logo')[0].src);

On the other end, getData() allows you to query for content by type (eg. text/html followed by text/plain). This, in turn, allows you to decide on acceptable content types at the time of the drop event or even during dragover to offer feedback for unacceptable types during the drag.

Here's another example from the receiving end of the example code:

var dt = ev.originalEvent.dataTransfer;    
$('.content_url .content').text(dt.getData('text/uri-list'));
$('.content_text .content').text(dt.getData('text/plain'));
$('.content_html .content').html(dt.getData('text/html'));

Where dataTransfer really shines, though, is that it allows your drop targets to receive content from sources outside your defined draggable elements and even from outside the browser altogether. Firefox accepts such drags, and attempts to populate dataTransfer with appropriate content types extracted from the external object.

Thus, you could select some text in a word processor window and drop it into one of your elements, and at least expect to find it available as text/plain content.

You can also select content in another browser window, and expect to see text/html appear in your events. Check out the outline editing demo and see what happens when you try dragging various elements (eg. images, tables, and lists) and highlighted content from other windows onto the items there.

Using Drag Feedback Images

An important aspect of the drag and drop interaction is a representation of the thing being dragged. By default in Firefox, this is a "ghost" image of the dragged element itself. But, the dataTransfer property of the original Event object exposes the method setDragImage() for use in customizing this representation.

There's a live demo of this feature, as well as associated JS code available. The gist, however, is sketched out in these code snippets:

var dt = ev.originalEvent.dataTransfer;    

dt.setDragImage( $('#feedback_image h2')[0], 0, 0);

dt.setDragImage( $('#logo')[0], 32, 32); 

var canvas = document.createElement("canvas");
canvas.width = canvas.height = 50;

var ctx = canvas.getContext("2d");
ctx.lineWidth = 8;
ctx.moveTo(25,0);
ctx.lineTo(50, 50);
ctx.lineTo(0, 50);
ctx.lineTo(25, 0);
ctx.stroke();

dt.setDragImage(canvas, 25, 25);

You can supply a DOM node as the first parameter to setDragImage(), which includes everything from text to images to canvas elements. The second two parameters indicate at what left and top offset the mouse should appear in the image while dragging.

For example, since the #logo image is 64x64, the parameters in the second setDragImage() method places the mouse right in the center of the image. On the other hand, the first call positions the feedback image such that the mouse rests in the upper left corner.

Using Drop Effects

As mentioned at the start of this article, the drag and drop interaction has been used to support actions such as copying, moving, and linking. Accordingly, the HTML 5 specification accomodates these operations in the form of the effectAllowed and dropEffect properties exposed by the Event object.

For a quick fix, check out the a live demo of this feature, as well as the associated JS code.

The basic idea is that the dragstart event listener can set a value for effectAllowed like so:

var dt = ev.originalEvent.dataTransfer;
switch (ev.target.id) {
    case 'effectdrag0': dt.effectAllowed = 'copy'; break;
    case 'effectdrag1': dt.effectAllowed = 'move'; break;
    case 'effectdrag2': dt.effectAllowed = 'link'; break;
    case 'effectdrag3': dt.effectAllowed = 'all'; break;
    case 'effectdrag4': dt.effectAllowed = 'none'; break;
}

The choices available for this property include the following:

none: no operation is permitted
copy: copy only
move: move only
link: link only
copyMove: copy or move only
copyLink: copy or link only
linkMove: link or move only
all: copy, move, or link

On the other end, the dragover event listener can set the value of the dropEffect property to indicate the expected effect invoked on a successful drop. If the value does not match up with effectAllowed, the drop will be considered cancelled on completion.

In the a live demo, you should be able to see that only elements with matching effects can be dropped into the appropriate drop zones. This is accomplished with code like the following:

var dt = ev.originalEvent.dataTransfer;
switch (ev.target.id) {
    case 'effectdrop0': dt.dropEffect = 'copy'; break;
    case 'effectdrop1': dt.dropEffect = 'move'; break;
    case 'effectdrop2': dt.dropEffect = 'link'; break;
    case 'effectdrop3': dt.dropEffect = 'all'; break;
    case 'effectdrop4': dt.dropEffect = 'none'; break;
}

Although the OS itself can provide some feedback, you can also use these properties to update your own visible feedback, both on the dragged element and on the drop zone itself.

Conclusion

The new first-class drag and drop events in HTML5 and Firefox make supporting this form of UI interaction simple, concise, and powerful in the browser. But beyond the new simplicity of these events, the ability to transfer content between applications opens brand new avenues for web-based applications and collaboration with desktop software in general.

Archived Comments

David Semeria
2009-07-16T17:56:52

Wow - thanks for a great introduction to D+D in HTML 5, I'm really looking forward to giving it a thorough road test.

The key part of the implementation are the target listeners, because, as I'm sure you're aware, onmouseover would historically never fire over the target element because the dragged item would always 'cover it up'. BTW, my proposed solution was to add an 'event transparency' property, which would have made the dragged item invisible from the point of view of selected events, eg onmouseover.

This implementation is going to take a lot of pain away. You have no idea how many hoops I had to jump through to get a fully generic D+D implementation working without these new calls.

If you're interested, you can see it working here: http://lmframework.com/page.php?id=vd_twig_short_2
Alex
2009-07-17T00:13:23

Hey,

Looks great and useful article. For dragging from external applications (such as the Desktop) is a normal webpage able to get the data from the drop? I was just playing with it and seem to always get what seems to be a security error with getData or mozGetDataAt. However, the documentation seems to state that on drop that data should be made available to the page. Likewise, in the same way an image can be dragged off the page onto the desktop, can an arbitrary element be dragged onto the desktop with whatever file content to be saved? Thanks for the help.

Alex
Paul Tarjan
2009-07-17T04:52:54

Very nice, I didn't know that FF 3.5 actually had this implemented. Time to start playing :)

Also, how did you do your code posts? Was it just pasting in HTML or do you have a better setup?
Animal
2009-07-17T14:35:21

Seems pretty pointless pressing ahead with this when you just can't write a web app to use it. There's no support. All a bunch of whizzy fun I'm sure, but sod-all use.
l.m.orchard
2009-07-17T16:23:10

@Animal: Why can't you write a web app to use it? It's in the HTML5 spec, works in Fx3.5 and Safari 4 / WebKit. It doesn't have universal support yet, or course, but new standards never do.
paul
2009-07-21T08:08:26

very nice indeed! i like this, it has definitely opened up new avenues for interaction without using flash. but it will take time before people upgrade to the new browser. a lot of people who browse the web wont realise what the technology will bring and will use their current browser because it does the job!
Joel
2009-07-21T14:36:04

Excellent demos of the new features.

When using a feedback image on a zoomed in page, should the image not also be scaled up automatically?
RenegadeX
2009-07-22T04:13:46

Dragging & dropping is as you say, a "one of the most fundamental interations" that users of computer graphical interfaces have come to know, and expect.

It therefore completes dumbfounds me how it is that we still can not natively drag & scroll (& then drop) in Firefox. We can only drag & drop an item within the currently visible portion of a Firefox webpage or tab, no further up, no further down.

That is should be limited makes absolutely no sense. Microsoft built the function into their Windows 3.x (file) Explorer, and then when visual browsers came along, naturally carried it through into Internet Explorer.

Firefox Bug 41708, "Should be able to scroll in the viewport while dragging" was opened in June, 2000 (yes, 9 years ago!) and remains open, and is disregarded by Firefox developers as little more than a trivial little annoyance, and therefore is and should be treated as a low-importance 'enhancement' rather than as the 'standard feature' it should really be.

If it weren't for the extension 'DragToScroll', which has been around for 3-1/2 years now (not updated in a long time but still works if you override version compatibility), I would have dumped Firefox and switched to a different browser (Maxthon 3, most likely).

So, I'm wondering (and hoping) -- does the new HTML5 drag and drop specification include anything that if implemented properly would make it possible to scroll & drag?
l.m.orchard
2009-07-22T13:57:42

@RenegadeX: Ugh. I don't work on the browser itself, but I noticed that unexpected behavior on drag & drop. The window really should scroll when you get toward the top or bottom while dragging - and, in fact, most pre-HTML5 JS frameworks do that in their own drag & drop abstractions.

Until / unless Firefox gets this fixed, the same sort of auto-scrolling could be hacked in to HTML5 drag and drop. Not perfect, but it's doable. That is, in the drag event, you can check if the mouse is near the upper or lower edge of the viewport - and if so, start scrolling appropriately. That's pretty much how the JS frameworks do it with old-school D&D
CopperBot
2009-07-23T21:41:07

Very cool article! Thanks for sharing. HTML5 really is going to change everything about how we use and develop for the web. Pretty awesome!
Alexos
2009-07-26T05:05:44

Hi, would DD work for uploading files? How can I display a progress bar? Thanks! I've been looking for that several days, and found only applets which I can't use in my proyect :-)
Sam
2009-07-28T13:54:26

Looks fantastic, going to try it out now (once I have upgraded my FF)
Dan Kantor
2009-07-30T03:10:25

Looks like effectAllowed and dropEffect are not working for FF 3.5 on the Mac. I see effects on Windows and Safari 4 on Mac. I've been playing around with adding borders to the drop target, but the + icon for copy really helps a lot. Hopefully Mozilla will fix this next update.
Ryan
2009-09-01T02:33:06

Great article it's good to see developments in this area. I wrote an article on the further extensions Firefox 3.6 has done with the dataTransfer method by adding the file attribute so you can access local files and upload them without the need for file inputs. http://www.thecssninja.com/javascript/drag-and-drop-upload
phil swenson
2009-09-03T22:58:36

Why not a first class WYSIWYG rich text editor with copy from clipboard image support?

NO ONE has built a decent text editor in JavaScript - At least not that I've seen. And browsers don't allow image paste for reasons I don't understand.

And by decent I mean just like you'd get in textmate or another editor. Have the standard text editor features everyone expects: tab, select indent/select unindent, resize image, home, end, duplicate line, delete line, styling, etc.

would push the browser to the next level in killing the desktop. non-techies hate wikis. They want a real editor. I do too actually.... would be great for forums, bug tracking system (imagine pasting screen shots in line with your bug desc), email, etc.
Jean-Denis
2009-09-04T00:23:39

Francisco Tomalski wrote up a nice piece on HTML 5 drag'n drop at http://www.alertdebugging.com/2009/08/16/on-html-5-drag-and-drop/

where he uncovers that the proposed standard is partially broken. Any comment on his piece?
Lisa
2009-09-09T07:33:08

This is a great move forward no more relying on 3rd party apps and extensions to play video or audio, i have been reading up on HTML 5 at the HTML 5 Tutorials website, i am now playing around with one of the free templates and was wondering how to embed audio, so thanks a lot, great information, lets hope more people lean towards HTML 5 and SOON!!!

The drag and drop feature i did not notice was already working in FF 3.5, i was told to get Safari to see HTML 5 in action. Thanks for a great post
Zoltan Hawryluk
2010-01-11T15:14:48

A million thank yous! This article was great introduction to HTML5 D+D. With it, I was able to extend it to other browsers. It was a little painful at first because the browser implementations diverge in significant, but manageable ways.

If you are interested, check out my article at http://www.useragentman.com/blog/2010/01/10/cross-browser-html5-drag-and-drop/ if you are interested in my results.

I noticed you haven't blogged in a while - I hope you haven't stopped totally and continue to share with the webdev community.
Raul
2011-04-26T23:56:02

Hi, Leslie. I've just come across a problem that's driving me nuts. I'm not fully comfortable with D&D, but managed to move a crosshairs image over a map to very precisely controlled positions. It worked great on FF 3.6 and FF 4. After a couple of days of successful testing, the image suddenly refused to de dropped after being dragged (it rather flew back to its previous position). Do you know if there is a bug in FF that might cause this?

BTW, during the programming process I updated Firebug, which also is getting a little cranky.

Thanks for your prompt answer, Raul
Alexander Plutov
2011-10-01T11:51:12

Good post about Drag & Drop http://plutov.by/post/html5_drag_and_drop

7 facts about me

2009-01-23T16:59:54+00:00

So, it finally happened—I've been tagged by Stephen Donner. I've not been one to follow memes in this blog, but this one's been going around the Mozillasphere for awhile now and has been kind of interesting. I'm half-tempted to bookmark and tag all the entries I've caught so far. Anyway, the rules:

Link to your original tagger(s) and list these rules in your post.
Share seven facts about yourself in the post.
~~Tag seven people at the end of your post by leaving their names and the links to their blogs.~~
~~Let them know they’ve been tagged.~~

Now, for your random facts, after the jump:

My wife is a nerd, who tracked me down via a user name shown in the webcam picture I used for my Yahoo! Personals ad, in order to avoid paying to reply. Dial-up BBSes and DOOR games played a part in our courtship.
We have two ocicats, littermates named Puck and Inanna. They're both very cute and are more awesome than anyone else's cats.
I have sleep apnea, and use a CPAP machine every night. If you snore, go get tested. Heart disease, of which my snoring father died, is one of many possible consequences of untreated sleep apnea.
Although I have been a computer nerd since the age of 7, I had the wild notion in starting college that I wanted to be a journalist or a psychologist. Luckily, the web took off during my freshman year, which let me be a little of all the above. I'm not sure why I ever thought I'd be anything other than a computer nerd.
I'm a complete introvert and though I work hard at being sociable, I need a day or two alone for every half-day or so spent being social. My main coping mechanism since even before my teenage years was to make friends from behind a keyboard and screen, and the internet has made that even more scalable as I've grown up. And now, I get to work that way too—HUGE SUCCESS!
In my senior year of High School, I won 2nd place in the Ayn Rand Institute's The Fountainhead essay contest. This was the culmination of having read every scrap of Ayn Rand's writing I could get my hands on throughout High School. I used the money to take a summer trip to visit a friend who'd moved to Texas and to buy my well used, now dead palmtop PC.
I like extremely hot spicy food, insanely hop-saturated beer, Irish whiskey, and fresh locally roasted coffee.

Per Ben Hearsum, I think I'll make this a leaf node in the meme spanning the Mozillasphere. In a quick 5 minute search, it looks like most blogging Mozillans have already been tagged, and I'm not going to spread the contagion outside the organization. :)

Archived Comments

David Tenser
2009-01-23T18:26:51

Re #5, I just realized how annoying my "les is more" jokes must be! I need to figure out how I can deliver these high class jokes in a more subtle way while still being able to entertain myself. Maybe I could just think about the phrase instead?

Re #7, I couldn't agree more!
l.m.orchard
2009-01-23T19:20:44

@David: Heh, heh. The "les is more" jokes aren't really annoying - they've just kind of graduated up to a bigger meta-joke that I've gotten used to. :) No worries!
Geof F. Morris
2009-01-24T00:15:47

I had always read you talking about CPAP and just thought you were crazy ... then I had an NPSG and got mine. HOLY CRAP YOU ARE NOT KIDDING.
http://pattimst3k.livejournal.
2009-01-24T03:59:30

When did you and your wife meet? I met my husband on line back in 1999 when people still thought that you only met psychopaths on the internet. No one could understand it...until Jeff explained how picky I am and then it made sense :) Hopheads rule!!</p TeacherPatti
Easysleeper
2011-06-14T10:01:28

I am so glad that you talk about your sleep apnea and CPAP machine as openly as you do. People shouldn't be ashamed of it!

Enter the LizardFeeder

2009-01-06T00:01:54+00:00

[caption id="attachment_1582" align="alignright" width="247" caption="The Mozilla Tree"][/caption]

Behind Firefox is Mozilla, and behind Mozilla is a community. And the Mozilla community acts a lot like an ecosystem, which can be visualized as a kind of living tree—not to confused with the mozilla-central tree. Oh yeah, and Mozilla is the name of both a Foundation and a Corporation.

Confused yet? If not, then we should talk so you can explain it to me, because it all looks pretty tangly and intertwingled to me. Nonetheless, it seems to work, and produces a good chunk of my favorite software and technologies.

There are many efforts to track what's going on—including planets and newsletters and bugzillas and wikis and repositories and tinderboxen. Some of these resources report on, or are driven by, the activity occurring in the others. Some are automated, and others are carefully stitched together by hand. None offer a full picture of what's going on in the Mozilla galaxy in a way that's casually comprehensible by a sane human being.

Of course, that's not a slight against any of these sites or the people maintaining them—extracting an overview from such an organic phenomenon is neither easy nor straightforward. But, it might be fun to try.

As an infovore and avid practitioner of continuous partial attention, my first impulse is to reach for a firehose and stick my head into the stream. Relax, defocus, and try to let my pattern recognizers do their thing—sometimes those pattern recognizers are in my head, and sometimes they're written in Python.

[caption id="attachment_1585" align="alignright" width="225" caption="Firefox Victory!"][/caption]

But, for Mozilla, I couldn't find a stream of sufficient volume or completeness to satisfy me or my robots. Happily, though, my feeding urge found itself aligned with a project to discover the patterns of contribution in the Mozilla community and to find a way to thank the contributors responsible.

So, while we're still working on the thank-you angle, allow me to introduce you to the Lizardfeeder. The LizardFeeder is a feed aggregator, whose source code is built atop Sam Ruby's Planet Venus. The LizardFeeder pulls together and archives activity streams from a wide variety of Mozilla community sources. Beyond the usual human-readable pages produced by a blog-gathering Planet, the LizardFeeder accumulates statistical and historical data meant for consumption and analysis by robots.

At present, the only robot navigating the LizardFeeder archives is an AJAX-ified user interface that animates the firehose as a near real-time or time-lapsed stream of events scrolling by.

This is just meant as a conversation starter, though. I'm hoping to gather feedback and find more sources, as well as to entice creative community members to come up with more sophisticated visualizations of this data.

So, take a look, check it out, and let me know what you think!

Archived Comments

dria
2009-01-06T01:26:18

Is there a list of what sources are already being read by the LizardFeeder anywhere? I scanned through the various links here but didn't see anything obvious :)
l.m.orchard
2009-01-06T01:37:15

Yeah, that part could use some improvement. There's a monster list here:

http://feeds.mozilla.com/sources.opml
Brian King
2009-01-06T10:11:46

Excellent stuff. I was going to ask about access to the list of sources for each category, but Dria beat me to it.
l.m.orchard
2009-01-06T15:26:50

Actually, it occurs to me that this config file might work as a more readable version of the list of sources:

https://svn.mozilla.org/projects/lizardfeeder/trunk/conf/config.ini-dist

There's also this, which is where most of the previous list came from:

https://svn.mozilla.org/projects/lizardfeeder/trunk/conf/hg-feeds.opml-dist
Austin King
2009-01-06T20:55:52

Thanks for the background surrounding lizard feeder. Great post!

+1 Dria and Brian

Maybe make the title of the link to the OPML more descriptive than just "Feeds" and/or link to it in the body of the UI too and write something around it to encourage other's visualizations.

Awesome work Les.
Sanjay Parekh
2009-02-04T20:38:04

Is the AJAX UI available anywhere for download? I'd like to hack it for another use altogether. Great visualization and great application. Good job.
Deen Seth.
2009-09-16T18:21:02

This is a very good idea. Can we accomplish the same result using Yahoo Pipe? Do you plan to add events from Bugzilla, and mailing list to the feed?

I am more interested in development activities. There aren't much in code category.

I am interested in analyzing development related events.

Improving my Delicious command for Ubiquity

2008-09-07T05:20:20+00:00

After writing up my first stab at a Delicious command for Ubiquity, I planned to continue revising it based on feedback and to work on exploring more of what Ubiquity enables. I started looking into writing my own nouns for tag suggestions, as well as playing with page load and browser startup hooks. And, I also started poking at a little bit of deeper extension development, which took up most of my time today.

I've updated my UbiquityCommands page and checked in my latest revision of the Delicious command.

The main new feature is a status bar item reporting bookmarks for the current page:

As you can see above, the command now comes with a status bar panel powered by the Delicious URL info JSON feed, providing bookmarking info on every page visited. It shows a bookmark count, a tooltip with further information, and sends the user to the URL info page on Delicious when clicked. It mostly works, but it could use some looking at. This is my first time really cracking open the hood on Firefox and XUL, and so I'm feeling around in the dark.

Specifically, I'm using Ubiquity's page load hook—but I'm also trying to augment that by tracking tab selection events, in order to keep the status bar info updated for the active tab. But then, that leads me to trying to track new windows, to attach the tab selection event handler for every newly opened window. Or I could just be barking up the wrong tree entirely. At any rate, the code is probably brain-dead dumb, so I hope someone can clue me into a better way.

Archived Comments

Gen Kanai
2008-09-07T08:10:39

Les, you may want to let R/RW know about your new command.

http://www.readwriteweb.com/archives/the_ultimate_list_of_custom_ubiquity_verbs.php
Abimanyu Raja
2008-09-07T08:55:52

I disagree... Your command doesn't fit on that list. It's way too good for that list. :)
Jeton
2008-09-07T21:56:43

Well, i was having problems understanding the "tagged" and "entitled". Didn't knew it was without brackets....

But i gotta say, it actually replaces the delicious firefox extension for its easy way of posting links on delicious.

Well done, and seeing that it's only the begining i don't doubt that it will improve much more!

Oh, and i had a WTF when i first saw that delicous icon on the status bar of Firefox. Didn't knew it was from your command ;)

Thanks for making this.
Jeton
2008-09-07T22:20:09

One question: Is there a chance to remove the quotation marks (i meant the same on the above comment as well, not brackets) when saving a note about a bookmark?

Everytime i bookmark a page via this command, it quotes the note.
l.m.orchard
2008-09-07T22:41:20
@Jeton: I guess the quoted thing is something I do by habit, to discriminate between what's my comment and what's summary straight from the page. One way you can force it unquoted is with the "noted" modifier. So, select some text on the page, and you can do something like:
```
sha noted this tagged foo bar baz
```
That should use the selected text without quotes. Also, you can just type arbitrary stuff:
```
sha noted These are notes entitled A title goes here tagged foo bar baz
```
l.m.orchard
2008-09-07T22:44:59

@Jeton: Actually, I just made a check-in that flips it around. There's now a "quoted" modifier to wrap the notes in quotes, leaving it unquoted otherwise. I might be the weirdo here in my quote usage :)
Jack Mottram
2008-09-09T10:04:12

I'm a weirdo too, and now I can't quite work out how to get quotes around stuff I'm quoting from a page. I assumed this would work:

sha this quoted

but it just adds the word 'quoted' to the end of the quoted text, without quotes.

Similarly:

sha quoted this tagged test

adds the words 'tagged' and 'test' to the end of the quoted text, though this time everything _is_ in quotes. (So it seems I can get quotes, but lose the ability to tag in the process.)

sha this quoted text

puts the word 'text' in quotes, but drops the selected text...

So, I give up - what's the secret to getting quotes around quoted text?!
Jack Mottram
2008-09-09T10:11:59

And of course I worked it out immediately after leaving that comment:

sha tagged test monkey whatever quoted this

works just fine (but feels the wrong way around to be tagging before quoting). And now I can't work out how to quote and note, so to speak. I tried adding

noted These are my comments on what I just quoted

but it doesn't work, it puts 'noted These are my &c.' inside the quotes...

(I should say, despite all this moaning, I really have been enjoying using the command - it's very slick compared to the official add-on.)
l.m.orchard
2008-09-09T14:30:06
Huh, weird - that's exactly how I use the command:
```
sha quoted this tagged foo bar baz
```
One major difference, though, is that I'm using a bleeding edge checkout of the Ubiquity extension. I wonder if there's a bug in the parser for the last release. :(

The definition of the modifiers is pretty simple, so there's not a whole lot of debugging I can do in the command itself. Ugh.

Oh, and also: RIght now it's either plain notes or quoted notes. Presence of the quoted modifier clobbers plain notes. But, how that I think about it I wonder if I should make those work together.
Frank
2008-09-09T17:40:39

Hello,

I was wondering if there was any chance you could put together a post annotating/explaining the changes/additions to your improved Delicious command. In particular, some explanations of the code dealing with the statusbar would be helpful for us beginners. Thanks.

Frank
Tony
2008-10-22T05:12:52

Not a big fan of things adding stuff my status bar. :/ Ubiquity commands should just focus on Ubiquity, not XUL changes in Firefox.
l.m.orchard
2008-10-29T16:04:28

@Tony: Actually, commands are just one of the goals of Ubiquity. Easier XUL hacking, ala Greasemonkey, is another goal, albeit under-explored so far.
http://dy-verse.blogspot.com/
2009-04-27T14:50:43

Here is a share-to-delicious ubiquity command that automatically generates tags using YAHOO pipes. It also uses the text selected on the page as notes and the page title as the title of the article.

http://gist.github.com/26425
l.m.orchard
2009-04-28T05:25:58

Auto-generating tags via Yahoo Pipes is a neat trick, but it's unfortunately very much frowned upon by delicious. Not that anything negative will happen to you, but it defeats the purpose of tagging altogether.

Try searching for "Lazy Sheep Bookmarklet" and possibly read:

http://www.mail-archive.com/ydn-delicious@yahoogroups.com/msg00853.html

In particular - "People add metadata so that 1) they can find things and 2) other people can find things. You are removing the step in which people add the tag metadata, thus making the system less valuable to themselves and others."

The key word repeated there is "people"

Writing a Delicious command for Ubiquity

2008-09-01T04:37:03+00:00

In my last post, I got all fluffy about how cool Ubiquity is but didn't share any code to prove the point. As it happens, I have come up with at least one useful command that I'm starting to use habitually in posting bookmarks to Delicious. You can subscribe to my command or check out the full source—this post will serve as a dissection of the thing. Since this will be fairly lengthy, follow along after the jump.

Oh, and it's been awhile since I posted something this in-depth around here, so feel free to let me know how this first draft works. And, bug reports and patches are of course welcome.

To begin, consider the following code starting off the command source code:

/**
 * share-on-delicious - an Ubiquity command for sharing bookmarks on
 * delicious.com
 *
 * l.m.orchard@pobox.com
 * http://decafbad.com/
 * Share and Enjoy!
 */
var uext = Application.extensions.get('ubiquity@labs.mozilla.com');

var cookie_mgr = Components.classes["@mozilla.org/cookiemanager;1"]
    .getService(Components.interfaces.nsICookieManager);

The first thing to note here is that a short header comment introduces the command. This isn't required, but it's a good idea. It's also something you can't really do with bookmarklets. On the other hand, Greasemonkey user scripts expect metadata about the script to be provided here, but Ubiquity doesn't use this convention.

Second, notice that the code accesses some chrome-level resources. Again, this is something unavailable to bookmarklets and Greasemonkey user scripts. Just take a look at the FUEL library documentation to get a quick sense of what's available using that simplified API, not to mention what's available using the lower-level browser APIs.

Now, check out this next chunk of code, which begins the construction of an Ubiquity command:

CmdUtils.CreateCommand({
    
    name:        
        'share-on-delicious',
    icon:
        'http://delicious.com/favicon.ico',
    description: 
        'Share the current page as a bookmark on delicious.com',
    help:        
        'Select text on the page to use as notes, or enter your own ' + 
        'text after the command word.  You can also assign tags to the '+ 
        'bookmark with the "tagged" modifier, and alter the bookmark ' + 
        'default page title with the "entitled" modifier.  Note that ' + 
        'you must also already be logged in at delicious.com to use ' +
        'this command.',

    homepage:   
        'http://decafbad.com',
    author: { 
        name: 'Leslie Michael Orchard', 
        email: 'l.m.orchard@pobox.com' 
    },
    license:
        'MPL/GPL/LGPL',

Whereas Greasemonkey scripts support metadata in the header comment, the Ubiquity command script API works a little differently.

The CmdUtils module provided by Ubiquity offers a CreateCommand function, which expects an object as a parameter. The object literal whose construction is begun in the code above serves as a self-contained package for the command, bearing metadata describing the command as well as containing all the code necessary to implement it.

So, in the above code block, you can see the machine-readable description of the command—including a command name, display icon, home page URL, author information, and license. The command name (share-on-delicious) will be used by the Ubiquity command parser, but the rest of the description will also be used in the list of commands available to the user, invoked by the command-list command, like so:

Moving along, this next chunk of code introduces the first functional bits of the command:

    takes: { notes: noun_arb_text },
    modifiers: { 
        tagged:  noun_arb_text,
        entitled: noun_arb_text
    },

Like smart keyword shortcut bookmarks, Ubiquity commands accept user-supplied input. But, what's unique to Ubiquity is that it employs a parser whose goal is to support something approximating natural language. At present, this results in commands that support a single primary argument—declared above with the takes property—and any number of additional keyword modifiers—declared above by the modifiers property.

For the command under construction here, this establishes a pattern something like the following:

share-on-delicious {notes} [tagged {tags} entitled {title}]

Content for the {notes} argument can either be typed directly by hand, or it can be supplied by text highlighted on the page. To use highlighted text, you can either issue the command alone, or use the word this for the {notes} argument before including further modifiers.

The modifiers tagged and entitled are optional, and can be used in any order. Each of these keywords signifies the start of a different argument—which unfortunately can collide with the literal data supplied for notes, which will hopefully be a rare occurrence.

All of this adds up command invocations including the following:

share-on-delicious
share-on-delicious I really like this page tagged nifty amusing
share-on-delicious this entitled This bookmark has no tags
sh this tagged osx software apple entitled This is good OS X software

That last example is important—since I have no other commands starting with "sh", I can abbreviate the full command. Ubiquity only requires enough of a command name to disambiguate it within your collection of commands.

Another thing to note is the use of the constant value noun_arb_text, which declares that these arguments should expect any arbitrary text as input.

This facility is not exploited for the present command, but Ubiquity defines noun types. These include concepts such as plain text, dates, address book contacts, browser tabs, bookmark tags, and more. You can define your own noun types, as well as implement suggestion schemes that help guide the user toward constructing useful input values in the command interface. You can read more about this in the official author tutorial.

Next up is a quick bit of command-specific configuration:

    /**
     * Command configuration settings.
     */
    _config: {
        // Base URL for the delicious v1 API
        api_base:      'https://api.del.icio.us',

        // Domain and name of the delicious login session cookie.
        cookie_domain: '.delicious.com',
        cookie_name:   '_user'
    },

Since this command will be posting to Delicious via the V1 API, it's handy to declare the base URL for the API in an easily changed spot. That way, you could change this value later on to point the command at another implementation of the API.

Additionally, this command will employ a little-known authentication trick supported by the Delicious API that accepts the user's login cookie set by the Delicious website—this "cookie god" auth is used by the official Delicious addon for Firefox. It's handy for piggybacking on the website login and removing the need to ask the user for their username and password again and possibly storing it in an insecure manner.

In fact, this next chunk of code defines a utility method to rummage through the cookie jar:

    /**
     * Dig up the Delicious login session cookie.
     */
    _getUserCookie: function() {
        var iter = cookie_mgr.enumerator;
        while (iter.hasMoreElements()) {
            var cookie = iter.getNext();
            if( cookie instanceof Components.interfaces.nsICookie && 
                cookie.host.indexOf(this._config.cookie_domain) != -1 && 
                cookie.name == this._config.cookie_name) {
                return decodeURIComponent(cookie.value);
            }
        }
    },

The method defined above, ._getUserCookie(), uses the browser's cookie manager and the values defined in the previous configuration section to find the login session cookie set for Delicious. Take note that this is far beyond the allowed capabilities of bookmarklets and Greasemoney user scripts—this is digging straight into the browser itself, skipping past the usual content-space security restrictions.

In other words: In Ubiquity, the gun is loaded and you should be careful.

Moving along, consider this next utility method:

    /**
     * Given input data and modifiers, attempt to assemble data necessary to
     * post a bookmark.
     */
    _extractBookmarkData: function(input_obj, mods) {
        return {
            _user:
                this._getUserCookie(),
            url:
                context.focusedWindow.location,
            description:
                mods.entitled.text || context.focusedWindow.document.title,
            extended: 
                input_obj.text,
            tags:
                mods.tagged.text
        };
    },

Named ._extractBookmarkData(), this utility method accepts the results of Ubiquity's parser interpreting the primary argument and modifier arguments supplied by the user. Using these data structures, it attempts to build a structure representing the fields of a Delicious bookmark.

The _user field is used for authentication via the site login cookie. The url is set from the location bar of the current page. The description, or title, field of the bookmark is taken from either the entitled modifier or the title of the current page. The tags, if any, come from the tagged modifier. And, finally, the extended notes for the bookmark are taken from the primary input argument of the command.

As you'll see shortly, this utility method will be used in both the preview and the execution of the command.

Next, there's one more utility method to cover:

    /**
     * Given an object, build a URL query string
     */
    _buildQueryString: function(data) {
        var qs = [];
        for (k in data) if (data[k]) 
            qs.push( encodeURIComponent(k) + '=' + 
                encodeURIComponent(data[k]) );
        return qs.join('&');
    },

In anticipation of using the Delicious V1 API, the ._buildQueryString() method accepts an object and constructs a URL query string from the encoded properties of the object. This will be paired with the ._extractBookmarkData() method to supply data for API calls.

Moving along, it's time to start digging into the meat of this Ubiquity command:

    /**
     * Present a preview of the bookmark under construction during the course
     * of composing the command.
     */
    preview: function(pblock, input_obj, mods) {

        var bm          = this._extractBookmarkData(input_obj, mods);
        var user_cookie = this._getUserCookie();
        var user_name   = (user_cookie) ? user_cookie.split(' ')[0] : '';

        var ns = { user_name: user_name, bm: bm };
        var tmpl;

With this code, the implementation of command method .preview() has begun. This method is used by Ubiquity to generate a live preview of the command. Called with a DOM node (pblock) and partially completed command input (input_obj and mods), this method is expected to build a representation of the command's results in the DOM node. As the user types, this method will be called over and over again, ideally offering feedback as the user composes a command.

Continuing on, consider this next chunk of code checking the validity of command input:

        if (!user_name) {

            // If there's no user name, there's no login, so this command won't work. 
            tmpl = [ 
                'No active user found - log in at ', 
                ' ',
                'delicious.com ', 
                'to use this command.'
            ].join('');

        } else if (!bm.description) {

            // If there's no title, then this is an error too.
            tmpl = [ 
                'A title is required for bookmarks on ', 
                ' ',
                'delicious.com ', 
                ''
            ].join('');

This chunk of code first checks for a user name, which can be extracted from a valid Delicious login cookie, if one was found. If not found, the command will fail—so the preview built here will instruct the user to login at Delicious before going further.

The second precondition for using the command is that the bookmark has been given a title. By default, this is the title of the current page—but, some pages don't offer titles. So, an error needs to be flagged if the user hasn't manually supplied a title in this case.

Finally, notice in both of these error cases, a string of HTML is composed in the variable tmpl. This will be used at the end of the method to populate the DOM node passed in as pblock.

Now, assuming that all the command's prerequisites have been met, it's time to try constructing a proper preview for the results of this command:

        } else {

            // Attempt to construct a vaguely delicious-esque preview of a bookmark.
            tmpl = [ 
                '',
                '',
                    'Share a bookmark at  ',
                        'delicious.com/${user_name}:',
                    '',
                        '',
                        '${bm.description}',
                        '${bm.url}',
                        bm.extended ? 
                            '${bm.extended}' : '',
                        bm.tags ?
                            'tags: ${bm.tags}' : '',
                    '',
                ''
            ].join("\n");

        }

        pblock.innerHTML = CmdUtils.renderTemplate(tmpl, ns);
    },

Building on the notion that previews are built in a DOM node, the code above uses both CSS and HTML to assemble a quick-and-dirty facsimile of a Delicious bookmark—which will be rendered like this:

Also note that Ubiquity provides a template engine for use in generating content—namely the JavaScript Templates engine from the TrimPath project. This engine may eventually be replaced with another, but the notion is that Ubiquity will provide tools to more easily generate previews and more.

The conclusion of the .preview() method uses the template engine with a call to CmdUtils.renderTemplate() to inject content into the preview element by way of the .innerHTML property.

Now that the preview is out of the way, it's time to get down to implementing the execution of the command:

    
    /**
     * Attempt to use the delicious v1 API to post a bookmark using the 
     * command input
     */
    execute: function(input_obj, mods) {
        var bm          = this._extractBookmarkData(input_obj, mods);
        var user_cookie = this._getUserCookie();
        var user_name   = (user_cookie) ? user_cookie.split(' ')[0] : '';

        if (!user_name) {
            // If there's no user name, there's no login, so this command won't work. 
            displayMessage('No active user found - log in at delicious.com ' +
                'to use this command.');
            return false;
        }

        if (!bm.description) {
            // If there's no title, somehow, then this is an error too.
            displayMessage("A title is required for bookmarks at delicious.com");
            return false;
        }

Mirroring the .preview() method, the .execute() method first checks for validity of the arguments given by the user. A missing user name or title result in a notification that the command has failed.

But, if the arguments are all valid, it's time to actually issue a request to the Delicious V1 API:

        var path = '/v1/posts/add';
        var url  = this._config.api_base + path;

        var req = Components.classes["@mozilla.org/xmlextras/xmlhttprequest;1"].
            createInstance();

        req.open('POST', url, true);

        req.onload = function(ev) { 
            displayMessage('Bookmark "' + bm.description + '" ' + 
                'shared at delicious.com/' + user_name);
        }

        req.onerror = function(ev) { 
            displayMessage('ERROR: Bookmark "' + bm.description + '" ' + 
                ' NOT shared on delicious.com/' + user_name);
        }

Using the base URL for the Delicious API declared earlier in the configuration section, the .execute() method constructs an API URL for the /v1/posts/add method. Then, it creates an instance of XMLHttpRequest from the browser to be used in sending the API request. Event handlers are registered with the object to present notifications to the user indicating whether or not the API request was successful.

At long last, it's time to wrap up this method and make the API request:

        req.setRequestHeader('Authorization', 'Basic Y29va2llOmNvb2tpZQ=='); // btoa('cookie:cookie')

        var mediator = Components.classes["@mozilla.org/appshell/window-mediator;1"].
            getService(Components.interfaces.nsIWindowMediator);
        var win = mediator.getMostRecentWindow(null);
        var user_agent = win.navigator.userAgent + ";Ubiquity-share-on-delicious";

        req.setRequestHeader("User-Agent", user_agent);      

        req.setRequestHeader('Content-Type', 'application/x-www-form-urlencoded');
        req.send(this._buildQueryString(bm));
    },

    EOF:null // I hate trailing commas
});

The login cookie authentication supported by the Delicious V1 API is triggered by supplying a user name / password pair of cookie, which is done by setting the Authorization request header. The login cookie is then expected to be passed in as the POST variable _user, which is done in the ._extractBookmarkData() utility method.

Another bit here that shows more access of browser resources is the construction of a unique User-Agent header for this API call based on the browser's own User-Agent string, something that's suggested in the guidelines for using the Delicious API.

Finally, the .execute() method—and the command itself—is wrapped up with by sending off the bookmark data encoded as POST form variables with the ._buildQueryString() utility method.

And, that's it—a command-driven Delicious browser extension in a little over 200 lines of code. There's still more to be done to really make this thing full-featured, but I think this shows off the basic features of Ubiquity. I'm hoping to dig in deeper and explore further, taking a look at running Greasemonkey-style code at browser startup and page load, as well as playing with some more browser chrome features.

Archived Comments

Jason Clark
2008-09-01T14:47:35
First off- fantastic post. Great to see a lengthy post here again, although I'm one to talk. This is an excellent introduction to Ubiquity command development, and tres useful to boot.

I'm wondering why you chose to construct and post the XMLHttpRequest manually instead of using jQuery, which is included with Ubiquity. I don't know that there's any benefit other than some simplicity, but I took a crack at converting your code to use jQuery, which works nicely. In the 'execute' function, replace everything after "var url = this._config.api_base + path;" with this (hope code blocks work in comments):

var win = context.focusedWindow; var user_agent = win.navigator.userAgent + ";Ubiquity-share-on-delicious";
```
    jQuery.ajax({
      type: "POST",
      url: url,
      data: this._buildQueryString(bm),
      username: "cookie",
      password: "cookie",
      beforeSend: function( req ) {
        req.setRequestHeader("User-Agent", user_agent); 
      },
      error: function() {
        displayMessage('ERROR: Bookmark "' + bm.description + '" ' + 
            ' NOT shared on delicious.com/' + user_name);
      },
      success: function() {
        displayMessage('Bookmark "' + bm.description + '" ' + 
            'shared at delicious.com/' + user_name);
      },
    });
```
Also, with both versions of the code, I'm seeing some unexpected behavior around authentication. Assume I'm logged in to delicious, with "stay logged in" checked, and I restart my browser. Trying to post with the command fails with a 401 unauthorized, even though I can see the cookie was sent (via Live HTTP Headers extension). Going to delicious.com shows me logged in, and once I've viewed the site, the command works. Except that now I can't reproduce; but I know it happened because I've got the headers. At any rate, it is working nicely, but the previous failure is bugging me... feel like I'm overlooking something.

Thanks again for an awesome post. Hope to see more of the same.
Ryan Scott Scheel
2008-09-01T15:07:01

You should be helping with the documentation, if you aren't already. Very nice job with this article;
Aza Raskin
2008-09-02T01:44:13

Hi Leslie,

This is a beautiful tutorial on writing a Ubiquity command. We'd love your help in making Ubiquity's documentation better (especially dev facing). You should totally link to this from the Ubiquity Wiki -- or even add the content in someway.

Anyway, just wanted to say thanks.

-- Aza
seth
2008-09-08T17:30:16

Am I doing something wrong?

When I enter in "sh this tagged tag1 tag2 entitled This is a title" everything after the tagged is added as a tag. It's not seeing the entitled part.
http://spyced.blogspot.com/
2008-09-15T19:12:58

Did something break? I'm getting a 404 accessing http://decafbad.com/hg/UbiquityCommands/file/tip/delicious.ubiq.js
l.m.orchard
2008-09-15T23:07:42

Yeah, looks like I had a small snafu with switching back from Lighttpd to Apache. Left out a rewrite rule - doh!
Tony
2008-10-22T04:56:03

Great article. This is replacing my delicious bookmarklet. Thanks!
dgtlchlk
2009-04-14T01:06:57

Great article and command. Wish it worked correctly with the latest 0.1.8 release though. No matter what text you put in it adds everything as the notes. The tagged and entitled modifiers don't work.
Nolan
2009-04-16T03:10:07

I second that comment. Delicious is actually saying the link given was "chrome://browser/content/browser.xul" and marking it as harmful inside delicious!
Justin
2009-07-06T12:39:39

Hi Leslie,

Thanks for the delicious ubiquity command. Unfortunately, as one of the commenters above mentions, the tagged modifier doesn't seem to work. I'm using Ubiquity 0.5 pre and typing the phrase:

share tagged foo

Adds the bookmark to Delicious with the note text "tagged foo"

Cheers, Justin
Eric
2009-07-20T22:43:17

Echoing others.. I'm using the latest ubiquity (err, next to latest.. 0.1.8?), and doing "share-on-delicious this is a note tagged testing" saves the link with notes "this is a note tagged testing" and no tags. :( Then again, the other delicious command I've found (by someone else) is having the same issue.

Ubiquity cracks open personal mashup tinkering

2008-08-31T04:07:22+00:00

When I was a wee hacker, I lived my digital life though a Commodore 64. I played games on it, did homework, talked to people far away—you know, all the stuff they showed in the pictures on the box. I also took things apart—both the machine itself and software running on it. I grew up learning that my digital environment was ultimately understandable, susceptible to tinkering, and open to being bent to my own purposes.

From the Commodore 64, I graduated eventually to terminals and text editors, opening portals mostly onto computers elsewhere via powerful UNIX command shells. And, of course, over the past decade, this has largely given way to life in a browser.

Yet, for a little while, particularly in the first few years of browsers, freedom to tinker seemed cramped. JavaScript had yet to arrive, and was a little messy when it did. There was no relatively easy addon development. And, though the portals opened by a browser were richer than those provided by terminals, the paths of navigation defined by links controlled by site owners offered less freedom of movement than UNIX commands. I could create my own pages, but I couldn't do much to others' pages.

But then, javascript: URLs came around, dots were connected, and bookmarklets were born. Suddenly, it was possible to customize my browsing environment with arbitrary JavaScript code having access to the current page—no matter whose page it was. And, through the various tricks of the AJAX trade, bookmarklets have only gotten more capable throughout the years.

Smart keyword shortcuts came around a little later, allowing quick access to bookmarks via simple keywords typed into the location bar. The smart part, though, came in the form of bookmarked URLs with placeholders and keywords given arguments to fill the placeholders—allowing not only quick access to bookmarked pages but also search engine forms bookmarked with late-bound fields.

Bookmarklets inherited the benefits of smart keyword shortcuts. The same placeholder in http: URLs can be inserted into the code of a javascript: URL, thus parameterizing the JavaScript code and incidentally turning the location bar into a kind of primitive command line. For example, one of my most heavily used "addresslets" is based on John Resig's original "Super-Fast Delicious Bookmarklet".

Another leap in prying open the browser tinkering space came in the form of Greasemonkey—an addon-powered environment created explicitly for the purpose of end-user scripting applied to others' pages. Greasemonkey user scripts can do more than bookmarklets, and with a much better development environment to boot. And, though a user script can't do quite as much as a proper browser addon, they're much easier to hack on and distribute.

Now, consider one of Mozilla Labs' newest projects, named Ubiquity. This rough and experimental addon for Firefox combines and improves upon everything I've described so far:

Ubiquity is a hackable command line environment, better than bookmarklets and smart keyword shortcuts;
Ubiquity enables persistent customization of others' pages, not unlike Greasemonkey;
Ubiquity facilitates live in-browser creation and web-based subscription to user commands and scripts;
Ubiquity gives access to browser chrome resources without a need for frequent restarts;

So far, most of the commands I see popping up since the 0.1 release have not accomplished much more than smart keyword shortcuts in the location bar could. But, it's early yet, and Ubiquity is far from limited to these commands.

Once the basics have been well-explored, I expect to see more people taking a crack at the broader capabilities offered by Ubiquity. Bookmarklets and Greasemonkey can't access browser chrome—but Ubiquity can. Ubiquity also offers a user interface that's so much more promising than keyword shortcuts, including command previews and typed parameters with suggestions.

Ubiquity promises web-wide mashups directed by a conversational command interface. All in all, the potential of this makes me feel like my digital environment—browser and web as a whole—is getting even more intimately, personally hackable.

It'll be very interesting to see where this project goes.

Archived Comments

Raul
2008-08-31T10:20:58

Hi, was using the original delicious command linked from the Ubiquity wiki, just tried yours and its definitely more polished and functional. Great job with the preview and the extra functionality. Only thing is 'share-to-delicious' is too much to type so I unsubscribed the previous command and changed the namespace in yours. This is clearly going to become a problems as the commands proliferate.
l.m.orchard
2008-08-31T14:43:39

@Raul: "share-to-delicious" is long, but keep in mind is that you only need to type enough of the command to disambiguate it. That is, all I type is "sh this tagged osx software apple" because I have no other commands starting with "sh". Watch the list of commands in the preview as you type. Those tell you what the parser thinks of what you're typing as you type. It's like automatic tab-completion.

Also, I think there's work planned to put some usage based sorting into the command parser, preferring the commands you use most in order of disambiguation.
Abi
2008-08-31T15:33:24

Nice post. I share your sentiment with regards to Ubiquity commands. A lot of commands that I see are just plain simple searches. I hope developers will work on more interesting things. For example, even things like auto-form filling for this comment (possibly even on page load without having to type a command) could be done by Ubiquity. There's still a lot more room for experimentation.
Abi
2008-08-31T15:45:59

But otherwise, I really like your command especially the preivew. We should include it as a builtin command, if you don't mind. :)
Abi
2008-08-31T15:51:20

I noticed a bug in your command. If I select some text in the awesomebar, the bookmark url is chrome. You should use something the command utils to get the url, instead.
l.m.orchard
2008-09-01T06:17:37

@Abi: Making this a built-in command is totally fine by me! It can use more work, though, for sure.

Also, I can reproduce that bug. Ugh. I can't find any methods in the CmdUtils to get the current page URL, though. I'll keep poking a bit though.
Abi
2008-09-01T06:50:23

Your blog seems to be rejecting code (that's why I had so many posts in the first place). You can get the current page url using (with dots):

CmdUtils getDocumentInsecure() location href
Blair McBride
2008-09-02T02:50:12

I recommend against using getDocumentInsecure() - its got "Insecure" in its name for a reason! Instead, you should use:

Application.activeWindow.activeTab.uri.spec

Moz08: Rockslides and Blackouts and Bears - Oh My!

2008-08-04T20:15:23+00:00

Since the beginning, I've been a Mozilla fanboy. But, up until very recently, I've only shown that as a user and a verbal supporter, with a shallow understanding or participation in what's really done in the Mozilla community. Sure, I've checked out source code from time to time going back to the first code release, built and poked at it, but I've never really contributed back.

But now, I'm an employee of the Mozilla Corporation, and I feel like I found a little-known cheat code for getting into the place where all the cool stuff happens. See, I just got back from this thing called the Mozilla 2008 Firefox Plus Summit—and if I was overwhelmed by the initial Mozilla Firehose after coming on-board, I'm entirely gobsmacked now.

The event—having taken place in the bear-infested, rockslide-prone, electricity-deficient wilds of Whistler, BC in Canada—was attended by not only employees of the corporation itself, but a selection of invitees from around the Mozilla community. Throughout the course of the event, I felt humbled by the efforts of everyone and have come away with a feeling that I really need to step up my game. Somehow, it seems I managed to sneak into the place without having first run the decade-old traditional gauntlet of meritocracy and contribution.

Being a newbie of just a few months, I barely know anyone at Mozilla, so I was quite overwhelmed socially. That the early-morning power loss also knocked out my CPAP and left me sick and surly on the last day didn't help my already introverted tendencies. Nonetheless, I talked to a lot of people, attended a lot of sessions, partied a bit, played some Rock Band, and have a lot that's percolating in my brain-meats.

There's plenty coming up to work on in my own assigned sphere of webdev—but I'm also pretty jazzed about the Mozilla Labs projects Prism, Ubiquity, and Weave. Not sure what I can do yet, but would like to do something to help out those efforts. And maybe by the time of the next summit, I'll still be in the Mozilla sphere and will have met and helped a few more Mozillans with whom I can connect face-to-face.

There was also a lot of talk about the future and continuing mission of Mozilla, all very inspirational and telling me that this is the center of things where I've always wanted to be. Now, I just have to work hard to make sure they're not sorry they hired me! I'm running much slower than I'd like, since I feel I have a lot to catch up on while in the midst a lot of disruption and change—between the intensity of the last few months (or even the last year) and readjusting to being back in the Mitten with homeownership all anew and many plans having taken unexpected turns.

But, things are shaping up in a sort of too-good-to-be-true configuration, and I suspect I'll be just fine if I sharpen up a bit and roll with the changes.

Archived Comments

Gen Kanai
2008-08-04T22:32:43

It was a pleasure finally meeting you and chatting, even if it was finally on the bus ride home.
danmactough
2008-08-05T00:28:35

Les, don't be too hard on yourself. Every company knows that new hires need some time to get up to speed (or at least the good companies know that). For example, my firm has calculated that it costs them about $250,000 (excluding salary, bonus, moving expenses, etc.) to hire a new associate. That consists partly of benefits expenses and administrative costs, but mostly just "getting up to speed" time. Relax. You'll think smarter if you do.
Stephen Donner
2008-08-05T03:26:49
Les -

You implemented the sexy theme browser -- need I say more?

:-)
- Stephen
Marty
2008-08-05T15:05:59

Now that you're back in the mitten - don't be a stranger :D