Building a Bookmark Management UI for Mozilla's Build Your Own Browser

Pondering the design

One of the first things I did in getting down to work on this thing was to take a look at the visual design.

The design of the bookmark organizer consists of two panes: folders on the left; folder contents on the right. And, in Firefox, there are two main places where bookmarks are found: the Bookmarks menu and the Bookmarks toolbar. So, each of these are represented as root folders in the folder pane.

Considering the requirements

Along with the design, there were also the business requirements to consider. These included the description in the bugzilla bug for the feature, as well as existing requirements from a previous implementation:

• Only Toolbar and Menu are allowed as root bookmark folders
• Only the root folders can contain sub-folders
• Folders may contain bookmarks and livemarks
• Up to 3 items in the Toolbar
• Up to 5 items in the Menu
• Between 1 and 10 items in any other folder

Note that though the design shows additional root folders beyond toolbar and menu, but the requirements disallow this. Resolving differences like this are just part of the webdev job. Usually, this is straightforward after a chat or two, but surprises are to be expected.

Staging the attack

My habit is to just start hacking iteratively, after just a little bit of forethought. Maybe a few doodles are done on a notepad, but I don't spend too much time on the details up front before starting to sketch with working code. It's best to fail early and often, and it's great to iterate on one small puzzle at a time.

While a good grasp of available tools is a minimum starting point, remember that tools change, documentation lies, and things tend not to work the way you think they should. I start trying things right away, and have the uncompressed, original source code of all libraries involved close at hand.

Oh, and make sure you've got Firebug installed; it could save your day.

Sketching with markup

Like many modern UI widgets, having chosen to use Sortables informs to some extent what markup I can use in the page. Luckily, the choice here is completely sane and not at all objectionable: I can just use <ul> lists with <li> children. No extra wrappers or shims just to make the code work.

Keeping markup clean and meaningful is more than just a nice-to-have: I've found it inevitably leads to cleaner JavaScript and CSS—as well as easier localization and accessibility. This should be almost obvious at this point in webdev history, but it's easy to forget when faced with new shiny webdev toys.

So, with a widget in mind, I started playing with the markup and CSS. I built out the two panes of my bookmark management UI with vaguely semantic HTML, threw a pair of lists into the folder pane and another list into the folder contents pane. I then mocked up a variety of folders, bookmarks, and livemarks and slapped together some CSS styles for a rough layout.

Calling in the sortables

With a first-draft DOM structure in place, I enhanced the page with Sortables. As it turns out, the widget worked as advertised on my three lists—at least for reordering items within the bounds of each individual list.

What I really wanted, though, is to be able to drag items between each of these lists. In the docs, I discovered the "connectWith" option, which establishes a one-way relationship to drag items from one sortable to another. Having found that, I connected each of the three with the other two.

And, it worked. Items dragged from one list to another moved in the DOM as expected.

Sketching with data

So, I had three Sortable-enhanced lists, in a style vaguely reminiscent of the visual design. I had a ways to go with the functionality, but I shifted gears to the bookmark data itself with an eye toward deriving more realistic markup from it.

JSON is the first thing I reach for when I think about a data format for roundtrips between a web server and a browser client. Once upon a time, that might have been XML, but the ease in consuming and producing JSON across platforms and its ample expressiveness make it a win.

I thought a bit about the kinds of items needed to describe bookmarks and what attributes each would have. And, thinking in terms of folders, I wanted to make sure nested folder structures were consistent, thinking ahead to recursive walking code I might be writing in the near future.

So, as I did with markup, I started sketching with data and came up with something like this:

{
    "toolbar": {
        "items": [
            { 
                "id":"bm-1", "type":"bookmark", "title":"Foo",
                "description":"Description of Foo",
                "link":"http://example.com/foo" 
            },
            { 
                "id":"bm-2", "type":"folder", "title":"Bar",
                "items": [
                    { 
                        "id":"bm-3", "type":"livemark", "title":"Baz",
                        "description":"Description of Baz",
                        "feedLink":"http://example.com/baz/feed",
                        "siteLink":"http://example.com/baz"
                    },
                ]
            }
        ]
    },
    "menu": {
        "items": [
        ]
    }
}

This isn't formal like drafting a schema or drawing out a class diagram in UML—this is just thinking things through in running code. Object literals in JSON make this sort of doodling easy, since you don't need to commit to class definitions up front while exploring an idea.

Generating page content with JS

There are many tools in the webdev kit for generating page content from JavaScript.

One of the most basic techniques is to use the createElement and setAttribute DOM methods to build up elements and insert them into the page structure.

You can also build up strings of HTML markup and use the innerHTML property to inject content, which works fairly well on modern browsers. Some webdevs will tell you that's non-standard and cheating, though.

An even more interesting technique—in terms of building an end-to-end web app—is to use one of the many template languages in JS, such as Mustache or an implementation of the Django Template Language. I even tried building Zope's Template Attribute Language for jQuery, once. In most cases, these utilities have implementations in multiple languages besides JS, letting you share the same templates between both the server and client sides of a web application.

Send in the clones

My favorite technique lately, though, is to use the cloneNode with a template node. For most purposes, it's both simple and capable enough to do everything I need, and seems more lightweight and performant than many alternatives. Additionally, I use plain PHP as the templating language on the server, so there's really no language to be shared with the client side here.

In my server-side HTML templates, I include markup to build templates for list items and the like. These templates in the page are hidden by CSS, and I include class names to tag placeholders for content. Then, in client-side JavaScript, I clone these nodes and use CSS selectors to find and fill the placeholder elements with content.

I've found that building these templates on the server often comes in handy when localizing a site: Text can be localized using server-side tools (eg. gettext), rather than trying to make localized strings available to client-side JS (eg. gettext-js). Doing localization in JS can often be a pain, so I'll try avoiding it whenever possible.

Since I've been using this approach so much lately with my own tweaks, I've bundled it up as a jQuery plugin called cloneTemplate. You can see a simple template for bookmark folders in the HTML template for the UI. And, over in the JS code, you can see it in use for updating the folder pane.

If you like it, you're of course free to use this plugin and send suggestions.

Populating the bookmarks view

So, with bookmark data and templates for page content in hand, it was time to put the two together and replace the markup I'd sketched by hand with more realistic elements generated from data.

I filled in list items based on the folders found in the data for the Toolbar and Menu root folders. I also filled in the bookmarks pane using the contents of the first folder in the list. Both of these updates were performed using the cloneTemplate plugin and hidden template nodes.

I used Event Delegation to attach a click handler to the <ul> folder list container. This handler catches any clicks on folder list items. As the user manipulates the bookmarks data, I expect the set of children in this list to change often. So, tracking individual list items is much less efficient than simply catching things at the stable parent level.

The click handler for folders, in turn, triggers the population of the bookmarks pane with the contents of the folder clicked. And, with that, I reached the point where the UI was fully data-driven and I could use it to navigate a set of bookmarks from JSON.

Reconsidering the Sortables

It's true that, with respect to bookmark data, the UI was read-only at the first milestone. But, since I'd hooked up the Sortables, the page and its DOM were mutable via user interactions—the changes just didn't get reflected in the underlying bookmark data itself.

So, that suggested the goal for the next milestone of work: Connecting changes in the page to changes in data. Or, at least, that had been my first thought.

I could drag items between the lists, and the Sortable widget automatically updated the DOM of the respective lists. And, Sortables also offer a toArray method that translates the current state of the DOM into an array of IDs taken from the list items. This makes sortables very useful for letting users rearrange simple lists, because the DOM directly corresponds to the data.

However, as I started working through the interactions afforded by the Sortable-enhanced lists, I realized that things didn't quite match up to what I wanted. For example:

• I could drag bookmarks from the right pane into the folder list in the left—but what I really wanted was to drag a bookmark into a folder in the left pane, not between the folders.
• I wanted to reorder Toolbar sub-folders on the left, and see the results of that action reflected in the right panel when it showed the Toolbar contents.

The map is not the territory

The map is not the territory—that is, the view is not the model. Had I spent more time thinking things through at the top, I might have anticipated this. (Of course, had I spent more time thinking, I might not have gotten as much done.)

So, I kind of backed into a Model-View-Controller design. I realized that I couldn't rely on the Sortable's toArray method to convert DOM state (ie. the view) directly to bookmark data (ie. the model). And that's because the UI gestures afforded to make changes in the data model didn't all directly correspond to changes in the browser view. For example:

• Dragging a bookmark from the right pane into a folder on the left should cause it to disappear—but it should still exist, just in a folder not currently selected.
• Reordering a folder on the left should be reflected on the right—but the right side includes bookmarks not displayed on the left.

So, I had to break the Sortable's default connection between the gesture (eg. drag-and-drop) and the displayed result (eg. DOM node moved within or between lists).

That's the true goal of the next milestone, then: Connecting gestures to changes in the data model. Model changes, in turn, should get reflected in the view. That separation there, between gestures and model and view, is the important bit.

This is where things start to get messy.

Looking under the Sortable hood

Since the default behavior of the Sortable widget turned out not to be the right fit for the UI interactions I wanted, I needed to take a closer look at how to customize that behavior.

The best way to do this is through the set of custom events generated by the Sortable widget, to which you can attach your own handlers. These events narrate the entire course of a drag and drop interaction involving Sortables. At each point, you can do things such as:

• detect a drag from list onto another connected list;
• trigger feedback while hovering over drop targets;
• react to the completion of a drop.

And, each event handler has access to a lot of context about the drag and drop operation, such as:

• the original list from which an item was dragged;
• the item being dragged;
• a helper element appearing under the mouse during the drag;
• a placeholder element that can appear in lists to show a destination preview;

Having access to all the above in event handlers means that, at any point during the interaction, you can tweak or override the out-of-box Sortable behavior.

In other words, having checked out the events generated by a Sortable widget, I worked out that I could interpret them as user gestures and completely revise their effects on the data model and browser view.

Making gestures

Once I started getting down to handling the UI gestures, the permutations began to make my head hurt. So, unlike most other steps in this process, I decided it was time to hit the whiteboard.

I considered drag-and-drop as a UI gesture, consisting of two major parts:

• the item dragged;
• the drop target.

The list of items available for drag was pretty short:

• folder in folder pane;
• folder in bookmark pane;
• bookmark/livemark in bookmark pane.

But, the list of available drop targets included not only all the things available for drag—but several relative positions within each of those items: above, below, and on top. These relative positions account for both list reordering and parent/child relationships—but they expand the list of drop targets to 9.

This meant that, paired with the draggable items, I had 27 distinct gestures to evaluate.

I probably could have generalized this more easily in my head, if I'd had enough coffee and if it hadn't been 2am. But, at that point, I just wanted to brute-force my way through the problem space so that I didn't miss anything.

In order to wrap my head around the combinations, I actually composed a little one-off JavaScript in the Firebug console on a scratch page to build a table with a results column to consider and fill in by hand. This was what I came up with:

From this, I figured out the basic set of data model operations:

• move item as sibling before target
• move item as sibling after target
• append item as last child of target

Again, with enough sleep or caffeine, the above might have just been a no-brainer. But, I'd also made explicit some constraints I'd had floating in my head, such as:

• bookmarks can't have child items, only folders can
• bookmarks don't belong in between folders in the folder pane

Armed with these realizations, I started writing handler logic for Sortable events. Although it's getting ahead of the story, you can see the end result for folders and for bookmarks in the JS code.

Cheating on the view

As I wrote the code to pair gestures in Sortables with changes to bookmark data, I needed to ensure the view was updated to reflect those changes.

In past projects, I've kept things granular by mirroring the model and view changes individually. But, here I got lazy and cheated: Rather than carefully moving DOM nodes around to reflect the same state as the model, I just blew away the UI contents and rebuilt the whole thing at the end of each gesture and data model update.

On the face of it, this sounds horrendously expensive and shouldn't perform well at all. But, practically speaking, none of the lists in the UI ever deal with more than 10 items, thanks to the business requirements. So, doing a full refresh of the UI every time is actually very fast. Additionally, it ensures that the view always reflects the state of the model after a gesture—and it saved my rapidly-degrading brain from thinking too much at 3am.

Smartening up the model

Getting closer to the milestone of editable bookmarks, I noticed that I was repeating myself a lot. My data model was just a dumb JSON structure, and I was writing the code to manipulate the model directly in the gesture handlers.

It seemed like a good idea at the time: At first, the repetition was just the single line or so it took to move an item between JS arrays. But, then I got around to thinking about enforcing the constraints demanded by the business requirements. Recall these from earlier:

• Only Toolbar and Menu are allowed as root bookmark folders
• Only the root folders can contain sub-folders
• Folders may contain bookmarks and livemarks
• Up to 3 items in the Toolbar
• Up to 5 items in the Menu
• Between 1 and 10 items in any other folder

Well, checking whether a particular folder is too full to allow a drop—ie. 0, 3, 5, or 10 maximum items, depending on what both the dragged item and drop target are—starts to get really repetitive if I copy-and-paste the same code to every spot where an item gets moved into a folder.

So, with those alarm bells going off in my bleary head, I decided it was long-past time to refactor things. I needed to make my dumb JSON model smart by wrapping it in prototyped objects bearing methods to take care of manipulations and constraint enforcement.

Since I wanted a little classical inheritance to handle things like the relationship between plain folders and the Toolbar / Menu root folders, I dragged in John Resig's "Simple JavaScript Inheritance" code. It did the job and seemed small enough to not cause much fuss.

In retrospect, I think I over-engineered it, but you can take a look at the resulting model code and see what you think. Some of the concerns expressed in this code include:

• all items should easily convert themselves and their children to a JSON structure (ala Sortable's toArray) for later submission;
• all items should be able to report what they are;
• structure manipulations should be simple method calls;
• folders should report whether they're full or not, depending on type;
• items should be able to validate their own properties against business requirements.

All of these concerns are aimed at making the code in the controller event handlers as simple and self-explanatory as possible, reducing the repetition and encapsulating the model concerns in one spot.

Simplifying things with a smarter model also freed me up a bit to think about adding in more UI feedback during gestures. That is, I could easily figure out what a drop target would do with a dragged item during the drag. So, for example, I could figure out that a folder would accept a bookmark as a child, and so highlight that folder when the mouse was dragging over it.

Progressive enhancement

Progressive Enhancement describes an approach to building web content that naturally degrades gracefully on web clients offering varied capabilities. In an ideal world, everything should be built this way.

In my day-to-day world, this has been a challenge to achieve for web applications like BYOB.

I'm not even sure how I'd build a user interface like this one, following the principles of Progressive Enhancement. I could imagine composing it out of a tedious sequence of form POSTs to manipulate the state of bookmarks data, just like something I might have built back in 1999. Then, I could gradually enhance this process with AJAX requests back to the server to perform in-place changes, and even enable drag-and-drop eventually.

But, I'd probably lose the conveniences of end-to-end JSON, among other things. The development process would have been a bear—and probably would've taken more time than I had to spare. And then, who would have genuinely benefitted from the work? Should I really expect people to be using something like BYOB from an under-powered mobile device or an older browser?

Still, failing at progressive enhancement and graceful degradation makes me hang my head as a webdev. So, I'll take that hit to my karma for now, and try to do better on future projects.

HTML5 Drag and Drop

And then, almost in direct opposition to graceful degradation, are shiny new toys like the HTML5 drag-and-drop events. I've written about these before, and have been pleased with them.

Still, I didn't want to entirely exclude browsers missing this new feature. But, as in the previous section, I'm not sure if that consideration will really benefit many real BYOB users. Other aspects of BYOB may exclude users of older browsers anyway: Since BYOB is a bit of a niche service, we honestly haven't spent a lot of time ensuring the site performs properly on browsers beyond the latest crop.

So, I wonder if settling on using the Sortable widget was the right decision? Rather than overriding its behavior, could I have saved a lot of time and hassle by just writing everything from scratch using the new HTML5 features? Possibly. Would it be worth rewriting everything to use them in the future? Probably not—at least, not unless it were part of another major refactoring of the UI for other reasons.

Accessibility

Accessibility is an aspect of web site development that empowers people with disabilities to use a site. Navigation and user interface cues embedded in the markup, among other things, allow assistive technologies to work with a site in ways more appropriate to a user's abilities.

I have to admit that my practical understanding in this area needs work. Consequently, BYOB could use a lot of work in the accessibility department. For this bookmark management UI specifically, I'd love to hear any suggestions on how it could be made more accessible.

Localization

Localization describes how web sites can be built in a way that makes them usable throughout the world. This includes translating text, but also encompasses issues from how sets of things are counted to the expression of dates and times.

At Mozilla, localization has become a huge part of our mission, so webdevs are strongly encouraged to account for it in all of our projects. And, to a large exten, we do that: For instance, the AMO team and an army of volunteers maintain over 40 locales for addons.mozilla.org.

But, alas, BYOB is available only in US English, at present. Again, I hang my head in shame.

Fortunately, the problem isn't so much a lack of know-how but a lack of time. I've set sites up for localization before, and have even put some hours into working on the AMO team. With a bit of elbow grease, I think BYOB could be retro-fit for localization, and I expect it'll be a quarterly goal for us very soon.

Optimizing page performance

There are a number of best practices for page performance that are widely used across the web. Since BYOB has been in heavy development mode, these optimizations are not applied very well here.

Again, this is more a matter of time than knowledge, so this will definitely be a goal for a near-future release.

Redundant validation

As I mentioned earlier, bookmark data validation occurs both on the client side in UI constraints and on the server side in a recursive inspection on submission. It would be nice to collapse this redundancy just on general principle.

But, most things I come up with sound horribly inefficient. For example, I could frequently submit the JSON data to the server via AJAX. I would display validation feedback from the server and just skip enforcing the constraints in the UI. But, that sounds like a horrible user experience and a lot of network traffic.

My hunch is that something better could be done here, but I just can't figure out what.