0xDECAFBAD

It's all spinning wheels and self-doubt until the first pot of coffee.

Enter the LizardFeeder

[caption id="attachment_1582" align="alignright" width="247" caption="The Mozilla Tree"]The Mozilla Tree[/caption]

Behind Firefox is Mozilla, and behind Mozilla is a community. And the Mozilla community acts a lot like an ecosystem, which can be visualized as a kind of living tree—not to confused with the mozilla-central tree. Oh yeah, and Mozilla is the name of both a Foundation and a Corporation.

Confused yet? If not, then we should talk so you can explain it to me, because it all looks pretty tangly and intertwingled to me. Nonetheless, it seems to work, and produces a good chunk of my favorite software and technologies.

There are many efforts to track what's going on—including planets and newsletters and bugzillas and wikis and repositories and tinderboxen. Some of these resources report on, or are driven by, the activity occurring in the others. Some are automated, and others are carefully stitched together by hand. None offer a full picture of what's going on in the Mozilla galaxy in a way that's casually comprehensible by a sane human being.

Of course, that's not a slight against any of these sites or the people maintaining them—extracting an overview from such an organic phenomenon is neither easy nor straightforward. But, it might be fun to try.

As an infovore and avid practitioner of continuous partial attention, my first impulse is to reach for a firehose and stick my head into the stream. Relax, defocus, and try to let my pattern recognizers do their thing—sometimes those pattern recognizers are in my head, and sometimes they're written in Python.

[caption id="attachment_1585" align="alignright" width="225" caption="Firefox Victory!"]Firefox Victory Robot[/caption]

But, for Mozilla, I couldn't find a stream of sufficient volume or completeness to satisfy me or my robots. Happily, though, my feeding urge found itself aligned with a project to discover the patterns of contribution in the Mozilla community and to find a way to thank the contributors responsible.

So, while we're still working on the thank-you angle, allow me to introduce you to the Lizardfeeder. The LizardFeeder is a feed aggregator, whose source code is built atop Sam Ruby's Planet Venus. The LizardFeeder pulls together and archives activity streams from a wide variety of Mozilla community sources. Beyond the usual human-readable pages produced by a blog-gathering Planet, the LizardFeeder accumulates statistical and historical data meant for consumption and analysis by robots.

At present, the only robot navigating the LizardFeeder archives is an AJAX-ified user interface that animates the firehose as a near real-time or time-lapsed stream of events scrolling by.

This is just meant as a conversation starter, though. I'm hoping to gather feedback and find more sources, as well as to entice creative community members to come up with more sophisticated visualizations of this data.

So, take a look, check it out, and let me know what you think!

Archived Comments

  • Is there a list of what sources are already being read by the LizardFeeder anywhere? I scanned through the various links here but didn't see anything obvious :)

  • Yeah, that part could use some improvement. There's a monster list here:

    http://feeds.mozilla.com/sources.opml

  • Excellent stuff. I was going to ask about access to the list of sources for each category, but Dria beat me to it.

  • Actually, it occurs to me that this config file might work as a more readable version of the list of sources:

    https://svn.mozilla.org/projects/lizardfeeder/trunk/conf/config.ini-dist

    There's also this, which is where most of the previous list came from:

    https://svn.mozilla.org/projects/lizardfeeder/trunk/conf/hg-feeds.opml-dist

  • Thanks for the background surrounding lizard feeder. Great post!

    +1 Dria and Brian

    Maybe make the title of the link to the OPML more descriptive than just "Feeds" and/or link to it in the body of the UI too and write something around it to encourage other's visualizations.

    Awesome work Les.

  • Is the AJAX UI available anywhere for download? I'd like to hack it for another use altogether. Great visualization and great application. Good job.

  • This is a very good idea. Can we accomplish the same result using Yahoo Pipe? Do you plan to add events from Bugzilla, and mailing list to the feed?

    I am more interested in development activities. There aren't much in code category.

    I am interested in analyzing development related events.