0xDECAFBAD

It's all spinning wheels and self-doubt until the first pot of coffee.

Introducing dbagg3, an Atom-powered client/server aggregator

Update: I've just dumped what code I have into my CVS repository. So, go ahead and poke fun at it:

Or, fetch it from CVS:

$ cvs -d:pserver:anoncvs@www.decafbad.com:/cvsroot login
(Logging in to anoncvs@www.decafbad.com)
CVS password: anoncvs
$ cvs -d:pserver:anoncvs@www.decafbad.com:/cvsroot co dbagg3

So, besides the funk, there's a little project in which I've gotten immersed. Here's a teaser diagram:

It's a feed aggregator, my third attempt at such. At present, things are roughly close to the diagram above. Everything is clunky and command-line driven at present--but I've got further plans, like a REST API for feed queries and manipulation of various things such as feed subscriptions and the read/unread state of items. Pair this with an XSLT-driven browser UI, and the possibility of other clients (not the least of include other Atom-consuming aggregators).

The goal is to make a Client/Server Aggregator. Somewhat serendipitously, I just caught Bob DuCharme's xml.com article on Amazon.com's web services, which I really like and have drawn inspiration from in thinking about this new aggregator. Eventually, I want to offer the same sort of XML+XSL style of service that they've put together, along with some futher inspiration from the Atom API.

Anyway, as another teaser, check out this early demo involving the XML this thing has been producing, coupled with some experimental XSL:

You'll notice that there are three URLs involved here:

The XSLT expects Atom (plus a few extensions of my own), so you can also do things like this:

These are the sorts of tricks I was looking forward to when I started thinking about things like a universal blog transfer format and rose-colored glasses. XSLT used like this could just as easily produce a blog or RSS 2.0 content.

Anyway, hoping to get some code into CVS by this weekend that's not entirely embarassing. So if you're interested in this stuff, stay tuned. I'm hoping that this thing can provide a base for others interested in feed aggregation--if you just want to play with UI, use the scanning and storage as-is and tinker with XSLT; if you want to play with storage and query, leave the scanning and UI alone; if you want to tinker with parsing... er, talk to Mark Pilgrim.

More soon!

(Oh yeah, and I will be working on coming up with a better name than dbagg3. Unfortunately, I probably won't be coming up with a more visually appealing design for the UI, since what you see is the best I can do. Heh, heh. Don't let your programmers do visual design...)

Archived Comments

  • Benjamin
    This is great. It seems that this could be the base, for what I've been dreaming/thinking about the last half year or so. In contrast to you, I haven't come up with any code yet (mostly due to the fact, that I don't python yet, and I found no other reasonably good feed parser, than the Universal Feed Parser), but with a name: Roosster Features I'd really like to have in such a ClientServerAggregator are: (a) search over entries the aggregator has seen to date (b) add and arbitrary number of notes to any entry (c) add multiple category-tags to entries (d) interlink entries (I know this could be solved by some 'Related Entries' stuff, but somtimes I don't trust these "automatic classifiers") Can't wait, until I can take a look at the code and see if I can extend it. This would finally be a real reason to learn python. By the way, I really like the interface. Might be, that's because I'm a programmer too?!
  • Christopher Drake
    Very nice. The demo has me salivating a little. Oddly enough, I like the UI. If you can slim it down, side-to-side-wise a bit, it would also be a great aggregator to rip whole chunks from and feed to a Palm Pilot or PocketPC, as well. They're not as big as they were in the past, but with decent strides every couple of weeks, they still have a lot of following. Your page-width is dynamic, and that means a lot to me, a Palm user. A feature I'd like to see is kind of like Bloglines, where you receive a small box of info from the RSS feed, or what-have-you to give you a taste of an article. If going 1 link deep from any link on the aggregator page linked to a small page with such a tidbit, it would make surfing the articles a lot faster. This would allow a PDA or cellphone to drag an aggregator page 1 link deep with specialised software and get all of the highlights from say, 20 blogs. That could be a major boon, especially for people that connect while at wireless access points on business trips, etc. Keep it up, this looks like it'll rock when you are done.
  • Christoph Görn
    Hi, and I will try to plug an jabber/XMPP bot into it, so it will deliver the aggragated content rigth to my beloved IM app...