0xDECAFBAD

It's all spinning wheels and self-doubt until the first pot of coffee.

Personal Web Proxies

Okay, now that I'm healthy, my girlfriend is healthy, my job is healthy, and my iBook is back to being healthy - I might just get a chance to sneak some time in for my Projects again.

At the moment, I'm considering building a PersonalWebProxy. I've been playing with them and thinking about them off and on for years now. You can see a short list of them that I've poked at in the PersonalWebProxy wiki topic - you're welcome to add to the list if you know of more. In particular, the WBI project at IBM got me initially hooked on things. I really thought it was nifty how they edited HTML on the fly to weave in the WBI UI and add indicators on links in the page. And the idea of storing a history of one's web browsing in a local cache, available for review and later search, has always seemed incredibly attractive.

Lately, I've been thinking of a few more things that might be useful in a personal web proxy:

  • Marking pages to be periodically checked for change notification.
  • A browsing "shopping cart", in which to collect pages now for later browsing.
  • Auto-harvest RSS, FOAF, and whatever other associated page metadata that might be useful now or later. Maybe suggest subscribing to the site after a few visits.
  • Use a ubiquitous rating mechanism, machine learning, and maybe some off-peak spidering to have new content of interest suggested.
  • Publish and share browsing patterns, generate "Friends who viewed this page today also viewed this today..."
  • Generate RSS feeds for all notification features
And then, of course, there are things that I've seen already in other projects:
* Rich browsing history
  • Collaborative annotation
  • Ad filtering & pop-up blocking
  • Content summarization
I'm thinking it would be nice to put together something like WBI and its modular API, maybe in Python, and make this thing super friendly to quick hacking. Could be some fun. What do you think?

shortname=ooocdf

Archived Comments

  • mike
    Hmmm, funny that you mention it - I've ben considering something similar. I had originally set up a DB for holding bookmarks and interesting links (http://fiftyfly.mine.nu/kb/links.php) but after some use have found that it doesn't really suit browsing style. I like to share differnt/funny/off beat links and it works fairly well for that, but the sorting/category system is too cumbersome for 'real' links. A proxy that might add headers (for quick insertion into the db) not to mention filtering and automatic google "relating" (I've been using that a lot recently) not too mentio a host of other things might be an elegant way to combine much, while requiring a minimal level of use i/o Thanks for the idea.... I'm going to have to give the whole thought a lot of, well, thought ;p
  • dj
    As you'll have the information flowing through such a proxy anyway, how about the facility to inspect the HTTP headers to facilitiate a simple HTTP conversation analysis? I find these kinds of things incredibly useful, specially in this age of REST and all things that use HTTP. Proxomitron, or even a simple Perl script are things that I use right now...
  • Steve Makofsky
    Would be interesting to merge a personal proxy w/ some sort of p2p technology (ala IBM's uServ: http://www.almaden.ibm.com/cs/people/bayardo/userv/)for sharing browse patterns, caches, etc.
  • Mark Paschal
    Sharing bookmarks and cookies between all the browsers you use would be neat.
  • Bill Seiz
    I think this is a really interesting idea. I'm hoping OSAF/Chandler will make a good base for this (ZODB, RDF, etc.). I also see this tying into RSS reading/prioritizing... (although that gets into issues of whether it's supposed to be running all the time...) http://webseitz.fluxent.com/wiki/PersonalWebArchive http://webseitz.fluxent.com/wiki/UniversalInbox http://webseitz.fluxent.com/wiki/OSAF
  • BillSeitz
    Oh, yeah, another use for this is to impose your own WikiName space (and StructuredText rendering) over everything you read. http://webseitz.fluxent.com/wiki/WikiProxy
  • l.m.orchard
    Bill: Actually, that's a lot along the lines I was thinking - I want to use RDF, and was thinking ZODB might be a good repository. Hmm, want to start hacking tonight, but I might just look at how something like this could hook into or be hosted within an OSAF/Chandler release. Oh, and I want to look into this WikiProxy idea of yours some more. You mean something like a filter for me so that every page I look at gets imbued with WikiWord links? *looking at your page* Neat! Hmm, yeah, would have to figure out in what wiki namespace words are linked.
  • BillSeitz
    Yeah, that WikiProxy can get kinda weird. One place where I definitely saw promise was for intranet discussions taking place outside a wiki (since I personally don't like wiki for that), but referencing a shared wiki namespace (since the sender and recipient share the same wiki). But there are definitely times I think about the idea of layering my namespace over other people's messages. But I'm not sure that makes semantic sense... But a great thought experiment. (And no stupider than that Microsoft SmartTags idea...)
  • Moss
    Ooh... hey, the MT plugin that's being used to wikify this weblog... it could also be used to tie more than one weblog into the same wiki, couldn't it? Might be a good way to do a shared wiki namespace for discussion.