0xDECAFBAD

It's all spinning wheels and self-doubt until the first pot of coffee.

A personal server & Multi-threaded vs Async / Event-driven programming

Okay after the vote of confidence from Sam Ruby about my thinking out loud about a "404 correction" proxy server, I've been thinking more about writing a Radio-like desktop app server. I want to do more than make a DesktopWebsite, though. I want to make a full-blown PersonalServer app, capable of hosting things like a PersonalMailServer and a slew of other little local web services & etc. I may end up giving up and working more within Radio, but as I noted before I have some issues with Radio's performance and stability, which though balanced by my appreciation of the elegance of the system, is being gradually outweighed by my fears of lock-in and pre-existing experience with other technologies.

Then again, this thing probably won't replace RadioUserLand for me. I use it daily, I bought it, and it's not as exciting to reimplement what I already have. Unless it is exciting. Make any sense?

So, speaking of technologies... which ones should I use to start working on a personal server? My main goals are mumbled over here.

I've got a large amount of experience with Perl, and have written desktop apps with it for Mac OS X and Win32. I'm having more fun with Python, however, and though I haven't written the same apps I imagine that it's on par with Perl.

The main thing I'm trying to decide right now is: multi-threaded vs async/event-driven.

See, I need some sort of concurrency to handle multiple network server hits, multiple agents running, a scheduler, and whatever else wants to take up residence inside the PersonalServer. RadioUserLand, of course, has all of this.

I've worked a lot with POE in Perl to make some event-driven multitasking apps, a few servers (HTTP, FTP, NNTP, etc) and a few things to replace a fleet of forked scripts. I've also started looking at Twisted in Python which I gather to be the analogous thing in their camp. Not the same, but they both are using the same basic idea of event-driven programs.

The problem is that, to take code that you would have written for a forking or multi-threaded program, and make it play nice within the event-driven environment, there's a bit of re-think you need to do. Loops need to be broken up into procedures, turned into self-running events, etc.

Hmm... trying to think of more re-think examples, but the main one I can think of is that long-running loops and processes need to be sliced and diced. I seem to remember more pain than that.

Anyway, I'd rather use threads. In threads, there needs to be a bit of re-think too, in terms of protecting resources from concurrency, but at least the main logic of my code can remain straightforward. Perl doesn't have threads that I want to touch yet. Python has threads, but I'm not sure how kosher they are. Of course, there's always Java, but I want to avoid Java I think.

Anyone tuned in out there with any thoughts? Mostly thinking out loud right now.

shortname=ooooio

Archived Comments

  • Ashutosh
    I've been playing around with exactly the same idea - of a PersonalWebServer with built-in scheduler, object database etc. And again, Perl is my comfort zone. And the lack of decent thread support has been a stumbling block. I have too much work done in top level apps in Perl to change over to Python or anything else. Anyway, I used threads to invoke a scheduler thread and a http daemon thread as an experiment, and they seem to work pretty fine together. I plan to keep the Perl object database (presently embedded in my top level apps - using Storable persistant objects indexed using an RDBMS for searches using pre-configured object attributes) as an external server, and plan to get the PersonalWebAppServer to talk to the database server using XML-RPC.