Varied feed polling times versus item urgency in aggregators
The problem with varying the polling interval is that the need varies. It's ok not to poll my little opensource website within 24 hours, but what about the announcements to the civil defence website or local municipal environment alerts, or the nuclear power plant news feed?
Definitely a good point there. For most of the feeds in my daily habit, what I use is an AIMD variation on my polling frequency per feed based on occurrence of new items. For feeds with low-frequency but high-urgency items, a different algorithm should come into play.
On the other hand... should incoming alerts with that much urgency really be conveyed via an architecture driven by polling? Here's an excellent case for tying instant messaging systems and pub/sub into the works.