0xDECAFBAD

It's all spinning wheels and self-doubt until the first pot of coffee.

Varied feed polling times versus item urgency in aggregators

The problem with varying the polling interval is that the need varies. It's ok not to poll my little opensource website within 24 hours, but what about the announcements to the civil defence website or local municipal environment alerts, or the nuclear power plant news feed?

Definitely a good point there. For most of the feeds in my daily habit, what I use is an AIMD variation on my polling frequency per feed based on occurrence of new items. For feeds with low-frequency but high-urgency items, a different algorithm should come into play.

On the other hand... should incoming alerts with that much urgency really be conveyed via an architecture driven by polling? Here's an excellent case for tying instant messaging systems and pub/sub into the works.

shortname=polling_and_urgency

Archived Comments

  • Why not just tweak the constants to be more biased towards rapid updating of the important sites?