Comments on: Dynamic feed polling times for news aggregators http://decafbad.com/blog/2003/09/25/dynamic-feed-scan-times It's all spinning wheels and self-doubt until the first pot of coffee. Wed, 07 Jan 2009 05:03:38 +0000 http://wordpress.org/?v=2.8-bleeding-edge hourly 1 By: Gnomon http://decafbad.com/blog/2003/09/25/dynamic-feed-scan-times/comment-page-1#comment-1054 Gnomon Tue, 30 Nov 1999 00:00:00 +0000 http://www.decafbad.com/blog/?p=483#comment-1054 <p>Why not just go the TCP/IP route - Additive Increase / Multiplicative Decrease? Start by setting the check-interval to one hour (or whatever). For each feed, if a new post is found, cut the check-interval for that feed by half; if no new post is found, increase the check-interval by an hour.</p> <p>It's not optimal, and it won't automagically zero in on the predicted post times of individual feeds, but it strikes a nice balance between bandwidth politeness, update rapidity and conceptual simplicity. The constant values (initial check-interval, check-interval increment, check-interval multiplier) can be tweaked for different behavioural styles.</p> Why not just go the TCP/IP route - Additive Increase / Multiplicative Decrease? Start by setting the check-interval to one hour (or whatever). For each feed, if a new post is found, cut the check-interval for that feed by half; if no new post is found, increase the check-interval by an hour.

It’s not optimal, and it won’t automagically zero in on the predicted post times of individual feeds, but it strikes a nice balance between bandwidth politeness, update rapidity and conceptual simplicity. The constant values (initial check-interval, check-interval increment, check-interval multiplier) can be tweaked for different behavioural styles.

]]> By: l.m.orchard http://decafbad.com/blog/2003/09/25/dynamic-feed-scan-times/comment-page-1#comment-1055 l.m.orchard Tue, 30 Nov 1999 00:00:00 +0000 http://www.decafbad.com/blog/?p=483#comment-1055 <p>Hmm... I knew that this was something that someone had already handled somewhere. :) I'm just not all that familiar with the details of TCP/IP, but this sounds pretty much like a workable approach I'd like to go with.</p> Hmm… I knew that this was something that someone had already handled somewhere. :) I’m just not all that familiar with the details of TCP/IP, but this sounds pretty much like a workable approach I’d like to go with.

]]> By: l.m.orchard http://decafbad.com/blog/2003/09/25/dynamic-feed-scan-times/comment-page-1#comment-1056 l.m.orchard Tue, 30 Nov 1999 00:00:00 +0000 http://www.decafbad.com/blog/?p=483#comment-1056 <p>Oh! For a second I thought that what you're suggesting was basically what I was already working on with my multiplying/dividing by a factor... But you're talking about something that ADDS to increase and MULTIPLIES to decrease, which is something much more biased to back off than ramp up, which seems very polite to me. Yay!</p> Oh! For a second I thought that what you’re suggesting was basically what I was already working on with my multiplying/dividing by a factor… But you’re talking about something that ADDS to increase and MULTIPLIES to decrease, which is something much more biased to back off than ramp up, which seems very polite to me. Yay!

]]> By: hebig.org/blog http://decafbad.com/blog/2003/09/25/dynamic-feed-scan-times/comment-page-1#comment-1057 hebig.org/blog Tue, 30 Nov 1999 00:00:00 +0000 http://www.decafbad.com/blog/?p=483#comment-1057 <p><strong>Quick Links, September 29</strong></p> <p>Holy Moly: One Million Weblogs Tracked - "Technorati is currently tracking about 7,000 new weblogs per day" Linus Torvalds...</p> Quick Links, September 29

Holy Moly: One Million Weblogs Tracked - “Technorati is currently tracking about 7,000 new weblogs per day” Linus Torvalds…

]]> By: hebig.org/blog http://decafbad.com/blog/2003/09/25/dynamic-feed-scan-times/comment-page-1#comment-1058 hebig.org/blog Tue, 30 Nov 1999 00:00:00 +0000 http://www.decafbad.com/blog/?p=483#comment-1058 <p><strong>Quick Links, September 29</strong></p> <p>Holy Moly: One Million Weblogs Tracked - "Technorati is currently tracking about 7,000 new weblogs per day" Linus Torvalds...</p> Quick Links, September 29

Holy Moly: One Million Weblogs Tracked - “Technorati is currently tracking about 7,000 new weblogs per day” Linus Torvalds…

]]>