Skip navigation

Using web services and XSLT to scrape RSS from HTML



After tinkering a bit with web services and XSLT-based scraping last week for generating RSS from HTML, I ripped out some work I was doing for a Java-based scraper I’d started working on last year and threw together a kit of XSLT files that does most everything I was trying to do.

I’m calling this kit XslScraper, and there’s further blurbage and download links avaiable in the Wiki. Check it out. I’ve got shell scripts to run the stuff from as a cron job, and CGI scripts to run it all from web services.

For quick gratification, check out these feeds:

shortname=xsl_scraper

This entry was written by l.m.orchard and posted on September 2, 2003 at 12:22 am and filed under Uncategorized. Bookmark the permalink. Follow any comments here with the RSS feed for this post. Both comments and trackbacks are currently closed.

One Comment

  1. Patrick Lioi
    Posted September 3, 2003 at 7:03 pm | Permalink

    The KurzweilAI.net feed is empty.

5 Trackbacks/Pingbacks

  1. By Notes from my terminal on 02 Sep 2003 at 3:58 am

    Wow. Using web services and XSLT to scrape RSS from HTML

    Les Orchard of 0xDECAFBAD has written an amazing piece of software called XslScraper. It’s written in Java and uses XSLT and HTMLTidy to scrape web pages in order to produce RSS. Better yet, the XSLT processor and Tidy are offered…

  2. By hebig.org/blog on 02 Sep 2003 at 10:21 am

    QuickLinks - September 02, 2003

    Changes As I still don’t have a solution for a Quick Links XML feed (the usual approach with one…

  3. By hebig.org/blog on 02 Sep 2003 at 10:24 am

    QuickLinks - September 02, 2003

    Changes As I still don’t have a solution for a Quick Links XML feed (the usual approach with one…

  4. By More Like This WebLog on 04 Sep 2003 at 1:05 pm

    XSL+Tidy RSS Scraper Service

    Alright, L. M. Orchard took my technique of RSS scraping via Tidy and XSLT, and produced a RESTful web service to do this. You provide a URL for the site to scrape, and the URL of an XSLT transform and it returns RSS.

  5. By 0xDECAFBAD » Greatest Hits Vol. 1, the Tinkery Bits on 26 Apr 2008 at 1:10 pm

    [...] Sep 02: Using web services and XSLT to scrape RSS from HTML Progressing from perl scraping hacks to cleaner XSLT-based feed [...]