Current Design / Summary

At the moment (as being implemented in PipeFilters):

• struct myapp.filterData(base64 data, string contentType, struct params):
  • Accepts:
    • data (base64): Data stream to be filtered
    • contentType (string): MIME type of data being passed
    • params (struct): Arbitrary parameters to the filter
  • Returns:
    • (struct):
      • data (base64): Filtered data stream
      • contentType (string): MIME type of filtered data

First version spec (as implemented in XmlRpcFilterService):

• base64 myapp.filterData(base64 data):
  • Accept a stream of data/text in base64 (data), process the data, return the results as a stream of data/text in base64.

Existing Implementations

• Blosxom plugin -- warning, hardcoded to "wiki." namespace.
• MovableType pluging -- also hardcoded to "wiki." namespace.
• XrFilter -- Python command line tool for piping data through an XmlRpcFilteringPipe

Concept

This seems like a simple enough idea that could use with a standard spec:

I have text. They have something that consumes text, colorizes the text in HTML, and produces text. You've got something like that too, only you turn my mentions of stock symbols into links to reports on the stock. I want to pass my text through each of your services in series. How about you two expose a simple XmlRpc method on your service, and I'll call them, feeding the output of one method to the input of the next?

So, in effect, we have something like:

• cat my_html_text.txt | their-funky-31337-colorizer | your-stock-symbol-linker > my_new_html_text.txt

Wherein my client app performs the role of piping shell.

How about this for a spec?

• base64 myapi.filterData(base64 data):
  • Accept a stream of data/text in base64 (data), return the results of processing as a stream of data/text in base64.

Seems cleanest to associate the filter method with the namespace of whatever API contains the filter. I was thinking that an argument to the method would be easier than dynamically changing the name of the method, but I actually haven't run into an XML-RPC implementation yet (considering RadioUserLand, Cocoa, Perl, and Python) that makes changing the method name difficult.

If we do want to consider introspection, let's say that for this concept, filtering pipe methods must have a signature like:

base64 *.filterData(base64 data)

So, if I ever implement some standard XML-RPC introspection methods on my APIs, I'll just look for those signatures if I happen to want to create a pipe-building GUI for instance.

-- LesOrchard - 13 Mar 2002

More thoughts while driving home from work last night. Thinking that I'll expand the simple spec into something more sophisticated:

• struct myapp.filterData(base64 data, string contentType, struct params):
  • Accepts:
    • data (base64): Data stream to be filtered
    • contentType (string): MIME type of data being passed
    • params (struct): Arbitrary parameters to the filter
  • Returns:
    • (struct):
      • data (base64): Filtered data stream
      • contentType (string): MIME type of filtered data

This second spec is a bit more sophisticated, assumes that the filter may change the format of the data on the way through, and accepts "command line arguments" like any program in a UNIX command line pipe might.

-- LesOrchard - 14 Mar 2002

Uses

Here are a few uses for which I might want to apply this idea:

• XmlRpcFilterService: Ubiquitous personal (desktop/laptop) text filtering service
  • Write a Service menu item (MacOSX) or clipboard-processing System Tray icon (Windows) that pipes selected text through XmlRpc filters and replaces the selected text with the results.
• WeblogWithWiki
  • Build this piping into a BloggerAPI client. Cause the client to dip weblog entries through a WikiEngine?'s XmlRpcToWiki filterData method to produce weblog entries imbued with WikiLinks? and WikiFormat?
• PipeFilters
  • Use filtering pipes in macros &etc in RadioUserLand
  • Remote page scripting / formatting
  • Expose a text-based macro or page scripting language, ala PHP or JSP or ASP. Process the incoming data through it as if it were a local page (with sandboxy restrictions). Return the results.

-- LesOrchard - 06 Mar 2002

Hm, very interesting approach. This could be really handy as a tool in producing a newsletter where you are constantly referring to things on the net but you don't want to clutter up the ascii text with links; the filter could pick up the links from one of your databases and then dump the annotations at the bottom.

Have you pointed PeterKaminski? and BillSeitz at this? They both run Wikis and weblogs & have done a bunch of thinking at the intersection between the two. (links from my page on Ryze)

-- EdwardVielmetti - 13 Mar 2002

I've just given my latest Local Names server code the XmlRpcFilteringPipe interface. It's in the OneBigSoup:SubversionServer at the moment.

EdwardVielmetti, Local Names meets your "links database" requirement. I've been working on this for a while; Please investigate at: http://ln.taoriver.net/ I've got a bunch of functional stuff, see http://ln.taoriver.net/links.html .

I'm so eager to see my MovableType and Blosxom friends able to use LocalNames?. :) I intend to write a WordPress? XmlRpcFilteringPipe. (Because: I use WordPress?, and I intend to use my own LocalNames? software.)

Questions & Concerns:

• Can/should we use HTTP Content-type info to denote string encodings?
• I'd like to extend the API, to allow for a function that documents the parameters. That is, you call a function, and it returns a dictionary, telling what the interface does, what it's all about, the FOAF of the person who wrote it, the name of the person who wrote it, what the parameters titles are, and a doc-string for each parameter.
• I have an XML-RPC server. It doesn't have any namespaces. Is it still possible to use the XmlRpcFilteringPipe?
• How do existing plugins know what namespace to use?

-- LionKimbro - 01 Jan 2005

In particular, I'm thinking, we should either:

• Designate "wiki." as the standard namespace,
• Designate "filter." as the standard namespace,
• Designate something else as the standard namespace,
• Designate that no namespace is used.

I'd prefer to just say, "No namespace," but if that's not okay with people, I'm happy to concede a particular namespace.

(If you wanted to host multiple filters, I'd just say: Assign them to different urls! That is, I don't believe that namespacing is the job of XML-RPC, I believe it's the job of the URL.)

What I think would really suck is making the person using the plugin specify the namespace as a parameter.

What is less sucky, but I am somewhat okay, is making a registry-retrieving function at the base namespace, which tells what namespace is available with the particular function. I suppose there may well be a standard for this already.

The reason I don't like it is because it complicates things.

In the meantime, I'm probably just going to use "wiki.", even though I'm not a wiki!

LesOrchard, I'd really like to talk this all over with you some time within the month.

-- LionKimbro - 01 Jan 2005