0xDECAFBAD

Please fix the XML-RPC spec

I've written before that I love XML-RPC, and that it has served me well in the past couple of years. I think it's the right tool for a broad range of jobs. But, after having studied the spec, and after having implemented it in a handful of languages for a handful of well-used apps, I think the spec needs just a bit of fixing.

In particular, the spec needs a tweak with regards to this "ASCII limitation". There is confusion about this, period. I've had to hash this out with clients before, this was an issue of note while working out an XML-RPC based Wiki API, and it's obviously an issue in many other projects. This, of course, includes the current hubbub surrounding weblog APIs and whatnot.

So, please fix the spec. It shouldn't take long to make this issue a non-issue by some simple clarification in the main XmlRpc to which everyone refers.

Yes, I know there's a bit of clarification at the end of the spec, involving escaping ( not encoding) < and & along with the statement that "A string can be used to encode binary data."

Well, yeah, I do that all the time with Base64. And, since the spec earlier had called for "ASCII", I assume that's what encoding binary data means in the context of this spec. To me, encoding implies a transformation from original form to some other form later requiring decoding.

But, apparently, my interpretation and the interpretation of others is wrong on that score. But still, I've been confused, and so have others. Consider this a bug report.

I've been referred by Fredrik Lundh (via Dave Winer), to "private conversations", "various public fora", and "early archives for the XML-RPC mailing list". And, again by Fredrik Lundh, I'm told:

But even if you don't know all this, it's not that hard to figure it out for yourself. Just make sure you read and digest the entire specification, apply some common sense to sort out the contradictions, and you'll find that it's pretty obvious that the intent is that you can use any character allowed by XML.

Well, let's see. I read the whole spec, more than once, and what I figured out for myself with my "common sense" is what I wrote above. I thought the spec called for ASCII (as in: ASCII), and assumed that encoding binary data called for something like Base64. Yes, I realize that XmlRpc is XML, but when a spec calls for ASCII as a particular part, I assume that there's a reason for it since that's what the specification specified.

In my experience, specifications are not about common sense, figuring it out, and connotation. Specifications are about declaration, clarity, and denotation.

Yes, I understand that no spec is perfect, and that many are steaming piles meeting none of the criteria I just mentioned, but that doesn't alter the goal. A spec can always be made better by revising with these things in mind, given the input of consumers of the spec. This is what a process of communication is all about, and specifications are intended as a form of communication.

So, instead of talking about intent and things that have been talked about somewhere at some time, with the implication that I should just go off and search for these things, can we just get a clarifying fix to the XmlRpc spec? I don't want to send my clients off to mailing list and discussion archives, or present XmlRpc with any corrections or caveats. I want to say, as I have been, "Here, go to xmlrpc.com, read the spec, implement to the API I emailed you, and get back to me." Only, it'd be nice if the first question is about my API, not about character encoding.

I've been confused, and so have others. I consider myself a smart person, and I consider most of the others who have been confused as even smarter. I apologize if my "common sense" is of a different sort, but that's what you have to deal with in the world of people. As young as I am, even I've discovered this already.

So, can we just get a clarifying revision of the spec? And if not, why not?

Update: Rock on. After catching up on a bit of banter over at Sam's place, I see that the same Fredrik Lundh I quoted before has already begun an XML-RPC errata page with the goal of clarification. (I just missed it in my daily reading thus far.) As Mark comments, I fear bumps in the road as any confused implementors find things weren't what they thought, but I'm happy to see the clarification accepted.

Update again: If you've stopped rocking, resume. Dave Winer updated the XML-RPC spec. It was a small change, could have been more, but had not been done at all until now. I doubt that my asking please really had much to do with it, but I couldn't guess that it hurt. Thanks!

shortname=plz_fix_xmlrpc_spec