Matt Mullenweg points out that the RSS feed validator is moving the goal posts, and not for the first time:

They’ve also started marking all uses of content:encoded as potentially causing problems, which is funny because it actually avoids a ton of problems and (again) people have been using it in RSS 2.0 feeds for 3+ years now, and I even asked Dave Winer about it in the past and he said that was fine.

I’ve been struggling with similar problems for some time, trying to improve Textpattern’s RSS support. Textpattern still only supports RSS 0.92¹. I decided some time ago to update to RSS 2.0 as soon as I was sure I could produce a feed that was not just valid (most of the time, at least), but also accurate and useful. That is to say, the contents of the feed should reflect the best possible representation of the article contents. The same kind of principle as used in modern, semantic, standards-compliant web design.

I could easily produce a perfectly valid RSS 2.0 feed the old-fashioned way: strip all tags and entities from the title and description, and just include the first 200 characters of plain ASCII text. That would be valid, but neither accurate nor useful.

The root of the problem is that the RSS 2.0 spec is far too vague. What should I do if the article title contains HTML code or entities? The spec doesn’t say. Should the description include HTML? The spec doesn’t say. Should I use <description>, <content:encoded>, or both? The spec doesn’t say. If I use both, should the description and content:encoded include identical content, or different? The spec doesn’t say. What do I do with an article that contains both a short excerpt and a long body? The spec doesn’t say. Can I safely include UTF-8 characters, or should I entity-encode them? The spec doesn’t say. Should I stick with core elements for things like dates, use namespace elements instead, or include both? The spec doesn’t say.

There is no shortage of conflicting opinions on these questions. The problem isn’t coming up with answers, it’s choosing the right ones.

Even those responsible for the spec seem to have given up hope of resolving these issues. An entire industry has sprung up around the service of simply interpreting and fixing all these little semantic differences between feeds.

So to Matt, and any other RSS application developers in earshot, I’d like to propose a modest solution: let’s agree on a simple set of answers. I’m not talking about rewriting the spec, just a brief statement of interpretations and best practices for RSS 2.0 feed producers. Things like:

• How to deal with entities and HTML in each of the title², description and content:encoded elements
• Which extensions to use, and for what
• When to duplicate content in both core elements and extensions
• How to deal with multi-part articles
• Multiple enclosures per item: yes or no

Never mind the spec lawyers or the feed validator guys. I’ll listen to what they have to say when they can agree on some basic answers to fundamental questions, instead of making endless arbitrary, conflicting decisions. Until then, let’s see if the application developers can show them how it’s done.

As I said, most of the answers are already available; it’s simply a matter of agreeing on which ones to choose.

¹ I notice Matt seems to have reverted to RSS 0.92, at least for now. A quiet protest?

² The title element is the most problematic, in my experience. For example, imagine an article with a title like this:

HTML Tutorial: the <br /> tag

How should I encode it? Single entity encoding works in some readers:

<title>HTML Tutorial: the &lt;br /&gt; tag</title>

But others think I’m trying to insert HTML markup, and strip the tag out, so it looks like this:

HTML Tutorial: the tag

For those readers, double encoding might work:

<title>HTML Tutorial: the &amp;lt;br /&amp;gt; tag</title>

But readers that expect single encoding will probably display this incorrectly:

HTML Tutorial: the <br /> tag

Here’s a sample feed including both test cases, plus a third case with numeric single-encoding. Google Reader strips the tag from the single and numeric encoded cases, but displays the double-encoded title correctly:

Bloglines also appears to interpret the double-encoded example the way we want, but interprets the single and numeric cases as actual HTML, inserting a line break in the title:

Ironically, the Feed Validator issues a warning about the double-encoded case—the only one that worked correctly in both readers. The Validator’s recommendation is to use numeric encoding instead, which didn’t work well in either reader. Yet the validator is the only source I’ve found that addresses the issue.

All three cases are considered valid, incidentally. Which proves the point that there’s a big difference between valid, useful and accurate.