dingbat@c... (Andy Dingley) wrote in message news:<28200a19.0405060335.601df2dd@p...>...
> > However, it still fails validation at
> > http://feeds.archive.org/validator/
> 
> The version this morning seems OK.  I suggest you keep an archive of
> these bad feeds under static URLs, so that we can more easily see the
> problems.
>  
> > The problem is "bad characters". I'm not sure how to start to debug
> > that. What is the first thing I should look for?
> 
> Most likely thing is the use of HTML entities (like &eacute; ) that
> aren't valid because RSS is an XML protocol and doesn't recognise
> these HTML-defined entities. A _very_ common error in RSS feeds.

I'm stumped when I think of all the possible user inputs that can mess
up an RSS feed. Myself and a friend have been working on some weblog
software for a year now - the weblog entries show up in the RSS feed.
Lots and lots of problems come up - people might copy and paste
something from Word in their weblog and then get a Windows character
that is not in the defined char set for the RSS feed. How to protect
against that. Or people can, as you say, add in some HTML entities, or
sometimes the software does that automatically.

It's true that I could write filters for each possible problem as it
comes up, but it seems like it would take a mountain of code and there
would never be an end to it - after I figure out how to convert
unexpected European accented characters, then I'd have all the Asian
alphabets to work on.

I guess I'm asking if there is an easy, elegant way to convert a set
of characters to some particular character set. I don't know of any
myself. What strategies have other programmers used when tackling this
problem?