Unescape HTML &#nn; sequences

r

Solution

Just for reference, here is the solution I came up with. It makes use of the great package `gsubfn`:

library(gsubfn)

I use a vector `htmlchars` for named html entities I scraped from Wikipedia. For brevity, I do not copy the vector in this answer here, but source it from pastebin:

source("http://pastebin.com/raw.php?i=XtzN1NMs") # creates variable htmlchars

Now the decoding function I was looking for is simply:

strdehtml <- function(s) {
    ret <- gsubfn("&#([0-9]+);", function(x) rawToChar(as.raw(as.numeric(x))), s)
    ret <- gsubfn("&([^;]+);", function(x) htmlchars[x], ret)
    return(ret)
}

Not sure if this covers all possible HTML characters, but it gets me working. For instance, it can be used thus:

test <- "My this &amp; last year&#39;s resolutions"
strdehtml(test)
[1] "My this & last year's resolutions"

Problem

My text has some HTML escaped characters, for instance, instead of `'` there is `&#39;`. Now I would like to unescape these sequences. Since I do not know which characters are escaped, I do not want to use a simple mapping such as in `c("&#39;"="'", ...)`. I understand that the number after the ampersand is the decimal unicode number. So `&#39;` is `\u27` since 27 is the hexidecimal representation of 39. So I thought a solution that involves ``` sprintf("\u%x", s) ``` where `s` is the extracted number between `&` and `;`. However, this results in an error: "\u used without hex numbers." What would be a better approach to convert HTML escaped sequences back to characters?

Original source