UTF-8 file output in R
cjk, r, unicode
Solution
The problem is due to some R-Windows special behaviour (using the default system coding / or using some system write functions; I do not know the specifics but the behaviour is actually known)
To write text UTF8 encoding on Windows one has to use the `useBytes=T` options in functions like writeLines or readLines:
txt <- "在"
writeLines(txt, "test.txt", useBytes=T)
readLines("test.txt", encoding="UTF-8")
[1] "在"
Find here a really well written article by Kevin Ushey: http://kevinushey.github.io/blog/2018/02/21/string-encoding-and-r/ going into much more detail.
Problem
I'm using R 2.15.0 on Windows 7 64-bit. I would like to output unicode (CJK) text to a file. The following code shows how a Unicode character sent to write on a UTF-8 file connection does not work as (I) expected: ``` rty <- file("test.txt",encoding="UTF-8") write("在", file=rty) close(rty) rty <- file("test.txt",encoding="UTF-8") scan(rty,what=character()) close(rty) ``` As shown by the output of scan: ``` Read 1 item [1] "<U+5728>" ``` The file was not written with the UTF character itself, but some kind of ANSI-compliant fallback. Can I make it work right the first time (i.e. with a text file that has "在" in it instead), or can I work some extra magic to convert the output to Unicode with the proper character replacing the code string? Thanks. [More info: the same code behaves properly in Cygwin, R 2.14.2, while 2.14.2 on Win7 is also broken. Is this on my end somewhere?]