Why does RMarkdown `render` behavior depend on whether it's called from RStudio Server or from a PHP shell?
knitr, r, r-markdown, rstudio
Solution
Usually, issues of this form (where unicode characters are converted to a unicode code point representation, e.g. `<U+00EB>` in this case) are caused by an attempt to run R within a non-UTF-8 locale.
Typically, this can be verified by checking the output of `Sys.getlocale("LC_ALL")`. If you see a `C` locale reported, then you likely need to enforce a UTF-8 locale with something like:
Sys.setlocale("LC_ALL", "en_US.UTF-8")
substituting the particular UTF-8 locale flavor based on your desired language. (For reference, the set of available locales can usually be queried from a terminal with something like `locale -a`).
Problem
I have an RMarkdown document that includes 'special characters', such as `ë`. If I render the document using RStudio Server's "knit document" button, it renders fine. When I render it by using the RStudio Server button to source another R script that calls `RMarkdown`'s `render` function, it also renders fine. However, from some reason that's beyond me (but hopefully not for long), I get different results when that same R script is called by `index.php` using: ``` $results = shell_exec("R --file='/home/username/public_html/some/subdirectories/process.R' --no-save 2>&1"); ``` When I do this, in the resulting .html file, the special symbols (I guess the unicode symbols) are replaced by `<U+00EB>`. I've tried to look up whether this is some kind of variation of HTML elements that I didn't know about yet, but I have been unable to find anything about this. (note: any link to a place where I can learn more about this (and, while we're at it, why my browser doesn't show it as, for example, the ë it represents, is also greatly appreciated!) Reproducable example Contents of `example.php`: ``` <?php shell_exec("R --file='/home/username/public_html/subdirectory/example.R' --no-save 2>&1"); ?> ``` Contents of `example.R` (this is what I needed on my server): ``` workingPath <- "/home/username/public_html/subdirectory"; ### Set path to RStudio's pandoc version Sys.setenv(PATH=paste(Sys.getenv("PATH"), "/usr/lib/rstudio-server/bin/pandoc", sep=":")); ### Set HOME and LANG Sys.setenv(HOME = '/home/username'); Sys.setenv(LANG = 'en_US.UTF-8'); require(rmarkdown); renderResults <- render(file.path(workingPath, 'example.Rmd'), output_file = file.path(workingPath, 'example.html'), intermediates_dir = file.path(workingPath, 'tmp'), encoding="UTF-8"); ``` Contents of `example.Rmd`: ```` --- title: 'Reproducable example' output: html_document --- ```{r} cat("This is an ë symbol."); ``` ```` Results of this example: When I run this from R Studio, I get: cat("This is an ë symbol."); ## This is an ë symbol. When I run this from PHP, I get: cat("This is an ë symbol."); ## This is an <U+00EB> symbol. (note how, interestingly, the `echo`'ed ë does show up normally...) I now resorted to doing a `str_replace` in the `index.php` file, but that's not ideal. I've checked the `render` manual, but I can't find anything about this behavior. I've also looked at specifying options for `pandoc` in the YAML header of the .Rmd file, but the only thing that seems to come close is the `--ascii` option, and that doesn't do anything. The R Studio RMarkdown page doesn't provide any hints, either. Could it perhaps have to do with environment variables that are set in RStudio? I already had to set: ``` Sys.setenv(HOME = '/home/oupsyusr'); Sys.setenv(LANG = 'en_US.UTF-8'); ``` in the R script to get Pandoc going in the first place when called in the R script called from the PHP shell; but if this is the problem, how can I figure out which settings RStudio sets to which values, or more accurately, which of those are important? I ran: ``` Sys.getenv() ``` From within R Studio, and that shows quite a list. I recognize none of the entries as having to do with encoding or so. Or, does knitr cause this? When I store and inspect the .md file, the Unicode element things already show up. However, the knitr help page with chunk options doesn't say anything about unicode or encoding in general. Does anybody know where this is documented, or does anybody happen to have encountered this situation before? I'm running RStudio 0.99.903 and R 3.3.1 on CentOS 6.8.