Characters Å Ä Ö is not getting displayed in my DDL, how can I tell restclient to utilize a specific charset?

asp.net-mvc, c#, json, json.net, xml

Solution

in the theory you got an charset encoding/decoding problem.

the Cause: the content you try to read has been encoded using a charset like iso-8859-1 or iso-8859-15. and you'll try to read (decode) it directly to an "UTF-8" character Model. Of course it won't work because UTF-8 because UTF-8 won't in a miracle recognize your special chars (Ä,Ü, Ö, and so on..). UTF-8 is no guesser for character coding.

Solution:

1- (Re)encode your content( e.g "Björn Nilsson") with its corresponding charset (iso-8859-1/iso-8859-15) into Byte collection.

2- Decode your content with into "UTF-8" based charset.

here an Helper Class as example:

using System;
using System.Collections.Generic;
using System.Text;

    namespace csharp.util.charset
    {
        public class SysUtil
        {
            /// <summary>
            /// Convert a string from one charset to another charset
            /// </summary>
            /// <param name="strText">source string</param>
            /// <param name="strSrcEncoding">original encoding name</param>
            /// <param name="strDestEncoding">dest encoding name</param>
            /// <returns></returns>
            public static String StringEncodingConvert(String strText, String strSrcEncoding, String strDestEncoding)
            {
                System.Text.Encoding srcEnc = System.Text.Encoding.GetEncoding(strSrcEncoding);
                System.Text.Encoding destEnc = System.Text.Encoding.GetEncoding(strDestEncoding);
                byte[] bData=srcEnc.GetBytes(strText);
                byte[] bResult = System.Text.Encoding.Convert(srcEnc, destEnc, bData);
                return destEnc.GetString(bResult);
            }

        }
    }

Usage:

in your (JSON-, XML, other) serializer/deserializer classes just convert your content like that

String content = "Björn Nilsson";
SysUtil.StringEncodingConvert(content, "ISO-8859-1","UTF-8");

you could try to make your calls in your deserializer (if they really do what they mean):

public class JsonNetSerializerFactory :ISerializerFactory 
{
    public ISerializer<T> Create<T>()
    {
        return new JsonNetSerializer<T>();
    }
    public class JsonNetSerializer<T> : ISerializer<T>
    {
        public T Deserialize(string input, String fromCharset, String toCharset)

        {
           String changedString = SysUtil.StringEncodingConvert(input, fromCharset,toCharset);

            return JsonConvert.DeserializeObject<T>(changedString  );
        }

        public IList<T> DeserializeList(string input, String fromCharset, String toCharset)
        {
         String changedString =  SysUtil.StringEncodingConvert(input, fromCharset,toCharset);

            return JsonConvert.DeserializeObject<IList<T>>(changedString);
        }
    }
}

that's for your

JsonNetSerializerFactory

please try to do same for other factories like

XmlSerializerFactory

and don't forget your setting in your HTML-page

<meta charset="utf-8"> <!--HTML 5 -->

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /> <!-- if HTML version < 5-->

Problem

Before I start here is the problem. It should be like this: Björn Nilsson, instead its displaying strange special characters, all values that have characters Å, Ä and Ö becomes like this. I fill my DDL with values from API in XML format that has all the values, and we are also using Linq2Rest for that. This how the process looks like ``` private readonly RestContext<ADConsultants> restContext; public ConsultantContext(Uri uri, Format format) { restContext = new RestContext<ADConsultants>(GetRestClient(uri, format), GetSerializerFactory(format)); } public enum Format { Pox, Json } private static readonly IEnumerable<Type> knownTypes = new[] {typeof (ADConsultants)}; public static IRestClient GetRestClient(Uri uri, Format format) { switch (format) { case Format.Pox: return new XmlRestClient(uri); case Format.Json: return new JsonRestClient(uri); default: throw new NotImplementedException(); } } private static ISerializerFactory GetSerializerFactory(Format format) { switch (format) { case Format.Pox: return new XmlSerializerFactory(knownTypes); case Format.Json: return new JsonNetSerializerFactory(); default: throw new NotImplementedException(); } } public IQueryable<ADConsultants> Consultant { get { return restContext.Query; } } ``` } This is my JsonNetSerializerFactory class: ``` public class JsonNetSerializerFactory :ISerializerFactory { public ISerializer<T> Create<T>() { return new JsonNetSerializer<T>(); } public class JsonNetSerializer<T> : ISerializer<T> { public T Deserialize(string input) { return JsonConvert.DeserializeObject<T>(input); } public IList<T> DeserializeList(string input) { return JsonConvert.DeserializeObject<IList<T>>(input); } } } ``` And this is inside my controller: ``` var consultants = new ConsultantContext( new Uri("http://adress:port/api/consultants"), ConsultantContext.Format.Json) .Consultant .Where(x => x.Office == "Örebro") .OrderBy(x => x.DisplayName) .ToList() .Select(x => new { name = x.DisplayName }); ``` I did a test by doing this: ``` name = "åäö" ``` and it worked fine, ddl values was "åäö" Any help is appreciated on how to fix so characters Å Ä Ö works fine as values in my DDL. HTTP header is utf-8, my html content have it aswell. It must be the XML that need to be set to a specific charset, How can I do that? Thanks in advance!

Original source