why is ASCII.GetBytes returning wrong bytes

Question

Im creating a class and converting it into xml. The problem is that when i convert the class xml string into bytes the `ASCII.GetBytes` return a byte array with an extra character in the beginning of the `ascArray` It's always a ? character so the xml starts like this ``` ?<?xml version="1.0" encoding="utf-8"?> ``` Why is this happening? This is the code: ``` WorkItem p = new WorkItem(); // Fill the class with whatever need to be sent to client OneItem posts1 = new OneItem(); posts1.id = "id 1"; posts1.username = "hasse"; posts1.message = "hej again"; posts1.time = "time1"; p.setPost(posts1); OneItem posts2 = new OneItem(); posts2.id = "id 2"; posts2.username = "bella"; posts2.message = "hej again again"; posts2.time = "time2"; p.setPost(posts2); // convert the class WorkItem to xml MemoryStream memoryStream = new MemoryStream(); XmlSerializer xs = new XmlSerializer(typeof(WorkItem)); XmlTextWriter xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8); xs.Serialize(xmlTextWri…

Accepted Answer

I strongly suspect that the data starts with a byte order mark, which can't be represented in ASCII.

It's not clear why you're doing what you're doing in the first place, particularly around the `MemoryStream`. Why are you creating a UTF-8 encoded byte array, then decoding that to a string (and we don't know what `UTF8ByteArrayToString` does), then converting it back to a byte array? Why not just write the byte array straight to the client to start with? If you need the data as a string, I'd use a subclass of `StringWriter` which advertises that it uses UTF-8 as the encoding. If you don't need it as a string, just stick to the byte array.

Note that even aside from this first character, the fact that you've got an XML document encoded in UTF-8 means there may well be other non-ASCII characters in the string. Why are you using ASCII at all here?

EDIT: Just to be clear, you're fundamentally applying a lossy transformation, and doing it needlessly. Even if you want a local copy of the data, you should have something like this:

```
// Removed bad try/catch block - don't just catch Exception, and don't
// just swallow exceptions
MemoryStream memoryStream = new MemoryStream();
XmlSerializer xs = new XmlSerializer(typeof(WorkItem));
XmlTextWriter xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8);
xs.Serialize(xmlTextWriter, p);

// Removed pointless conversion to/from string
// Removed pointless BinaryWriter (just use the stream)

// An alternative would be memoryStream.WriteTo(clientStream);
byte[] data = memoryStream.ToArray();
clientStream.Write(data, 0, data.Length);
Console.WriteLine(" send.." + data);

// Removed Close calls - you should use "using" statements to dispose of
// streams automatically.
```

why is ASCII.GetBytes returning wrong bytes

Solution

Problem