Audio streaming via TCP socket on Android

Question

I am streaming mic input from a C Server via socket. I know the stream works because it does with a C client and I am getting the right values on my Android client. I am streaming a 1024 floatarray. One float are 4 bytes. So I got a incoming stream with 4096 bytes per frame. I am getting the floats out of this bytes and I know this floats are the ones I sent, so that part should work. Now I want to get that stream directly to the phones speakers by using AudioTrack. I tried to input the bytes I received directly: just noise. I tried to cast it back to a byte array, still the same. I tried to cast that float into short (because AudioTrack takes bytes or short). I could get something that could have been my mic input (knocking), but very scratchy and and extremely laggy. I would understand if there was a lag between the frames, but I can't even get one clear sound. I can, however, output a sin sound clearly that I produce locally and put into that shortarray. Now I wonder if I got some…

Accepted Answer

Sooo...I just solved this only hours after I desperatly put bounty on it, but thats worth it.

I decided to start over. For the design thing with threads etc. I took some help from this awesome project, it helped me a lot. Now I use only one thread. It seems like the main point was the casting stuff, but I am not too sure, it also may have been the multithreading. I don't know what kind of bytes the byte[] constructor of AudioTracker expects, but certainly no float bytes. So I knew I need to use the short[] constructor. What I did was -put the bytes in a byte[] -take 4 of them and cast them to a float in a loop -take each float and cast them to shorts

Since I already did that before, I am not too sure what the problem was. But now it works. I hope this can help someone who wents trough the same pain as me. Big thanks to all of you who participated and commented.

Edit: I just thought about the changes and figured that me using CHANNEL_CONFIGURATION_STEREO instead of MONO earlier has contributed a lot to the stuttering. So you might want to try that one first if you encounter this problem. Still for me it was only a part of the solution, changing just that didn't help.

```
    static final int frequency = 44100;
    static final int channelConfiguration = AudioFormat.CHANNEL_CONFIGURATION_MONO;
    static final int audioEncoding = AudioFormat.ENCODING_PCM_16BIT;
    boolean isPlaying;
    int playBufSize;
    Socket socket;
    AudioTrack audioTrack;

playBufSize=AudioTrack.getMinBufferSize(frequency, channelConfiguration, audioEncoding);
    audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC, frequency, channelConfiguration, audioEncoding, playBufSize, AudioTrack.MODE_STREAM);

new Thread() {
        byte[] buffer = new byte[4096];
        public void run() {
            try { 
                socket = new Socket(ip, port); 
            }
            catch (Exception e) {
                e.printStackTrace();
            }
            audioTrack.play();
            isPlaying = true;
            while (isPlaying) {
                int readSize = 0;
                try { readSize = socket.getInputStream().read(buffer); }
                catch (Exception e) {
                    e.printStackTrace();
                }
                short[] sbuffer = new short[1024];
                for(int i = 0; i < buffer.length; i++)
                {

int asInt = 0;
                    asInt = ((buffer[i] & 0xFF) << 0) 
                            | ((buffer[i+1] & 0xFF) << 8) 
                            | ((buffer[i+2] & 0xFF) << 16) 
                            | ((buffer[i+3] & 0xFF) << 24);
                    float asFloat = 0;
                    asFloat = Float.intBitsToFloat(asInt);
                    int k=0;
                    try{k = i/4;}catch(Exception e){}
                    sbuffer[k] = (short)(asFloat * Short.MAX_VALUE);

i=i+3;
                }
                audioTrack.write(sbuffer, 0, sbuffer.length);
            }  
            audioTrack.stop();
            try { socket.close(); }
            catch (Exception e) { e.printStackTrace(); }
        }
    }.start();
```

Audio streaming via TCP socket on Android

Solution

Problem