Why does reading a file into memory takes 4x the memory in Java?
file, file-io, java, memory, performance
Solution
It's an interesting question, but rather than stress over why Java is using so much memory, why not try a design that doesn't require your program to load the entire file into memory?
Problem
I have the following code which reads in the follow file, append a \r\n to the end of each line and puts the result in a string buffer: ``` public InputStream getInputStream() throws Exception { StringBuffer holder = new StringBuffer(); try{ FileInputStream reader = new FileInputStream(inputPath); BufferedReader br = new BufferedReader(new InputStreamReader(reader)); String strLine; //Read File Line By Line boolean start = true; while ((strLine = br.readLine()) != null) { if( !start ) holder.append("\r\n"); holder.append(strLine); start = false; } //Close the input stream reader.close(); }catch (Throwable e){//this is where the heap error is caught up to 2Gb System.err.println("Error: " + e.getMessage()); } return new StringBufferInputStream(holder.toString()); } ``` I tried reading in a 400Mb file, and I changed the max heap space to 2Gb and yet it still gives the out of memory heap exception. Any ideas?