Java creates HUGE file when writing via Windows Remote Desktop (tsclient)

file-io, java, remote-desktop, windows

Solution

I do not have a setup (No Windows) to verify this effect :( so just thoughts:

2GB sounds like Filesystem related max file size. 32bit Windows Operating system at your client side?

The behaviour sounds like clever Filesystem caching on bad block FS: rapid file write access of big blocks remotely tries to cleverly preoccupy the file in an attempt to fasten future writes to the file having blocks together. Try a different FS to verify? Tried FreeRDP?

Keep the file open. Re-opening for write of huge blocks could hint clever systems to cache.

Update:

FileChannelImpl.java:248

// in append-mode then position is advanced to end before writing
p = (append) ? nd.size(fd) : position0(fd, -1);

leads finally to FileDispatcherImpl:136

static native long More ...size0(FileDescriptor fd) throws IOException;

what as being native can hold any bug. When it comes to protocolls inbetween. I would file this rather as bug in nio/Windows, as They might not have foreseen any funny thing with RDP underneath.

It looks like the returned size is `Integer.MAX_VALUE` and the file pointer is moved there…

Alternate implementation `java.io.FileWriter` and no encoding to reduce lines of code:

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.util.Collections;

/**
 * Demonstrates weird issue when writing (appending) to a file over TsClient (Microsoft Remote Desktop).
 *
 * @author Martin
 */
public class WriteOverTsClientDemo
{
   // private static final File FILE_TO_WRITE = new File("\\\\tsclient\\C\\Temp\\TestFile.txt");
   private static final File FILE_TO_WRITE = new File("/tmp/TestFile.txt");

   private static final String ROW_DATA = "111111111122222222223333333333444444444555555555566666666667777777777888888888899999999990000000000";

   public static void main(final String[] args) throws IOException
   {
      if (!FILE_TO_WRITE.getParentFile().exists())
      {
         throw new RuntimeException("\nPlease create directory C:\\Temp\\ on your local machine and run this application via RemoteDesktop with C:\\ as a 'Local resource'.");
      }
      FILE_TO_WRITE.delete();
      new WriteOverTsClientDemo().execute();
   }

   private void execute() throws IOException
   {
      System.out.println("Writing to file: " + FILE_TO_WRITE);
      System.out.println();

      for (int i = 1; i <= 20; i++)
      {
         System.out.println("Writing batch " + i + "...");
         writeDataToFile(i);
         System.out.println("Size of file after batch " + i + ": " + FILE_TO_WRITE.length());
         System.out.println();
      }
      System.out.println("Done!");
   }

   private void writeDataToFile(final int batch) throws IOException
   {
      try (BufferedWriter writer = new BufferedWriter(new FileWriter(FILE_TO_WRITE, batch > 1)))
      {
         writeData(batch, writer);
      }
   }

   private void writeData(final int batch, final BufferedWriter writer) throws IOException
   {
      for (final String data : createData())
      {
         writer.append(Integer.toString(batch));
         writer.append(" ");
         writer.append(data);
         writer.append("\n");
      }
   }

   private Iterable<String> createData()
   {
      return Collections.nCopies(100, ROW_DATA);
   }

}

Problem

A client of ours reported a very weird issue when our Swing application is writing a file to the users local machine via Windows Remote Desktop (the application is hosted on a terminal server where users connects). The flow is: - Users logon and run the application via remote desktop (with their `C:\` included as a "Local resource") - While working they export data from the database into files - The user chooses what data to export - The user selects a destination file on their local computer like `\\tsclient\C\Temp\TestFile.txt` - Files could be big so 1000 rows are fetched from database and written to file per batch - On the second batch, when Java opens the file and write to it again, something really weird starts to happen! - The file increases rapidly in size and stops at around 2 GB - Then data continues to be written to the file I'm not sure if this is a problem in the core Java libraries, the Remote Desktop implementation or a combination. Our application is also hosted via Citrix which works fine, and writing to local disk or UNC network paths works fine as well. I've created a SSCCE demonstrating the problem, connect to a computer with Remote Desktop (make sure `C:\` is a "local resource") and run the program to see some really strange behavior! I'm using JDK-7u45. ``` import static java.nio.file.StandardOpenOption.APPEND; import static java.nio.file.StandardOpenOption.CREATE; import static java.nio.file.StandardOpenOption.TRUNCATE_EXISTING; import static java.nio.file.StandardOpenOption.WRITE; import java.io.BufferedWriter; import java.io.File; import java.io.IOException; import java.io.OutputStream; import java.io.OutputStreamWriter; import java.nio.charset.Charset; import java.nio.charset.CharsetEncoder; import java.nio.file.Files; import java.nio.file.OpenOption; import java.util.Collections; /** * Demonstrates weird issue when writing (appending) to a file over TsClient (Microsoft Remote Desktop). * * @author Martin */ public class WriteOverTsClientDemo { private static final File FILE_TO_WRITE = new File("\\\\tsclient\\C\\Temp\\TestFile.txt"); //private static final File FILE_TO_WRITE = new File("C:\\Temp\\TestFile.txt"); private static final String ROW_DATA = "111111111122222222223333333333444444444555555555566666666667777777777888888888899999999990000000000"; public static void main(String[] args) throws IOException { if (!FILE_TO_WRITE.getParentFile().exists()) { throw new RuntimeException("\nPlease create directory C:\\Temp\\ on your local machine and run this application via RemoteDesktop with C:\\ as a 'Local resource'."); } FILE_TO_WRITE.delete(); new WriteOverTsClientDemo().execute(); } private void execute() throws IOException { System.out.println("Writing to file: " + FILE_TO_WRITE); System.out.println(); for (int i = 1; i <= 10; i++) { System.out.println("Writing batch " + i + "..."); writeDataToFile(i); System.out.println("Size of file after batch " + i + ": " + FILE_TO_WRITE.length()); System.out.println(); } System.out.println("Done!"); } private void writeDataToFile(int batch) throws IOException { Charset charset = Charset.forName("UTF-8"); CharsetEncoder encoder = charset.newEncoder(); try(OutputStream out = Files.newOutputStream(FILE_TO_WRITE.toPath(), CREATE, WRITE, getTruncateOrAppendOption(batch)); BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(out, encoder))) { writeData(batch, writer); } } private void writeData(int batch, BufferedWriter writer) throws IOException { for (String data : createData()) { writer.append(Integer.toString(batch)); writer.append(" "); writer.append(data); writer.append("\n"); } } private Iterable<String> createData() { return Collections.nCopies(100, ROW_DATA); } /** * @return option to write from the beginning or from the end of the file */ private OpenOption getTruncateOrAppendOption(int batch) { return batch == 1 ? TRUNCATE_EXISTING : APPEND; } } ```

Original source