Why is .NET's File.Open with a UNC path making excessive SMB calls?

.net, c#, c++, file-io, windows-networking

Solution

In short, File.Open calls `new FileStream()` and `new FileStream()` does a lot of calls:

NormalisePath.

String filePath = Path.NormalizePath(path, true, maxPath); // fullCheck: true

leads to this code:

1.a: Get full path:

    if (fullCheck) { ... 
        result = newBuffer.GetFullPathName();

GetFullPathName() calls `Win32Native.GetFullPathName` one or two times (depending on the lentgh of resulting path).

1.b. Trying to expand short path. Your path contains `~` char, so it looks like a candidate for a path expanding:

    if (mightBeShortFileName) {
        bool r = newBuffer.TryExpandShortFileName();

as a result, Win32Native.GetLongPathName() is called.

FileIoPermission.Demand() (for non-trusted only):

// All demands in full trust domains are no-ops, so skip 
if (!CodeAccessSecurityEngine.QuickCheckForAllDemands()) {
    ...
    new FileIOPermission(secAccess, control, new String[] { filePath }, false, false).Demand();

Open fileStream (floppy strikes back;)):

 // Don't pop up a dialog for reading from an emtpy floppy drive
int oldMode = Win32Native.SetErrorMode(Win32Native.SEM_FAILCRITICALERRORS);
try {
    ...
    _handle = Win32Native.SafeCreateFile(tempPath, fAccess, share, secAttrs, mode, flagsAndAttributes, IntPtr.Zero);

Win32Native.GetFileType()

Not all of them would lead to smb request, but some will do. I've tried to reproduce chatty requests by debugging with source step-by-step (here's manual for enabling the .net source debugging) and checking the log after each step. Resuts are more similar to your's first listing. If you're really interested in finding the real issue, you'll have to do it yourself.

UPD Note that I've checked current (.net 4.5.2) behavior. It was changed multiple times since 2.0 (e.g. `FileIOPermission.Demand()` originally was called for full-trusted code too), so it depends:)

Problem

I have a block of code that needs to open and read a lot of small text files from a NAS server using UNC paths. This code is part of a module that was originally written in C++ but is now being converted to C#. The C# version is significantly slower. I determined that the call to open the file accounts for nearly all of the performance difference. Using WireShark I found that this is because the System.IO.File.Open call makes far more SMB network requests than similar C++ code. The C++ code makes this call: ``` FILE *f = _wfsopen(fileName, L"r", _SH_DENYWR); ``` This results in the following sequence of SMB requests: ``` NT Create AndX Request, FID: 0x0004, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0004 Trans2 Request, QUERY_FILE_INFO, FID: 0x0004, Query File Basic Info Trans2 Response, FID: 0x0004, QUERY_FILE_INFO Read AndX Request, FID: 0x0004, 1327 bytes at offset 0 Read AndX Response, FID: 0x0004, 1327 bytes Close Request, FID: 0x0004 Close Response, FID: 0x0004 NT Create AndX Request, FID: 0x0005, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0005 ``` The C# code makes this call: ``` FileStream f = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.Read); ``` This results in the following sequence of SMB requests: ``` Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i Trans2 Response, FIND_FIRST2, Files: i Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a\\q Trans2 Response, FIND_FIRST2, Files: q Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a\\q\\~141106162638847.nmd Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i Trans2 Response, FIND_FIRST2, Files: i Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a Trans2 Response, FIND_FIRST2, Files: a Trans2 Request, QUERY_PATH_INFO, Query File Basic Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, QUERY_PATH_INFO, Query File Standard Info, Path: \\a\\i\\a Trans2 Response, QUERY_PATH_INFO Trans2 Request, FIND_FIRST2, Pattern: \\a\\i\\a\\q Trans2 Response, FIND_FIRST2, Files: q Close Request, FID: 0x000f Close Response NT Create AndX Request, FID: 0x0018, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0018 Trans2 Request, QUERY_FILE_INFO, FID: 0x0018, Query File Basic Info Trans2 Response, FID: 0x0018, QUERY_FILE_INFO Read AndX Request, FID: 0x0018, 1327 bytes at offset 0 Read AndX Response, FID: 0x0018, 1327 bytes Close Request, FID: 0x0018 Close Response, FID: 0x0018 NT Create AndX Request, FID: 0x0019, Path: \\a\\i\\a\\q\\~141106162638847.nmd NT Create AndX Response, FID: 0x0019 ``` Why does System.IO.File.Open make all these extra SMB requests? Is there any way to change this code to avoid all these extra requests?

Original source