Runtime: Support long file names on Windows

[JeremyKuhne]

Now would seem to be the time to finally tackle the MAX_PATH nightmare. With the Win32FileSystem and other abstractions we can convert incoming paths to their long path equivalent if needed before handing off to the actual Win32 APIs. It appears that all of the APIs currently in use support the extended syntax (\?), with the exception of Get/SetDirectory.

[ebickle]

A bit of classic reading from the BCL team for anyone digging into this issue:
http://blogs.msdn.com/b/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx
http://blogs.msdn.com/b/bclteam/archive/2007/03/26/long-paths-in-net-part-2-of-3-long-path-workarounds-kim-hamilton.aspx
http://blogs.msdn.com/b/bclteam/archive/2008/06/10/long-paths-in-net-part-3-of-3-kim-hamilton.aspx

Any creative workarounds for the path normalization issue?

[anaisbetts]

Now would seem to be the time to finally tackle the MAX_PATH nightmare

This will never happen, and should never happen. Please do not add implicit long path support to .NET, it will be a nightmare for users, who now have files that they have created in one app are inexplicably unopenable in other apps.

MAX_PATH being a #define in Win32 means that it is hard-coded into tens of thousands of applications, in a way that is almost impossible to change after-the-fact (i.e. via AppCompat shims, etc). You can't even set this in _new_ versions of the SDK because a bunch of apps have made wire / IPC protocols which would include MAX_PATH in the definition, and now they disagree and you've broken things. Using extended syntax _also_ bypasses all of the path correction that Win32 does and means that path semantics are now different between apps, depending on their .NET version. Attempting to "fix" MAX_PATH is simply not tenable in any sane way, _especially_ from a non-OS component like the CLR.

[smbecker]

The reality is that you already have tons if files that are unopenable by .Net applications because the framework doesn't let you bypass MAX_PATH. Or you have to resort to rewriting System.IO classes using P/Invoke like in the case of AlphaFS. This should be possible to enable long path support from within .Net. Should it be implicit? I could be convinced either way. However, it is possible in Windows and should at least be possible in .Net.

[Alexx999]

@paulcbetts I believe that this change is _really_ needed because backwards-compatibility with some ancient software is actually _not_ a goal. Even worse, if this kind of backwards compatibility is enforced in such ways.
.NET should not enforce MAX_PATH if this is not required. It's legacy platform-specific stuff that shouldn't be present in modern platform.
If what you say is important - this should be removed from WinAPI so no one ever will create such file by accident.
This is leftover from FAT times, nothing more. And I've encountered situations where files in some directory could be read using _anything_ (well, of tools I used at a time) but my .NET program, which was frustrating at least. (that was part of build system, and there was lots of subprojects... so paths were long).

And that posts by BCL team state that they understood that enforcing MAX_PATH is not really long-term option even back in 2007.
Now, we don't have to support Windows XP (and even no Vista support), so we can just ignore issues that are present there and look if its doable using WIn7+ APIs

[anaisbetts]

I believe that this change is really needed because backwards-compatibility with some ancient software is actually not a goal.

This is leftover from FAT times, nothing more.

The Windows 8.1 SDK defines MAX_PATH to be 260 characters. That's _brand new_ software, written _today_, that has all of the issues that I described. The situation between 15 years ago when Win2k came out, and software written 10 minutes ago, is 100% identical - trying to fix this won't break ancient software, it'll break software written today too

[Alexx999]

Yes, Windows 8.1 SDK has MAX_PATH defined to 260.
But it doesn't make it any less legacy - .NET BCL still contains non-generic collections for the very same reason. That doesn't mean that anyone will use them (actually, I hope that no-one uses them this days).

And this doesn't mean that we shouldn't support something new only because old applications will not be able to open resulting files. Using this logic we might as well revert back to DOS and FAT16, because some old applications don't work well with Windows and non-8.3 names.

[Yomodo]

Whatever you choose, we're right here:
Your GitHub neighbor: https://github.com/alphaleonis/AlphaFS

[smbecker]

Yes Windows defines MAX_PATH at 260 but it at least gives you a means to go beyond that if you need to and has for a number of years. Today, core .Net gives you no options but to bypass System.IO almost entirely and drop down to P/Invoke. It is pointless to debate whether or not you should be able to go beyond MAX_PATH or whether that will break backwards compatibility because Windows already decided that you can. The question then becomes "how can you access those files from within .Net?" Today you can't from the BCL. Your only option is to use something like AlphaFS, which is a great library but it solves a problem that frankly shouldn't exist in .Net.

[JeremyKuhne]

To make sure I'm super clear- I'm not suggesting that .NET return \?\ paths. Just that it convert internally when talking to Win32 APIs that require it and strip the prefix off again.

I struggled with all of this years ago when I was writing the project system for Expression Blend. It was super common to get relative paths that were longer than 260 characters when combined with their base source location (notably where the project file is). While you could set the working directory, it isn't wise as any other thread can also do so (consistent bug for an inconsistent one isn't the best of trades).

I discovered that Win32's GetFullPathName could care less about the path length (at least on XP and later) and made my own call to attempt to get the path name below 260 before letting the System.IO.Path.GetFullPath method get it. (System.IO does a little _extra_ normalizing, primarily expanding all path segments that are expandable from 8.3, even if the full path doesn't exist.)

The ability of GetFullPathName to take long strings is a part of the answer to the normalization issue mentioned above. But even if this didn't work, the normalization that gets skipped by \?\ isn't that terribly hard to replicate.

I'm happy to contribute towards snuffing out the MAX_PATH and other Path.IO related problems.
I actually spoke at some length with Kim a year or so before she wrote those posts regarding my IO struggles. It has been a long time, but I'm _still_ anxious to see this get better. :)

[poizan42]

System.IO is in mscorlib. Shouldn't this issue be tracked at CoreCLR instead?

[Alexx999]

@poizan42 see dev guide for coreclr - it states that API review should happen in corefx, and this item definitely requires review

[poizan42]

@Alexx999 well I think it says that cases which requires changes to mscorlib will be addressed on a case-by-case basis.

[mikedn]

The relevant IO stuff is no longer in mscorlib for .NET Core, it's in System.IO.FileSystem and that's already part of this repository:

https://github.com/dotnet/corefx/tree/master/src/System.IO.FileSystem/src/System/IO

[JeremyKuhne]

What exactly is the deal with mscorlib System.IO vs. System.IO.FileSystem? Looking at the implementation in FileSystem is what caused me to kick off this thread here. System.IO.FileSystem seems mostly isolated from mscorlib's System.IO and implements most of the functionality (GetFullPath being one of the few calls I see going to System.IO.Path).

[ericstj]

MSCorlib contains the old desktop & phone implementations. We are trying to move as much as possible out of mscorlib that doesn't need to be there. Getting things out of mscorlib lets us ship them independently and enables more pay-for-play eventually when we can slim down mscorlib.

The long path work will need to touch the Path class primarily, which is still in mscorlib, but we do have plans to pull it out. I'd prefer that we wait for that to happen so that we can do this potentially destabilizing work in isolation without risking the phone & desktop platforms which ship the mscorlib implementation inbox.

This is all good discussion. I agree that we need to support long paths. It _will_ be done and _must_ be done in a way that doesn't break existing apps. We'll likely have an implementation that lets you put an app in "max path mode" that would put up guard rails needed if you are working with legacy components that aren't long path aware. We'd need to identify the trigger for such a mitigation (opt-in-api vs opt-out-api vs quirk) and the right scope (app, context, or instance). I'd be curious to hear folks opinion on this.

[JeremyKuhne]

I'm not sure how helpful the Framework can be for interacting with components that don't handle >MAX_PATH. Ultimately there is nothing preventing you from sending a really long path to some random component. While I can imagine that IO.FileSystem throwing PathToLong might help in some cases- that seems more likely to be accidental than intentional. If callers do robust error handling on System.IO, why wouldn't they do it on other, presumably more sensitive, components?

I guess one argument would be around mitigating buffer overruns in unmanaged components. If the framework throws you're probably less likely to get to the unmanaged component. That is, of course, presuming code usually hits some System.IO API beforehand.

One thought is that having iterators (GetFiles) never return long paths might be a reasonable compat option to have? Outside of that Path.Combine and Path.GetFullPath are the most likely choke points.

[Alexx999]

@JeremyKuhne I don't think that having restrictions on select APIs will really help - I don't see real difference between long path from iterator vs long path from some user input. It may go to some native code or be written to some location where it will cause problems anyway.
Moreover, in many cases having iterators with long path is required - for example some resource compilation when you go through directory tree and process what you find.

[JeremyKuhne]

@Alexx999 That is sort of my main point. I don't really think restrictions would be that useful as there are no guarantees people will call them on all paths before passing them on. And, to be clear, I would not recommend any of these APIs do anything but support long filenames by default. Just brainstorming possible opt-in behaviors.

Along the lines of mitigation- Kim pointed out that the Windows shell collapses paths to short equivalents to get them to fit under 260. While that is interesting, that behavior breaks the desire for paths to be normalized. In addition, it seems that this feature can be disabled and isn't on by default for all drives. (Anyone know of any links that describes the expected behavior? It seems like only the system drive gets this turned on in Win8- or perhaps there is a drive size gate?)

I think it would be useful to provide an API that tries to create these compacted paths for legacy component compat. In addition, potentially provide APIs that help you create hard links (with shorter paths)?

[mikedn]

I think the compatibility concerns may be a bit exaggerated. It's already possible to create files with long paths. Granted, the mechanism that can achieve this isn't common but I done this once or twice by accident and end up with files that cannot be opened and that's simply illogical.

``` C#
var longDirPath = @"D:\" + new string('x', 200);
var shortDirPath = @"D:\" + new string('x', 40);
var fileName = "filesdjhaskjdhkashdkjshdkjahsdkahskdhakjsdhkd.txt";

Directory.CreateDirectory(shortDirPath);
File.WriteAllText(Path.Combine(shortDirPath, fileName), "");

Directory.CreateDirectory(longDirPath);

// create a file with a long path...
Directory.Move(shortDirPath, Path.Combine(longDirPath, Path.GetFileName(shortDirPath)));

// ...enumeration works fine...
foreach (var filePath in Directory.EnumerateFiles(longDirPath, "*", SearchOption.AllDirectories)) {
// .. but the file cannot be read because FileStream can't open files with long paths
Console.WriteLine(File.ReadAllText(filePath));
}
```

[gulbanana]

it's worth mentioning that the need to deal with long paths is becoming more common. for example, consider visual studio kproj (asp.net 5) projects: they have built-in npm package installation. npm uses recursive paths for dependencies, such that installing even a single package can tip you over 260 - grunt-browserify, last I checked, included long enough paths that not even storing your solution at c:\ would save you. and that's a common package.

the implications of this right now are that your build tooling cannot be written in .net if using npm packages, and the git provider for VS doesn't correctly handle the directory structure. it's still a niche use case, but it's growing, and .net will have to provide a way to do this sooner or later.

[nvivo]

+1 to supporting long file paths. Lots of issues dealing with node_modules folders that go well beyond 260 chars.

[glen-84]

Has any work been done on this yet?

I'm not able to open my project in VS 2015 RC:

(note the grammatical errors as well)

Log:

=====================
2015/05/07 14:25:45
Crippling
System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: The item metadata "%(FullPath)" cannot be applied to the path "wwwroot\jspm_packages\github\glen-84\[email protected]\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\". D:\Programming\Projects\ASP.NET\Apex\src\Apex.Web\wwwroot\jspm_packages\github\glen-84\[email protected]\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInvalidOperation(String resourceName, Object[] args)
   at Microsoft.Build.Shared.ErrorUtilities.VerifyThrowInvalidOperation(Boolean condition, String resourceName, Object arg0, Object arg1, Object arg2)
   at Microsoft.Build.Shared.FileUtilities.ItemSpecModifiers.GetItemSpecModifier(String currentDirectory, String itemSpec, String definingProjectEscaped, String modifier, String& fullPath)
   at Microsoft.Build.Evaluation.BuiltInMetadata.GetMetadataValueEscaped(String currentDirectory, String evaluatedIncludeBeforeWildcardExpansionEscaped, String evaluatedIncludeEscaped, String definingProjectEscaped, String name, String& fullPath)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetBuiltInMetadataEscaped(String name)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadataEscaped(String metadataName)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadata(String metadataName)
   at Microsoft.VisualStudio.ProjectSystem.ProjectInstanceItemProperties.GetEvaluatedPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.PropertyPages.PageRuleBase.GetPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.GetItemMetadata(IImmutableDictionary`2 basis, IRule rule)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.BuildItemsSnapshot(IRule rule, IProjectCatalogSnapshot projectCatalogSnapshot, ImmutableDictionary`2 previousVersion, ImmutableList`1 recentItemRenames, ImmutableHashSet`1& addedItems, ImmutableHashSet`1& removedItems, ImmutableHashSet`1& changedItems, ImmutableDictionary`2& renamedItems)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.UpdateRuleSnapshotCore(IProjectVersionedValue`1 source, String ruleName, IRule rule, ImmutableDictionary`2 items, ImmutableDictionary`2 properties, ImmutableList`1 recentItemRenames, IProjectChangeDiff& diff)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectRuleSubscriptionServiceBase`1.UpdateSnapshotCoreAsync(TSource source, IImmutableSet`1 subscribedElements, IProjectVersionedValue`1 previousResult)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<UpdateSnapshotAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<>c__DisplayClass27_0.<<Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__72.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask`1.<JoinAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<<Initialize>b__27_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Utilities.DataflowExtensions.<>c__DisplayClass22_0`2.<<CreateSelfFilteringTransformBlock>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Package.UnconfiguredProjectHostBridge`3.<ApplyAsync>d__18.MoveNext()
   --- End of inner exception stack trace ---
---> (Inner Exception #0) System.InvalidOperationException: The item metadata "%(FullPath)" cannot be applied to the path "wwwroot\jspm_packages\github\glen-84\[email protected]\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\". D:\Programming\Projects\ASP.NET\Apex\src\Apex.Web\wwwroot\jspm_packages\github\glen-84\[email protected]\node_modules\gulp-clone\node_modules\gulp-util\node_modules\dateformat\node_modules\meow\node_modules\indent-string\node_modules\repeating\node_modules\
   at Microsoft.Build.Shared.ErrorUtilities.ThrowInvalidOperation(String resourceName, Object[] args)
   at Microsoft.Build.Shared.ErrorUtilities.VerifyThrowInvalidOperation(Boolean condition, String resourceName, Object arg0, Object arg1, Object arg2)
   at Microsoft.Build.Shared.FileUtilities.ItemSpecModifiers.GetItemSpecModifier(String currentDirectory, String itemSpec, String definingProjectEscaped, String modifier, String& fullPath)
   at Microsoft.Build.Evaluation.BuiltInMetadata.GetMetadataValueEscaped(String currentDirectory, String evaluatedIncludeBeforeWildcardExpansionEscaped, String evaluatedIncludeEscaped, String definingProjectEscaped, String name, String& fullPath)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetBuiltInMetadataEscaped(String name)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadataEscaped(String metadataName)
   at Microsoft.Build.Execution.ProjectItemInstance.TaskItem.GetMetadata(String metadataName)
   at Microsoft.VisualStudio.ProjectSystem.ProjectInstanceItemProperties.GetEvaluatedPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.PropertyPages.PageRuleBase.GetPropertyValueAsync(String propertyName)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.GetItemMetadata(IImmutableDictionary`2 basis, IRule rule)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.BuildItemsSnapshot(IRule rule, IProjectCatalogSnapshot projectCatalogSnapshot, ImmutableDictionary`2 previousVersion, ImmutableList`1 recentItemRenames, ImmutableHashSet`1& addedItems, ImmutableHashSet`1& removedItems, ImmutableHashSet`1& changedItems, ImmutableDictionary`2& renamedItems)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectEvaluationSubscriptionService.UpdateRuleSnapshotCore(IProjectVersionedValue`1 source, String ruleName, IRule rule, ImmutableDictionary`2 items, ImmutableDictionary`2 properties, ImmutableList`1 recentItemRenames, IProjectChangeDiff& diff)
   at Microsoft.VisualStudio.ProjectSystem.Designers.ProjectRuleSubscriptionServiceBase`1.UpdateSnapshotCoreAsync(TSource source, IImmutableSet`1 subscribedElements, IProjectVersionedValue`1 previousResult)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<UpdateSnapshotAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<>c__DisplayClass27_0.<<Initialize>b__1>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask.<JoinAsync>d__72.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Threading.JoinableTask`1.<JoinAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Designers.CustomizableBlockSubscriberBase`3.<<Initialize>b__27_0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.Utilities.DataflowExtensions.<>c__DisplayClass22_0`2.<<CreateSelfFilteringTransformBlock>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.ProjectSystem.VS.Implementation.Package.UnconfiguredProjectHostBridge`3.<ApplyAsync>d__18.MoveNext()<---

===================

Can this be made a higher priority?

[JeremyKuhne]

I'm starting on implementing this support. My initial plan of attack is roughly the following:

1.Allow extended syntax (\?) for common file operations (file info/navigation first, then stream handling) [CoreFX]
2.Allow longer than MAX_PATH for common file operations (will also unblock Unix) [CoreFX]
3.Unblock MAX_PATH constraints with assembly and resource loading (will also unblock Unix) [CoreCLR]
4.Allow using "normal" long paths (no extended syntax needed) to avoid the need to write custom code to use long paths on Windows

I'll be opening a variety of subtasks on this over the next day or so.

[be5invis]

https://github.com/npm/npm/issues/3697
I am waiting powershell to support UNC and long paths natively, since the IO is done by CLR.

[whoisj]

:+1: thank (deity), finally!

[mlidbom]

What will this mean for existing .net applications? Will they just start working correctly with long paths once this is implemented?

Of particular interest: What about visual studio and msbuild?

[OlegKi]

+1 to supporting long file paths.

I want just add very _simple realistic scenario_, which shows how files with long path names could be created. Let us you have the following structure of directories:

C:\Users\User1
C:\Users\User2
...

and both C:\Users and C:\Users\User1 have shares. The user User1 could have his home directory connected as H: in the login script. So if we saves document with the long name under H:\ (H:\longFilename.doc for example), then the document will be saved in C:\Users\User1 which have more long file name (C:\Users\User1\longFilename.doc). If one access the same file on C:\ or if one used U: connected to the shared directory C:\Users then one can have path with is longer as MAX_PATH characters.

So one have to support paths, which are longer as MAX_PATH. In any way one have to implement such support in all server based applications, in all software which makes backup or synchronization of data, all virus scanner and so on. It would be very helpful if .Net would support \\?\" prefixes in the filenames to have 32K limit existing in the file system.

[JeremyKuhne]

@mlidbom For existing .NET Desktop apps the hope is to make it "just work" for most apps. First priority, however, is getting implementation in place for .NET Core. We'll then look at exactly how and when we pull support back to the desktop.

For VS and MSBuild there may (will probably) be some work that has to be done for existing unmanaged code and P/Invokes.

[JeremyKuhne]

Extended syntax should now work throughout CoreFx (\?). :tada: (as of dotnet/corefx#2700)

[whoisj]

:+1:

[gulbanana]

that's great news!

[JeremyKuhne]

Now you can use long paths without extended syntax! (as of dotnet/corefx#3001 & dotnet/corefx#3010)

[be5invis]

:tada:

[whoisj]

:tada: :trophy:

[be5invis]

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

[whoisj]

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

These improvements will be shipped with the next version of the .NET Framework. This has little, to nothing to do with Windows 10. Long path names have been supported on Windows since, at least, Vista -- maybe XP.

[poizan42]

@whoisj: Long before XP... Here is an excerpt from the NT4.0 SDK:

_lpFileName_
Points to a null-terminated string that specifies the name of the object (file, pipe, mailslot, communications resource, disk device, console, or directory) to create or open.
If *lpFileName is a path, there is a default string size limit of MAX_PATH characters. This limit is related to how the CreateFile function parses paths.
Windows NT: You can use paths longer than MAX_PATH characters by calling the wide (W) version of CreateFile and prepending “\?\” to the path. The “\?\” tells the function to turn off path parsing. This lets you use paths that are nearly 32,000 Unicode characters long. You must use fully-qualified paths with this technique. This also works with UNC names. The “\?\” is ignored as part of the path. For example, “\?C:\myworld\private” is seen as “C:\myworld\private”, and “\?\UNC\tom_1\hotstuff\coolapps” is seen as “\tom_1\hotstuff\coolapps”.

The documentation for CreateFile in the NT3.1 SDK does not mention any size restrictions at all, though that's probably an oversight. Nt/ZwCreateFile isn't documented in the NT3.1 WDK, so not hint there about whether the underlying system supported long paths.

[smbecker]

Will this be included in .Net 4.6.1?

[weshaggard]

@smbecker No it will not be in .NET 4.6.1 however we do plan to get it added to a future release of full .NET framework, not sure which one yet.

[Ashtonian]

:+1:

[JeremyKuhne]

https://github.com/dotnet/coreclr/issues/1776 tracks the work for the runtime. CoreFx is fully implemented.

[kumarharsh]

So, when these improvements will be shipped with Windows 10, to bring long path support for existing applications, like Powershell?

These improvements will be shipped with the next version of the .NET Framework. This has little, to nothing to do with Windows 10. Long path names have been supported on Windows since, at least, Vista -- maybe XP.

@whoisj what people generally mean by "support for Windows 10" is that the Explorer and Powershell/Command-Prompt be able to open/delete these files or folders.

[whoisj]

@kumarharsh, while I appreciate what you're saying. if people are asking about the Windows Explorer Shell, then they should be looking for avenues to query the owners of the Windows Explorer Shell. The CRL/NetFX developers have little information and influence in that arena.

As an aside, if CoreFX supports long file names then it will not be long before PowerShell does as well.

[be5invis]

@whoisj I am care about the time: Since CoreFX now supports long path, when will the change published to end users, to make at least PowerShell work? The key question is the “when”.

[Jaykul]

As I understand it, the process for that is so complicated, that you're
never going to get an answer on a timeline.

First of all, because right now we have this awkward dual-mode .Net where
there's legacy .Net and then there's .Net Core, and products (like Asp.NET)
are *picking one *to ship against. PowerShell ships against legacy .Net,
but CoreFX is ... well, currently it's never been shipped in legacy.

So first they have to actually decide (how) to ship the new CoreFX code in
an upgrade to legacy .Net

Then they have to decide if it's a minor point release upgrade (like 4.5.1
or 4.5.2 were, versus the way 4.5 or 4.0 were)

Then they have to ship that version of .Net in Windows ... but in the
client world, that's a complete unknown: will they ever _force_ a new
version of .Net as a Windows 10 update? That's never been done outside of a
"R2" release, as far as I know?

Then, if they decided it was a major release of .Net, you have to get a new
version of PowerShell that's compiled to target that version of .Net ...

Of course, in the meantime, on Nano Server they're shipping a Core
PowerShell which is compiled against .Net Core and might get that long path
fix _incidentally,_ so it _could just show up_ on Nano Server ...

Joel "Jaykul" Bennett
http://HuddledMasses.org
PowerShell MVP, Software Dev., Speaker
585-563-9632 | Jaykul @ GitHub
https://github.com/jaykul | Twitter https://twitter.com/Jaykul | Linked
_In_ http://www.linkedin.com/in/jaykul/ http://PowerShellGroup.org

On Wed, Mar 23, 2016 at 4:37 AM, Belleve Invis [email protected]
wrote:

@whoisj https://github.com/whoisj I am care about the time: Since
CoreFX now supports long path, _when will the change published to end
users, to make at least PowerShell work_? The key question is the “when”.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/dotnet/corefx/issues/645#issuecomment-200247235

[bugith]

So the key question turns in “Why so many skilled-guys work so hard wasted-efforts for... _They_?” Do they hope a sweet?

[fostersanders]

Hello,

I had the same problem. I didn´t know what to do so I searched on the internet for some solutions.

And I read about [b]Long Path Tool[/b], which is a great tool in these type of cases. :o

It worked really well. Hope it works for you too :p

[JeremyKuhne]

@whoisj, @be5invis, @Jaykul The anniversary update (RS1) of Windows 10 has PowerShell support for long paths. 4.6.2 is part of that release and PowerShell turns on the .NET functionality. The only caveat is that Windows long path policy needs enabled as (for now) it is off by default. I go over these desktop details in one of my blog posts.

Anything that relies on versions of CoreFX will support long paths without any policy changes or other updates to the consuming code as we do the work to make the paths work with (\\?\). We didn't do that in 4.6.2 (yet) due to constraints around Code Access Security (canonicalizing paths when you allow device paths is difficult and costly due to the need to allow for paths that don't fully exist).

[whoisj]

@JeremyKuhne interesting that you were looking at this issue, I was just looking too.

Have you ever look at this nice write up on long-path support and the canonicalization nightmare?

https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html

[JeremyKuhne]

@whoisj No, I hadn't seen that write up- it is quite good. Thanks for sharing the link.

FYI, I did a dump of my own around that time as well:

Path Format Overview
Path Normalization
DOS to NT: A Path’s Journey

I've made the various .NET APIs much more friendly to the range of path formats. Mostly by not trying to second-guess what a "bad" path was. :)

[whoisj]

@JeremyKuhne I shared your with some folks looking to improve Msys, they seem to think it was rather valuable - so thanks for that!