Runtime: Directory.EnumerateDirectories doesn't return directory list on rhel7.2

Created on 13 Apr 2018 · 26Comments · Source: dotnet/runtime

_From @jainaashish on April 7, 2018 0:18_

I've created a simple application to demonstrate the issue where Directory.EnumerateDirectories api doesn't return the list of child directories on rhel 7.2, although it works fine on Windows or Ubuntu.

Sample Project - https://github.com/jainaashish/CoreClrIssue/tree/master/EnumerateDirectoriesIssue/EnumerateDirectoriesIssue

_Copied from original issue: dotnet/coreclr#17465_

area-System.IO needs more info

Source

jkotas

All 26 comments

_From @danmosemsft on April 7, 2018 2:14_

@jeremykuhne @anipik

jkotas on 13 Apr 2018

@Anipik can you take a look?

@jainaashish sample output would be great. Presumably you were running against public Core 2.0 bits?

JeremyKuhne on 13 Apr 2018

@JeremyKuhne I just checked on rhel7 and its Directory.Enumerate Directories are returning the list of directories for corefx repo.
Test output

  Discovering: System.IO.FileSystem.Tests
  Discovered:  System.IO.FileSystem.Tests
  Starting:    System.IO.FileSystem.Tests
     System.IO.Tests.EnumerableTests.EnumerateDirectories_Rhel [FAIL]
        Assert.Equal() Failure
        Expected: String[] []
        Actual:   FileSystemEnumerable<String> ["/tmp/EnumerableTests_0olhl43x.apn/EnumerateDirecto"..., "/tmp/EnumerableTests_0olhl43x.apn/EnumerateDirecto"...]
        Stack Trace:
           /git/corefx/src/System.IO.FileSystem/tests/Directory/EnumerableTests.cs(54,0): at System.IO.Tests.EnumerableTests.EnumerateDirectories_Rhel()
  Finished:    System.IO.FileSystem.Tests

  === TEST EXECUTION SUMMARY ===
/git/corefx/Tools/tests.targets(492,5): warning :    System.IO.FileSystem.Tests  Total: 3993, Errors: 0, Failed: 1, Skipped: 0, Time: 25.061s [/git/corefx/src/System.IO.FileSystem/tests/System.IO.FileSystem.Tests.csproj]
  /git/corefx/src/System.IO.FileSystem/tests
  ----- end 15:31:59 ----- exit code 1 ----------------------------------------------------------
  Looking around for any Linux dump...
  ... found no dump in /git/corefx/bin/tests/System.IO.FileSystem.Tests/netcoreapp-Linux-Debug-x64
/git/corefx/Tools/tests.targets(492,5): warning MSB3073: The command "/git/corefx/bin/tests/System.IO.FileSystem.Tests/netcoreapp-Linux-Debug-x64/RunTests.sh /git/corefx/bin/testhost/netcoreapp-Linux-Debug-x64/" exited with code 1. [/git/corefx/src/System.IO.FileSystem/tests/System.IO.FileSystem.Tests.csproj]
/git/corefx/Tools/tests.targets(500,5): error : One or more tests failed while running tests from 'System.IO.FileSystem.Tests' please check /git/corefx/bin/tests/System.IO.FileSystem.Tests/netcoreapp-Linux-Debug-x64/testResults.xml for details! [/git/corefx/src/System.IO.FileSystem/tests/System.IO.FileSystem.Tests.csproj]

Here is the version for my rhel machine

[root@75ea72ed8713 corefx]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.2 (Maipo)
[root@75ea72ed8713 corefx]#

Anipik on 13 Apr 2018

@Anipik what are you trying to say? You're showing me a failing test that isn't in the repo. Is there a failure or not? Have you tried the repro project on both 2.0 and 2.1? Can you share output? If you can't repro please include all of the info, mark this as blocked, and move it to Future for now.

JeremyKuhne on 13 Apr 2018

@JeremyKuhne the error is reproducable in 2.0 but not in 2.1.
I added in a new test in the both repo (master(2.1) and release/2.0.0). https://github.com/Anipik/corefx/blob/enumerate/src/System.IO.FileSystem/tests/Directory/EnumerableTests.cs#L48

It returned the list of the subDirectories in 2.1

FileSystemEnumerable<String> ["/tmp//EnumerableTests_0olhl43x.apn/EnumerateDirecto"..., "/tmp/EnumerableTests_0olhl43x.apn/EnumerateDirecto"...]

whereas in 2.0 it returned

FileSystemEnumerable<String> []

Anipik on 14 Apr 2018

None of the other tests failed? Or are other tests failing in 2.0? If none of the other tests are failing, what is different about your test that doesn't already get covered?

JeremyKuhne on 14 Apr 2018

None Of the other tests failed in 2.0.0. I am just trying to figure what is different about my test

Anipik on 14 Apr 2018

None Of the other tests failed in 2.0.0. I am just trying to figure what is different about my test

Thanks for the clarification. I've marked this as 2.0.x for now unless you find an issue with the 2.1 implementation.

JeremyKuhne on 14 Apr 2018

if I modify the test to

  [Fact]
  [SkipOnTargetFramework(TargetFrameworkMonikers.NetFramework)]
  public void EnumerateDirectories_Rhel()
  {
        DirectoryInfo rootDirectory = Directory.CreateDirectory(GetTestFilePath());
        rootDirectory.CreateSubdirectory("hello");
        rootDirectory.CreateSubdirectory("hi");
        Assert.Equal(new string[] { }, Directory.EnumerateDirectories(rootDirectory.FullName));
    }

Then the output is same as in 2.1 i.e

FileSystemEnumerable<String> ["/tmp//EnumerableTests_0olhl43x.apn/EnumerateDirecto"..., "/tmp/EnumerableTests_0olhl43x.apn/EnumerateDirecto"...]

Anipik on 14 Apr 2018

We change string.empty to "*" in 2.1 at https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEnumerableFactory.cs#L44
whereas in 2.0 we were not converting it to anything
https://github.com/dotnet/corefx/blob/release/2.0.0/src/System.IO.FileSystem/src/System/IO/PathHelpers.Windows.cs#L87

Anipik on 14 Apr 2018

@anipik Please try and repro the reported customer issue as linked. I've included it here for your convenience:

``` C#
static void Main(string[] args)
{
if (args == null || args.Length == 0)
{
Console.WriteLine("Please pass a dirextory path");
return;
}

var dir = args[0];

Console.WriteLine("Dir Name : " + dir);

Console.WriteLine("Directory exists : " + Directory.Exists(dir));

if (Directory.Exists(dir))
{
    Console.WriteLine("Below is the list of child directories found in this directory:");

    foreach (var subDir in Directory.EnumerateDirectories(dir))
    {
        Console.WriteLine(subDir);
    }
}

}
```

Looking at 2.0 I see that we definitely add the *:

https://github.com/dotnet/corefx/blob/release/2.0.0/src/System.IO.FileSystem/src/System/IO/Directory.cs#L337-L344

Moving back into 2.1 until you've validated the reported code.

JeremyKuhne on 15 Apr 2018

I am not able to reproduce it in 2.0.0. I installed dotnet 2 on rhel7.2, cloned his repo and did dotnet run on the directory

[root@5e1ddb79f054 EnumerateDirectoriesIssue]# dotnet run /git
Dir Name : /git
Directory exists : True
Below is the list of child directories found in this directory:
/git/CoreClrIssue
[root@5e1ddb79f054 EnumerateDirectoriesIssue]#

Anipik on 17 Apr 2018

@jainaashish, we can't reproduce the problem as you've described it. Is there anything else you can give us that might help?

JeremyKuhne on 17 Apr 2018

@JeremyKuhne I can repro it on the msbuild repo: https://github.com/dotnet/cli/issues/8819#issuecomment-386130496

cdmihai on 2 May 2018

@cdmihai, can you repro it with a standalone .NET Core app? Can you work with @Anipik to try and get us a repro we can debug?

JeremyKuhne on 3 May 2018

I don't have a rhel72 machine so if someone can provide me a vm, then I can try to give the repro. Send me the details at asja @ Microsoft . com

jainaashish on 3 May 2018

@Anipik can you please make this your first priority as we need to know whether there is a 2.1 bug.

danmosemsft on 3 May 2018

👍1

There is some suspicion this may be the case for the linux failing leg of https://github.com/dotnet/core-setup/pull/4115. Ping me offline and I can share a repo machine.

weshaggard on 3 May 2018

So here is the output of the same sample code ran on rhel72 machine.

[dotnet-bot@rhel72-20171003-238400 project3]$ ../.dotnet/dotnet EXEC EnumerateDirectoriesIssue.dll /home/dotnet-bot/.nuget/packages/microsoft.netcore.platforms
Dir Name : /home/dotnet-bot/.nuget/packages/microsoft.netcore.platforms
Directory exists : True
Below is the list of child directories found in this directory:

And the content of the same folder:

[dotnet-bot@rhel72-20171003-238400 project3]$ cd /home/dotnet-bot/.nuget/packages/microsoft.netcore.platforms/
[dotnet-bot@rhel72-20171003-238400 microsoft.netcore.platforms]$ ls -al
total 28
drwxrwxr-x.   6 dotnet-bot dotnet-bot   85 Oct  3  2017 .
drwxrwxr-x. 140 dotnet-bot dotnet-bot 8192 Oct  3  2017 ..
drwxrwxr-x.   3 dotnet-bot dotnet-bot 4096 Oct  3  2017 1.0.1
drwxrwxr-x.   3 dotnet-bot dotnet-bot 4096 Oct  3  2017 1.0.1-rc4-24217-01
drwxrwxr-x.   3 dotnet-bot dotnet-bot 4096 Oct  3  2017 1.1.0
drwxrwxr-x.   3 dotnet-bot dotnet-bot 4096 Oct  3  2017 2.0.0-preview1-25305-02
[dotnet-bot@rhel72-20171003-238400 microsoft.netcore.platforms]$

dotnet --info

[dotnet-bot@rhel72-20171003-238400 project3]$ ../.dotnet/dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   2.2.100-preview1-008636
 Commit:    6c9942bae6

Runtime Environment:
 OS Name:     rhel
 OS Version:  7
 OS Platform: Linux
 RID:         rhel.7-x64
 Base Path:   /home/dotnet-bot/.dotnet/sdk/2.2.100-preview1-008636/

Host (useful for support):
  Version: 2.2.0-preview1-26503-05
  Commit:  f6bf0a83c6

.NET Core SDKs installed:
  2.2.100-preview1-008636 [/home/dotnet-bot/.dotnet/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.0-preview2-30475 [/home/dotnet-bot/.dotnet/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.0-preview2-30475 [/home/dotnet-bot/.dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.0-preview3-26411-06 [/home/dotnet-bot/.dotnet/shared/Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.0-preview1-26503-05 [/home/dotnet-bot/.dotnet/shared/Microsoft.NETCore.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

jainaashish on 3 May 2018

@JeremyKuhne I can also repro it with a standalone netcoreapp:

[micodoba@micodoba-az2-rhel enumeration]$ ~/projects/msbuild/artifacts/.dotnet/2.1.300-rc1-008673/dotnet restore
  Restoring packages for /home/micodoba/projects/enumeration/enumeration.csproj...
/home/micodoba/projects/enumeration/enumeration.csproj : error NU1102: Unable to find package Microsoft.NETCore.App with version (>= 2.1.0-rc1-26428-03)
/home/micodoba/projects/enumeration/enumeration.csproj : error NU1102:   - Found 31 version(s) in nuget.org [ Nearest version: 2.1.0-preview2-26406-04 ]
  Generating MSBuild file /home/micodoba/projects/enumeration/obj/enumeration.csproj.nuget.g.props.
  Generating MSBuild file /home/micodoba/projects/enumeration/obj/enumeration.csproj.nuget.g.targets.
  Restore failed in 198.91 ms for /home/micodoba/projects/enumeration/enumeration.csproj.
[micodoba@micodoba-az2-rhel enumeration]$ cat obj/project.assets.json | head -10
{
  "version": 3,
  "targets": {
    ".NETCoreApp,Version=v2.1": {}
  },
  "libraries": {},
  "projectFileDependencyGroups": {
    ".NETCoreApp,Version=v2.1": [
      "Microsoft.NETCore.App >= 2.1.0-rc1-26428-03"
    ]
[micodoba@micodoba-az2-rhel enumeration]$ cat enumeration.csproj
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp2.1</TargetFramework>
  </PropertyGroup>

</Project>

cdmihai on 4 May 2018

The problem is we are seeing directories as files. We call readdir and check whether the d_type is DT_DIR.

The issue is that these machines have XFS file system, on which readdir returns a d_type of DT_UNKNOWN for both files and directories.

In 2.0 encountering DT_UNKNOWN we then fell back to calling lstat to find out whether it's a directory - as the docs for readdir say we should.
https://github.com/dotnet/corefx/blob/release/2.0.0/src/System.IO.FileSystem/src/System/IO/UnixFileSystem.cs#L614-L630

In 2.1 we only do this for DT_LNK.
https://github.com/dotnet/corefx/blob/AddAlpine3.8/src/System.IO.FileSystem/src/System/IO/Enumeration/FileSystemEntry.Unix.cs#L47

danmosemsft on 4 May 2018

🚀1

Need 2.1 port

danmosemsft on 4 May 2018

@Anipik what is the filesystem on the drive you were testing on? Try df

danmosemsft on 4 May 2018

For future reference, this only reproes on XFS if ftype=0, eg in my case:

$ xfs_info /
meta-data=/dev/sda2              isize=256    agcount=4, agsize=1934016 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=0        finobt=0
data     =                       bsize=4096   blocks=7736064, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=0
log      =internal               bsize=4096   blocks=3777, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

This is set at the time the volume is created with mkfs.

danmosemsft on 4 May 2018

👍1

I used the Docker image available in MS repository for RHel7.2 . I will post the file system as soon as I am able to use my desktop

Anipik on 5 May 2018

@anipik I think we can set this issue aside now. It is fully understood and fixed.

danmosemsft on 6 May 2018

👍1

Was this page helpful?

0 / 5 - 0 ratings