Emscripten: opendir using NODERAWFS=1 always returns invalid argument

[JanWielemaker]

Using emscripten 1.38.18 and the following program

#include <dirent.h>
#include <stdio.h>

DIR *dir;

int
main(int argc, char **argv)
{ printf("Trying %s\n", argv[1]);

  if ( (dir=opendir(argv[1])) )
  { struct dirent *e;

    for(e=readdir(dir); e; e = readdir(dir))
    { printf("%s\n", e->d_name);
    }
  } else
  { perror("opendir");
    return 1;
  }

  return 0;
}

And compile using

emcc -s NODERAWFS=1 t.c

We get

node a.out.js /
Trying /
opendir: Invalid argument

(host is Ubuntu 18.04, amd64)

[JanWielemaker]

I guess the issue is probably in the open method in library_fs.js? I can edit that file, but what do I need to do to get my program using the modified code?

[rla]

Recompilation will include the modified code.

Anyway, I analysed the filesystem layer code and found couple of issues:

Musl sets O_DIRECTORY on open() call in readdir. It also sets O_CLOEXEC but this is handled in NODEFS.flagsForNode(). NODEFS.flagsForNode() does not handle O_DIRECTORY as it is missing in the flags mapping in the same file. Adding it into the mapping object will solve it.

NODERAWFS readdir does return nothing. Looks like just a missing return. After fixing it, syscall220 (SYS_getdents64) has issues:

var child = FS.lookupNode(stream.node, name)

This has stream.node undefined and FS.lookupNode does not seem to like it.

This is caused by open() in NODERAWFS not adding node to stream:

var stream = { fd: fd, nfd: nfd, position: 0, path: path, flags: flags, seekable: true };

After this I gave up. I tried various ways to add a filesystem node here including FS.lookupPath. Someone more familiar with the Emscripten FS design might help to solve it.

[IngwiePhoenix]

I just ran into this while inspecting an issue I have had compiling a CLI program which used opendir() and friends. While -s NODERAWFS=1 will work with single files, it won't work for reading folders, which is very unfortunate.

Has there been a fix to this yet?

$ emcc --version
emcc (Emscripten gcc/clang-like replacement) 1.38.25 ((unknown revision))
Copyright (C) 2014 the Emscripten authors (see AUTHORS.txt)
This is free and open source software under the MIT license.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

[Milerius]

we should have an update from the team about it no ?

[kripken]

It would be great if someone has time to look into this! I looked a little and wrote this:

https://github.com/emscripten-core/emscripten/compare/raw?expand=1

it fixes one issue, see the library_noderawfs.js part - the return was missing. But there is more that needs to be fixed. If someone continues the investigation, let me know if you have any questions on the emscripten side that I can help with!

[Milerius]

Is it on upstream already ? I will give it a try tomorrow if it's the case

[kripken]

No, that's a side branch, "raw".

[Milerius]

is it possible to build emscripten with the SDK tool's on a specific branch ? like "raw"

[Milerius]

So I can try it

[kripken]

The sdk can't do it for you, but you can do this: emsdk install and activate tot-upstream (tip of tree build, very latest), then get emscripten from github, checkout that branch, and use the emcc from that checkout. It will use binaries for llvm etc. from the tot build, but the specific emscripten you use.

[AlexanderOMara]

I took a stab at trying to finish fixing this issue, but I'm kinda at a loss for what's supposed to be different.

First issue, the if (name[0] === '.') { statement doesn't make much sense, because a file name that starts with a . isn't necessarily a file.

Second issue, the stream variable passed to FS.lookupNode doesn't have the mode and node_ops properties that FS.nodePermissions, FS.mayLookup, and probably some other functions expect.

https://github.com/emscripten-core/emscripten/blob/a8dfdd0be4989bdf76e668f82844080b14ee58f9/src/library_syscall.js#L1040-L1059

[AlexanderOMara]

I ended up poking at it more and using @kripken's raw branch as a basis, I think I have implemented the missing pieces, and removed the broken one.

https://github.com/emscripten-core/emscripten/pull/10162

I was able to compile and run my test program successfully.

#include <stdio.h>
#include <dirent.h>
#include <string.h>

int main() {
    DIR * dir;
    struct dirent * dp;
    char * file_name;
    dir = opendir("sub/dir");
    printf("%p\n", dir);
    while ((dp=readdir(dir)) != NULL) {
        printf("%s\n", dp->d_name);
    }
    closedir(dir);
    return 0;
}

I'm totally new to the emscripten codebase, so let me know if there is anything I should do to improve it.

[stale[bot]]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

[sbc100]

bump.. I guess this will get fixed once #7487 lands?