Deno: Deno.run with piped stdout gets stuck in specific scenarios

[lucacasonato]

Reproduction

1. Generate test file:

echo "hello world" > file.txt
for i in {1..10}; do cat file.txt file.txt > file2.txt && mv file2.txt file.txt; done

1. create test.js

const proc = Deno.run({
  cmd: ["cat", "test.json"],
  stdout: "piped",
  stderr: "piped"
});

const status = await proc.status();

1. run test.ts

deno --allow-run test.js

1. Deno gets stuck on await proc.status().

[ry]

I can't repeat it.

[lucacasonato]

Managed to narrow it down to only happening when the file that is being opened is stored on an NTFS drive - this is really weird. It's not happening on macOS for me either, just on Linux + file.txt on NTFS drive. If the file.txt is on ext4 its fine. I'll try on Windows and report back. This might be a weird NTFS driver bug on Linux though - had some of those before.

[lucacasonato]

Checked on windows - not happening there. I'll close this and reopen if I have any more exact information about what is going on.

[lucacasonato]

Ok I have a reliable reproduction, on macOS now:

1. Save this file as lib.deno.d.ts: https://gist.github.com/lucacasonato/9a81ef88fa7f43ba8fc1297a04187ac9

2. Create test.js:

const proc = Deno.run({
  cmd: ["deno", "doc", "lib.deno.d.ts", "--json", "--reload"],
  stdout: "piped",
  stderr: "piped"
});

const status = await proc.status();

1. Run deno --allow-run test.js

2. Deno gets stuck on await proc.status().

[bartlomieju]

@lucacasonato does error happen if you await proc.output() and await proc.stderrOutput() before awaiting on status? It seems that unit_test_runner produces similar sized output and works without a problem.

[lucacasonato]

Everything works as expected when I add await proc.output() between the run call and the proc.status()call. await proc.stderrOutput() has no effect.

[youurayy]

This happens when the stdout is bigger than (?) 64KiB, consider:

const p = Deno.run({

  // this will work in any case:
  // cmd: [ 'curl', '-s', 'https://deno.land' ], // 4KB

  // this will work only if stdout is drained below:
  cmd: [ 'curl', '-s', 'https://en.wikipedia.org/wiki/Deno_(software)' ], // 79KB

  stdout: 'piped',
  stderr: 'piped'
})

console.log('here 1')

// await p.output() // drain the stdout here
console.log('here 1.1')

await p.stderrOutput()
console.log('here 1.2')

const { code } = await p.status()
console.log('here 2, code: ' + code)

You have to uncomment the // drain the stdout here line to make the Wikipedia link work.
Tested on Windows.
Run with deno run --allow-run test.js.

[lucacasonato]

I just ran into this again on the registry. It is a very annoying bug because it is so not logical. I think many people have and will run into it. I also ran into it for doc.deno.land. We should really address this one soon.

[exside]

@lucacasonato I think I did run into some form of it: https://github.com/denoland/deno/issues/7208#issuecomment-687523182 but can't explain it...

[ebebbington]

@lucacasonato I get the same issue when the sub process is asking for stdin (similar to what i mentioned before), it just hangs on the await p.status(), not sure in this scenario if it's possible to address or even a bug

[rok-star]

encountered same issue again! here is my work around:

const proc = Deno.run({ cmd, stderr: 'piped', stdout: 'piped' });
const [ stderr, stdout, status ] = await Promise.all([ proc.stderrOutput(), proc.output(), proc.status() ]);