React-native: [CI] Run tests on Windows

Created on 18 Jul 2018  ·  33Comments  ·  Source: facebook/react-native

For Discussion

Our suite of tests does not currently run on Windows machines. This results in a blind spot that has allowed issues affecting Windows users to creep into the code base (https://github.com/facebook/metro/issues/181).

Circle CI does not support running jobs on Windows at the moment. I'd like to open a discussion and get feedback from the community on how to proceed.

Windows Locked Discussion 🔩Test Infrastructure
Source
picture hramos
❤6 👍4

Most helpful comment

Please try past https://github.com/facebook/react-native/commit/e327f88f028579023ce93f04d66b9606ec476547, hopefully that adresses issues with the fs mock that were happening on Windows.

picture jeanlauliac on 17 Sep 2018
👍2

All 33 comments

A while back @matthargett suggested that to fix the CircleCI "Windows absence" we could integrate AppVeyor CI

kelset picture kelset on 18 Jul 2018
👍1

Recently babel add Appveyor too.

gengjiawen picture gengjiawen on 18 Jul 2018

I agree that AppVeyor CI would be a good alternative and it looks like from their homepage that Facebook is already one of their great customers.

As starting point, maybe we can introduce some basic Windows tests like:

  1. Clone the repository and use the cli (from local repository) to init a fresh React Native project.
  2. From this fresh project, bundle the JavaScript (local-cli/cli.js bundle).

With this we should be able to catch issues like #20015.

charpeni picture charpeni on 18 Jul 2018
👍1

I have created a pr https://github.com/facebook/react-native/pull/20281, hope it helps.

gengjiawen picture gengjiawen on 18 Jul 2018

Yeah I think that's a "good enough" start, and then we can ask Windows dev to jump in and add more tests as the platform becomes more widely used.

kelset picture kelset on 18 Jul 2018

Okay, the basic setup is okay, and we can wait for https://github.com/facebook/metro/issues/181 to be fixed.
Current build log: https://ci.appveyor.com/project/gengjiawen/react-native/build/job/x476xw6q46hg5h3e.

gengjiawen picture gengjiawen on 18 Jul 2018

So what test should on windows platform run ? Aside from basic android build. I also add some npm run test, but some test case are failed.

Personally I want to add metro debug server bundle works as expected.

gengjiawen picture gengjiawen on 20 Jul 2018

@hramos Can you check the appveyor config or the badge url. Looks like README.md now has not show any build in appveyor.

gengjiawen picture gengjiawen on 24 Jul 2018

The windows issue (https://github.com/facebook/react-native/issues/20353) still exists on master branch now. So we do need a test case to test the request to http://localhost:8081/RNTester/js/RNTesterApp.android.delta?platform=android&dev=true&minify=false works on windows.

I will see what I can do after my two prs got merged.

gengjiawen picture gengjiawen on 24 Jul 2018

Currently the android build works on appveyor, but we have some jest test failure.

Test Suites: 16 failed, 2 skipped, 90 passed, 106 of 108 total
Tests:       54 failed, 4 skipped, 545 passed, 603 total
Snapshots:   42 passed, 42 total
Time:        12.803s

Most of those related to On a win32 FS, the options' cwd() must return a valid win32 absolute path.

Error: On a win32 FS, the options' `cwd()` must return a valid win32 absolute path. This happened while trying to resolve: `c:\empty`

      33 |       continue;
      34 |     }
    > 35 |     fs.mkdirSync(entPath);
         |        ^
      36 |     mockDir(entPath, ent);
      37 |   }
      38 | }

      at MemoryFs._resolve (node_modules/metro-memory-fs/src/index.js:830:15)
      at MemoryFs.mkdirSync (node_modules/metro-memory-fs/src/index.js:994:5448)
      at mockDir (local-cli/__mocks__/fs.js:35:8)
      at MemoryFs.setMockFilesystem [as __setMockFilesystem] (local-cli/__mocks__/fs.js:17:3)
      at Object.<anonymous> (local-cli/core/__tests__/android/getDependencyConfig.spec.js:23:19)

cc @rafeca

gengjiawen picture gengjiawen on 2 Sep 2018

Just an FYI: I just enabled facebook/react-native on Appveyor. Builds should be running now.

hramos picture hramos on 4 Sep 2018

I have a fix for the fs-related failure, it's due to a problem in the way the test are setup and not specific to metro-memory-fs. Will send soon.

jeanlauliac picture jeanlauliac on 11 Sep 2018
👍2

@jeanlauliac Hopefully we can fix the windows ci via your pr :)

gengjiawen picture gengjiawen on 11 Sep 2018

Please try past https://github.com/facebook/react-native/commit/e327f88f028579023ce93f04d66b9606ec476547, hopefully that adresses issues with the fs mock that were happening on Windows.

jeanlauliac picture jeanlauliac on 17 Sep 2018
👍2

@jeanlauliac Good job. Test case fail number on windows from 54 to 7.

gengjiawen picture gengjiawen on 19 Sep 2018
❤1

I have create a pr to fix remaining failed case: https://github.com/facebook/react-native/pull/21203.
With the pr, we only have one failed case:

 FAIL  local-cli/link/__tests__/ios/writePlist.spec.js
  ● Test suite failed to run

    TypeError: require(...).join is not a function

      24 | const writePlist = require('../../ios/writePlist');
      25 | 
    > 26 | const projectPath = path.join(__dirname, '../../__fixtures__/project.pbxproj');
         |                                   ^
      27 | const infoPlistPath = path.join(__dirname, '../../__fixtures__/Info.plist');
      28 | 
      29 | fs.readFileSync = jest.fn(() => readFileSync(projectPath).toString());

cc @jeanlauliac Can you help on this ?

gengjiawen picture gengjiawen on 19 Sep 2018

Yeah that's because path doesn't get mocked early enough. Changeset on the way.

jeanlauliac picture jeanlauliac on 24 Sep 2018

9c242c8a2fd57de929b7f1357d160326c86fc638 should fix the issue. Let me know.

jeanlauliac picture jeanlauliac on 24 Sep 2018

@jeanlauliac It looks like cause more problem, now it have three fails, plz check https://ci.appveyor.com/project/facebook/react-native/branch/master/job/9s95x3eb1f937qrk.

gengjiawen picture gengjiawen on 26 Sep 2018

The errors on the report are not related to the JS tests are they? It looks like build errors.

jeanlauliac picture jeanlauliac on 27 Sep 2018

Nope, it's test case.

gengjiawen picture gengjiawen on 27 Sep 2018

We're not looking at the same thing then. In the link you sent, here's the only error I can see:

FAILURE: Build failed with an exception.
* What went wrong:
A problem occurred configuring root project 'react-native'.
> Could not resolve all files for configuration ':classpath'.
   > Could not resolve com.sun.xml.fastinfoset:FastInfoset:1.2.13.
     Required by:
         project : > com.android.tools.build:gradle:3.1.4 > com.android.tools.build:gradle-core:3.1.4 > com.android.tools.build:builder:3.1.4 > com.android.tools:sdklib:26.1.4 > com.android.tools:repository:26.1.4 > org.glassfish.jaxb:jaxb-runtime:2.2.11
      > Could not resolve com.sun.xml.fastinfoset:FastInfoset:1.2.13.
         > Could not get resource 'https://jcenter.bintray.com/com/sun/xml/fastinfoset/FastInfoset/1.2.13/FastInfoset-1.2.13.pom'.
            > Could not GET 'https://jcenter.bintray.com/com/sun/xml/fastinfoset/FastInfoset/1.2.13/FastInfoset-1.2.13.pom'. Received status code 504 from server: Gateway Time-out
   > Could not resolve javax.xml.bind:jaxb-api:2.2.12-b140109.1041.
     Required by:
         project : > com.android.tools.build:gradle:3.1.4 > com.android.tools.build:gradle-core:3.1.4 > com.android.tools.build:builder:3.1.4 > com.android.tools:sdklib:26.1.4 > com.android.tools:repository:26.1.4 > org.glassfish.jaxb:jaxb-runtime:2.2.11 > org.glassfish.jaxb:jaxb-core:2.2.11
      > Skipped due to earlier error
   > Could not resolve org.glassfish.jaxb:txw2:2.2.11.
     Required by:
         project : > com.android.tools.build:gradle:3.1.4 > com.android.tools.build:gradle-core:3.1.4 > com.android.tools.build:builder:3.1.4 > com.android.tools:sdklib:26.1.4 > com.android.tools:repository:26.1.4 > org.glassfish.jaxb:jaxb-runtime:2.2.11 > org.glassfish.jaxb:jaxb-core:2.2.11
      > Skipped due to earlier error
   > Could not resolve com.sun.istack:istack-commons-runtime:2.21.
     Required by:
         project : > com.android.tools.build:gradle:3.1.4 > com.android.tools.build:gradle-core:3.1.4 > com.android.tools.build:builder:3.1.4 > com.android.tools:sdklib:26.1.4 > com.android.tools:repository:26.1.4 > org.glassfish.jaxb:jaxb-runtime:2.2.11 > org.glassfish.jaxb:jaxb-core:2.2.11
      > Skipped due to earlier error
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 1m 34s
Command exited with code 1

What am I missing? 🙂

jeanlauliac picture jeanlauliac on 27 Sep 2018

This link https://ci.appveyor.com/project/Facebook/react-native/build/1.0.397/job/kya41akupssp9wdf

gengjiawen picture gengjiawen on 27 Sep 2018

Here's the error report from the above link:

Summary of all failing tests
FAIL Libraries/Interaction/__tests__/InteractionManager-test.js
  ● promise tasks › should also work with a deadline
    expect(received).toBe(expected) // Object.is equality
    Expected: 1
    Received: 0
      14 | 
      15 | function expectToBeCalledOnce(fn) {
    > 16 |   expect(fn.mock.calls.length).toBe(1);
         |                                ^
      17 | }
      18 | 
      19 | describe('InteractionManager', () => {
      at expectToBeCalledOnce (Libraries/Interaction/__tests__/InteractionManager-test.js:16:32)
      at Timeout._onTimeout (Libraries/Interaction/__tests__/InteractionManager-test.js:294:7)
FAIL local-cli/link/__tests__/ios/writePlist.spec.js
  ● ios::writePlist › should write a `.plist` file
    ENOENT: no such file or directory, open 'C:\projects\__fixtures__\project.pbxproj'
      27 | const infoPlistPath = path.join(__dirname, '../../__fixtures__/Info.plist');
      28 | 
    > 29 | fs.readFileSync = jest.fn(() => readFileSync(projectPath).toString());
         |                                                          ^
      30 | 
      31 | const {writeFileSync} = fs;
      32 | fs.writeFileSync = jest.fn(writeFileSync);
      at Object.<anonymous> (local-cli/link/__tests__/ios/writePlist.spec.js:29:58)
      at pbxProject.Object.<anonymous>.pbxProject.parseSync (node_modules/xcode/lib/pbxProject.js:44:28)
      at Object.<anonymous> (local-cli/link/__tests__/ios/writePlist.spec.js:43:13)
  ● ios::writePlist › should write a `.plist` file
    ENOENT: `/Basic/Info.plist`: no such file or directory
      27 |   // indentation of the plist.
      28 |   // Ref: https://github.com/facebook/react-native/issues/11668
    > 29 |   return fs.writeFileSync(
         |                        ^
      30 |     plistPath,
      31 |     plistParser.build(plist, {indent: '\t', offset: -1}) + '\n',
      32 |   );
      at makeError (node_modules/metro-memory-fs/src/index.js:1239:15)
      at MemoryFs._resolveEnt (node_modules/metro-memory-fs/src/index.js:877:13)
      at MemoryFs._resolve (node_modules/metro-memory-fs/src/index.js:850:12)
      at MemoryFs._open (node_modules/metro-memory-fs/src/index.js:736:10)
      at Object.writeFileSync (node_modules/metro-memory-fs/src/index.js:994:5210)
      at writePlist (local-cli/link/ios/writePlist.js:29:24)
      at Object.<anonymous> (local-cli/link/__tests__/ios/writePlist.spec.js:49:49)
  ● ios::writePlist › when plistPath is null it should return null
    ENOENT: no such file or directory, open 'C:\projects\__fixtures__\project.pbxproj'
      27 | const infoPlistPath = path.join(__dirname, '../../__fixtures__/Info.plist');
      28 | 
    > 29 | fs.readFileSync = jest.fn(() => readFileSync(projectPath).toString());
         |                                                          ^
      30 | 
      31 | const {writeFileSync} = fs;
      32 | fs.writeFileSync = jest.fn(writeFileSync);
      at Object.<anonymous> (local-cli/link/__tests__/ios/writePlist.spec.js:29:58)
      at pbxProject.Object.<anonymous>.pbxProject.parseSync (node_modules/xcode/lib/pbxProject.js:44:28)
      at Object.<anonymous> (local-cli/link/__tests__/ios/writePlist.spec.js:43:13)
  ● ios::writePlist › when plistPath is null it should return null
    expect(jest.fn()).not.toHaveBeenCalled()
    Expected mock function not to be called but it was called with:
      ["/Basic/Info.plist", "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
    <!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">
    <plist version=\"1.0\">
    <dict>
        <key>CFBundleDevelopmentRegion</key>
        <string>en</string>
        <key>UISupportedInterfaceOrientations</key>
        <array>
            <string>UIInterfaceOrientationPortrait</string>
        </array>
    </dict>
    </plist>
    "]
      55 |     plistPath = null;
      56 |     expect(writePlist(project, '/', plist)).toBeNull();
    > 57 |     expect(fs.writeFileSync).not.toHaveBeenCalled();
         |                                             ^
      58 |   });
      59 | });
      60 | 
      at Object.<anonymous> (local-cli/link/__tests__/ios/writePlist.spec.js:57:45)
Test Suites: 2 failed, 2 skipped, 104 passed, 106 of 108 total
Tests:       3 failed, 4 skipped, 601 passed, 608 total
Snapshots:   42 passed, 42 total
Time:        31.172s
Ran all test suites.
npm ERR! Test failed.  See above for more details.

A few of these appear to be issues reading from the filesystem on Windows. For example:

ENOENT: no such file or directory, open 'C:\projects\__fixtures__\project.pbxproj'
hramos picture hramos on 27 Sep 2018

@jeanlauliac ping

gengjiawen picture gengjiawen on 2 Oct 2018

The changeset partly fixes things only, I think :) for the rest I'd have to grab a windows VM and try out.

As for the InteractionManager-test.js this may be flakiness and I do not believe this is related to the fs mock. I have no context on that test I'm afraid.

jeanlauliac picture jeanlauliac on 10 Oct 2018

@jeanlauliac Glad to hear, I hope we can fix this soon. Maybe @rafeca can help you with the windows part, he fix some metro issue on windows machine.

gengjiawen picture gengjiawen on 14 Oct 2018

Since this issue was created, Microsoft released this:

https://azure.microsoft.com/en-ca/blog/announcing-azure-pipelines-with-unlimited-ci-cd-minutes-for-open-source/

Which may be an option if appveyor is being too troublesome. That InteractionManager issue is really odd, because I cannot reproduce it on my windows VM at all.

empyrical picture empyrical on 17 Oct 2018

@empyrical Windows 10 ? I have the issue on my win10 too.

gengjiawen picture gengjiawen on 17 Oct 2018

Appveyor is still red -- what do you folks think about removing Appveyor.yml from the master branch until this is fixed? It makes it hard to see when Circle CI tests have broken when looking at the list of recent commits. I hope Appveyor does let you run tests on a branch that includes Appveyor.yml, similar to how Circle CI works, as that would help verify the status of those tests.

hramos picture hramos on 17 Oct 2018

I've used AppVeyor without issue on my React Native fork, not sure about other branches on the same repo though.

But if someone wants to try and fix this last remaining test on their fork it would work fine.

empyrical picture empyrical on 17 Oct 2018

@empyrical Can you share your success link in your react native fork ?

gengjiawen picture gengjiawen on 18 Oct 2018

Oh, sorry, I didn't get the test working on my fork.

What I mean is, if someone wants to make an attempt at fixing the failing InteractionManager test, AppVeyor will work fine for testing on their personal React Native fork

empyrical picture empyrical on 18 Oct 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

jobsamuel picture jobsamuel  ·  3Comments
phongyewtong picture phongyewtong  ·  3Comments
josev55 picture josev55  ·  3Comments
oney picture oney  ·  3Comments
vikeri picture vikeri  ·  3Comments