Jest: ☂️TypeScript migration tracking issue

[SimenB]

This issue is meant to track Jest's internal migration to TypeScript.

Process

We would _love_ to get help from the community here. I've compiled a list of all the packages in this repo below - feel free to claim any package not claimed and submit a PR migrating it.

List

| Package Name | Task Owner | PR | Status | Notes |
| --------------------------- | ------------- | ----- | ------- | -------------- |
| @jest/core | @SimenB | #7998 | MERGED | |
| @jest/reporters | @loryman | #7994 | MERGED | |
| @jest/transform | @SimenB | #7918 | MERGED | |
| babel-jest | @SimenB | #7862 | MERGED | |
| babel-plugin-jest-hoist | @deneuv34 | #7898 | MERGED | |
| babel-preset-jest | - | - | - | Not transpiled |
| diff-sequences | @loryman | #7820 | MERGED | |
| eslint-config-fb-strict | - | - | - | Not transpiled |
| expect | @natealcedo | #7919 | MERGED | |
| jest | @SimenB | #8024 | MERGED | |
| jest-changed-files | @loryman | #7827 | MERGED | |
| jest-circus | @doniyor2109 | #7916 | MERGED | |
| jest-cli | @SimenB | #8024 | MERGED | |
| jest-config | @SimenB | #7944 | MERGED | |
| jest-diff | @SimenB | #7824 | MERGED | |
| jest-docblock | @SimenB | #7836 | MERGED | |
| jest-each | @mattphillips | #8007 | MERGED | |
| jest-environment-jsdom | @lirbank | #8003 | MERGED | |
| jest-environment-node | @lirbank | #7985 | MERGED | |
| jest-get-type | @SimenB | #7818 | MERGED | |
| jest-haste-map | @jeysal | #7854 | MERGED | |
| jest-jasmine2 | @doniyor2109 | #7970 | MERGED | |
| jest-leak-detector | @r3nya | #7825 | MERGED | |
| jest-matcher-utils | @SimenB | #7835 | MERGED | |
| jest-message-util | @SimenB | #7834 | MERGED | |
| jest-mock | @thymikee | #7847 | MERGED | |
| jest-phabricator | @r3nya | #7965 | MERGED | |
| jest-regex-util | @loryman | #7822 | MERGED | |
| jest-repl | @natealcedo | #8000 | MERGED | |
| jest-resolve | @SimenB | #7871 | MERGED | |
| jest-resolve-dependencies | @jeysal | #7922 | MERGED | |
| jest-runner | @natealcedo | #7968 | MERGED | |
| jest-runtime | @SimenB | #7964 | MERGED | |
| jest-serializer | @thymikee | #7841 | MERGED | |
| jest-snapshot | @doniyor2109 | #7899 | MERGED | |
| jest-util | @SimenB | #7844 | MERGED | |
| jest-validate | @SimenB | #7991 | MERGED | |
| jest-watcher | @SimenB | #7843 | MERGED | |
| jest-worker | @SimenB | #7853 | MERGED | |
| pretty-format | @SimenB | #7809 | MERGED | |

After all packages are migrated, we can start to migrate our integration tests. Depending on how this migration goes, we can track that in this issue as well, or we can track it separately later.

How

Order of packages to migrate

One thing to note is that because this repo is a monorepo, we have dependencies between packages. So we have to migrate leaf packages (without dependencies) first, then walk up the dependency tree until everything is migrated. Which means jest-cli will be the last package migrated.

You can use yarn to figure out which packages depend on which internally: yarn --silent workspaces info. This will output a JSON object of all packages in the workspace. An example looks like this:

{
  "babel-jest": {
    "location": "packages/babel-jest",
    "workspaceDependencies": ["babel-preset-jest"],
    "mismatchedWorkspaceDependencies": []
  }
}

The interesting part here is workspaceDependencies. If that array is empty, that's a perfect package to start migrating. If it is _not_ empty, you'll want to make sure that every package listed in the array has been migrated already.

Steps

1. Claim a package in this issue
2. Copy tsconfig.json from an already migrated package
   
   
   
   1. If the package you're migrating have dependencies on another package in this repo, use references
   
   
3. If a type file exists in types/ in the root of the repo for the package, move that into the package's src directory as a file named types.ts
4. Add "types": "build/index.d.ts" to package.json below main
5. Rename all files with js extension to ts (or tsx if they have jsx in them), fixing type errors as you go
6. Make sure tests and lint (including flow) pass
7. Ensure that the JS after compilation is essentially the same as before migrating*
8. Open up a PR

To build, you can run yarn build or yarn watch in the root of the repository.

You can look at my PR for pretty-format for a look at how a migration might look.

You can use flow-to-typescript to help in migration. However, since the syntax between Flow and Typescript is so similar, I personally only used it for the type definition files in types - for normal source files it was easier to rename the file to ts(x) and fix the syntax errors that my IDE showed me.

*) Do this by comparing git diffs before and after migration (also please include this diff in the PR after opening it)

1. run yarn build on master
2. git add -f packages/MY_PACKAGE/build*
3. git commit -m 'before migration'
4. run yarn build on your branch with the migration
5. rm packages/MY_PACKAGE/build*/**/*.ts packages/MY_PACKAGE/build*/**/*.map
6. git add -f packages/MY_PACKAGE/build*
7. git commit -m 'after migration'
8. git diff master packages/MY_PACKAGE/build*
9. On macOS (there probably exists similar tools on Linux and Windows), this can be copied and included in the PR git --no-pager diff master packages/MY_PACKAGE/build* | pbcopy. Stick that in a code fence with diff syntax in the PR.

Things to look out for during migration

The config doesn't allow implicit any

Out current setup with flow allows this - just add an explicit any (or a stricter type if you're able) in those cases. If possible, please use unknown instead of any.

The module exports CommonJS

Convert require to import Use exports = to replace modules.exports - this allows TypeScript to understand the export. We include a babel-plugin which transpiles this into module.exports for the distributed code (and tests).

Potential gotchas

Probably more, but I'll write down the ones I know of

• Jest currently imports quite a lot of types from types/ in the root of this repo, allowing us to use types across packages without dependencies on them (typically modules will depend on types/Config which allow them to have ProjectConfig as arguments). Since we'll be distributing the types, we need those dependencies to be explicit.
  
  
  
  • A solution in this concrete case is probably to create a separate packages that has everything jest-config has today except for the default configs (so it can drop babel-jest, the test environments etc)
  
  
• Similarly, types/TestResult is used by jest-jasmine2, jest-circus and jest-cli (for reporters). We need to figure out how to share that type effectively between packages.

Another idea on how to solve "type sharing" is to use a separate jest-types project that's _just_ types that are shared between modules.

Ideas here are very much welcome!

EDIT: As of #7834, I've created a @jest/types package

---

PS: This will not affect anyone using Jest (unless you use the modules exported by Jest directly, which will become typed). This is strictly an internal refactor. We will _not_ be considering adding types to be a breaking change.

PPS: It is currently a non-goal to distribute TS types for using Jest - that is excellently handled in @types/jest already. At some point we might want to distribute that with jest as well - but that's a separate issue.

PPPS: This issue is _not_ for discussions about the merits of the migration in and of itself - that feedback can be posted in the RFC. However, if you have experience migrating, building, testing or distributing a module written in TS, feel free to chime in with learnings from that process in this issue.

[lorenzorapetti]

I would love to contribute on this! Can i take diff-sequences for now? I've already ported it in my branch and all the tests for the module are passing.

[SimenB]

Yeah, go for it!

[r3nya]

Hey @SimenB!
I'd be happy to help. Can you recommend a package to start with? :)

[SimenB]

That's awesome @r3nya!

I'd say the easiest are babel-plugin-jest-hoist, jest-leak-detector and jest-phabricator.

Others that should be pretty straightforward are jest-changed-files, jest-diff, jest-docblock, jest-message-util, jest-regex-util and jest-serializer.

[lorenzorapetti]

Yeah, go for it!

Thank you! I could take jest-changed-files and jest-regex-util as well.

[SimenB]

🎉 I've updated the OP

[r3nya]

@SimenB let me do a jest-leak-detector and jest-phabricator for now. ;)

[thymikee]

@r3nya go for it! :)

[lorenzorapetti]

So, jest-changed-files imports from types/Config and types/ChangedFiles which are imported by several packages as well. What do you think is the right way to go from here?

[SimenB]

Path is just string, so that should be fine to replace. You can to type Path = string to keep the diff down 🙂

types/ChangedFiles can be copied and moved into jest-changed-files (similar to what I did with types/PrettyFormat)

[lorenzorapetti]

Got it, thanks! I've created the PR :)

[MohamadKh75]

I'll take jest-repl 😁

[SimenB]

@MohamadKh75 thanks! However, that depends on jest-runtime, jest-config and jest-validate, all of which have to be migrated first. You can try jest-validate?

[mattphillips]

Happy to migrate jest-each over :smile:

[SimenB]

Woo, do it 😀

[MohamadKh75]

@MohamadKh75 thanks! However, that depends on jest-runtime, jest-config and jest-validate, all of which have to be migrated first. You can try jest-validate?

Sure! then i start jest-validate

[SimenB]

awesome!

[jeysal]

I'll try my luck with haste map 😄

[lirbank]

I'd be happy to do jest-environment-node and jest-environment-jsdom.

[SimenB]

Awesome, thanks! OP updated 🙂

[deneuv34]

Hi, can I do the first try for babel-plugin-jest-hoist?

[SimenB]

For sure!

[doniyor2109]

Happy to take jest-snapshot 😄

[SimenB]

That's great @doniyor2109! Make sure to work on top of #7871 (will hopefully merge it today) 🙂

[natealcedo]

Calling dibs on jest-config :)

[SimenB]

Great @natealcedo! Should coordinate with @MohamadKh75 who's working on jest-validate

[natealcedo]

oh wait, I just realized I was looking at the wrong package. jest-config still has dependencies which havent been migrated. Lemme call dibs on expect instead. Sorry bout that haha

[SimenB]

👍

[natealcedo]

Hey what do we do in the scenario that we import types from types but they don't necessarily have their own package. In my case, expect uses types from types/Matchers. Do I migrate most of them into the types.ts file of expect?

[SimenB]

@natealcedo I think it makes sense to stick them in expect - that's where all matchers live. It's currently only expect, jest-jasmine2 and jest-circus that uses it, and they both import from expect already.

However, since we do module.exports, it's hard (impossible) to also export types, so we might need to stick it in @jest/types for now, and move it into expect whenever we ditch CJS

[jeysal]

Will do jest-resolve-dependencies once the jest-snapshot PR is complete :) except this time the weekend is coming up so I'll actually have the time to get to it quickly enough 😅

[doniyor2109]

Ready to take jest-circus when its dependencies are migrated 💪

[SimenB]

Awesome @doniyor2109, wrote it down.

---

@mattphillips @lirbank @r3nya @MohamadKh75 hey, how's it going with the packages you've claimed? 🙂 Feel free to use GitHub's awesome new Draft PR feature to gather some early feedback if you're stuck on something.

https://github.blog/changelog/2019-02-14-draft-pull-requests/

[MohamadKh75]

@SimenB thanks for that 😁
I'll use it soon...
One thing, the __tests__ also need to be converted to .ts?

[thymikee]

@MohamadKh75 it would be nice to do so, but it's not required at the moment, especially that they're not currently typed (I'm speaking of jest-validate)

[SimenB]

Agreed - we can do tests in a follow up if they prove to be tricky. The important thing is to get the source code converted to keep the migration going. We know that the current code _works_, which is why I'd like to see a diff against master in the PRs

I'll be setting up a new issue for tests, utils etc (as well as following up on @ts-ignores) later.

[SimenB]

For anyone else following along, we've added 2 new packages to the repo that needs to be converted as well - @jest/transform & @jest/reporters (split out from jest-runtime and jest-cli, respectively). They should be pretty straight-forward to convert if anybody wants to pick them up 🙂

[doniyor2109]

Feel free to use GitHub's awesome new Draft PR feature to gather some early feedback if you're stuck on something.

github.blog/changelog/2019-02-14-draft-pull-requests

Strange. It is not working for my PR 🙈

[SimenB]

Huh, I guess I should have tested it rather than blindly trust the blog 😅

Just stick [WIP] in the title then, until the feature works as advertised 🙂

[r3nya]

Hey @SimenB!
Sorry for my delay. WIll do it soon!

[natealcedo]

Hey how do you guys verify that the build is actually working? Tried building off of master and the build:ts script is failing for me

$ yarn build:ts
$ node scripts/buildTs.js
Building TypeScript definition files
Building..............................................................error 
TS5055: Cannot write file '/Users/natealcedo/open-source/jest/packages/jest-
diff/build/index.d.ts' because it would overwrite input file.

error TS5055: Cannot write file '/Users/natealcedo/open-source/jest/packages
/jest-diff/build/types.d.ts' because it would overwrite input file.


Found 2 errors.


Unable to build TypeScript definition files
Error: Command failed: tsc -b /Users/natealcedo/open-source/jest/packages/ba
bel-jest /Users/natealcedo/open-source/jest/packages/babel-plugin-jest-hoist
 /Users/natealcedo/open-source/jest/packages/diff-sequences /Users/natealced
o/open-source/jest/packages/jest-changed-files /Users/natealcedo/open-source
/jest/packages/jest-diff /Users/natealcedo/open-source/jest/packages/jest-do
cblock /Users/natealcedo/open-source/jest/packages/jest-get-type /Users/nate
alcedo/open-source/jest/packages/jest-haste-map /Users/natealcedo/open-sourc
e/jest/packages/jest-leak-detector /Users/natealcedo/open-source/jest/packag
es/jest-matcher-utils /Users/natealcedo/open-source/jest/packages/jest-messa
ge-util /Users/natealcedo/open-source/jest/packages/jest-mock /Users/natealc
edo/open-source/jest/packages/jest-regex-util /Users/natealcedo/open-source/
jest/packages/jest-resolve /Users/natealcedo/open-source/jest/packages/jest-
serializer /Users/natealcedo/open-source/jest/packages/jest-snapshot /Users/
natealcedo/open-source/jest/packages/jest-types /Users/natealcedo/open-sourc
e/jest/packages/jest-util /Users/natealcedo/open-source/jest/packages/jest-w
atcher /Users/natealcedo/open-source/jest/packages/jest-worker /Users/nateal
cedo/open-source/jest/packages/pretty-format
    at makeError (/Users/natealcedo/open-source/jest/node_modules/execa/inde
x.js:174:9)
    at Function.module.exports.sync (/Users/natealcedo/open-source/jest/node
_modules/execa/index.js:338:15)
    at Object.<anonymous> (/Users/natealcedo/open-source/jest/scripts/buildT
s.js:29:9)
    at Module._compile (module.js:653:30)
    at Object.Module._extensions..js (module.js:664:10)
    at Module.load (module.js:566:32)
    at tryModuleLoad (module.js:506:12)
    at Function.Module._load (module.js:498:3)
    at Function.Module.runMain (module.js:694:10)
    at startup (bootstrap_node.js:204:16)
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this 
command.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this 
command.

[doniyor2109]

@natealcedo It also happened to me. After deleting build files, it works

[natealcedo]

Calling dibs on jest-runner! :)

[lorenzorapetti]

I'd be happy to take jest-cli once all the dependencies are migrated if anyone hasn't claimed it yet :)

[SimenB]

Awesome @loryman, updated the OP! Note that jest-cli depends on everything else, so it might be a bit of time before all of its dependencies are migrated 🙂

[lorenzorapetti]

Yeah no problem! In the meantime i can do @jest/reporters since all its dependencies are migrated

[doniyor2109]

Let's finish jest-jasmine2 💪. I will try one more time

[natealcedo]

Hey quick question, do I migrate types/TestRunner to jest-runner or jest-types.

[SimenB]

@natealcedo jest-runner makes sense to me. Although the exported Reporter can be moved to @jest/reporters, which @loryman is working on migrating.

If it proves difficult (due to CJS or circular dependencies or something else), just move it to @jest/types and we can clean it up later 🙂

[natealcedo]

Sweet! Sounds like a plan. @SimenB

[natealcedo]

Hey what's left that can be picked up? :)

[SimenB]

Woah, we're almost done! Awesome 😀 If you want, you can start converting the e2e tests? Just the ones in this directory, not the tests themselves: https://github.com/facebook/jest/tree/master/e2e/__tests__
It essentially involves converting Utils to TypeScript I think, after which the tests themselves should just be renaming the file

I'll ping the ones that have claimed packages but not opened up a PR yet, might be more available.

---

@mattphillips @lirbank @MohamadKh75 hey, one more ping. If you don't have time to migrate that's totally fine, but if so it'd be nice if someone else could pick up the package you've claimed 🙂

[natealcedo]

The link you provided returns a 404. But you're referring to the contents of the __tests__ directory I presume?

If that's the case I'll gladly take it up 👍

[SimenB]

Yes! Not sure why GH messes up the link, what I pasted is https://github.com/facebook/jest/tree/master/e2e/__tests__. Seems like a bug 🤷‍♂️

[natealcedo]

Am I going to need to have a tsconfig.json in the __tests__ directory?

[SimenB]

I don't know. I don't think so (we won't be generating typings files here)? Try without 🙂

[natealcedo]

Sure thing. 💯

[doniyor2109]

@SimenB There is also jest-repl left

[SimenB]

It has dependencies that have not been migrated. But I guess we can just do ts-ignore on those imports for now, I think we've got a good enough of a setup to support that now. It should take like 5 minutes, though 🙂

[lirbank]

@SimenB Sorry for the lag, got sidetracked and was out all week. I'll get it done today!

[SimenB]

Awesome. I've added a @jest/environment package which is supposed to export an interface to be implemented by the environments. I haven't tested it though, so it might not work properly. Feel free to open an early PR if it gives you difficulties

[MohamadKh75]

Woah, we're almost done! Awesome If you want, you can start converting the e2e tests? Just the ones in this directory, not the tests themselves: https://github.com/facebook/jest/tree/master/e2e/__tests__
It essentially involves converting Utils to TypeScript I think, after which the tests themselves should just be renaming the file

I'll ping the ones that have claimed packages but not opened up a PR yet, might be more available.

@mattphillips @lirbank @MohamadKh75 hey, one more ping. If you don't have time to migrate that's totally fine, but if so it'd be nice if someone else could pick up the package you've claimed

I'm so sorry... i just had some unexpected problems... maybe i can help in other sections... :disappointed:

[SimenB]

No problem @MohamadKh75, thanks for reaching out! 🙂 You can do jest-repl if you wanna do a package? We're at the point where we don't have to wait for dependencies. If you don't have time, that's no problem at all, of course - we're all doing this in our spare time

[natealcedo]

Hey can I take up jest-repl since @MohamadKh75 isn't available to pick up the task? :)

[SimenB]

Sure, go for it 🙂

[SimenB]

Hey @loryman, I picked up jest-cli since you're busy with @jest/reporters 🙂

[lorenzorapetti]

Hey @loryman, I picked up jest-cli since you're busy with @jest/reporters 🙂

Sure, go ahead!

[SimenB]

Holy shit, we're done! 🎉

Thank you so much to everyone who helped out. I did _not_ think we'd be able to do this in just 4 weeks.

Jest is now officially a TS repo.

$ cloc packages --vcs git --exclude-dir __tests__,__mocks__
     462 text files.
     408 unique files.
      76 files ignored.

github.com/AlDanial/cloc v 1.80  T=0.45 s (896.0 files/s, 98149.4 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
TypeScript                     286           4581           4052          29135
JSON                            82              0              0           2106
Markdown                        19            571              0           1697
JavaScript                      16            217            213           1497
PHP                              1             22             14            150
-------------------------------------------------------------------------------
SUM:                           404           5391           4279          34585
-------------------------------------------------------------------------------

---

Next steps are as follows:

• [x] Release an alpha to verify we didn't break anything (done as [email protected])
• [ ] Setup a replacement for @types/jest
• [ ] Convert more of the unit tests to TS
• [ ] Setup tests to verify our types (especially around inferring the type of jest spies)

I'll be opening issues for these soon(ish).

Thank you again to everybody who chipped in!

[SimenB]

Alpha released! Please do yarn add -D jest@beta or npm i -D jest@beta (currently [email protected]) and complain if it explodes horribly 🙂