Redwood: TypeScript tracking issue

[peterp]

We're going to add first-class support for TypeScript and be an even better experience for JavaScript as a result.

Make the project that's created by yarn create redwood-app support TypeScript:

• [ ] Convert the *.js files in create-redwood-app repo to TypeScript.
• [ ] Transpile the new TypeScript files into JavaScript if the target is JavaScript. (JavaScript will be the default.)
• [ ] Add a flag to switch the target to TypeScript in the create-redwood-app package.
• [ ] Check for presence of tsconfig.json file in a project

Make the generators TS/ JS aware:
👉See sub-Tracking Issue #523

• [ ] Make the generators understand that the target is TS/ JS.
• [ ] Rewrite the templates from JS into TS.
• [ ] When the target is JS transpile them.

The api side:

• [x] Verify that we import .ts files via the importAll.macro (tests!)
• [x] We auto import context, is that a problem in the ts world?
• [x] Verify that the dev server works well with .ts files.

The bridge between api and web:

• [ ] Share type definitions from SDL in api with gql`` in web, fixed via #1321
• [ ] Share the currentUser returned from api/src/lib/user.ts with getCurrentUser in useAuth

The web side:

• [x] Do we properly handle .ts/ .tsx files?
• [x] Do our auto-imported routes work properly with tslint?

Framework/Packages

• [x] makeMergedSchema #563

Tooling

• [x] Does linting work?
• [x] Does jest work?

[Domino987]

Is there a Branch?

[mohsen1]

Are you planning to move the core itself to TS? The router needs to have types to work well in TS projects.

[thedavidprice]

Currently in process by Peter:

• https://github.com/redwoodjs/redwood/pull/264
• https://github.com/redwoodjs/create-redwood-app/pull/37

Are you planning to move the core itself to TS?

Yes indeed. I believe @jeremyscatigna might have a start on this already. True, Jeremy?

[johnslemmer]

In this adding of "first class typescript support", is there a line item for generating types for gql queries? Something like apollo-codgen? This is currently one of the pain points when integrating with any API (whether REST or graphql) and if that would be included in the Redwood framework that would be amazing and make for an awesome and very type-safe coding experience.

[divideby0]

+1 on generating types for gql queries. The graphql-code-generator project might also worth considering as it generates typesafe React Hooks, HOCs, etc for pre-defined query fragments.

[marceloverdijk]

Is there a rough timeline for TypeScript support? E.g months, 1/2 year, 1 year?

[thedavidprice]

Hi @marceloverdijk it’s a top 3 priority (on the list with Auth and Jest tests, for example), but the progress has admittedly been slow. I know we _want_ it to be “months”. But that’s all the certainty I can commit to regarding a rough timeline.

What I _could_ commit to, personally, is helping organize a collaborative effort around this. I know:

• @mohsen1 is and will continue to contribute (and has more experience with TS than the entire core team combined),
• there was an early offer to help on our forum,
• and @Domino987 was looking to help (above)…

…but we haven’t really rounded up critical mass. So taking this as a nudge in that direction, and as a next-step I just this created a Forum topic (below). Chime in you’re available + interested:
https://community.redwoodjs.com/t/team-typescript-for-those-who-want-to-help-add-ts-support/369

[marceloverdijk]

Thx @thedavidprice for the honest answer. Much appreciated!
I'm mainly a Java developer, taking some side paths in the evening, like JS/TS. But I'm not a level to contribute unfortunately. But I will follow the thread with a lot of interest.

[peterp]

Our development configurations are now "TypeScript aware" via #438, this means that if you rename the .js files to .ts and add .tsx files that the development servers will understand how to work with those - and you should not be personally blocked by developing in TypeScript in a Redwood project.

The next steps are to figure out if building a project is working, and to add types to the redwood router.

We'll add support for starting a redwood project in typescript, and making sure the generators generate TypeScript.

[aldonline]

• If we want to make Redwood TypeScript friendly,
  then we need to guarantee that a Redwood project works with the TypeScript compiler out of the box. In other words, Redwood code MUST be valid TypeScript code.
• There is only one TypeScript compiler, and one set of rules. So the goal is clear. In fact, the TypeScript compiler is what most IDEs use to enable language features (Including code completion for JavaScript!). So enabling TypeScript support has many benefits, even if you don't use TypeScript.
• If Redwood has some "magical" tricks going on, then they need to be mapped into TypeScript. If this fit doesn't come naturally, there are a few things that can be done:
  
  
  
  • Edit tsconfig.json
  
  
  • Modifying tsconfig.json is usually the simplest solution. Unfortunately, configuration via tsconfig.json is very restrictive when compared to webpack/babel. This means that there are many things that can be achieved with babel/webpack that _cannot_ be mapped to TypeScript compiler settings.
  
  
  • Use code generation
  
  
  • Create a TypeScript Language Service Plugin
  
  
  • This is powerful, but not easy to maintain. Deno has recently popularized this approach (to enable their special import syntax and other things)
    
    
    
    
    
    • This is the Deno Language Service Plugin implementation (It eventually delegates module resolution to module_resolver.ts)
    
    
    
  
  
  • Modify the compiler itself
  
  
  • Not recommended. But adding it here for completion. Scroll to the end of this comment for more details.
  
  
  • Reconsider the feature (is it really worth breaking normal TS behavior?)
  
  

EDIT: added an example that showcases some of the techniques discussed here

Those are the tools at our disposal. But what are the challenges?

Challenges

Using path aliases (src/*) instead of relative imports

• This can be easily be represented by settings in tsconfig.json, which are actually equivalent to the settings in jsconfig.json. (in fact, jsonfig.json is a subset of tsconfig.json)
• While this is not a big challenge, it is still worth pointing out that some tools might have trouble processing these imports, as they require a special kind of resolution algorithm. For example, some testing tools like Wallaby.js break (this will be fixed at some point)

Directory Name convention

Redwood uses the DirectoryName convention extensively.
This convention is meant to improve Developer Experience by allowing more significant file names (instead of having multiple files named "index.js")
This is enabled via a webpack plugin and, functionally, it extends webpack's module resolution logic by adding an extra alternative:

For the following import:

import MyComponent from "src/MyComponent"

Normal Lookup (simplification)

src/MyComponent.(t|j)sx?
src/MyComponent/index.(t|j)sx?
node_modules...

With DirectoryName convention:

src/MyComponent.(t|j)sx?
src/MyComponent/index.(t|j)sx?
src/MyComponent/MyComponent.(t|j)sx? <---- extra step
node_modules...

Can it be mapped to TypeScript?

• We could build a Language Service Plugin like Deno does
• We could automatically add "path" entries to tsconfig.json - one per file. This should give us a lot of flexibility since we can "alias" imports. TODO: test how this affects autocomplete/editing experience (it will most likely result in duplicates). The obvious downside is that we would be overwriting a config file constantly.
• Code generation can get us half way: we could inject generated declarations for module "src/MyComponent" so the TypeScript compiler doesn't complain. At runtime we don't change the webpack loader logic, since these are two different things. The only issue here is that "go to definition" would take users to the generated code instead of the desired component.
• The only complete solution would be to provide a Custom TypeScript compiler (more info on this at the end of this comment). Internally, this compiler would provide its own "ts.resolveModuleNames" implementation

Auto Imports on Routes.js

• Page components are automatically imported and made available to Routes.js
• This is not standard TypeScript (or even ES) behavior. The compiler will complain that the Identifiers are not found (since they should be explicitly imported)

Possible Solution: Language Service Plugin

• It can filter out diagnostics
• It doesn't solve the "go to definition" problem (this link will be lost). This can be solved by a VSCode extension or a Language Server (via provideDefinition(...))

Possible Solution: Code Generation

If we're already generating code, we could "declare global var"iables for each of these pages.
This would make the identifiers available so the compiler would not complain, but it would not
allow "jump to definition" to work correctly, since it would only take us to the generated declaration.
The runtime bundling magic would still be left to webpack.

Possible Solution: Routes.js must always be JS (not TS)

• Even if you're writing a TypeScript project, Routes.js must be a .js file and special rules apply
• We can still recover/add some level of IDE features with specialized extensions

Named Routes

• We could easily use code generation to enable strongly typed routes.*
• I wrote a bit more about code generation here

importAll.macro

• If babel is still used for transpiling (instead or in addition to the TypeScript compiler), then this should still work as expected. (challenge: the processing pipeline needs to be set in the right order)
• If we wish the output of calling importAll() to be correctly typed at design time, we can do that via code-generation and overloading, with a few border-cases.

Cells

• Cells are automatically wrapped as a React Component via a custom webpack loader
• This means that there is a mismatch between the static type of a Cell file module and the type
  you end up using when you import it.
• TypeScript has no way of knowing about this change, so when you try to use the cell as a React component, the compiler will complain

Solution?

• With code generation, we could generate a wrapper for each cell
• This generated file could be named in such a way that the TypeScript compiler would pick it up (MyCell/index.ts). Just like with every "generated code" based solution, things like "go to definition" would take users to the generated code. Which is probably not what they want.

(*) Providing a Custom TypeScript Compiler

If everything else fails, we can change the way TypeScript works :). This might sound crazy, but I'll bring it up to be exhaustive.

• If we need to do things that cannot be achieved via tsconfig.json + code generation, then Redwood could provide its own "wrapped/custom" TypeScript compiler
• This is a version of the compiler with minor changes to account for the special features

How can this be distributed?
I've seen two approaches:

Publish a separate NPM module

• It can be published to NPM as "@redwood/typescript", and added as dev dependency to all redwood projects (instead of the vanilla typescript)
• When using VSCode, for example, users would configure the IDE to use this bundled version
• This is somewhat officially supported (you can tell VSCode which TypeScript SDK folder to use via config, or we could automatically output/create a .vscode/settings.json file)

Challenges:

• It is obviously a very extreme measure to take. If there is no clear justification, we'll get some backlash from the community.
• Also, other tools (outside of redwood and VSCode) might still look up the "typescript" module by its name (expecting it to be in node_modules/typescript or equivalent).

Monkey Patch

• We can "monkey patch" the official compiler with a post-installation script.
• This is not as tricky as it sounds. The way the TypeScript compiler codebase is designed allows for some things to be "overriden" easily by injecting a few lines of code that run before the rest of the modules are loaded. Specially module resolution logic.
• Without going into details, the way this works is by replacing tsc.js, server.js, etc with a thin loader that injects some code, which then loads the original files.

Challenges:

• I am not sure if all the use cases can be covered this way
• How reliable are post-install hooks? I have seen them being used for "important" stuff by "important" teams, (including the vscode team using it to download vscode.d.ts until two releases ago).

[koolamusic]

Maybe I could take a hit at writing type definitions for <Route>, <Link>, and <Private> at the moment

[thedavidprice]

@koolamusic that would be fantastic! Looping in @peterp here.

Maybe open a separate Issue (linking back to this one) to get started? And definitely don't hesitate to let us know what questions + clarification + help you need along the way.

🚀

[tomitrescak]

@thedavidprice do you have any developer guidelines for adding types? Do you want to store them in one big file, or separate definitions into linked files? Do you want to keep definitions in Redwood repo, or shall it be tracked in the @types repo?

[thedavidprice]

Hi @tomitrescak! Thanks for checking in here. @peterp is the one to provide an update as this issue is (in a good way) out of date. And your question is a bit of "it depends" 😆. For the Framework (this repo redwoodjs/redwood) we have a mix of approaches. For the application development template (redwoodjs/create-redwood-app) there's a lot more customization involved and (I believe) includes storing in a big file.

[koolamusic]

@tomitrescak I think it's okay to start out with one file for some packages, while others could use a type-per-module approach to separate the definitions. With regards to the developer guidelines, have you seen https://github.com/sindresorhus/typescript-definition-style-guide ? would love to know what your thoughts are

[peterp]

I've added the type defs for the router over here: https://github.com/redwoodjs/redwood/pull/1110

[lobosan]

Once this is finished, maybe you can coin the term TAMStack (TypeScript, API, Markdown) :D