Tfjs: Feature Request: Please pack `src/*.ts` with NPM package for easy debuging

[huan]

TensorFlow.js version: 1.0

Browser version: N/A

Describe the problem or feature request

When we developing with tfjs, we will see bugs and we need to understand the underlying logic of the tfjs modules.

Current, the @tensorflow/tfjs and @tensorflow/tfjs-layers published the JS code with source map to TS, we can get the error trace in the source code of TS.

However, according to the current NPM packing behavior, we did not pack the src/*.ts with the NPM package, so that I can not find any TS source code files in my node_modules directory, which make me can not easy follow the following error trace message:

(node:9419) UnhandledPromiseRejectionWarning: Error: Cannot find any save handlers for URL '/tmp/translation.keras'
    at new ValueError (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/src/errors.ts:48:5)
    at Model.<anonymous> (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/src/engine/training.ts:1610:15)
    at step (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/dist/engine/training.js:42:23)
    at Object.next (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/dist/engine/training.js:23:53)
    at /home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/dist/engine/training.js:17:71
    at new Promise (<anonymous>)
    at __awaiter (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/dist/engine/training.js:13:12)
    at Model.save (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/dist/engine/training.js:889:16)
    at /home/zixia/git/javascript-concist-chit-chat/bin/translation.ts:387:9
    at Generator.next (<anonymous>)
(node:9419) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:9419) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:9419) UnhandledPromiseRejectionWarning: Error: Error when checking : expected input2 to have shape [null,59,93] but got array with shape [1,1,93].
    at new ValueError (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/src/errors.ts:48:5)
    at checkInputData (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/src/engine/training.ts:325:19)
    at Model.predict (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/src/engine/training.ts:1063:5)
    at decode_sequence (/home/zixia/git/javascript-concist-chit-chat/bin/translation.ts:312:49)

What I want to do is I need see what's there in the at Model.<anonymous> (/home/zixia/git/javascript-concist-chit-chat/node_modules/@tensorflow/tfjs-layers/src/engine/training.ts:1610:15).

It will make the developer life be easier if we can trace the TS files when we developing, and publish tye TS source file with the NPM will archive that.

So I'd like to suggest us to include the TypeScript source files (src/*/.ts) when we publishing the @tensorflow/* NPM module.

cc @wangtz

[dsmilkov]

Hi @huan , I'm curious about your development setup. Are you using node interpreter and run it server-side? Asking since we do ship *.js.map to npm alongside every .js file. The *.js.map file has the ts source code embedded in it.

Then it becomes a matter of the toolchain to link the stack trace to the typescript source code. I'm not sure what package would do that for node. Browsers do this automatically.

[huan]

Hi @dsmilkov, thank you for the question.

Yes, I'm using node interpreter and run it server-side.

I understand that the *.js.map will include the ts source code, and the browser can do this automatically.

However, In my setup, I develop my code with tfjs in TypeScript and run it directly with ts-node(or node -r ts-node/register). In this case, I will get the error trace with the above log, which caused I need the src directory to inspect it.

[dsmilkov]

I did some research and none of the popular TypeScript libraries (angular, typescript itself, etc), publish their ts source code on npm. Publishing ours will add ~3.5MB to the npm package. I'm not ready to make that decision yet.

How about using GitHub release tree to browser the code? For example, if you depend on tfjs-core 1.0.0, you can browse the source code using this url: https://github.com/tensorflow/tfjs-core/tree/v1.0.0/src (note the v1.0.0).

[huan]

Using the GitHub release tree to browse is do-able. However, it's not very convenience comparing with running a vim command directly inside the console.

From what I'm thinking:

1. It's not a big deal that we add another 3.5MB to npm package, because the node.js binding is almost 100MB already.
2. The angular need not publish the source code because it mainly runs in the browser, it's a web application.

It's ok that we kept not publish the src/ directory with npm. It's just ... stoping me to dig inside the source when I'm working with the terminal.

[dsmilkov]

Thanks for the context. I'll run it by the team tomorrow and will let you know.

[wangtz]

I'm not sure what would be the best practice. But is there a chance to have a developer friendly version of npm such as tfjs-dev?

[nsthorat]

We chatted about this -- we've decided to ship the .ts files!

[nsthorat]

Just for posterity: src/ only adds 400K to the .tar.gz NPM file of tfjs-core, so this won't have too much impact total.

[huan]

@nsthorat Brilliant! Thanks for bringing the TS source code with the NPM!

[nsthorat]

1.0.1 now has TypeScript src code in the NPM package: https://unpkg.com/@tensorflow/tfjs@1.0.1/src/

[huan]

@nsthorat sweet!

I'll close this issue because it had been fulfilled.

Thank you very much!