Typescript: Emit does not preserve empty lines

[NoelAbrahams]

Hi,

TS Version: 1.1

Given

function foo() {

    var x = 10;

    var y = 11;
}

We used to get

function foo() {
    var x = 10;

    var y = 11;
}

In the new compiler the line break is missing

function foo() {
    var x = 10;
    var y = 11;
}

(Both compilers removed the first empty line, but the new compiler has gone a step further.)

This can affect the experience when debugging JavaScript in the browser.

[ahejlsberg]

Adding some background info... The reason the new compiler removes all blank lines is that this is the only thing we can do _consistently_. Blank line preservation is always going to be a hit or miss thing when it comes to constructs that are subject to rewriting, e.g. class and module declarations. We face exactly the same issues with comment preservation. So, while I'm sympathetic to resolving this, it's not an easy thing to do.

@NoelAbrahams I'm curious what issues you see when debugging?

[CyrusNajmabadi]

@ahejlsberg @NoelAbrahams I created a prototype emitted in the original CodePlex project that actually did a terrific job with comment and newline preservation. Even on constructs that got heavily rewritten (like arrow-functions to function expressions) it would almost always do what you intuitively expected it to do.

Line preservation mostly a function of just reusing the line breaks when preserving old code, and using relative spacing when doing transformations. i.e. if you have:

module M {
}
module N {
}

Then when you're emitting the IIFE for 'N' you say "we should keep the trivia between the module we're rewriting and the previous syntactic element".

Here is the before/after for how my emitter prototype worked that preserved comments/newlines with high levels of fidelity:

https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter2/ecmascript5/Parser.ts
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter2/ecmascript5/Parser.ts.expected

You can see a lot of examples here as well:
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter/ecmascript5/

[CyrusNajmabadi]

The only feature i didn't implement was 'alignment'. i.e. if you had code that was aligned in some way in the original (very common with parameter declarations), we'd want the emitted code to preserve that as well.

But it would have been very trivial to do.

That said, conversion of contructs from TS to JS did try to preserve indentation. You can see that here:
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter/ecmascript5/ClassDeclaration/ClassDeclaration2.ts
https://typescript.codeplex.com/SourceControl/latest#tests/Fidelity/emitter/ecmascript5/ClassDeclaration/ClassDeclaration2.ts.expected

Note how the statements are properly indendented (even when they spread over multiple lines) even after we convert the nested modules and class into IIFEs

[NoelAbrahams]

@ahejlsberg, there are no significant issues when debugging in the browser. It just makes it easer to navigate the JavaScript code and locate lines for setting breakpoints when there is an exact correspondence with the actual TypeScript source code.

I could personally live without the empty lines, but since TS has gone to such lengths to preserve and emit _beautiful_ JavaScript (:smile:) it seems only natural that this is implemented.

[mtraynham]

@ahejlsberg @NoelAbrahams There is one issue that exists with debugging in the browser that is slightly related to this conversation. When using setter/getter chains (like with jquery) or chaining promises, the new line feeds are lost during translation. That being said, it is a huge pain point when working with Arrow functions.

As an example:

(<any> x).a('#test')
    .b('test')
    .c(() => 'foo')
    .d(() => 'bar')
    .e(() => 5)
    .f(() => 6);

Becomes:

x.a('#test').b('test').c(function () { return 'foo'; }).d(function () { return 'bar'; }).e(function () { return 5; }).f(function () { return 6; });

Using Chrome and sourceMaps, the breakpoints are still skipped.

http://www.typescriptlang.org/Playground#src=(%3Cany%3E%20x).a('%23test')%0A%20%20%20%20.b('test')%0A%09.c(()%20%3D%3E%20'foo')%0A%09.d(()%20%3D%3E%20'bar')%0A%09.e(()%20%3D%3E%205)%0A%09.f(()%20%3D%3E%206)%3B

[NoelAbrahams]

@mtraynham, actually I think the problem you highlight is a slightly different one.

In previous versions the body of an inline function was always emitted on new lines:

// TS
var x = () => 'foo';

// JS - old
var x = function () { 
             return 'foo'; 
       };

// JS - new
var x = function () { return 'foo'; };

I too have found this to be a problem - having to occasionally go back and create a function so that I can set a breakpoint when debugging in the browser.

[mtraynham]

@NoelAbrahams Ahh yes, I have been using this exact same temp solution... I wasn't sure if this was an appropriate bug to contribute this problem to (erasure of line feeds), or should I open another?

I created #2259 to the separate issue.

[timjmartel]

As an engineering director exploring moving our javascript development community to typescript the new line capability would really help. One of the primary appeal of typescript was maintaining the readability and structure of the code created in typescript generatedJavascript.

Great to see comments are kept in the recent update and addition of the “--removeComments" command line directive.

[ghost]

I would also like this for a similar reason as @timjmartel - when developers see the emitted JS looks _nice_ they are less resistant to adopt. Preserving (at least vertical) whitespace makes the code look less like it was generated by a machine and more like idomatic JS code written by a human.

If our team ever decided to abandon TS and instead continue with the transpiled JS it would be much easier to adopt the emitted JS sources if they had human-friendly whitespace.

[fehmud]

In regard to empty lines, would it be possible - for now - to have them emitted, even if they are hit or miss? Such experimental feature could be asked with a “--keepEmptyLines" option. It would be not so much for having nice JS, but for having more readable JS.

In regard to chained function calls, would it be possible to have one call for line, to let users set breakpoints? Again, this feature could be asked with a "--oneCallForLine" option, if it is another "hit or miss" thing.

Thanks for your attention.

[fehmud]

Actually, chained function calls could be split by a source code beautifier, therefore it doesn't make sense to embed such feature in TypeScript.

[ORESoftware]

This shouldn't be that hard, shouldn't there just be an option in tsconfig.json

preserveWhitespace: true/false

?

[ORESoftware]

But I am not seeing this as a compiler option: https://www.typescriptlang.org/docs/handbook/compiler-options.html

[ORESoftware]

I just realized. One good reason to not keep whitespace is that it will really help prevent you from accidentally editing the .js instead of the .ts. I guess one thing to do would be to write out the .js files as readonly/execute only. So maybe this is a non-issue.

That being said, I don't think tsc writes out .js files as readonly/execute only, is there a way to configure tsc to do this?

[DanielRosenwasser]

No, not at the moment. Feel free to open up a separate issue for that though.

[ORESoftware]

@DanielRosenwasser you're saying if we want to preserve whitespace that we should open a separate issue? Is this issue not enough? Nevermind LOL more than a month later I realize you were saying to open a separate issue for read/write/execution permissions on transpiled files :)

[byelims]

It would be nice to have this.

[cwallsfdc]

+1

[angeloocana]

+1

[jaguar7021]

+1

[ppsimatikas]

The crowd wants preserveWhitespace: true/false
@ORESoftware ++

[Ciantic]

Reason this is important is that TypeScript is supposed to "degrade gracefully" to JS. Right now it can't preserve new lines making the JS a bit dense to read, especially if you write TypeScript, but you are supposed to deliver JS to some place else.

[bril-andrew]

+1 preserveWhitespace: true/false

Temporary hack

Use esformatter to add linebreaks.

With the following configuration file:

{
  "lineBreak": {
    "before": {
      "FunctionDeclaration": ">=2",
      "FunctionDeclarationOpeningBrace": 0,
      "FunctionDeclarationClosingBrace": 1,
      "MethodDefinition": ">=2",
      "ClassDeclaration": ">=2"
    },
    "after": {
      "FunctionDeclaration": ">=2",
      "FunctionDeclarationOpeningBrace": 1,
      "MethodDefinitionClosingBrace": ">=2",
      "ClassClosingBrace": ">=2"
    }
  }
}

[valera-rozuvan]

@mtraynham Your example:

(<any> x).a('#test')
    .b('test')
    .c(() => 'foo')
    .d(() => 'bar')
    .e(() => 5)
    .f(() => 6);

with the latest compiler produces this JS:

x.a('#test')
    .b('test')
    .c(function () { return 'foo'; })
    .d(function () { return 'bar'; })
    .e(function () { return 5; })
    .f(function () { return 6; });

(see TS PlayGround https://goo.gl/JViurr )

A lot has changed since TS Version 1.1 (the version of TypeScript for which this issue was created). Maybe this issue can be closed? @ahejlsberg ?

[mtraynham]

@valera-rozuvan I had opened #2259 instead. My issue was related to line feeds, but not the exact same issue described by this bug. #2259 was closed a while back (May 2015).

[fermmm]

This is the bril-andrew esformatter configuration but with a bug fixed, (when the class declaration contained the word import there was no line break):

{
    "lineBreak": {
        "before": {
            "FunctionDeclaration": ">=2",
            "FunctionDeclarationOpeningBrace": 0,
            "FunctionDeclarationClosingBrace": 1,
            "MethodDefinition": ">=2",
            "ClassDeclaration": ">=2",
            "ExportNamedDeclaration": 2
        },
        "after": {
            "FunctionDeclaration": ">=2",
            "FunctionDeclarationOpeningBrace": 1,
            "MethodDefinitionClosingBrace": ">=2",
            "ClassClosingBrace": ">=2"
        }
    }
}

[DanielSWolf]

I don't think using esformatter solves the whole problem. Sure, it can automatically insert blank lines around functions etc. But for me, blank lines within functions are even more crucial. We use blank lines like paragraphs in prose: to group individual thoughts.

Those blank lines within functions help to communicate the structure of the function. Without them, I find that readability suffers.

[JimLynchCodes]

@ahejlsberg I am seeing incorrect line numbers in my unit testing output when using ts-jest, and this issue seems to be caused by the empty lines being removed in the js output. I am curious why it is so difficult to leaves these lines in the final js. Is there any more information out there on this? Can we help somehow to make this happen? :)

Or has it been merged already and not released yet? --> V4 Next Big Version #3143

[dragomirtitian]

@JimTheMan If you use source maps then maybe the source-map-support package can help you get the correct stack traces in the output.

[zeroliu]

I also run into this issue. I came up with a workaround by creating a diff patch and revert whitespace changes in the patch. jsdiff allows you to create a structured patch object and manipulate it as you wish.

import * as diff from 'diff';

const patch =
      diff.parsePatch(diff.createPatch('file', oldText, newText, '', ''));
const hunks = patch[0].hunks;
for (let i = 0; i < hunks.length; ++i) {
  let lineOffset = 0;
  const hunk = hunks[i];
  hunk.lines = hunk.lines.map(line => {
    if (line === '-') {
      lineOffset++;
      return ' ';
    }
    return line;
  });
  hunk.newLines += lineOffset;
  for (let j = i + 1; j < hunks.length; ++j) {
    hunks[j].newStart += lineOffset;
  }
}
return diff.applyPatch(oldText, patch);

With this workaround, you can preserve all the line breaks from the original file.

[valera-rozuvan]

@zeroliu Does it introduce a noticeable time delay in compile step?

[valera-rozuvan]

@ahejlsberg Do you think it's worth fixing this issue?

[zeroliu]

@valera-rozuvan depending on the size of your project. For my use cases where I transpile 10-ish files of 100-1000 LOC, it doesn't introduce any noticeable delay.

[BiosBoy]

Any solutions here yet? I run in this trouble too...

[mbroadst]

I was part way into trying to fix this in the compiler itself when my teammate @emadum reminded me that tsc can preserve comments. Here's a little gulp pipeline that seems to do a fairly decent job of preserving newlines:

const gulp = require('gulp');
const ts = require('gulp-typescript');
const through = require('through2');

function preserveNewlines() {
  return through.obj(function(file, encoding, callback) {
    const data = file.contents.toString('utf8');
    const fixedUp = data.replace(/\n\n/g, '\n/** THIS_IS_A_NEWLINE **/');
    file.contents = Buffer.from(fixedUp, 'utf8');
    callback(null, file);
  });
}

function restoreNewlines() {
  return through.obj(function(file, encoding, callback) {
    const data = file.contents.toString('utf8');
    const fixedUp = data.replace(/\/\*\* THIS_IS_A_NEWLINE \*\*\//g, '\n');
    file.contents = Buffer.from(fixedUp, 'utf8');
    callback(null, file);
  });
}

gulp.task('default', function () {
  return gulp.src('src/**/*.ts')
    .pipe(preserveNewlines())
    .pipe(ts({
      removeComments: false
    }))
    .pipe(restoreNewlines())
    .pipe(gulp.dest('lib'));
});

I think a more clever comment would prevent some false positives, but it seems to work well for us today. I've only tested this on a relatively small file, but the performance overhead there was minimal.

hth

[CarabusX]

@mbroadst

I ended up using your idea as a base, and eventually expanded upon it until it became an npm module:
https://www.npmjs.com/package/gulp-preserve-typescript-whitespace

I credited your post in the Readme, hopefully you don't mind :)