Typescript: Lamda Expressions

[jaredjbarnes]

First I would like to say that I don't want to overwhelm those who are diligently making TypeScript a reality. I really appreciate this project. I also believe TypeScript is the future.

The company that I work for deals with a lot of data. In order to deal with this need we built a DbContext (Entitites in C#) for javascript. We also built Queryables (Linq in C#) backed by Expressions Trees for javascript. This has been very useful, and all of our programs today are using this technology. The problem is that today our syntax for this is a bit wonky. Here is what the syntax looks like today.

dbContext.people.asQueryable().filter(function(e){
    return e.property(“firstName”).isEqualTo(“Jared”);
});

It gets worse with “AND”s and “OR”s.

dbContext.people.asQueryable().filter(function(e){
    return e.and(e.property(“firstName”).isEqualTo(“Jared”),  e.property(“lastName”).isEqualTo(“Barnes”));
});

Our wish is that we could have the filter function look like we were acting on local data with Array.prototype.filter.

dbContext.people.asQueryable().filter(function(person: Person){
    return person.firstName == “Jared” && person.lastName == “Barnes”;
});

Or even better would be.

dbContext.people.asQueryable().filter((person: Person) => {
    return person.firstName == “Jared” && person.lastName == “Barnes”;
});

So here is what I would propose to answer this problem in TypeScript. I would like feedback and open discussion about this. I don’t think that I have all the answers, so please tell me if I’m completely misguided.

I like the idea that we could have a get Property on a function that would return the expression tree representing the function. If we do this, we have the function that could be used to filter local arrays as well as the expression tree (Abstract Syntax Tree) to filter remote data. Here is how it would look.

var fn = (person: Person) => {
    return person.firstName == “Jared”;
};

// Now I can turn this into anything I want. Odata, SQL you name it.
var expressionTree = fn.expression; 

someArray.filter(fn);
dbContext.people.asQueryable().filter(fn).toArray().then((array)=>{
    // Do somthing.
});

The problem with this approach is that every function would be unnecessarily bloated with this Expression Tree. My thought was maybe we could create a syntax for Lamda Expressions. That way would could only have the bloat when it's necessary.

Should the syntax be like C#’s?

person:Person => person.firstName == “Jared”;

I have already hacked the ArrowFunctionExpression to do it for our company today. But I think that it would be better if I had more feedback on how others would like it, or if they would like lambdas at all. I also don’t want to have it on the ArrowFunctionExpression because I find myself using this syntax because its a better way to describe a function anyways (less verbose). Not to mention that its made it into the javascript spec.

On a deeper note this is how its compiled today for us.

Typescript:

var fn = (person: Person) => { 
    return person.firstName == “Jared”;
};

Javascript:

var fn = (function(){

    var _fn = function(person){
        return person.firstName == “Jared”;
    };

    var _scopeInspector = function(variableName){
        return eval(variableName);
    };

    // The ExpressionTree needs a way to access
    // scoped variables found in the expression tree.
    // This is why we are using the evil eval.
    // expressionJSON is generated from the AST during compilation.
    var expTree = new ExpressionTree(_scopeInspector, expressionJSON );   

    Object.defineProperty(_fn, “expression”, {
        get: function(){
            return expTree;
        }
    });

    return _fn;
}());

If you made it this far, thank you for reading.

And please share your ideas.

[jonathandturner]

Since TypeScript already has lambdas (you can see them in the docs here: https://github.com/Microsoft/typescript/wiki/Functions), there already is a smaller expression for first class functions based on the ECMAScrip 6 recommendation.

Had you just not see that, or were you recommending something in addition to it?

[BSick7]

I think @jaredjbarnes wants lambdas to work similar to C# where you get an AST simply by changing the declaration type:

Func<int> = () => 1; //compiled function
Expression<Func<int>> = () => 1; //expression tree

This appears like a job for an auxiliary library; however, that library would end up replicating the work already done by Typescript.

[jaredjbarnes]

@BSick7 is exactly right. I just would like feedback on how we could signify that its a lambda expression not a compiled function.

For example we could use a thin arrow instead of a fat arrow.

() -> {} // Lambda Expression
() => {} // Just a shorthand for a functions.

Also I have written a library for this very purpose, but when writing the library I have to choose between scope, or pretty syntax. I can't have both, unless its part of the language. Jaydata has written something similar to our Linq system, but they chose pretty syntax over having local scope. I personally feel its a shame to not be able to have both.

JayData is like this:

(function(){
    var someName = "Jared"

    context.people.filter(function(person){
        return person.firstName == someName;
   },{ someName: someName});

}());

Note how I had to pass an object literal into the filter function because scope doesn't exist in jaydata's filter functions, and that is because they toString the function then parse it into a AST and then use the expression tree at runtime.

My Company's Linq

(function(){
    var someName = "Jared";

    dbContext.people.asQueryable().filter(function(e){
        return e.property(“firstName”).isEqualTo(someName);
    });

}());

Note how our syntax is wonky because we chose to have local scope.

I just want to be able to have both pretty syntax as well as scope. :)

Anyways I hope this clarifies a little bit more.

[omidkrad]

This can enable interesting use cases. I don't know if it's possible but maybe instead of introducing a new semantic like -> we could do it in a more generic way with language extensions so that we could make it work with the existing thick arrow =>, just by giving it a context.

var fn: ExpressionTree = (person: Person) => { 
    return person.firstName == "Jared";
};

[RyanCavanaugh]

This would be a huge amount of complex code gen. Because you can already get the string representation of a function's implementation, it's possible to write this kind of functionality as an external library without getting the compiler involved.

See also https://github.com/Microsoft/TypeScript/wiki/TypeScript-Design-Goals for a overview of where we're trying to spend our complexity budget

[jaredjbarnes]

I understand why Typescript wouldn't want to support this idea after reading the TypeScript design goals. Thanks though for the discussion.

[n9]

OT @jaredjbarnes In case you need more features from C#, you may try C# to JavaScript compiler, which has different design goals (https://github.com/erik-kallen/SaltarelleCompiler).

[jogibear9988]

@RyanCavanaugh:
I've tried to implement a functionality like this, via parsing the function string. This would work very well, but the problem is, I can not access the closures of the function! (Example Code is here: http://stackoverflow.com/questions/35958913/javascript-typescript-access-previous-scope-of-a-function)

Is there a way so, that TypeScript compiler detects a closures of a function, and for example add them as a parameter?

[RyanCavanaugh]

No

[knightlore]

I really like to see this feature too, it's almost essential if you want to write a good ORM query support (with better support intellisense and refactoring) or even smaller things like filters, and I think that converting a piece of the AST to some sort of representation don't add much complexity.

To achieve this, I wrote a rough and simple Babel plugin to convert some expressions into tree where calling specific functions:

"use strict";

//ToDo: ConditionalExpression

module.exports = function (babel) {
    var t = babel.types;

    var InsideWhere = false;
    var par = [];
    const Methods = ["Where", "Select"];

    return {
        visitor: {

            "ArrowFunctionExpression|FunctionExpression": {
                enter: function enter(path) {
                    if (InsideWhere) {
                        path.node.params.forEach(function (element) {
                            par.push(element.name);
                        }, this);
                    }
                },
                exit: function exit(path) {
                    if (InsideWhere) { }
                }
            },

            UnaryExpression: function UnaryExpression(path) {
                if (InsideWhere) {
                    path.replaceWith(t.objectExpression([
                        t.objectProperty(t.identifier("Type"), t.stringLiteral("U")),
                        t.objectProperty(t.identifier("Argument"), path.node.argument),
                        t.objectProperty(t.identifier("Operator"), t.stringLiteral(path.node.operator)),
                    ]))
                }
            },

            BinaryExpression: function BinaryExpression(path) {
                if (InsideWhere) {
                    path.replaceWith(t.objectExpression([
                        t.objectProperty(t.identifier("Type"), t.stringLiteral("B")),
                        t.objectProperty(t.identifier("Left"), path.node.left),
                        t.objectProperty(t.identifier("Operator"), t.stringLiteral(path.node.operator)),
                        t.objectProperty(t.identifier("Right"), path.node.right)
                    ]))
                }
            },

            MemberExpression: function MemberExpression(path) {
                if (InsideWhere) {
                    var isp = false;
                    var pos = 0;
                    for (var n = 0; n < par.length; n++) {
                        var element = par[n];
                        if (element == path.node.object.name) {
                            isp = path.node.object.name;
                            pos = n;
                            break;
                        };
                    }
                    if (isp) {
                        path.replaceWith(t.objectExpression([
                            t.objectProperty(t.identifier("Type"), t.stringLiteral("P")),
                            t.objectProperty(t.identifier("Parameter"), path.node.object),
                            t.objectProperty(t.identifier("Name"), t.stringLiteral(isp)),
                            t.objectProperty(t.identifier("Pos"), t.numericLiteral(pos)),
                            t.objectProperty(t.identifier("Field"), t.stringLiteral(path.node.property.name))
                        ]))
                    }
                }
            },

            LogicalExpression: function LogicalExpression(path) {
                if (InsideWhere) {
                    path.replaceWith(t.objectExpression([
                        t.objectProperty(t.identifier("Type"), t.stringLiteral("B")),
                        t.objectProperty(t.identifier("Left"), path.node.left),
                        t.objectProperty(t.identifier("Operator"), t.stringLiteral(path.node.operator)),
                        t.objectProperty(t.identifier("Right"), path.node.right)
                    ]))
                }
            },

            CallExpression: {
                enter: function enter(path) {
                    if (t.isMemberExpression(path.node.callee)) {
                        if (Methods.indexOf(path.node.callee.property.name) + 1) InsideWhere = true;
                    }
                },
                exit: function exit(path) {
                    if (t.isMemberExpression(path.node.callee)) {
                        if (Methods.indexOf(path.node.callee.property.name) + 1) InsideWhere = false
                    }
                }
            }

        }
    };
};

Which convert this:

var evalu = { a: 2, b: { c: 2 } };
var result = Query(Customers).Where((e) => -e.id > evalu.a && e.Name == "A" && e.id + evalu.a > 2 && e.id in [1, 2, 3, 4] && MySQL.Day(e.Date) == 3)

Into this:

var evalu = { a: 2, b: { c: 2 } };
var result = Models_1.Query(Customers).Where(function (e) {
  return {
    Type: 'B',
    Left: {
      Type: 'B',
      Left: {
        Type: 'B',
        Left: {
          Type: 'B',
          Left: {
            Type: 'B',
            Left: {
              Type: 'U',
              Argument: {
                Type: 'P',
                Parameter: e,
                Name: 'e',
                Pos: 0,
                Field: 'id'
              },
              Operator: '-'
            },
            Operator: '>',
            Right: evalu.a
          },
          Operator: '&&',
          Right: {
            Type: 'B',
            Left: {
              Type: 'P',
              Parameter: e,
              Name: 'e',
              Pos: 0,
              Field: 'Name'
            },
            Operator: '==',
            Right: "A"
          }
        },
        Operator: '&&',
        Right: {
          Type: 'B',
          Left: {
            Type: 'B',
            Left: {
              Type: 'P',
              Parameter: e,
              Name: 'e',
              Pos: 0,
              Field: 'id'
            },
            Operator: '+',
            Right: evalu.a
          },
          Operator: '>',
          Right: 2
        }
      },
      Operator: '&&',
      Right: {
        Type: 'B',
        Left: {
          Type: 'P',
          Parameter: e,
          Name: 'e',
          Pos: 0,
          Field: 'id'
        },
        Operator: 'in',
        Right: [1, 2, 3, 4]
      }
    },
    Operator: '&&',
    Right: {
      Type: 'B',
      Left: MySQL.Day({
        Type: 'P',
        Parameter: e,
        Name: 'e',
        Pos: 0,
        Field: 'Date'
      }),
      Operator: '==',
      Right: 3
    }
  };
}); 

Where P is Parameter, U is Unary Expression and B is Binary Expression, and the references to each parameter passed in the arrow function.

This can be used to create a Select function as well, a SQL query generator (like my case) or many other things.

The problem is that I have to use Babel and gulp to build, merge sourcemaps and a lot of things that I like to avoid if just Typescript add this feature :)

I hope this will be implemented soon (or the new emit system) and for now I hope that this small plugin helps others.

[santialbo]

Since Typescript has changed a lot in the last 3 years, can we have a second look and reconsider this @RyanCavanaugh? Adding Expressions a la C# that emit the AST instead of the function can give a lot of power to 3rd party libraries and ORMs.

[RyanCavanaugh]

This remains very much outside of our design goals (see earlier text on that).

[jogibear9988]

@RyanCavanaugh maybe it would be possible to create ExtensionPoints in Typescript Codegen, so we can create a Extension wich can do this?
I'd really like to have IQueryable & Linq in Typescript... (with C# Backend)

[masaeedu]

There is a hacky way to do this that doesn't rely on feeding TypeScript metadata into the runtime emitted code. For simple lambda expressions that only access properties of an object fed to them, you can invoke them with a dummy object that has get accessors that record property access and returns itself for any accessed property. This will tell you how the lambda accesses the passed objects at minimal runtime cost, and you can use it to construct whatever sort of query you want.

You wouldn't be able to do ===, but you would be able to do something like person => and(equals(person.firstName, “Jared”), equals(person.lastName, “Barnes”));.