Bluebird: Add flatMap (also known as concatMap, mapcat, mapcons, ...)

[Zirak]

Often times I find myself returning an array of array of promises, and flattening it myself with a finishing reduce. A contrived example:

getJSON('products.json')
// we receive an array of articles:
// [{}, {}, {}]
    .then(articles => articles.map(getRecommendations))
// for each article, we receive an array of recommendations:
// [[{}, {}], [{}], [{}]]
// but we want it as a single array, so we need to concat them
    .reduce(ret, recommends => ret.concat(recommends))
// we now have a flat array, yay!
// [{}, {}, {} ,{}]
    .map(recommendation => recommendation.date);

In collections-land that's known as flatMap, mapcat, concatMap and so forth.

Is it useful enough to include in core, or niche and simple enough that flattening be left to the user?

[bergus]

This should not be a promise method. What you want indeed is

getJSON(…).then(articles => articles.concatMap(getRecommendations).get("date")).…
//                                   ^^^^^^^^^

It's an array/list/collection method, not a promise one.

It's too confusing given that promises are a monad as well, but a very different one. Btw, we tend to call it chain in JS.

[benjamingr]

I don't care about the theoretical implications of mixing two monads. I think flatMap is useful, I just wonder if it should be in core.

[Zirak]

@bergus Good point, the example was indeed overly simplistic. Let's look at a scenario where the user is not in charge of the nested array, but it's given by an external resource:

var tags = ['horror', 'fiction', 'sci-fi'];
// we have a list:
// []
Promise.map(tags, queryImdb)
// for every tag, imdb returns a list of movies, so we again have a list of lists:
// [ [{}, {}], [{}], [{}] ]
// which we need to flatten :(

[bergus]

OK, I'd be fine with Promise.concatMap (the "_concat_" part makes it clear that it's about arrays), though I guess that Promise.map(tags, queryImdb).then(_.flatten) isn't too bad either. Is it really that common that it would warrant an extra core method?

[spion]

My opinion is that we should stop adding array-related methods to core and start planning how to take the existing ones out (preferably into a companion Stream library :)

[benjamingr]

My opinion is that we should stop adding array-related methods to core and start planning how to take the existing ones out (preferably into a companion Stream library :)

99% of our users don't care about how sound or how nice the theory or the types are. People care about usefulness.

In practice, map filter and other methods are seeing a ton of use in bluebird. Removing them would be a spit in the face of our users telling them that theoretical niceness trumps actual usefulness.

If abstract references (bind operator, or whatever @zenparsing is calling it today) make it in that changes my opinion since it would let us extend stuff from the outside.

[benjamingr]

Although frankly virtually all my new code uses async/await (or at least .coroutine) anyway. So there's that.

[spion]

@benjamingr your argument is irrelevant, I'm not talking about theory but about practical benefits. I'm not saying to remove them but to replace them with something better. And by better I mean more performant, uses less memory (avoids allocations), maybe even "supports back pressure" and other niceties.

See https://github.com/cujojs/most/blob/master/test/perf/README.md

With a sufficiently nice API that plays well with promises, it will be just as easy and convenient to use.

[petkaantonov]

If you have lodash or such you can do:

getJSON('products.json')
// we receive an array of articles:
// [{}, {}, {}]
    .map(getRecommendations).then(_).call("flatten").call("value")
    .map(recommendation => recommendation.date);

[grantila]

Adding arbitrary array helpers for fun without actually solving problems regarding _promises_ doesn't make any sense at all.

Bluebird's map, reduce and filter functions are not just nice array helpers. After all, we already have this in ES5. Bluebird's versions work with arrays of promises and values mixed, as well as with operators (the function to map, filter and reduce) which might return promises. This logic is non-trivial, and the support in bluebird (or in a support library) makes perfect sense.

If this actually _would_ respect promises in both levels - the outer and inner arrays - it could potentially be bluebird-material, but It would affect the interpretation of the concurrency option in non-obvious ways. In many situations you would like to have different weights to the concurrency of the outer and inner mapping jobs, and then all of a sudden it becomes a bit messy.

[djanowski]

Array.flatMap() is part of Javascript today and widely supported. Could this live in core?