Bluebird: Bluebird enumerates entire iterable in `.map` and `.forEach` (and probably any other method that accepts iterables)

Created on 24 Jun 2016  路  5Comments  路  Source: petkaantonov/bluebird

1) What version of bluebird is the issue happening on?
3.4.1

2) What platform and version? (For example Node.js 0.12 or Google Chrome 32)
Node.js 6.2.0

3) Did this issue happen with earlier version of bluebird?
3.0 onwards (as earlier versions did not support iterable).

(Write description of your issue here, stack traces from errors and code that reproduces the issue are helpful)

Given the following code:

const bluebird = require('bluebird');

function* test() {
    let i = 0;
    while (i++ < 10) {
        console.log('yield ' + i);
        yield i;
    }
}

bluebird.each(
  test(), 
  function(i) { 
    console.log('process ' + i); 
    return Promise.resolve(i); 
  }
);

Bluebird will enumerate the generator until finished (which I think internally it must be just converting it to an array).

This causes the following problems:

  • You want to iterate over something of infinite value forever
  • Your generator yields a promise for an asynchronous operation, which leads to confusion when using concurrency in the map option

Most helpful comment

I do not see the point in supporting _iterables_ if it just drains the whole iterator into an array. The client could already do that itself. I have an iterable that eventually produces hundreds of thousands of values, and I'd like them processed asynchronously and sequentially.

All 5 comments

These are features not bugs.

You want to iterate over something of infinite value forever

Don't do that, and what else should Bluebird sensibly do as an alternative to the current behaviour?

Your generator yields a promise for an asynchronous operation, which leads to confusion when using concurrency in the map option

Being able to return promises from these helper methods is the entire point of the methods :)

Bluebird converts the passed-in iterable to an array.

We discussed what you are proposing but we never implemented it. I'm +1 on it.

I do not see the point in supporting _iterables_ if it just drains the whole iterator into an array. The client could already do that itself. I have an iterable that eventually produces hundreds of thousands of values, and I'd like them processed asynchronously and sequentially.

I wonder why @petkaantonov closed this issue.

For anyone else needing to iterate an iterable without enumerating it in it's entirety first, try: https://github.com/leahciMic/promise-each-concurrency

Was this page helpful?
0 / 5 - 0 ratings

Related issues

SimonSchick picture SimonSchick  路  6Comments

julien-f picture julien-f  路  5Comments

sgerace picture sgerace  路  3Comments

wearhere picture wearhere  路  5Comments

chrisprobst picture chrisprobst  路  5Comments