Node-sqlite3: Add the ability to get rows one by one

Created on 21 Jan 2017  路  2Comments  路  Source: mapbox/node-sqlite3

Hi, and thanks for this helpfull project.
It would be nice to have the ability to handle rows one by one.
Currently, if we have an asynchronous function running in .each(), node-sqlite3 has no mean to know when we finished our asynchronous processing of one row, and immediately calls the function again with the next row. So if we have one million rows, then we will have one million asynchronous operation running simultaneously.

This makes it impossible to work with result sets that don't fit into memory.

Most helpful comment

Hi,
you can use Statement.get for retrieving the rows one by one (just don't submit any parameters, and you will get the next row). I have used this for creating an .each() function that don't load all rows into memory (see https://gist.github.com/erikman/1ad6c5dc57edfedde038082d0c54644d for an example), this is also discussed in #686

All 2 comments

Hi,
you can use Statement.get for retrieving the rows one by one (just don't submit any parameters, and you will get the next row). I have used this for creating an .each() function that don't load all rows into memory (see https://gist.github.com/erikman/1ad6c5dc57edfedde038082d0c54644d for an example), this is also discussed in #686

Thank you!

Maybe that would be worth documenting...
Currently, the docs say

Runs the SQL query with the specified parameters and calls the callback with the first result row afterwards.

(https://github.com/mapbox/node-sqlite3/wiki/API#databasegetsql-param--callback)

Was this page helpful?
0 / 5 - 0 ratings

Related issues

sampsongao picture sampsongao  路  48Comments

sveinnM picture sveinnM  路  25Comments

milu2003 picture milu2003  路  16Comments

jpillora picture jpillora  路  18Comments

j-berman picture j-berman  路  20Comments