Version: 7.10.0
Platform: macOS Sierra 10.12.5
I have a file that contains 9000+ lines,read this file by createReadStream line by line,every line is a json,I parse this json and insert the data to mysql,and 110000+ SQLs will execute,
but i get this error :
node(92378,0x1025313c0) malloc: * error for object 0xfffffffc: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
[1] 92378 abort NODE_ENV=dev node src/crawler/insertMetaAndImage.js
I have 16G memory~ when the error occur, the memory only used 8G-,
and every connection of mysql was release correctly
what's wrong?
Forgive my broken English!
thanks for any reply
The code of insertMetaAndImage.js like this (reduce many code):
const lineReader = require('readline').createInterface({
input: require('fs').createReadStream(
file || path.resolve('./src/crawler/data', 'ajson.5')
),
output: process.stdout,
terminal: false
})
lineReader.on('line', json => {
let article = JSON.parse(json)
const { images, aid } = article
DB.exec(
`DELETE FROM tableName WHERE aid = ${aid}`
).then(data => {
for (let image of images) {
DB.exec(`INSERT INTO tableName SET ?`, image).then(data => {
console.log(`the ID ${aid} insert success ....`)
})
.catch(err => {
console.log(err)
})
}
})
})
@pod4g hello,
On first look, since node has a much wider user base than just the mysql, and we don't see this bug often, I would guess that the bug is in mysql, so you might get better help in https://github.com/mysqljs/mysql/issues.
The best way to go about figuring this out is to build node or even just mysql with debug symbols, and trying to debug the C++ side of this.
Another good option will be if you could create a minimal test file (just expand the above snippet) that reproduces this error. Something we can just run with node test-mysql-pod4g.js and get the error.
Thanks @refack
I'm closing this issue. Feel free to reopen if https://github.com/mysqljs/mysql/issues/1741 doesn't pan out.
mysql is a pure JS module, IIRC, so it shouldn't be possible for it to hit a fatal error like that.
There may be other native add-ons loaded, of course. Look for .node files in your node_modules directory and exclude those.
Just had the same type of error occur during a webpack build. MacOS Sierra 10.12.4 here. Node version 7.4.0. No .node files in node_modules.
11% building modules 3/4 modules 1 active ...frontend/jekyll/sass/pages/index.scssnode(51830,0x70000c1b5000) malloc: *** error for object 0x106081020: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
abort trap: 6
Edit: Just want to leave a note here saying to disregard my entry. We found that node-sass can choke on empty files in our case, causing the cited error.
I just hit the same issue. Running node program which does many http requests reading parsing json and inserts it into a mongodb database. Program was running for approx 24 hours.
Same for me too MacOS Sierra, node version V8.4.0
node(34419,0x7fffed1f43c0) malloc: * error for object 0xb000000010412a4c: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
Abort trap: 6
Just ran into the same issue. Processing ~7 million lines of text files into objects then using those to update DynamoDB tables. Have done the same process before with no issues (albeit with about a million less lines). Also fun to note, is that the process completed on the second try. Just thought I'd point out the issue can still occur. Using v8.6.0
$ node -v
v8.6.0
node(45791,0x7fffa135b340) malloc: *** error for object 0x104c6e760: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
[1] 45791 abort node --max-old-space-size=16000 index.js
Most helpful comment
I just hit the same issue. Running node program which does many http requests reading parsing json and inserts it into a mongodb database. Program was running for approx 24 hours.
Same for me too MacOS Sierra, node version V8.4.0
node(34419,0x7fffed1f43c0) malloc: * error for object 0xb000000010412a4c: pointer being freed was not allocated
* set a breakpoint in malloc_error_break to debug
Abort trap: 6