Rocketmq: while not use FileChannel.write()? rocketmq use mmap file write , may cause use a lot of share memory
here is a test for FileChannel.write() , and mmap file write
https://github.com/jkreps/valencia/blob/master/src/test/java/valencia/TestLinearWritePerformance.java
what,s more , MaxDirectMemorySize dont work for memory hold by mmap file .
Ah39
All 4 comments
MaxDirectMemorySize dont work for memory hold by mmap file .SHR memory is equals the mmap file size .
Ah39
on 4 Dec 2018
@Ah39 Compared with FileChannel, MMAP can write to the page cache directly, but FileChannel with directbuffer needs to write to memory first, then write to the page cache. so MMAP has an advantage when writing a lot of small data into a file.
duhengforever
on 5 Dec 2018
@duhengforever but the rocketmq may hold 100G even more SHR memory .
how to control the memory use by rocketmq ,by mmap file ??
Ah39
on 5 Dec 2018
@duhengforever but the rocketmq may hold 100G even more SHR memory .
how to control the memory use by rocketmq ,by mmap file ??
Maybe using the dio in the future will be a good choice, memory management is entirely done by the application layer. However, This is a major change, and if you have any ideas, feel free to discuss them in the [email protected] mailing list.
duhenglucky
on 12 Mar 2020
Related issues
baihezhuo
·
3Comments
gchaoxue
·
5Comments
Switch-vov
·
3Comments
seaHeater
·
5Comments
zzwlstarby
·
5Comments
Most helpful comment
@Ah39 Compared with FileChannel, MMAP can write to the page cache directly, but FileChannel with directbuffer needs to write to memory first, then write to the page cache. so MMAP has an advantage when writing a lot of small data into a file.