radare2 is unusably slow for large binaries

[crowell]

steps to reproduce

• r2 /bin/bash
• aaa
• try to use r2
• cry

i know "dont use auto analysis" but the thing is by the time you've annotated a large bin, its the same.

link performance bugs under this one.

[lionaneesh]

I think a work around for thing would be a complete revamp of the RBinAPI. Right now we load the whole binary in memory, parse it and save these humongous structures (RBin, RBinObject) on the heap, which I think is the cause of this issue.

[radare]

rbin is only part of the problem, this only affects to load times. the main problem appears when analyzing and having tons of functions which makes many operations slow because of the number of flags and basic blocks to iterate.

[radare]

skiplist doesnt solves the problem

[radare]

what's the next bottleneck here? can anyone profile and get some results pasted here?

[Maykeye]

I've done analyzing libxul from firefox on

radare2 1.2.0-git 13411 @ linux-x86-64 git.1.1.0-103-g8e59afb
commit: 8e59afb91c4adb6347ec63c991889d97eafcf021 build: 2017-01-03

running under callgrind.

~25 trillions instructions and ~3 days later, most time was spend in these functions:

• r_flag_get_at (28.53%, ~7 trillions Irs)

• r_anal_get_fcn_at (11.64% ~2.9 trillion Irs)

• and __GI_strchr from libc itself! (10% ~2.6 trillions Irs)

Full callgrind file is in attachement callgrind_result.zip

(I also accidentally ran newer binary with older libraries, for comparison, before optimization in r_anal_get_fcn_at, it took 39 trillions instructions and 5 days to complete libxul)

r2 was called as r2 libxul.so.from.firefox -c aaa -s -i exit -q where exit is a file with single line quit (I don't remember why I didn't pass -c quit)

[radare]

Interesting! i bet we can optimize the r_flag_get_at by using Sdb instead of iterating over all the flags. this seems like a nice optimization to do, i though main issue was in r_anal_get_fcn_at.. but i guess that after the optimizations this is no longer an issue.

we should focus on r_flags now imho :)

On 06 Jan 2017, at 06:59, Maykeye [e-mail] wrote:

I've done analyzing libxul from firefox on

radare2 1.2.0-git 13411 @ linux-x86-64 git.1.1.0-103-g8e59afb
commit: 8e59afb https://github.com/radare/radare2/commit/8e59afb91c4adb6347ec63c991889d97eafcf021 build: 2017-01-03

running under callgrind.

~25 trillions instructions and ~3 days later, most time was spend in these functions:

r_flag_get_at (28.53%, ~7 trillions Irs)

r_anal_get_fcn_at (11.64% ~2.9 trillion Irs)

and __GI_strchr from libc itself! (10% ~2.6 trillions Irs)

Full callgrind file is in attachement callgrind_result.zip https://github.com/radare/radare2/files/689059/callgrind_result.zip
(I also accidentally ran newer binary with older libraries, for comparison, before optimization in r_anal_get_fcn_at, it took 39 trillions instructions and 5 days to complete libxul)

r2 was called as r2 libxul.so.from.firefox -c aaa -s -i exit -q where exit is a file with single line quit (I don't remember why I didn't pass -c quit)

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/radare/radare2/issues/5241#issuecomment-270837935, or mute the thread https://github.com/notifications/unsubscribe-auth/AA3-lvVtgdqVqNy-KwYzhAYj2fvoFVSrks5rPdhbgaJpZM4JB1zP.

[radare]

closing this as long as its a known issue and we improve from time to time in this direction.

[radare]

cc @Maijin