Netty: Netty 4 PooledByteBufAllocator returns incorrect information about allocations/deallocations

[codekrolik]

The following test outputs numbers from PooledByteBufAllocator arenas, and when I run it the numbers don't match the actual allocations/deallocations.
Actual allocations: 50000
Actual deallocations: 50000
Active buffers: 0

the output shows the following:
directActive 1 directAlloc 1 directDealloc 0
heapActive 0 heapAlloc 0 heapDealloc 0

import io.netty.buffer.ByteBuf;
import io.netty.buffer.PoolArenaMetric;
import io.netty.buffer.PooledByteBufAllocator;
import io.netty.util.ResourceLeakDetector;
import io.netty.util.ResourceLeakDetector.Level;

import org.junit.Test;

public class NettyTest2 {
    public static PooledByteBufAllocator alloc = new PooledByteBufAllocator(true); 

    @Test
    public void test() throws InterruptedException {
        ResourceLeakDetector.setLevel(Level.PARANOID);

        for (int i = 0; i < 50000; i++) {
            ByteBuf b = alloc.buffer(1000, 1000);
            b.release();
        }

        Thread.sleep(100);

        //alloc.freeThreadLocalCache();

        showActiveDirectBuffers();
        showActiveHeapBuffers();
    }

    void showActiveDirectBuffers() {
        int directActive = 0, directAlloc = 0, directDealloc = 0;
        for (PoolArenaMetric arena : alloc.directArenas()) {
            directActive += arena.numActiveAllocations();
            directAlloc += arena.numAllocations();
            directDealloc += arena.numDeallocations();
        }
        System.out.println("directActive " + directActive + " directAlloc " + directAlloc + " directDealloc " + directDealloc);
    }

    void showActiveHeapBuffers() {
        int heapActive = 0, heapAlloc = 0, heapDealloc = 0;
        for (PoolArenaMetric arena : alloc.heapArenas()) {
            heapActive += arena.numActiveAllocations();
            heapAlloc += arena.numAllocations();
            heapDealloc += arena.numDeallocations();
        }
        System.out.println("heapActive " + heapActive + " heapAlloc " + heapAlloc + " heapDealloc " + heapDealloc);
    }
}

[taojiaenx]

When you remove the code "b.release();" the program will show this:
Recent access records: 0
Created at:
io.netty.buffer.PooledByteBufAllocator.newDirectBuffer(PooledByteBufAllocator.java:280)
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:177)
io.netty.buffer.AbstractByteBufAllocator.buffer(AbstractByteBufAllocator.java:113)
TestAllocator.main(TestAllocator.java:18)
maindirectActive 43750 directAlloc 50000 directDealloc 6250
And there is a big memory leak
so maybe the reason is the buffer has been released

[johnou]

Isn't that the point of the _pooled_ allocator that bytebufs are reused?

Random rnd = new Random();

for (int i = 0; i < 50000; i++) {
    ByteBuf b = alloc.buffer(1000, 10000);
    int randomNum = 200 + rnd.nextInt((10000 - 200) + 1);
    b.writeBytes(new byte[randomNum] );
    b.release();
}

directActive 5 directAlloc 5 directDealloc 0
heapActive 0 heapAlloc 0 heapDealloc 0

[normanmaurer]

@codekrolik the problem is that you not "disabled" the caches so it makes perfect sense to see the numbers you are reporting as the cache is still holding the buffers. If you use something like this it will work:

PooledByteBufAllocator alloc = new PooledByteBufAllocator(true, 1, 1, 8192, 11, 0, 0, 0);

[taojiaenx]

@normanmaurer Would you tell me that what is the role of "cache" ?

[normanmaurer]

These are thread local caches which allow to do allocations without any synchronization if something is in the cache. If there are a lot of unused buffers in the caches these will be freed up at an interval.

For more details about it please see the jemalloc paper.

Am 24.05.2016 um 07:16 schrieb taojiaenx [email protected]:

@normanmaurer Would you tell me that what is the role of "cache" ?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub