Redis: "Raw" byte slice command

[wyndhblb]

Under large load (many thousands of commands/second), the command -> string interface of the commands incurs huge GC issues (lots and lots of string copies that need collecting). For 95% of uses this is ok, but that this load, a better solution is to keep things as byte slices and allow for those slices to get directly piped into either the pipeline buffer or sent to the pool connection directly. As a start one can simply introduce a "raw" function that simply copies a byte slice to the buffer rather then creating the command from arguments.

For example rather then

conn.ZAdd(key, redis.Z{Score: Number), Member: MyKey})

allow for

conn.Raw(mySlice)

where the mySlice is composed before hand (usually using sync.Pool byte buffers and custom custom optimized encoders). This will be particularly good for pipelines

[vmihailenco]

I am not sure I fully understand what you are proposing. conn.ZAdd(key, redis.Z{Score: Number, Member: MyKey}) allocates like 100 bytes and that does not involve copying strings. I don't think that conn.Raw will make huge difference.

https://github.com/go-redis/redis/pull/475 should reduce number of allocations in some cases, but it will not make a huge difference too.

the command -> string interface of the commands incurs huge GC issues (lots and lots of string copies that need collecting)

Can you please point me to these allocations?

[wyndhblb]

Hi .. when doing a 10k/s command into redis, the GC went insane (basically 50% of all CPU usage was GC mark/collecting). I pprof traced the issue to this module where it was taking ~40% of all allocated memory. Basically the serialization of the redis CMDs objects down to "strings" then down to bytes. In normal use cases, this is probably not a issue.

ZAdd is just what i was doing, (but i did a few other commands w/ the same result Sadd, Set, etc) imagine any commands suffer from the same issue at that kind of volume. So i was just proposing that there is a mechanism to bypass the cmd serialization process and insert basically a []byte into a [][]byte command list. Or direct to the base io.Writer of the socket buffer itself.

Basically i had to revert to raw TCP socket io.Writer in order to not have this issue (but in clustering mode, or ring mode, things need more fancy logic.

Another suggestion is to get that io.Writer from the module connection itself so the pooling and cluster/ring/sentinel goodies are still in play.

...gopkg.in/redis.v5/commands.go
         .          .   1241:}
         .          .   1242:
         .          .   1243:// Redis `ZADD key score member [score member ...]` command.
         .          .   1244:func (c *cmdable) ZAdd(key string, members ...Z) *IntCmd {
         .          .   1245:   const n = 2
   1588008    1588008   1246:   a := make([]interface{}, n+2*len(members))
   2950235    2950235   1247:   a[0], a[1] = "zadd", key
         .    4819391   1248:   return c.zAdd(a, n, members...)
         .          .   1249:}
         .          .   1250:
         .          .   1251:// Redis `ZADD key NX score member [score member ...]` command.
         .          .   1252:func (c *cmdable) ZAddNX(key string, members ...Z) *IntCmd {
         .          .   1253:   const n = 3

...gopkg.in/redis.v5/commands.go
         .          .   1230:   Aggregate string
         .          .   1231:}
         .          .   1232:
         .          .   1233:func (c *cmdable) zAdd(a []interface{}, n int, members ...Z) *IntCmd {
         .          .   1234:   for i, m := range members {
   1468867    1468867   1235:       a[n+2*i] = m.Score
         .          .   1236:       a[n+2*i+1] = m.Member
         .          .   1237:   }
         .    3330344   1238:   cmd := NewIntCmd(a...)
         .      20180   1239:   c.process(cmd)
         .          .   1240:   return cmd
         .          .   1241:}
         .          .   1242:
         .          .   1243:// Redis `ZADD key score member [score member ...]` command.
         .          .   1244:func (c *cmdable) ZAdd(key string, members ...Z) *IntCmd {



md5-c78dfb81ca19e91db2d30078caa5fa3c



/gopkg.in/redis.v5/commands.go
         .          .   1069:}
         .          .   1070:
         .          .   1071://------------------------------------------------------------------------------
         .          .   1072:
         .          .   1073:func (c *cmdable) SAdd(key string, members ...interface{}) *IntCmd {
   3248694    3248694   1074:   args := make([]interface{}, 2+len(members))
   2943985    2943985   1075:   args[0] = "sadd"
   3112748    3112748   1076:   args[1] = key
         .          .   1077:   for i, member := range members {
         .          .   1078:       args[2+i] = member
         .          .   1079:   }
         .    6207506   1080:   cmd := NewIntCmd(args...)
         .          .   1081:   c.process(cmd)
         .          .   1082:   return cmd
         .          .   1083:}
         .          .   1084:

[vmihailenco]

I see 3 types of allocations there:

• slice of interfaces for _args []interface{}
• interface{} itself via runtime.convT2E
• memory for Cmd struct

All those allocations are very small <= 64bytes and I don't see how it is possible to avoid them. It is also hard to imagine the type of program that will suffer from such allocations, but obviously you have just such a program...

I will try to add RawCmd, but it looks like a micro-optimization to me. You probably just need a bigger / more servers to run your app...

[dim]

We have a similar application while Go's GC pauses are improving from release to release, they also out

[dim]

Sorry, writing on my phone, let's try this again :)

We have a similar application while Go's GC pauses are improving from release to release, they also put additional pressure on CPU resources. I too toying with a RawCmd idea, but never had the time to look into it properly.

[vmihailenco]

RawCmd will require exporting some private API to get cmd name and first key:

func NewRawCmd(name, firstKey string, payload []byte) {}

That will avoid some overhead at the cost that user has to write all the serialization code himself, which sounds almost like requiring user to use assembler...

Another idea would be to move []interface{} allocation into command struct like this:

type baseCmd struct {
    bootstrap [10]interface{}
    _args []interface{}
}

and allocate args for most commands from bootstrap:

cmd._args = cmd.bootstrap[:4]

That will slightly increase memory usage, but decrease number of allocations. And hopefully help GC.

[vmihailenco]

That will slightly increase memory usage, but decrease number of allocations. And hopefully help GC.

I just tried and it removes exactly 1 allocation by increasing memory usage by almost 90 bytes. Does not look too impressing...

[vmihailenco]

@wyndhblb I wonder what is the performance difference between go-redis & raw TCP socket in your case?

The only idea that seems worth a try is to add WriteBuffer to Cmd:

var cmd redis.StringCmd
wb := cmd.WriteBuffer() // uses sync.Pool to allocate buffer
wb.AppendArgsNum(3)
wb.AppendString("set")
wb.AppendString("foo")
wb.AppendBytes([]byte("bar"))
client.Process(cmd) // returns buffer to sync.Pool

The only dubious part is using sync.Pool to allocate WriteBuffer which looks like a manual memory management...

[vmihailenco]

Closing since there is no much interest in this.

[liilac]

This may also be used with redis modules that have no client support, such as rejson.

[vmihailenco]

I don't see how this can be used with rejson, but I don't have a plan for rejson so maybe...

[youngjoon-lee]

I also really want to get []byte from Get(). In many cases, we store byte arrays instead of string. For example, I use Redis as a cache for the gRPC client. It means I use the unmarshalled protobuf message as Redis key/value.
If I call []byte(stringFromGet) whenever I call Get(), it will definitely cause lots of GCs.
I know we have the Bytes() function, but it does the same thing with the type casting.
Also, it would be so useful to use []byte as a key for Get() and Set().
Please consider this new feature. It will improve performance a lot.

[vmihailenco]

There is already rdb.Get("foo").Bytes() which I've just made faster in https://github.com/go-redis/redis/pull/1106. We can also add SetBytes to save 1 interface{} allocation but I wonder if that is measurable...

[youngjoon-lee]

There is already rdb.Get("foo").Bytes() which I've just made faster in #1106. We can also add SetBytes to save 1 interface{} allocation but I wonder if that is measurable...

Thank you for your reply. I didn’t know you’ve used unsafe for performance. I think it is enough for now. Thank you.