Redisearch: A special FT.SEARCH with "on-the-spot" can make cpu full load

[zhangkyou]

Steps to reproduce the bug:
1."FT.CREATE" "myIndex" "SCHEMA" "body" "TEXT" "ibody" "TEXT" "url" "TEXT" "title" "TEXT" "WEIGHT" "5" "SORTABLE"
2."FT.ADD" "myIndex" "d_739e6d2a73723ec7b1919fa5a51f9b07" "0.99994844" "FIELDS" "title" "Analysis" "body" "New" "ibody" "New" "url" "http"
3."FT.SEARCH" "myIndex" "on-the-spot" "LIMIT" "0" "30" "RETURN" "3" "title" "ibody" "url"

On the step 3,the cpu load becomes full, and then the redis crash.

By a few test,the search string that likes "on-the*" seem all can cause this bug.

[salt4pommes]

I was able to reproduct this on redis 4.0.6 and redisearch 1.0.4 using the three steps above.

I'll try to debug and find the loop :)

[salt4pommes]

Interesting Fact:

• There is no need for a document in index, you can ignore step 2

Seems we have the loop at the intersect iterator:

index.c in function: int II_Read(void ctx, RSIndexResult *hit)

I've enabled the debugging output. There is only 1 document in the index, but it seems to found unlimited documents :)

II 0x7fffd8000bb0 last docId 0, it 0 read docId 1(1), rc 1
II 0x7fffd8000bb0 last docId 1, it 1 read docId 1(0), rc 1
II 0x7fffd8000bb0 HIT @ 1
II 0x7fffd8000bb0 last docId 2, it 0 read docId 2(1), rc 1
II 0x7fffd8000bb0 last docId 2, it 1 read docId 2(0), rc 1
II 0x7fffd8000bb0 HIT @ 2
II 0x7fffd8000bb0 last docId 3, it 0 read docId 3(1), rc 1
II 0x7fffd8000bb0 last docId 3, it 1 read docId 3(0), rc 1
II 0x7fffd8000bb0 HIT @ 3
II 0x7fffd8000bb0 last docId 4, it 0 read docId 4(1), rc 1
II 0x7fffd8000bb0 last docId 4, it 1 read docId 4(0), rc 1
II 0x7fffd8000bb0 HIT @ 4
II 0x7fffd8000bb0 last docId 5, it 0 read docId 5(1), rc 1
II 0x7fffd8000bb0 last docId 5, it 1 read docId 5(0), rc 1
II 0x7fffd8000bb0 HIT @ 5
II 0x7fffd8000bb0 last docId 6, it 0 read docId 6(1), rc 1
II 0x7fffd8000bb0 last docId 6, it 1 read docId 6(0), rc 1
II 0x7fffd8000bb0 HIT @ 6
II 0x7fffd8000bb0 last docId 7, it 0 read docId 7(1), rc 1
II 0x7fffd8000bb0 last docId 7, it 1 read docId 7(0), rc 1
II 0x7fffd8000bb0 HIT @ 7
II 0x7fffd8000bb0 last docId 8, it 0 read docId 8(1), rc 1
II 0x7fffd8000bb0 last docId 8, it 1 read docId 8(0), rc 1
II 0x7fffd8000bb0 HIT @ 8
II 0x7fffd8000bb0 last docId 9, it 0 read docId 9(1), rc 1
II 0x7fffd8000bb0 last docId 9, it 1 read docId 9(0), rc 1
II 0x7fffd8000bb0 HIT @ 9
II 0x7fffd8000bb0 last docId 10, it 0 read docId 10(1), rc 1
II 0x7fffd8000bb0 last docId 10, it 1 read docId 10(0), rc 1
II 0x7fffd8000bb0 HIT @ 10
II 0x7fffd8000bb0 last docId 11, it 0 read docId 11(1), rc 1
II 0x7fffd8000bb0 last docId 11, it 1 read docId 11(0), rc 1
II 0x7fffd8000bb0 HIT @ 11
II 0x7fffd8000bb0 last docId 12, it 0 read docId 12(1), rc 1
II 0x7fffd8000bb0 last docId 12, it 1 read docId 12(0), rc 1
II 0x7fffd8000bb0 HIT @ 12
II 0x7fffd8000bb0 last docId 13, it 0 read docId 13(1), rc 1
II 0x7fffd8000bb0 last docId 13, it 1 read docId 13(0), rc 1
II 0x7fffd8000bb0 HIT @ 13
II 0x7fffd8000bb0 last docId 14, it 0 read docId 14(1), rc 1
II 0x7fffd8000bb0 last docId 14, it 1 read docId 14(0), rc 1
II 0x7fffd8000bb0 HIT @ 14
II 0x7fffd8000bb0 last docId 15, it 0 read docId 15(1), rc 1
II 0x7fffd8000bb0 last docId 15, it 1 read docId 15(0), rc 1
II 0x7fffd8000bb0 HIT @ 15
II 0x7fffd8000bb0 last docId 16, it 0 read docId 16(1), rc 1
II 0x7fffd8000bb0 last docId 16, it 1 read docId 16(0), rc 1
II 0x7fffd8000bb0 HIT @ 16
II 0x7fffd8000bb0 last docId 17, it 0 read docId 17(1), rc 1
II 0x7fffd8000bb0 last docId 17, it 1 read docId 17(0), rc 1
II 0x7fffd8000bb0 HIT @ 17
II 0x7fffd8000bb0 last docId 18, it 0 read docId 18(1), rc 1
II 0x7fffd8000bb0 last docId 18, it 1 read docId 18(0), rc 1
II 0x7fffd8000bb0 HIT @ 18
II 0x7fffd8000bb0 last docId 19, it 0 read docId 19(1), rc 1
II 0x7fffd8000bb0 last docId 19, it 1 read docId 19(0), rc 1
II 0x7fffd8000bb0 HIT @ 19
II 0x7fffd8000bb0 last docId 20, it 0 read docId 20(1), rc 1
II 0x7fffd8000bb0 last docId 20, it 1 read docId 20(0), rc 1
II 0x7fffd8000bb0 HIT @ 20
II 0x7fffd8000bb0 last docId 21, it 0 read docId 21(1), rc 1
II 0x7fffd8000bb0 last docId 21, it 1 read docId 21(0), rc 1
II 0x7fffd8000bb0 HIT @ 21
II 0x7fffd8000bb0 last docId 22, it 0 read docId 22(1), rc 1
II 0x7fffd8000bb0 last docId 22, it 1 read docId 22(0), rc 1
II 0x7fffd8000bb0 HIT @ 22
II 0x7fffd8000bb0 last docId 23, it 0 read docId 23(1), rc 1
II 0x7fffd8000bb0 last docId 23, it 1 read docId 23(0), rc 1
II 0x7fffd8000bb0 HIT @ 23
II 0x7fffd8000bb0 last docId 24, it 0 read docId 24(1), rc 1
II 0x7fffd8000bb0 last docId 24, it 1 read docId 24(0), rc 1
II 0x7fffd8000bb0 HIT @ 24
II 0x7fffd8000bb0 last docId 25, it 0 read docId 25(1), rc 1
II 0x7fffd8000bb0 last docId 25, it 1 read docId 25(0), rc 1
II 0x7fffd8000bb0 HIT @ 25
II 0x7fffd8000bb0 last docId 26, it 0 read docId 26(1), rc 1
II 0x7fffd8000bb0 last docId 26, it 1 read docId 26(0), rc 1
II 0x7fffd8000bb0 HIT @ 26
II 0x7fffd8000bb0 last docId 27, it 0 read docId 27(1), rc 1
II 0x7fffd8000bb0 last docId 27, it 1 read docId 27(0), rc 1
II 0x7fffd8000bb0 HIT @ 27
II 0x7fffd8000bb0 last docId 28, it 0 read docId 28(1), rc 1
II 0x7fffd8000bb0 last docId 28, it 1 read docId 28(0), rc 1
II 0x7fffd8000bb0 HIT @ 28
II 0x7fffd8000bb0 last docId 29, it 0 read docId 29(1), rc 1
II 0x7fffd8000bb0 last docId 29, it 1 read docId 29(0), rc 1
II 0x7fffd8000bb0 HIT @ 29
II 0x7fffd8000bb0 last docId 30, it 0 read docId 30(1), rc 1
II 0x7fffd8000bb0 last docId 30, it 1 read docId 30(0), rc 1
II 0x7fffd8000bb0 HIT @ 30
II 0x7fffd8000bb0 last docId 31, it 0 read docId 31(1), rc 1
II 0x7fffd8000bb0 last docId 31, it 1 read docId 31(0), rc 1
II 0x7fffd8000bb0 HIT @ 31
II 0x7fffd8000bb0 last docId 32, it 0 read docId 32(1), rc 1
II 0x7fffd8000bb0 last docId 32, it 1 read docId 32(0), rc 1
II 0x7fffd8000bb0 HIT @ 32
II 0x7fffd8000bb0 last docId 33, it 0 read docId 33(1), rc 1
II 0x7fffd8000bb0 last docId 33, it 1 read docId 33(0), rc 1
II 0x7fffd8000bb0 HIT @ 33
II 0x7fffd8000bb0 last docId 34, it 0 read docId 34(1), rc 1
II 0x7fffd8000bb0 last docId 34, it 1 read docId 34(0), rc 1
II 0x7fffd8000bb0 HIT @ 34
II 0x7fffd8000bb0 last docId 35, it 0 read docId 35(1), rc 1
II 0x7fffd8000bb0 last docId 35, it 1 read docId 35(0), rc 1
II 0x7fffd8000bb0 HIT @ 35
II 0x7fffd8000bb0 last docId 36, it 0 read docId 36(1), rc 1
II 0x7fffd8000bb0 last docId 36, it 1 read docId 36(0), rc 1
II 0x7fffd8000bb0 HIT @ 36
II 0x7fffd8000bb0 last docId 37, it 0 read docId 37(1), rc 1
II 0x7fffd8000bb0 last docId 37, it 1 read docId 37(0), rc 1
II 0x7fffd8000bb0 HIT @ 37
II 0x7fffd8000bb0 last docId 38, it 0 read docId 38(1), rc 1
II 0x7fffd8000bb0 last docId 38, it 1 read docId 38(0), rc 1
II 0x7fffd8000bb0 HIT @ 38
II 0x7fffd8000bb0 last docId 39, it 0 read docId 39(1), rc 1
II 0x7fffd8000bb0 last docId 39, it 1 read docId 39(0), rc 1
II 0x7fffd8000bb0 HIT @ 39
II 0x7fffd8000bb0 last docId 40, it 0 read docId 40(1), rc 1
II 0x7fffd8000bb0 last docId 40, it 1 read docId 40(0), rc 1
II 0x7fffd8000bb0 HIT @ 40
II 0x7fffd8000bb0 last docId 41, it 0 read docId 41(1), rc 1
II 0x7fffd8000bb0 last docId 41, it 1 read docId 41(0), rc 1
II 0x7fffd8000bb0 HIT @ 41
II 0x7fffd8000bb0 last docId 42, it 0 read docId 42(1), rc 1
II 0x7fffd8000bb0 last docId 42, it 1 read docId 42(0), rc 1
II 0x7fffd8000bb0 HIT @ 42
II 0x7fffd8000bb0 last docId 43, it 0 read docId 43(1), rc 1
II 0x7fffd8000bb0 last docId 43, it 1 read docId 43(0), rc 1
II 0x7fffd8000bb0 HIT @ 43
II 0x7fffd8000bb0 last docId 44, it 0 read docId 44(1), rc 1
II 0x7fffd8000bb0 last docId 44, it 1 read docId 44(0), rc 1
II 0x7fffd8000bb0 HIT @ 44
II 0x7fffd8000bb0 last docId 45, it 0 read docId 45(1), rc 1
II 0x7fffd8000bb0 last docId 45, it 1 read docId 45(0), rc 1
II 0x7fffd8000bb0 HIT @ 45
II 0x7fffd8000bb0 last docId 46, it 0 read docId 46(1), rc 1
II 0x7fffd8000bb0 last docId 46, it 1 read docId 46(0), rc 1
II 0x7fffd8000bb0 HIT @ 46
II 0x7fffd8000bb0 last docId 47, it 0 read docId 47(1), rc 1
II 0x7fffd8000bb0 last docId 47, it 1 read docId 47(0), rc 1
II 0x7fffd8000bb0 HIT @ 47
II 0x7fffd8000bb0 last docId 48, it 0 read docId 48(1), rc 1
II 0x7fffd8000bb0 last docId 48, it 1 read docId 48(0), rc 1
II 0x7fffd8000bb0 HIT @ 48
II 0x7fffd8000bb0 last docId 49, it 0 read docId 49(1), rc 1
II 0x7fffd8000bb0 last docId 49, it 1 read docId 49(0), rc 1
II 0x7fffd8000bb0 HIT @ 49
II 0x7fffd8000bb0 last docId 50, it 0 read docId 50(1), rc 1
II 0x7fffd8000bb0 last docId 50, it 1 read docId 50(0), rc 1
II 0x7fffd8000bb0 HIT @ 50
II 0x7fffd8000bb0 last docId 51, it 0 read docId 51(1), rc 1
II 0x7fffd8000bb0 last docId 51, it 1 read docId 51(0), rc 1
II 0x7fffd8000bb0 HIT @ 51
II 0x7fffd8000bb0 last docId 52, it 0 read docId 52(1), rc 1
II 0x7fffd8000bb0 last docId 52, it 1 read docId 52(0), rc 1
II 0x7fffd8000bb0 HIT @ 52
II 0x7fffd8000bb0 last docId 53, it 0 read docId 53(1), rc 1
II 0x7fffd8000bb0 last docId 53, it 1 read docId 53(0), rc 1
II 0x7fffd8000bb0 HIT @ 53
II 0x7fffd8000bb0 last docId 54, it 0 read docId 54(1), rc 1
II 0x7fffd8000bb0 last docId 54, it 1 read docId 54(0), rc 1
II 0x7fffd8000bb0 HIT @ 54
II 0x7fffd8000bb0 last docId 55, it 0 read docId 55(1), rc 1
II 0x7fffd8000bb0 last docId 55, it 1 read docId 55(0), rc 1
II 0x7fffd8000bb0 HIT @ 55
II 0x7fffd8000bb0 last docId 56, it 0 read docId 56(1), rc 1
II 0x7fffd8000bb0 last docId 56, it 1 read docId 56(0), rc 1
II 0x7fffd8000bb0 HIT @ 56
II 0x7fffd8000bb0 last docId 57, it 0 read docId 57(1), rc 1
II 0x7fffd8000bb0 last docId 57, it 1 read docId 57(0), rc 1
II 0x7fffd8000bb0 HIT @ 57
II 0x7fffd8000bb0 last docId 58, it 0 read docId 58(1), rc 1
II 0x7fffd8000bb0 last docId 58, it 1 read docId 58(0), rc 1
II 0x7fffd8000bb0 HIT @ 58
II 0x7fffd8000bb0 last docId 59, it 0 read docId 59(1), rc 1
II 0x7fffd8000bb0 last docId 59, it 1 read docId 59(0), rc 1
II 0x7fffd8000bb0 HIT @ 59
II 0x7fffd8000bb0 last docId 60, it 0 read docId 60(1), rc 1
II 0x7fffd8000bb0 last docId 60, it 1 read docId 60(0), rc 1
II 0x7fffd8000bb0 HIT @ 60
II 0x7fffd8000bb0 last docId 61, it 0 read docId 61(1), rc 1
II 0x7fffd8000bb0 last docId 61, it 1 read docId 61(0), rc 1
II 0x7fffd8000bb0 HIT @ 61
II 0x7fffd8000bb0 last docId 62, it 0 read docId 62(1), rc 1
II 0x7fffd8000bb0 last docId 62, it 1 read docId 62(0), rc 1
II 0x7fffd8000bb0 HIT @ 62
II 0x7fffd8000bb0 last docId 63, it 0 read docId 63(1), rc 1
II 0x7fffd8000bb0 last docId 63, it 1 read docId 63(0), rc 1
II 0x7fffd8000bb0 HIT @ 63
II 0x7fffd8000bb0 last docId 64, it 0 read docId 64(1), rc 1
II 0x7fffd8000bb0 last docId 64, it 1 read docId 64(0), rc 1
II 0x7fffd8000bb0 HIT @ 64
II 0x7fffd8000bb0 last docId 65, it 0 read docId 65(1), rc 1
II 0x7fffd8000bb0 last docId 65, it 1 read docId 65(0), rc 1
II 0x7fffd8000bb0 HIT @ 65
II 0x7fffd8000bb0 last docId 66, it 0 read docId 66(1), rc 1
II 0x7fffd8000bb0 last docId 66, it 1 read docId 66(0), rc 1
II 0x7fffd8000bb0 HIT @ 66
II 0x7fffd8000bb0 last docId 67, it 0 read docId 67(1), rc 1
II 0x7fffd8000bb0 last docId 67, it 1 read docId 67(0), rc 1
II 0x7fffd8000bb0 HIT @ 67
II 0x7fffd8000bb0 last docId 68, it 0 read docId 68(1), rc 1
II 0x7fffd8000bb0 last docId 68, it 1 read docId 68(0), rc 1
II 0x7fffd8000bb0 HIT @ 68
II 0x7fffd8000bb0 last docId 69, it 0 read docId 69(1), rc 1
II 0x7fffd8000bb0 last docId 69, it 1 read docId 69(0), rc 1
II 0x7fffd8000bb0 HIT @ 69
II 0x7fffd8000bb0 last docId 70, it 0 read docId 70(1), rc 1
II 0x7fffd8000bb0 last docId 70, it 1 read docId 70(0), rc 1
II 0x7fffd8000bb0 HIT @ 70
II 0x7fffd8000bb0 last docId 71, it 0 read docId 71(1), rc 1
II 0x7fffd8000bb0 last docId 71, it 1 read docId 71(0), rc 1
II 0x7fffd8000bb0 HIT @ 71
II 0x7fffd8000bb0 last docId 72, it 0 read docId 72(1), rc 1
II 0x7fffd8000bb0 last docId 72, it 1 read docId 72(0), rc 1
II 0x7fffd8000bb0 HIT @ 72
II 0x7fffd8000bb0 last docId 73, it 0 read docId 73(1), rc 1
II 0x7fffd8000bb0 last docId 73, it 1 read docId 73(0), rc 1
II 0x7fffd8000bb0 HIT @ 73
II 0x7fffd8000bb0 last docId 74, it 0 read docId 74(1), rc 1
II 0x7fffd8000bb0 last docId 74, it 1 read docId 74(0), rc 1
II 0x7fffd8000bb0 HIT @ 74
II 0x7fffd8000bb0 last docId 75, it 0 read docId 75(1), rc 1
II 0x7fffd8000bb0 last docId 75, it 1 read docId 75(0), rc 1
II 0x7fffd8000bb0 HIT @ 75
II 0x7fffd8000bb0 last docId 76, it 0 read docId 76(1), rc 1
II 0x7fffd8000bb0 last docId 76, it 1 read docId 76(0), rc 1
II 0x7fffd8000bb0 HIT @ 76
II 0x7fffd8000bb0 last docId 77, it 0 read docId 77(1), rc 1
II 0x7fffd8000bb0 last docId 77, it 1 read docId 77(0), rc 1
II 0x7fffd8000bb0 HIT @ 77
II 0x7fffd8000bb0 last docId 78, it 0 read docId 78(1), rc 1
II 0x7fffd8000bb0 last docId 78, it 1 read docId 78(0), rc 1
II 0x7fffd8000bb0 HIT @ 78
II 0x7fffd8000bb0 last docId 79, it 0 read docId 79(1), rc 1
II 0x7fffd8000bb0 last docId 79, it 1 read docId 79(0), rc 1
II 0x7fffd8000bb0 HIT @ 79
II 0x7fffd8000bb0 last docId 80, it 0 read docId 80(1), rc 1
II 0x7fffd8000bb0 last docId 80, it 1 read docId 80(0), rc 1
II 0x7fffd8000bb0 HIT @ 80
II 0x7fffd8000bb0 last docId 81, it 0 read docId 81(1), rc 1
II 0x7fffd8000bb0 last docId 81, it 1 read docId 81(0), rc 1
II 0x7fffd8000bb0 HIT @ 81
II 0x7fffd8000bb0 last docId 82, it 0 read docId 82(1), rc 1
II 0x7fffd8000bb0 last docId 82, it 1 read docId 82(0), rc 1
II 0x7fffd8000bb0 HIT @ 82
II 0x7fffd8000bb0 last docId 83, it 0 read docId 83(1), rc 1
II 0x7fffd8000bb0 last docId 83, it 1 read docId 83(0), rc 1
II 0x7fffd8000bb0 HIT @ 83
II 0x7fffd8000bb0 last docId 84, it 0 read docId 84(1), rc 1
II 0x7fffd8000bb0 last docId 84, it 1 read docId 84(0), rc 1
II 0x7fffd8000bb0 HIT @ 84
II 0x7fffd8000bb0 last docId 85, it 0 read docId 85(1), rc 1
II 0x7fffd8000bb0 last docId 85, it 1 read docId 85(0), rc 1
II 0x7fffd8000bb0 HIT @ 85
II 0x7fffd8000bb0 last docId 86, it 0 read docId 86(1), rc 1
II 0x7fffd8000bb0 last docId 86, it 1 read docId 86(0), rc 1
II 0x7fffd8000bb0 HIT @ 86
II 0x7fffd8000bb0 last docId 87, it 0 read docId 87(1), rc 1
II 0x7fffd8000bb0 last docId 87, it 1 read docId 87(0), rc 1
II 0x7fffd8000bb0 HIT @ 87
II 0x7fffd8000bb0 last docId 88, it 0 read docId 88(1), rc 1
II 0x7fffd8000bb0 last docId 88, it 1 read docId 88(0), rc 1
II 0x7fffd8000bb0 HIT @ 88
II 0x7fffd8000bb0 last docId 89, it 0 read docId 89(1), rc 1
II 0x7fffd8000bb0 last docId 89, it 1 read docId 89(0), rc 1
II 0x7fffd8000bb0 HIT @ 89
II 0x7fffd8000bb0 last docId 90, it 0 read docId 90(1), rc 1
II 0x7fffd8000bb0 last docId 90, it 1 read docId 90(0), rc 1
II 0x7fffd8000bb0 HIT @ 90
II 0x7fffd8000bb0 last docId 91, it 0 read docId 91(1), rc 1
II 0x7fffd8000bb0 last docId 91, it 1 read docId 91(0), rc 1
II 0x7fffd8000bb0 HIT @ 91
II 0x7fffd8000bb0 last docId 92, it 0 read docId 92(1), rc 1
II 0x7fffd8000bb0 last docId 92, it 1 read docId 92(0), rc 1
II 0x7fffd8000bb0 HIT @ 92
II 0x7fffd8000bb0 last docId 93, it 0 read docId 93(1), rc 1
II 0x7fffd8000bb0 last docId 93, it 1 read docId 93(0), rc 1
II 0x7fffd8000bb0 HIT @ 93
II 0x7fffd8000bb0 last docId 94, it 0 read docId 94(1), rc 1
II 0x7fffd8000bb0 last docId 94, it 1 read docId 94(0), rc 1
II 0x7fffd8000bb0 HIT @ 94
II 0x7fffd8000bb0 last docId 95, it 0 read docId 95(1), rc 1
II 0x7fffd8000bb0 last docId 95, it 1 read docId 95(0), rc 1
II 0x7fffd8000bb0 HIT @ 95
II 0x7fffd8000bb0 last docId 96, it 0 read docId 96(1), rc 1
II 0x7fffd8000bb0 last docId 96, it 1 read docId 96(0), rc 1
II 0x7fffd8000bb0 HIT @ 96
II 0x7fffd8000bb0 last docId 97, it 0 read docId 97(1), rc 1
II 0x7fffd8000bb0 last docId 97, it 1 read docId 97(0), rc 1
II 0x7fffd8000bb0 HIT @ 97
II 0x7fffd8000bb0 last docId 98, it 0 read docId 98(1), rc 1
II 0x7fffd8000bb0 last docId 98, it 1 read docId 98(0), rc 1
II 0x7fffd8000bb0 HIT @ 98
II 0x7fffd8000bb0 last docId 99, it 0 read docId 99(1), rc 1
II 0x7fffd8000bb0 last docId 99, it 1 read docId 99(0), rc 1
II 0x7fffd8000bb0 HIT @ 99
II 0x7fffd8000bb0 last docId 100, it 0 read docId 100(1), rc 1
II 0x7fffd8000bb0 last docId 100, it 1 read docId 100(0), rc 1
II 0x7fffd8000bb0 HIT @ 100

[dvirsky]

Thanks a lot. I'll fix ASAP and release a bugfix version.

[dvirsky]

Version 1.0.5 is out with a fix to this bug. Thanks again for reporting.