Dgraph: Data Races in Alpha & Zero

Created on 14 May 2019  路  13Comments  路  Source: dgraph-io/dgraph
  • [x] Data Race 1 in Alpha
  • [x] Data Race 2 in Alpha
  • [x] Data Race 3 in Alpha
  • [x] Data Race 4 in Alpha
  • [x] Data Race 5 in Alpha #3432
  • [x] Data Race 1 in Zero #3417
  • [x] Data Race 2 in Zero #3417
  • [x] Data Race 3 in Zero #3417

Version: b4fb07777b6929ee873bfaa07d62cbb0f583acb7

Data Race 1 in Alpha

alpha3    | ==================
alpha3    | WARNING: DATA RACE
alpha3    | Read at 0x0000028b9ec0 by goroutine 315:
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateFacetMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1395 +0x48f
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateUidMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1407 +0x3c
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).populateVarMap()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1448 +0x35c
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2722 +0x15c8
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).Process()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2771 +0x88
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:653 +0xb23
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha3    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha3    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha3    |   google.golang.org/grpc.(*Server).handleStream()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha3    | 
alpha3    | Previous write at 0x0000028b9ec0 by goroutine 346:
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateFacetMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1402 +0x3a0
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateUidMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1407 +0x3c
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).populateVarMap()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1448 +0x35c
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2722 +0x15c8
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).Process()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2771 +0x88
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:653 +0xb23
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha3    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha3    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha3    |   google.golang.org/grpc.(*Server).handleStream()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha3    | 
alpha3    | Goroutine 315 (running) created at:
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha3    |   google.golang.org/grpc.(*Server).serveStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha3    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
alpha3    | 
alpha3    | Goroutine 346 (finished) created at:
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha3    |   google.golang.org/grpc.(*Server).serveStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha3    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
alpha3    | ==================
kinbug
Source
picture mangalaman93

All 13 comments

Date Race 2 in Alpha

alpha3    | WARNING: DATA RACE
alpha3    | Read at 0x0000028b9fc0 by goroutine 315:
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateFacetMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1396 +0x4be
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateUidMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1407 +0x3c
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).populateVarMap()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1448 +0x35c
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2722 +0x15c8
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).Process()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2771 +0x88
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:653 +0xb23
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha3    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha3    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha3    |   google.golang.org/grpc.(*Server).handleStream()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha3    | 
alpha3    | Previous write at 0x0000028b9fc0 by goroutine 346:
alpha3    |   github.com/dgraph-io/dgraph/algo.IntersectWith()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/algo/uidlist.go:141 +0x3c7
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateUidMatrix()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1417 +0x14b
alpha3    |   github.com/dgraph-io/dgraph/query.(*SubGraph).populateVarMap()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1448 +0x35c
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2722 +0x15c8
alpha3    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).Process()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2771 +0x88
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:653 +0xb23
alpha3    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha3    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha3    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha3    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha3    |   google.golang.org/grpc.(*Server).handleStream()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha3    | 
alpha3    | Goroutine 315 (running) created at:
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha3    |   google.golang.org/grpc.(*Server).serveStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha3    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
alpha3    | 
alpha3    | Goroutine 346 (finished) created at:
alpha3    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha3    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha3    |   google.golang.org/grpc.(*Server).serveStreams()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha3    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha3    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
mangalaman93 picture mangalaman93 on 14 May 2019

Data Race 3 in Alpha

alpha1    | WARNING: DATA RACE
alpha1    | Write at 0x0000028b9fc0 by goroutine 336:
alpha1    |   github.com/dgraph-io/dgraph/algo.IntersectWith()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/algo/uidlist.go:156 +0x214
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateUidMatrix()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1417 +0x14b
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).populateVarMap()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1448 +0x35c
alpha1    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2722 +0x15c8
alpha1    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).Process()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2771 +0x88
alpha1    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:653 +0xb23
alpha1    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha1    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha1    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha1    |   google.golang.org/grpc.(*Server).handleStream()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha1    | 
alpha1    | Previous read at 0x0000028b9fc0 by goroutine 239:
alpha1    |   github.com/dgraph-io/dgraph/algo.MergeSorted()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/algo/uidlist.go:363 +0x141
alpha1    |   github.com/dgraph-io/dgraph/query.ProcessGraph()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2065 +0x3a09
alpha1    | 
alpha1    | Goroutine 336 (running) created at:
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha1    |   google.golang.org/grpc.(*Server).serveStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha1    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
alpha1    | 
alpha1    | Goroutine 239 (finished) created at:
alpha1    |   github.com/dgraph-io/dgraph/query.ProcessGraph()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2211 +0x13e9
mangalaman93 picture mangalaman93 on 14 May 2019

Data Race 4 in Alpha

alpha1    | WARNING: DATA RACE
alpha1    | Read at 0x0000028b9ec0 by goroutine 381:
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).preTraverse()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:522 +0x1d01
alpha1    |   github.com/dgraph-io/dgraph/query.processNodeUids()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/outputnode.go:459 +0x3ad
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).toFastJSON()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/outputnode.go:507 +0x17b
alpha1    |   github.com/dgraph-io/dgraph/query.ToJson()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/outputnode.go:53 +0x561
alpha1    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:674 +0x12ec
alpha1    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha1    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha1    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha1    |   google.golang.org/grpc.(*Server).handleStream()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha1    | 
alpha1    | Previous write at 0x0000028b9ec0 by goroutine 214:
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateFacetMatrix()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1402 +0x3a0
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).updateUidMatrix()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1407 +0x3c
alpha1    |   github.com/dgraph-io/dgraph/query.(*SubGraph).populateVarMap()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:1448 +0x35c
alpha1    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).ProcessQuery()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2722 +0x15c8
alpha1    |   github.com/dgraph-io/dgraph/query.(*QueryRequest).Process()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/query/query.go:2771 +0x88
alpha1    |   github.com/dgraph-io/dgraph/edgraph.(*Server).doQuery()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:653 +0xb23
alpha1    |   github.com/dgraph-io/dgraph/edgraph.(*Server).Query()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/edgraph/server.go:562 +0xbb
alpha1    |   github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api._Dgraph_Query_Handler()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/vendor/github.com/dgraph-io/dgo/protos/api/api.pb.go:1917 +0x2fc
alpha1    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha1    |   google.golang.org/grpc.(*Server).handleStream()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha1    | 
alpha1    | Goroutine 381 (running) created at:
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha1    |   google.golang.org/grpc.(*Server).serveStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha1    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
alpha1    | 
alpha1    | Goroutine 214 (running) created at:
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha1    |   google.golang.org/grpc.(*Server).serveStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha1    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
mangalaman93 picture mangalaman93 on 14 May 2019

Data Race 1 in Zero

zero1     | WARNING: DATA RACE
zero1     | Write at 0x00c00b24c020 by goroutine 16:
zero1     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).Oracle()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/oracle.go:432 +0x33d
zero1     |   github.com/dgraph-io/dgraph/protos/pb._Zero_Oracle_Handler()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4468 +0x135
zero1     |   google.golang.org/grpc.(*Server).processStreamingRPC()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1202 +0x1669
zero1     |   google.golang.org/grpc.(*Server).handleStream()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1282 +0x12e5
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
zero1     | 
zero1     | Previous write at 0x00c00b24c020 by goroutine 161:
zero1     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).Oracle()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/oracle.go:432 +0x33d
zero1     |   github.com/dgraph-io/dgraph/protos/pb._Zero_Oracle_Handler()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4468 +0x135
zero1     |   google.golang.org/grpc.(*Server).processStreamingRPC()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1202 +0x1669
zero1     |   google.golang.org/grpc.(*Server).handleStream()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1282 +0x12e5
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
zero1     | 
zero1     | Goroutine 16 (running) created at:
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
zero1     |   google.golang.org/grpc.(*Server).serveStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
zero1     |   google.golang.org/grpc.(*Server).handleRawConn.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
zero1     | 
zero1     | Goroutine 161 (running) created at:
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
zero1     |   google.golang.org/grpc.(*Server).serveStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
zero1     |   google.golang.org/grpc.(*Server).handleRawConn.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
mangalaman93 picture mangalaman93 on 14 May 2019

Data Race 2 in Zero

zero3     | WARNING: DATA RACE
zero3     | Read at 0x00c000e7f320 by goroutine 330:
zero3     |   github.com/dgraph-io/dgraph/protos/pb.(*OracleDelta).MarshalTo()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:7323 +0x17a
zero3     |   github.com/dgraph-io/dgraph/protos/pb.(*OracleDelta).Marshal()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:7294 +0x87
zero3     |   google.golang.org/grpc/encoding/proto.codec.Marshal()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/encoding/proto/proto.go:70 +0x224
zero3     |   google.golang.org/grpc/encoding/proto.(*codec).Marshal()
zero3     |       <autogenerated>:1 +0x62
zero3     |   google.golang.org/grpc.encode()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/rpc_util.go:543 +0x71
zero3     |   google.golang.org/grpc.prepareMsg()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/stream.go:1503 +0x98
zero3     |   google.golang.org/grpc.(*serverStream).SendMsg()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/stream.go:1396 +0x17e
zero3     |   github.com/dgraph-io/dgraph/protos/pb.(*zeroOracleServer).Send()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4481 +0x6a
zero3     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).Oracle()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/oracle.go:433 +0x373
zero3     |   github.com/dgraph-io/dgraph/protos/pb._Zero_Oracle_Handler()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4468 +0x135
zero3     |   google.golang.org/grpc.(*Server).processStreamingRPC()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1202 +0x1669
zero3     |   google.golang.org/grpc.(*Server).handleStream()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1282 +0x12e5
zero3     |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
zero3     | 
zero3     | Previous write at 0x00c000e7f320 by goroutine 360:
zero3     |   runtime.mapassign_fast32()
zero3     |       /usr/local/go/src/runtime/map_fast32.go:92 +0x0
zero3     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).groupChecksums()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/zero.go:242 +0x18f
zero3     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).Oracle()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/oracle.go:432 +0x317
zero3     |   github.com/dgraph-io/dgraph/protos/pb._Zero_Oracle_Handler()
zero3     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4468 +0x135
zero3     |   google.golang.org/grpc.(*Server).processStreamingRPC()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1202 +0x1669
zero3     |   google.golang.org/grpc.(*Server).handleStream()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1282 +0x12e5
zero3     |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
zero3     | 
zero3     | Goroutine 330 (running) created at:
zero3     |   google.golang.org/grpc.(*Server).serveStreams.func1()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
zero3     |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
zero3     |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
zero3     |   google.golang.org/grpc.(*Server).serveStreams()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
zero3     |   google.golang.org/grpc.(*Server).handleRawConn.func1()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
zero3     | 
zero3     | Goroutine 360 (running) created at:
zero3     |   google.golang.org/grpc.(*Server).serveStreams.func1()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
zero3     |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
zero3     |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
zero3     |   google.golang.org/grpc.(*Server).serveStreams()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
zero3     |   google.golang.org/grpc.(*Server).handleRawConn.func1()
zero3     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
mangalaman93 picture mangalaman93 on 14 May 2019

Data Race 3 in Zero

zero1     | WARNING: DATA RACE
zero1     | Read at 0x00c00193b140 by goroutine 274:
zero1     |   github.com/dgraph-io/dgraph/protos/pb.(*OracleDelta).Size()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:8912 +0xd7
zero1     |   github.com/dgraph-io/dgraph/protos/pb.(*OracleDelta).Marshal()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:7292 +0x3c
zero1     |   google.golang.org/grpc/encoding/proto.codec.Marshal()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/encoding/proto/proto.go:70 +0x224
zero1     |   google.golang.org/grpc/encoding/proto.(*codec).Marshal()
zero1     |       <autogenerated>:1 +0x62
zero1     |   google.golang.org/grpc.encode()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/rpc_util.go:543 +0x71
zero1     |   google.golang.org/grpc.prepareMsg()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/stream.go:1503 +0x98
zero1     |   google.golang.org/grpc.(*serverStream).SendMsg()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/stream.go:1396 +0x17e
zero1     |   github.com/dgraph-io/dgraph/protos/pb.(*zeroOracleServer).Send()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4481 +0x6a
zero1     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).Oracle()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/oracle.go:433 +0x373
zero1     |   github.com/dgraph-io/dgraph/protos/pb._Zero_Oracle_Handler()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4468 +0x135
zero1     |   google.golang.org/grpc.(*Server).processStreamingRPC()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1202 +0x1669
zero1     |   google.golang.org/grpc.(*Server).handleStream()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1282 +0x12e5
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
zero1     | 
zero1     | Previous write at 0x00c00193b140 by goroutine 232:
zero1     |   runtime.mapassign_fast32()
zero1     |       /usr/local/go/src/runtime/map_fast32.go:92 +0x0
zero1     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).groupChecksums()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/zero.go:242 +0x18f
zero1     |   github.com/dgraph-io/dgraph/dgraph/cmd/zero.(*Server).Oracle()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/zero/oracle.go:432 +0x317
zero1     |   github.com/dgraph-io/dgraph/protos/pb._Zero_Oracle_Handler()
zero1     |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4468 +0x135
zero1     |   google.golang.org/grpc.(*Server).processStreamingRPC()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1202 +0x1669
zero1     |   google.golang.org/grpc.(*Server).handleStream()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1282 +0x12e5
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
zero1     | 
zero1     | Goroutine 274 (running) created at:
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
zero1     |   google.golang.org/grpc.(*Server).serveStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
zero1     |   google.golang.org/grpc.(*Server).handleRawConn.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
zero1     | 
zero1     | Goroutine 232 (running) created at:
zero1     |   google.golang.org/grpc.(*Server).serveStreams.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
zero1     |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
zero1     |   google.golang.org/grpc.(*Server).serveStreams()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
zero1     |   google.golang.org/grpc.(*Server).handleRawConn.func1()
zero1     |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
mangalaman93 picture mangalaman93 on 14 May 2019

Manish and I looked at the entries in zero. All of them seem to be caused by the same issue. https://github.com/dgraph-io/dgraph/pull/3417 should fix it (refer to the description for an explanation).

For the race conditions in the query package, the Subgraph struct does not have a lock because the subroutines were supposed to operate on different parts of the data. The problem seems to be that populateVarMap is executed multiple times but the code is written under the assumption that it will only be executed once.

I am going to dig into the code and see if I can find the issue. I'll post updates here. Feel free to take a look yourselves. The more eyes the better.

martinmr picture martinmr on 15 May 2019
👀1 👍1

@mangalaman93 Can you run flock again with #3417 to verify the race conditions in zero are gone?

martinmr picture martinmr on 15 May 2019

Will do, today. Thanks for looking into it.

mangalaman93 picture mangalaman93 on 15 May 2019

Data Race 5 in Alpha

alpha1    | WARNING: DATA RACE
alpha1    | Write at 0x00c0000f9238 by goroutine 233:
alpha1    |   github.com/dgraph-io/dgraph/conn.(*Node).SetConfState()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/conn/node.go:196 +0xe3
alpha1    |   github.com/dgraph-io/dgraph/worker.(*node).applyConfChange()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/worker/draft.go:142 +0x1cb
alpha1    |   github.com/dgraph-io/dgraph/worker.(*node).Run()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/worker/draft.go:914 +0x1c4c
alpha1    | 
alpha1    | Previous read at 0x00c0000f9238 by goroutine 298:
alpha1    |   github.com/dgraph-io/dgraph/conn.(*RaftServer).IsPeer()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/conn/raft_server.go:141 +0x136
alpha1    |   github.com/dgraph-io/dgraph/protos/pb._Raft_IsPeer_Handler()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/protos/pb/pb.pb.go:4197 +0x2fc
alpha1    |   google.golang.org/grpc.(*Server).processUnaryRPC()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:998 +0x936
alpha1    |   google.golang.org/grpc.(*Server).handleStream()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:1278 +0x1351
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1.1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:717 +0xac
alpha1    | 
alpha1    | Goroutine 233 (running) created at:
alpha1    |   github.com/dgraph-io/dgraph/worker.(*node).InitAndStartNode()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/worker/draft.go:1431 +0x788
alpha1    |   github.com/dgraph-io/dgraph/worker.StartRaftNodes()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/worker/groups.go:137 +0xbf6
alpha1    |   github.com/dgraph-io/dgraph/dgraph/cmd/alpha.run.func5()
alpha1    |       /home/aman/gocode/src/github.com/dgraph-io/dgraph/dgraph/cmd/alpha/run.go:562 +0x73
alpha1    | 
alpha1    | Goroutine 298 (finished) created at:
alpha1    |   google.golang.org/grpc.(*Server).serveStreams.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:715 +0xb8
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).operateHeaders()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:421 +0x1622
alpha1    |   google.golang.org/grpc/internal/transport.(*http2Server).HandleStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/internal/transport/http2_server.go:461 +0x37a
alpha1    |   google.golang.org/grpc.(*Server).serveStreams()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:713 +0x170
alpha1    |   google.golang.org/grpc.(*Server).handleRawConn.func1()
alpha1    |       /home/aman/gocode/src/google.golang.org/grpc/server.go:675 +0x50
mangalaman93 picture mangalaman93 on 16 May 2019

The data race in Zero has been fixed. I'll take a look at the latest data race in Alpha.

martinmr picture martinmr on 16 May 2019

https://github.com/dgraph-io/dgraph/pull/3432 fixes the latest alpha race condition.

martinmr picture martinmr on 16 May 2019

This is a duplicate of #3685 at this point. Hence I am closing this.

pawanrawal picture pawanrawal on 7 Aug 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

pepoospina picture pepoospina  路  3Comments
djdoeslinux picture djdoeslinux  路  4Comments
andrewsmedina picture andrewsmedina  路  4Comments
ShawnMilo picture ShawnMilo  路  4Comments
pjebs picture pjebs  路  4Comments