Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Data race on ImportTask.segmentsInfo #34089

Closed
1 task done
congqixia opened this issue Jun 24, 2024 · 0 comments
Closed
1 task done

[Bug]: Data race on ImportTask.segmentsInfo #34089

congqixia opened this issue Jun 24, 2024 · 0 comments
Assignees
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.

Comments

@congqixia
Copy link
Contributor

Is there an existing issue for this?

  • I have searched the existing issues

Environment

- Milvus version: a1248a1
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Data race on ImportTask.segmentsInfo when calling UpdateSegmentInfo and GetSegmentsInfo

Expected Behavior

No data race

Steps To Reproduce

No response

Milvus Log

2024-06-24T04:30:25.6056736Z ==================
2024-06-24T04:30:25.6056977Z WARNING: DATA RACE
2024-06-24T04:30:25.6057255Z Write at 0x00c004116fc0 by goroutine 48690:
2024-06-24T04:30:25.6057601Z   runtime.mapassign_fast64()
2024-06-24T04:30:25.6057949Z       /usr/local/go/src/runtime/map_fast64.go:93 +0x0
2024-06-24T04:30:25.6058782Z   github.com/milvus-io/milvus/internal/datanode/importv2.(*ImportTask).importFile.UpdateSegmentInfo.func2()
2024-06-24T04:30:25.6059738Z       /go/src/github.com/milvus-io/milvus/internal/datanode/importv2/task.go:148 +0x604
2024-06-24T04:30:25.6060540Z   github.com/milvus-io/milvus/internal/datanode/importv2.(*taskManager).Update()
2024-06-24T04:30:25.6061386Z       /go/src/github.com/milvus-io/milvus/internal/datanode/importv2/task_manager.go:54 +0x241
2024-06-24T04:30:25.6062236Z   github.com/milvus-io/milvus/internal/datanode/importv2.(*ImportTask).importFile()
2024-06-24T04:30:25.6063099Z       /go/src/github.com/milvus-io/milvus/internal/datanode/importv2/task_import.go:197 +0x589
2024-06-24T04:30:25.6063947Z   github.com/milvus-io/milvus/internal/datanode/importv2.(*ImportTask).Execute.func1()
2024-06-24T04:30:25.6064812Z       /go/src/github.com/milvus-io/milvus/internal/datanode/importv2/task_import.go:138 +0xa30
2024-06-24T04:30:25.6065661Z   github.com/milvus-io/milvus/internal/datanode/importv2.(*ImportTask).Execute.func2()
2024-06-24T04:30:25.6066513Z       /go/src/github.com/milvus-io/milvus/internal/datanode/importv2/task_import.go:153 +0x4d
2024-06-24T04:30:25.6067861Z   github.com/milvus-io/milvus/pkg/util/conc.(*Pool[go.shape.interface {}]).Submit.func1()
2024-06-24T04:30:25.6068899Z       /go/src/github.com/milvus-io/milvus/pkg/util/conc/pool.go:81 +0x251
2024-06-24T04:30:25.6069792Z   github.com/panjf2000/ants/v2.(*goWorker).run.func1()
2024-06-24T04:30:25.6070665Z       /go/pkg/mod/github.com/panjf2000/ants/[email protected]/worker.go:67 +0xe1
2024-06-24T04:30:25.6071049Z 
2024-06-24T04:30:25.6071266Z Previous read at 0x00c004116fc0 by goroutine 49743:
2024-06-24T04:30:25.6071959Z   github.com/samber/lo.Values[go.shape.int64,go.shape.*uint8]()
2024-06-24T04:30:25.6072811Z       /go/pkg/mod/github.com/samber/[email protected]/map.go:16 +0xa9
2024-06-24T04:30:25.6074098Z   github.com/milvus-io/milvus/internal/datanode/importv2.(*ImportTask).GetSegmentsInfo()
2024-06-24T04:30:25.6075653Z       /go/src/github.com/milvus-io/milvus/internal/datanode/importv2/task_import.go:105 +0x79
2024-06-24T04:30:25.6077054Z   github.com/milvus-io/milvus/internal/datanode.(*DataNode).QueryImport()
2024-06-24T04:30:25.6078317Z       /go/src/github.com/milvus-io/milvus/internal/datanode/services.go:532 +0xd16
2024-06-24T04:30:25.6079725Z   github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).QueryImport()
2024-06-24T04:30:25.6081207Z       /go/src/github.com/milvus-io/milvus/internal/distributed/datanode/service.go:399 +0xab
2024-06-24T04:30:25.6082735Z   github.com/milvus-io/milvus/internal/proto/datapb._DataNode_QueryImport_Handler.func1()
2024-06-24T04:30:25.6084297Z       /go/src/github.com/milvus-io/milvus/internal/proto/datapb/data_coord.pb.go:9593 +0x96
2024-06-24T04:30:25.6086288Z   github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).startGrpcLoop.ServerIDValidationUnaryServerInterceptor.func9()
2024-06-24T04:30:25.6088330Z       /go/src/github.com/milvus-io/milvus/pkg/util/interceptor/server_id_interceptor.go:54 +0x1ad
2024-06-24T04:30:25.6090489Z   github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).startGrpcLoop.ChainUnaryServer.func10.1.1()
2024-06-24T04:30:25.6092205Z       /go/pkg/mod/github.com/grpc-ecosystem/[email protected]/chain.go:25 +0x88
2024-06-24T04:30:25.6094185Z   github.com/milvus-io/milvus/internal/distributed/datanode.(*Server).startGrpcLoop.ClusterValidationUnaryServerInterceptor.func8()
2024-06-24T04:30:25.6096209Z       /go/src/github.com/milvus-io/milvus/pkg/util/interceptor/cluster_interceptor.go:48 +0x166

Anything else?

No response

@congqixia congqixia added kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on. labels Jun 24, 2024
@congqixia congqixia self-assigned this Jun 24, 2024
congqixia added a commit to congqixia/milvus that referenced this issue Jun 24, 2024
sre-ci-robot pushed a commit that referenced this issue Jun 25, 2024
congqixia added a commit to congqixia/milvus that referenced this issue Jun 25, 2024
sre-ci-robot pushed a commit that referenced this issue Jun 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues or changes related a bug triage/accepted Indicates an issue or PR is ready to be actively worked on.
Projects
None yet
Development

No branches or pull requests

1 participant