Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How do I specify partitions for production messages #2844

Open
gopherhiro opened this issue Mar 30, 2024 · 6 comments
Open

How do I specify partitions for production messages #2844

gopherhiro opened this issue Mar 30, 2024 · 6 comments

Comments

@gopherhiro
Copy link

No description provided.

@gopherhiro
Copy link
Author

	msg := &sarama.ProducerMessage{
		Topic:     "example-topic",
		Partition: 0,
		Value:     sarama.StringEncoder("Hello, Kafka! - 1, i: " + string(i)),
	}

I specified it explicitly, but it didn't work

@gopherhiro
Copy link
Author

2024/03/30 21:54:46 Message sent to partition 0 at offset 28640
2024/03/30 21:54:47 Message sent to partition 0 at offset 28642
2024/03/30 21:54:48 Message sent to partition 1 at offset 15024
2024/03/30 21:54:49 Message sent to partition 1 at offset 15025
2024/03/30 21:54:50 Message sent to partition 1 at offset 15027

@abhipranay
Copy link
Contributor

abhipranay commented May 12, 2024

@gopherhiro Can you confirm whether you set config.Producer.Partitioner = NewManualPartitioner when creating producer ?
Setting this partitioner will use the value from msg.Partition to decide partition for produced message.

@Mengsen-W
Copy link

@gopherhiro您能否确认config.Producer.Partitioner = NewManualPartitioner在创建生产者时是否设置了? 设置此分区器将使用的值来msg.Partition决定生产消息的分区。

What if I want to make partitions at runtime instead of specifying them at initialization time? SendMessage function or ProducerMessage strcut, Can I set it.

@abhipranay
Copy link
Contributor

@gopherhiro您能否确认config.Producer.Partitioner = NewManualPartitioner在创建生产者时是否设置了? 设置此分区器将使用的值来msg.Partition决定生产消息的分区。

What if I want to make partitions at runtime instead of specifying them at initialization time? SendMessage function or ProducerMessage strcut, Can I set it.

Check struct sarama.ProducerMessage. You can set Partition field at runtime. The value of this field will be considered only when you set config.Producer.Partitioner = NewManualPartitioner

@Mengsen-W
Copy link

Mengsen-W commented Jun 26, 2024

@gopherhiro您能否确认config.Producer.Partitioner = NewManualPartitioner在创建生产者时是否设置了? 设置此分区器将使用的值来msg.Partition决定生产消息的分区。

What if I want to make partitions at runtime instead of specifying them at initialization time? SendMessage function or ProducerMessage strcut, Can I set it.

Check struct sarama.ProducerMessage. You can set Partition field at runtime. The value of this field will be considered only when you set config.Producer.Partitioner = NewManualPartitioner

Greate Help !
I think the comment for this parameter should be rewritten. That way I know straight away how to set up the partition.

// ProducerMessage is the collection of elements passed to the Producer in order to send a message.
type ProducerMessage struct {
	...
        // ------------- !!!THIS PARAMETER COMMENT!!! -------------- //
	// Partition is the partition that the message was sent to. This is only
	// guaranteed to be defined if the message was successfully delivered.
	Partition int32
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants