Feature request - support default values for topic config items after expert mode edit

[Noel-Jones]

When a topic is created it is created with 3 config values set iirc. When it is edited in expert mode this rises to 12.

      # These are the default values as if edited in expert mode and only retention bytes changed.
      "cleanup.policy"                      = "delete"
      "delete.retention.ms"                 = "86400000"
      "max.compaction.lag.ms"               = "9223372036854775807"
      "max.message.bytes"                   = "2097164"
      "message.timestamp.difference.max.ms" = "9223372036854775807"
      "message.timestamp.type"              = "CreateTime"
      "min.compaction.lag.ms"               = "0"
      "min.insync.replicas"                 = "2"
      "retention.bytes"                     = "1073741824" # 1GB (default -1)
      "retention.ms"                        = "604800000"
      "segment.bytes"                       = "104857600"
      "segment.ms"                          = "604800000"

When I set up Terraform in the development phase I was ok with this, though it took a long time to work out what was causing such large numbers of differences between topics. I also had to take a leap of faith that it was ok to update all topics with these values.

Now however there appear to be 2 more. A topic has been edited in the console to set the retention.ms and now Terraform reports that it it needs to make the following changes:

          - "message.timestamp.after.max.ms"      = "9223372036854775807" -> null
          - "message.timestamp.before.max.ms"     = "9223372036854775807" -> null
          ~ "retention.ms"                        = "-1" -> "604800000"

But the provider does not support setting it back to null. I am assuming the API does not allow for this. When I run Terraform it fails with this error:

Error: error updating Kafka Topic "lkc-6zrxj/dlq-oracle-erp.fct.customer.updates": reset to topic setting's default value operation (in other words, removing topic settings from 'configs' block) is not supported at the moment. Instead, find its default value at https://registry.terraform.io/providers/confluentinc/confluent/latest/docs/resources/confluent_kafka_topic and set its current value to the default value.

Per the advice I updated the code but that now reports that 263 topics need to be updated. This is somewhat excessive! In practice I know that nothing will be different if I do this but it is another leap of faith and given that we are now in production it is not something I can simply do.

So, I would like a feature to be able to set a config item to null (or not specify it) and for Terraform to not attempt to change it if the config item value is null or not specified or equals the default value.

In this case I could specify "message.timestamp.after.max.ms" = null or leave it out of the config.

This feature will greatly simplify onboarding and code maintenance. The only fly in the ointment is how to keep the provider up to date with new defaults.