Failed to acquire SASL OAUTHBEARER token: SSL certificate problem: unable to get local issuer certificate

[kishkulk]

Description

I am trying to set sasl.oauthbearer.token.endpoint.url to https endpoint

if (rd_kafka_conf_set(conf, "sasl.oauthbearer.token.endpoint.url", "https://<IP>:<https_port>/realms/master/protocol/openid-connect/token", errstr, sizeof(errstr)) != RD_KAFKA_CONF_OK) {
    fprintf(stderr, "Failed to set sasl.oauthbearer.token.endpoint.url: %s\n", errstr);
    return 1;
}

the above code fails while trying to connect the oauth token provider with error

%3|1718691008.556|OIDC|rdkafka#producer-1| [thrd:background]: Failed to retrieve OIDC token from "https://<IP>:<https_port>/realms/master/protocol/openid-connect/token": SSL certificate problem: unable to get local issuer certificate (-1)
%3|1718691008.561|ERROR|rdkafka#producer-1| [thrd:background]: Failed to acquire SASL OAUTHBEARER token: SSL certificate problem: unable to get local issuer certificate

When using the same system, the curl command functions correctly when I configure CURL_CA_BUNDLE to the CA certificate file from the OAuth token provider. I attempted to execute the librdkafka sample with the CURL_CA_BUNDLE environment variable, but unfortunately, it did not work. Consequently, I had to make corrections to the librdkafka code, specifically in rdhttp.c, where I needed to

 rd_http_error_t *rd_http_req_init(rd_http_req_t *hreq, const char *url) {
 const char *ca_bundle_path = getenv("CURL_CA_BUNDLE");
 if (ca_bundle_path) {
  curl_easy_setopt(hreq->hreq_curl, CURLOPT_CAINFO, ca_bundle_path);
 }
 ...
 }

since the librdkafka uses the curl I was thinking the curl environment variables should work.

How to reproduce

set sasl.oauthbearer.token.endpoint.url to a https endpoint

Checklist

• librdkafka version (release number or git tag): v2.4.0
• Apache Kafka version: ``
• librdkafka client configuration: sasl.oauthbearer.token.endpoint.url=https://url
• Operating system: ubuntu
• Provide logs (with debug=.. as necessary) from librdkafka

 %7|1718710484.974|SASL|rdkafka#producer-1| [thrd:app]: Selected provider OAUTHBEARER (builtin) for SASL mechanism OAUTHBEARER
%7|1718710484.974|OPENSSL|rdkafka#producer-1| [thrd:app]: Using OpenSSL version OpenSSL 3.0.2 15 Mar 2022 (0x30000020, librdkafka built with 0x30000020)
%7|1718710484.977|SSL|rdkafka#producer-1| [thrd:app]: Loading CA certificate(s) from file ./ca.crt
%7|1718710484.979|INIT|rdkafka#producer-1| [thrd:app]: librdkafka v2.4.0-RC2-dirty (0x20400ff) rdkafka#producer-1 initialized (builtin.features gzip,snappy,ssl,sasl,regex,lz4,sasl_gssapi,sasl_plain,sasl_scram,plugins,sasl_oauthbearer,http,oidc, GCC GXX PKGCONFIG INSTALL GNULD LDS C11THREADS LIBDL PLUGINS ZLIB SSL SASL_CYRUS CURL HDRHISTOGRAM SYSLOG SNAPPY SOCKEM SASL_SCRAM SASL_OAUTHBEARER OAUTHBEARER_OIDC CRC32C_HW, debug 0x200)
% Type some text and hit enter to produce message
% Or just hit enter to only serve delivery reports
% Press Ctrl-C or Ctrl-D to exit
%3|1718710485.039|OIDC|rdkafka#producer-1| [thrd:background]: Failed to retrieve OIDC token from "https://<IP>:<https_port>/realms/master/protocol/openid-connect/token": SSL certificate problem: unable to get local issuer certificate (-1)
%3|1718710485.039|ERROR|rdkafka#producer-1| [thrd:background]: Failed to acquire SASL OAUTHBEARER token: SSL certificate problem: unable to get local issuer certificate
%3|1718710496.041|OIDC|rdkafka#producer-1| [thrd:background]: Failed to retrieve OIDC token from "https://<IP>:<https_port>/realms/master/protocol/openid-connect/token": SSL certificate problem: unable to get local issuer certificate (-1)

• Provide broker log excerpts
• Critical issue