HIVE-28340: Test concurrent JDBC connections with Kerberized cluster, impersonation, and HTTP transport #5315
Conversation
… impersonation, and HTTP transport The new test case simulates a scenario with two JDBC clients doing the following in parallel: client 1, continuously opens and closes connections (short-lived connection) client 2, opens connection, sends fixed number of simple queries, closes connection (long-lived connection) Since the clients are running in parallel we have one long-lived session in HS2 interleaved with many short ones. The test case aims to increase test coverage and guard against regressions in the presence of many interleaved HS2 sessions. In older versions, without HIVE-27201, this test fails (with the exception outlined below) when the cluster is Kerberized, and we are using HTTP transport mode with impersonation enabled.
asf-ci-hive
added
tests pending
tests unstable
and removed
tests pending
tests unstable
labels
| // Simulate a client application that uses many short-lived connections. | ||
| Future<Boolean> openCloseConnectionTask = service.submit(() -> { | ||
| do { | ||
| try (Connection c = DriverManager.getConnection(hs2.getJdbcURL())) { |
There was a problem hiding this comment.
I'm confusing, the hs2.getJdbcURL() should be a Kerberized connection, right?
There was a problem hiding this comment.
Yes, based on the current setup the JDBC URL is:
jdbc:hive2://localhost:33961/default;transportMode=http;httpPath=cliservice;principal=hive/[email protected]
| @@ -194,9 +194,6 @@ public static MiniHS2 getMiniHS2WithKerb(MiniHiveKdc miniHiveKdc, HiveConf hiveC | |||
| .withConf(hiveConf) | |||
| .withMiniKdc(hivePrincipal, hiveKeytab) | |||
| .withAuthenticationType(authType); | |||
| if (HiveServer2.isHttpTransportMode(hiveConf)) { | |||
There was a problem hiding this comment.
why do we remove it here? what if the HS2 reads this property and opens the http server, while the client is trying to establish the connection with the binary port?
There was a problem hiding this comment.
The JDBC URL is usually obtained from MiniHS2#getJdbcURL() so the server/client configs should be aligned.
Nevertheless to simplify the review of this PR I reverted the changes here and in MiniHS2 and added the same isHttpTransportMode logic in the the other static factory method that is used in this test (add40a8 ).
…bWithRemoteHMSWithKerb Leave MiniHS2 and other methods unchanged and mimic other APIs in MiniHiveKdc for setting HTTP mode.
asf-ci-hive
added
tests pending
tests failed
and removed
tests unstable
tests pending
tests failed
labels
|
|
Tests are green can you have another look @dengzhhu653 ? Thanks! |
What changes were proposed in this pull request?
MiniHiveKdc#getMiniHS2WithKerbWithRemoteHMSWithKerb.Why are the changes needed?
Increase test coverage and guard against regressions. More info under HIVE-28340.
Does this PR introduce any user-facing change?
No
Is the change a dependency upgrade?
No
How was this patch tested?