[Improve] refactor clickhouse connector, using Factory to create source/sink

[liunaijie]

Purpose of this pull request

refactor clickhouse connector, using Factory to create source/sink

Does this PR introduce any user-facing change?

no

How was this patch tested?

using existing test

Check list

• If any new Jar binary package adding in your PR, please add License Notice according
  New License Guide
• If necessary, please update the documentation to describe the new feature. https://github.com/apache/seatunnel/tree/dev/docs
• If you are contributing the connector code, please check that the following files are updated:
  1. Update change log that in connector document. For more details you can refer to connector-v2
  2. Update plugin-mapping.properties and add new connector information in it
  3. Update the pom file of seatunnel-dist
• Update the release-note.

[EricJoy2048]

add try.. catch and finally to close proxy?

[liunaijie]

done

[EricJoy2048]

same as above.

[Carl-Zhou-CN]

@Override
public <T, SplitT extends SourceSplit, StateT extends Serializable>
        TableSource<T, SplitT, StateT> createSource(TableSourceFactoryContext context)
                throws PrepareFailException {
    ReadonlyConfig config = context.getOptions();

    List<ClickHouseNode> servers =
            ClickhouseUtil.createNodes(
                    config.get(HOST),
                    config.get(DATABASE),
                    config.get(SERVER_TIME_ZONE),
                    config.get(USERNAME),
                    config.get(PASSWORD));

    String sql = config.get(SQL);
    ClickHouseNode currentServer =
            servers.get(ThreadLocalRandom.current().nextInt(servers.size()));

    CatalogTable catalogTable;
    try (ClickHouseClient client = ClickHouseClient.newInstance(currentServer.getProtocol());
            ClickHouseResponse response =
                    client.connect(currentServer)
                            .format(ClickHouseFormat.RowBinaryWithNamesAndTypes)
                            .query(modifySQLToLimit1(sql))
                            .executeAndWait()) {

        TableSchema.Builder builder = TableSchema.builder();
        List<ClickHouseColumn> columns = response.getColumns();
        columns.stream()
                .forEach(
                        column -> {
                            PhysicalColumn physicalColumn =
                                    PhysicalColumn.of(
                                            column.getColumnName(),
                                            TypeConvertUtil.convert(column),
                                            Long.valueOf(column.getEstimatedLength()),
                                            column.getScale(),
                                            column.isNullable(),
                                            null,
                                            null);
                            builder.column(physicalColumn);
                        });
        String catalogName = "clickhouse_catalog";
        catalogTable =
                CatalogTable.of(
                        TableIdentifier.of(catalogName, config.get(DATABASE), "default", "default"),
                        builder.build(),
                        new HashMap<>(),
                        new ArrayList<>(),
                        "",
                        catalogName);
    } catch (ClickHouseException e) {
        throw new ClickhouseConnectorException(
                SeaTunnelAPIErrorCode.CONFIG_VALIDATION_FAILED,
                String.format(
                        "PluginName: %s, PluginType: %s, Message: %s",
                        factoryIdentifier(), PluginType.SOURCE, e.getMessage()));
    }

A reference

[liunaijie]

thanks, will update the code.
and one more thing i want say. now clickhouse source doesn't has table_name parameter, only has query. So we can't know the table name we queryed (or need parse the sql).
Maybe we can add table_name is next version, then when we generate the CatalogTable, we can pass the real name instead of default

[Carl-Zhou-CN]

Yes, we can emulate the jdbc source approach

[liunaijie]

done, and i check the jdbc code, if only pass query will generate the default.default table won't parse sql.

[Carl-Zhou-CN]

Close when this pr merges #6416

[Carl-Zhou-CN]

good job

[Hisoka-X]

I am a little worried whether clickhousefile can still work properly because we lack test cases for this.

[Carl-Zhou-CN]

@Hisoka-X Do we need to add it in this PR

[liunaijie]

@Carl-Zhou-CN PTAL when you have time

[Hisoka-X]

I think we should add a test case to cover this part. To make sure the sink can be created normally. We can do it in Clickhouse E2E (there has clickhouse container). CC @liunaijie @Carl-Zhou-CN

[Carl-Zhou-CN]

support

[Hisoka-X]

tip: When users use the clickhousefile connector, they need to download a clickhouse binary file themselves. So if we want add e2e for clickhousefile. We should download clickhouse-local program in container too. Please check https://clickhouse.com/docs/en/operations/utilities/clickhouse-local#download-clickhouse-local