jdk1.8
seatunnel2.3.3
想实现MYSQL到CLICKHOUSE同步
问题一:执行seatunnel本地案例提示如下
且在 apache-seatunnel-2.3.3/connectors/seatunnel中存在FakeSource的connector-fake-2.3.3.jar包
原因:缺少包 在安装目录/usr/local/seatunnel/apache-seatunnel-2.3.3/lib下加入如下包(可以通过maven仓库获取)
报错二:Caused by: java.lang.NoSuchMethodError: 'java.lang.String com.mysql.cj.CharsetMapping.getStaticJavaEncodingForMysqlCharset(java.lang.String)
?解决方案:
之前用的mysql驱动mysql-connector-java-8.0.25.jar? 需要升级到? ?mysql-connector-java-8.0.30.jar
问题三:Caused by: org.apache.seatunnel.engine.common.exception.JobDefineCheckException: 'MySQL-CDC' source don't support off-line job
问题四:mysql-cdc不支持离线模式 job.mode = "BATCH",需要改成? job.mode = "STREAMING"
env {
? ? ? ? execution.parallelism = 1
? ? ? ? job.mode = "STREAMING"
? ? ? ? checkpoint.interval = 2000
}
source {
? ? ? ? MySQL-CDC {
? ? ? ? base-url = "jdbc:mysql://192.168.85.128:3307/gisqbpm"
? ? ? ? username = "root"
? ? ? ? password = "wxy123456"
? ? ? ? table-names = ["gisqbpm.test"]
? ?}
}
sink {
? ? ? Clickhouse {
? ? ? ? ? ? ? ? host = "192.168.85.128:8123"
? ? ? ? ? ? ? ? database = "gisqbpm"
? ? ? ? ? ? ? ? table = "test"
? ? ? ? ? ? ? ? username = "default"
? ? ? ? ? ? ? ? password = "wxy123456"
? ? }
}