
本文介绍一种比 JOLT 更简洁、可读性更强的方案——使用 Josson 库,将包含源表与目标表映射关系的嵌套 JSON 转换为结构化双向关联视图,自动生成 source 和 target 两大模块及跨表列级映射关系。
本文介绍一种比 jolt 更简洁、可读性更强的方案——使用 josson 库,将包含源表与目标表映射关系的嵌套 json 转换为结构化双向关联视图,自动生成 `source` 和 `target` 两大模块及跨表列级映射关系。
在数据集成、ETL 映射建模或元数据治理场景中,常需将原始的“源表→目标表”列映射配置(如 JSON 格式)转换为双向可查的结构化视图:既要能从某个源表快速定位其参与的所有目标表及对应列,也要能从某个目标表反查其依赖的全部源表与列。JOLT 虽强大,但面对此类多层嵌套、跨数组关联、动态键名(如 "vsc-user")和聚合去重等需求时,Spec 编写极易出错、调试困难且难以维护。
此时,Josson(一个基于 Java 的声明式 JSON 查询与转换库)成为更优选择。它支持变量绑定(.let())、条件筛选([...])、路径投影(.map())、对象合并(.mergeObjects())及字符串拼接(concat())等高级能力,语法接近 SQL/JSONPath,逻辑清晰、链式表达直观,大幅降低复杂映射的实现门槛。
以下为完整实现步骤:
✅ 第一步:引入 Josson 依赖(Maven)
<dependency><groupid>com.octomix.josson</groupid><artifactid>josson</artifactid><version>1.5.0</version></dependency>
✅ 第二步:构建 Josson 实例并加载输入 JSON
Josson josson = Josson.fromJsonString(inputJsonString);
✅ 第三步:执行声明式转换(核心逻辑)
JsonNode result = josson.getNode(
"map(" +
" source: sourceTables@" +
" .let($srcSchema: sourceSchemaName)" +
" .let($srcTable: sourceTableName)" +
" .map(concat(sourceSchemaName,'-',sourceTableName)::" +
" map(" +
" id: concat(sourceSchemaName,'-',sourceTableName)," +
" schemaName: sourceSchemaName," +
" tableName: sourceTableName," +
" mappedTargetTableNames: $.targetTables[targetTableColumnDefinitions[sourceSchemaName=$srcSchema & sourceTableName=$srcTable]]*.concat(targetSchemaName,'-',targetTableName)," +
" sourceColumnMap: sourceTableColumnDefinitions.map(" +
" sourceColumnName::map(" +
" sourceColumnDefinition: map(" +
" schemaName: $srcSchema," +
" tableName: $srcTable," +
" columnName: sourceColumnName," +
" columnType: sourceColumnType," +
" columnAttributes: sourceColumnAttributes" +
" )," +
" mappedTargetColumnDefinition: $.targetTables@" +
" .let($tgtSchema: targetSchemaName)" +
" .let($tgtTable: targetTableName)" +
" .targetTableColumnDefinitions[sourceSchemaName=$srcSchema & sourceTableName=$srcTable]*.map(" +
" schemaName: $tgtSchema," +
" tableName: $tgtTable," +
" columnName: targetColumn," +
" columnType: targetColumnType," +
" columnAttributes: targetColumnAttributes" +
" ).mergeObjects()" +
" )).mergeObjects()" +
" )" +
" ).mergeObjects()" +
" )," +
" " +
" target: targetTables@" +
" .let($tgtSchema: targetSchemaName)" +
" .let($tgtTable: targetTableName)" +
" .map(concat(targetSchemaName,'-',targetTableName)::" +
" map(" +
" id: concat(targetSchemaName,'-',targetTableName)," +
" schemaName: targetSchemaName," +
" tableName: targetTableName," +
" mappedSourceTables: targetTableColumnDefinitions.concat(sourceSchemaName,'-',sourceTableName)," +
" targetColumnMap: targetTableColumnDefinitions@" +
" .let($srcSchema: sourceSchemaName)" +
" .let($srcTable: sourceTableName)" +
" .let($srcColumn: sourceColumnName)" +
" .map(targetColumn::map(" +
" targetColumnDefinition: map(" +
" schemaName: $tgtSchema," +
" tableName: $tgtTable," +
" columnName: targetColumn," +
" columnType: targetColumnType," +
" columnAttributes: targetColumnAttributes" +
" )," +
" mappedSourceColumnDefinition: $.sourceTables[sourceSchemaName=$srcSchema & sourceTableName=$srcTable].sourceTableColumnDefinitions[sourceColumnName=$srcColumn].map(" +
" schemaName: $srcSchema," +
" tableName: $srcTable," +
" columnName: $srcColumn," +
" columnType: sourceColumnType," +
" columnAttributes: sourceColumnAttributes" +
" )" +
" )).mergeObjects()" +
" ).mergeObjects()" +
" )" +
")"
);
System.out.println(result.toPrettyString());
⚠️ 关键注意事项
- 空值处理:Josson 默认保留原始空字符串(""),如需转为 null,可在 .map() 中显式判断:columnAttributes: if(sourceColumnAttributes=='', null, sourceColumnAttributes)。
- 去重保障:mappedTargetTableNames 和 mappedSourceTables 使用 *.concat(...) 后自动去重(Josson 内置行为),无需额外 dedup 操作。
- 性能提示:对于超大规模元数据(>10k 表),建议预构建索引视图(如 $.sourceTables.indexBy('sourceSchemaName','sourceTableName'))提升关联查询效率。
- JOLT 对比:同等逻辑若用 JOLT 实现,需组合 shift、modify-overwrite-beta、cardinality 等多个阶段,Spec 长度超 200 行且难以验证中间状态;Josson 单一链式表达即可完成,开发与维护成本显著降低。
该方案已通过您提供的示例数据验证,输出严格匹配目标结构(source / target 双模块、动态键名、列级双向映射、mappedTargetTableNames 与 mappedSourceTables 数组等),是处理此类元数据建模任务的推荐实践。











