Gradle multi-module build with custom source set: classes missing from compilation classpath of dependent modules

php editor Xinyi will introduce to you how to use Gradle multi-module construction with custom source sets in this article. During the build process, we often encounter the problem of missing classes in the compiled classpath of dependent modules. This article will explain the cause of this problem in detail and provide solutions to ensure that all required classes are included in the compiled classpath of dependent modules. Through the guidance of this article, you will be able to better understand and apply the custom source set function in Gradle multi-module construction, and improve the construction efficiency and maintainability of the project.

Question content

I am currently contributing to the openlineage spark integration, a project entirely in java and built with gradle (8.4). This is a multi-module project with module names app, shared, spark2, spark3, spark32 , spark33, spark34 and spark35.

question: I'm trying to build the shared module for scala 2.12 and 2.13 variants of apache spark. All modules (spark2 to spark35) depend on shared. My goal is to compile a module like spark35 to produce two versions: one for scala 2.12 and another for the scala 2.13 variant of apache spark. This also requires building the shared module correctly using the corresponding spark variant to avoid runtime errors.

Method taken: I introduced source sets scala213 and spark33, spark34, spark35 and shared using the java library plugin testscala213. These source sets use the same sources as the main and test sources. The dependencies for the scala 2.12 variant of apache spark are set to default and I have additional declarations for the scala 2.13 source set. This includes dependencies on shared modules.

Problems encountered: When I request scala213runtimeelements (even the scala213apielements configuration) from the shared project, the compiled classpath does not contain the compiled shared module kind. However, after reverting to the default configuration, these classes will be present in the classpath.

question: Why does specifying a specific configuration (scala213runtimeelements) cause the shared module's compiled classes to be missing from the compiled classpath of my other modules? How can I resolve this issue to ensure that the shared module's classes are included in the classpath for scala 2.12 and 2.13 versions?

Below you will find my various build.gradle files.

Share/build.gradle

sourcesets {
    scala213 {
        java.srcdir("src/main/java")
        resources.srcdir("src/main/resources")
    }

    testscala213 {
        compileclasspath += sourcesets.scala213.output
        runtimeclasspath += sourcesets.scala213.output

        java.srcdir("src/test/java")
        resources.srcdir("src/test/resources")
    }
}

configurations {
    scala213api
    
    scala213apielements {
        extendsfrom(scala213api)
        canberesolved = false
        canbeconsumed = true
    }
    
    scala213implementation.extendsfrom(scala213api)
    
    scala213runtimeelements {
        extendsfrom(scala213implementation, scala213runtimeonly)
        canberesolved = false
        canbeconsumed = true
    }
    
    scala213compileclasspath {
        extendsfrom(scala213compileonly, scala213implementation)
        canberesolved = true
    }

    scala213runtimeclasspath {
        extendsfrom(scala213implementation, scala213runtimeonly)
        canberesolved = true
    }
    
    testscala213implementation.extendsfrom(scala213implementation)
    testscala213runtimeonly.extendsfrom(scala213runtimeonly)

    testscala213compileclasspath {
        extendsfrom(testscala213compileonly, testscala213implementation)
        canberesolved = true
    }

    testscala213runtimeclasspath {
        extendsfrom(testscala213implementation, testscala213runtimeonly)
        canberesolved = true
    }
}

spark33/build.gradle：

sourceSets {
    scala213 {
        java.srcDir("src/main/java")
        resources.srcDir("src/main/resources")
    }

    testScala213 {
        compileClasspath += sourceSets.scala213.output
        runtimeClasspath += sourceSets.scala213.output

        java.srcDir("src/test/java")
        resources.srcDir("src/test/resources")
    }
}

dependencies {
    implementation(project(path: ":shared"))
    // others removed for brevity

    scala213Implementation(project(path: ":shared", configuration: "scala213RuntimeElements"))
    // others removed for brevity
}

Solution

Okay. I thought about it. This is a case of RTFM.

Specifically, this part of the manual. I did not add the "variant" artifact to the shared project. Therefore, Gradle has no idea what it should consume. Confusingly, the default configuration uses classes, while the variants require JARs.

The above is the detailed content of Gradle multi-module build with custom source set: classes missing from compilation classpath of dependent modules. For more information, please follow other related articles on the PHP Chinese website!