用Java编写kooperator

本教程专门针对具有 Java 背景、想要学习如何快速编写第一个 kubernetes 运算符的开发人员。为什么是运营商？有以下几个优点：

• 显着减少维护，节省击键次数
• 弹性内置于您创建的任何系统中
• 学习的乐趣，认真了解 Kubernetes 的具体细节

我会尝试将理论限制在最低限度，并展示一个万无一失的食谱如何“烤蛋糕”。我选择 Java 是因为它比较接近我的工作经验，而且说实话它比 Go 更容易（但有些人可能不同意）。

让我们直接跳到它。

理论与背景

没有人喜欢阅读冗长的文档，但让我们快速了解一下，好吗？

什么是 Pod？

Pod 是一组具有共享网络接口（并给定唯一的 IP 地址）和存储的容器。

什么是副本集？

副本集控制 Pod 的创建和删除，以便在每个时刻都有指定模板数量的 Pod。

什么是部署？

Deployment 拥有副本集并间接拥有 Pod。当您创建部署时，Pod 就会被创建，当您删除它时，Pod 就会消失。

什么是服务？

服务是一组 Pod 的单一互联网端点（它在它们之间平均分配负载）。您可以将其公开为从集群外部可见。它自动创建端点切片。

kubernetes 的问题在于它从一开始就被设计为无状态的。副本集不会跟踪 Pod 的身份，当特定 Pod 消失时，就会创建新的 Pod。有一些用例需要状态，例如数据库和缓存集群。有状态集只能部分缓解这个问题。

这就是为什么人们开始编写运算符来减轻维护负担的原因。我不会深入讨论该模式和各种 sdks — 您可以从这里开始。

控制器和调节

kubernetes 中工作的一切、机器的每个微小齿轮都基于控制循环的简单概念。因此，此控制循环对于特定资源类型的作用是检查是什么以及应该是什么（如清单中所定义）。如果存在不匹配，它会尝试执行一些操作来修复该问题。这就是所谓的和解。

运算符的真正含义是相同的概念，但适用于自定义资源。自定义资源是将 kubernetes api 扩展到您定义的某些资源类型的方法。如果您在 kubernetes 中设置了 crd，则可以在此资源上执行所有操作，例如获取、列出、更新、删除等。实际工作会做什么？没错——我们的运营商。

激励示例和 Java 应用程序

作为第一次测试技术的典型，您选择最基本的问题。因为概念特别复杂，所以本例中的 hello world 会有点长。无论如何，在大多数来源中，我看到最简单的用例是设置静态页面服务。

所以项目是这样的：我们将定义代表我们想要服务的两个页面的自定义资源。应用该资源后，操作员将自动在 Spring Boot 中设置服务应用程序，创建包含页面内容的配置映射，将配置映射装载到 apps pod 中的卷中，并为该 pod 设置服务。有趣的是，如果我们修改资源，它将动态重新绑定所有内容，并且新页面更改将立即可见。第二个有趣的事情是，如果我们删除资源，它将删除所有内容，使我们的集群保持干净。

提供 Java 应用程序

这将是 Spring Boot 中非常简单的静态页面服务器。您只需要 spring-boot-starter-web，因此请继续使用 spring 初始化程序并选择：

• 行家
• java 21
• 最新稳定版本（对我来说是3.3.4）
• graal 虚拟机
• 和 Spring Boot 入门网站

应用程序就是这样：

@SpringBootApplication
@RestController
public class WebpageServingApplication {

    @GetMapping(value = "/{page}", produces = "text/html")
    public String page(@PathVariable String page) throws IOException {
        return Files.readString(Path.of("/static/"+page));
    }

    public static void main(String[] args) {
        SpringApplication.run(WebpageServingApplication.class, args);
    }

}

无论我们作为路径变量传递什么，都将从 /static 目录中获取（在我们的例子中为 page1 和 page2）。因此静态目录将从配置映射中挂载，但稍后再说。

所以现在我们必须构建一个原生镜像并将其推送到远程存储库。

提示1

<plugin>
    <groupId>org.graalvm.buildtools</groupId>
    <artifactId>native-maven-plugin</artifactId>
    <configuration>
        <buildArgs>
            <buildArg>-Ob</buildArg>
        </buildArgs>
    </configuration>
</plugin>

像这样配置 GraalVM，您将以最低的内存消耗（大约 2GB）实现最快的构建。对我来说这是必须的，因为我只有 16GB 内存并且安装了很多东西。

提示2

<plugin>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-maven-plugin</artifactId>
    <configuration>
        <image>
            <publish>true</publish>
            <builder>paketobuildpacks/builder-jammy-full:latest</builder>
            <name>ghcr.io/dgawlik/webpage-serving:1.0.5</name>
            <env>
                <BP_JVM_VERSION>21</BP_JVM_VERSION>
            </env>
        </image>
        <docker>
            <publishRegistry>
                <url>https://ghcr.io/dgawlik</url>
                <username>dgawlik</username>
                <password>${env.GITHUB_TOKEN}</password>
            </publishRegistry>
        </docker>
    </configuration>
</plugin>

• use paketobuildpacks/builder-jammy-full:latest while you are testing because -tiny and -base won’t have bash installed and you won’t be able to attach to container. Once you are done you can switch.
• publish true will cause building image to push it to repository, so go ahead and switch it to your repo
• BP_JVM_VERSION will be the java version of the builder image, it should be the same as the java of your project. As far as I know the latest java available is 21.

So now you do:

mvn spring-boot:build-image

And that’s it.

Operator with Fabric8

Now the fun starts. First you will need this in your pom:

<dependencies>
   <dependency>
      <groupId>io.fabric8</groupId>
      <artifactId>kubernetes-client</artifactId>
      <version>6.13.4</version>
   </dependency>
   <dependency>
      <groupId>io.fabric8</groupId>
      <artifactId>crd-generator-apt</artifactId>
      <version>6.13.4</version>
      <scope>provided</scope>
   </dependency>
</dependencies>

crd-generator-apt is a plugin that scans a project, detects CRD pojos and generates the manifest.

Since I mentioned it, these resources are:

@Group("com.github.webserving")
@Version("v1alpha1")
@ShortNames("websrv")
public class WebServingResource extends CustomResource<WebServingSpec, WebServingStatus> implements Namespaced {
}

public record WebServingSpec(String page1, String page2) {
}

public record WebServingStatus (String status) {
}

What is common in all resource manifests in Kubernetes is that most of them has spec and status. So you can see that the spec will consist of two pages pasted in heredoc format. Now, the proper way to handle things would be to manipulate status to reflect whatever operator is doing. If for example it is waiting on deployment to finish it would have status = “Processing”, on everything done it would patch the status to “Ready” and so on. But we will skip that because this is just simple demo.

Good news is that the logic of the operator is all in main class and really short. So step by step here it is:

KubernetesClient client = new KubernetesClientBuilder()
    .withTaskExecutor(executor).build();

var crdClient = client.resources(WebServingResource.class)
    .inNamespace("default");


var handler = new GenericResourceEventHandler<>(update -> {
   synchronized (changes) {
       changes.notifyAll();
   }
});

crdClient.inform(handler).start();

client.apps().deployments().inNamespace("default")
     .withName("web-serving-app-deployment").inform(handler).start();

client.services().inNamespace("default")
   .withName("web-serving-app-svc").inform(handler).start();

client.configMaps().inNamespace("default")
    .withName("web-serving-app-config").inform(handler).start();

So the heart of the program is of course Fabric8 Kuberenetes client built in first line. It is convenient to customize it with own executor. I used famous virtual threads, so when waiting on blocking io java will suspend the logic and move to main.

How here is a new part. The most basic version would be to run forever the loop and put Thread.sleep(1000) in it or so. But there is more clever way - kubernetes informers. Informer is websocket connection to kubernetes api server and it informs the client each time the subscribed resource changes. There is more to it you can read on the internet for example how to use various caches which fetch updates all at once in batch. But here it just subscribes directly per resource. The handler is a little bit bloated so I wrote a helper class GenericResourceEventHandler.

public class GenericResourceEventHandler<T> implements ResourceEventHandler<T> {

    private final Consumer<T> handler;

    public GenericResourceEventHandler(Consumer<T> handler) {
        this.handler = handler;
    }


    @Override
    public void onAdd(T obj) {
        this.handler.accept(obj);
    }

    @Override
    public void onUpdate(T oldObj, T newObj) {
        this.handler.accept(newObj);
    }

    @Override
    public void onDelete(T obj, boolean deletedFinalStateUnknown) {
        this.handler.accept(null);
    }
}

Since we only need to wake up the loop in all of the cases then we pass it a generic lambda. The idea for the loop is to wait on lock in the end and then the informer callback releases the lock each time the changes are detected.

Next:

for (; ; ) {

    var crdList = crdClient.list().getItems();
    var crd = Optional.ofNullable(crdList.isEmpty() ? null : crdList.get(0));


    var skipUpdate = false;
    var reload = false;

    if (!crd.isPresent()) {
        System.out.println("No WebServingResource found, reconciling disabled");
        currentCrd = null;
        skipUpdate = true;
    } else if (!crd.get().getSpec().equals(
            Optional.ofNullable(currentCrd)
                    .map(WebServingResource::getSpec).orElse(null))) {
        currentCrd = crd.orElse(null);
        System.out.println("Crd changed, Reconciling ConfigMap");
        reload = true;
    }

If there is no crd then there is nothing to be done. And if the crd changed then we have to reload everything.

var currentConfigMap = client.configMaps().inNamespace("default")
        .withName("web-serving-app-config").get();

if(!skipUpdate && (reload || desiredConfigMap(currentCrd).equals(currentConfigMap))) {
    System.out.println("New configmap, reconciling WebServingResource");
    client.configMaps().inNamespace("default").withName("web-serving-app-config")
            .createOrReplace(desiredConfigMap(currentCrd));
    reload = true;
}

This is for the case that ConfigMap is changed in between the iterations. Since it is mounted in pod then we have to reload the deployment.

var currentServingDeploymentNullable = client.apps().deployments().inNamespace("default")
        .withName("web-serving-app-deployment").get();
var currentServingDeployment = Optional.ofNullable(currentServingDeploymentNullable);

if(!skipUpdate && (reload || !desiredWebServingDeployment(currentCrd).getSpec().equals(
        currentServingDeployment.map(Deployment::getSpec).orElse(null)))) {

    System.out.println("Reconciling Deployment");
    client.apps().deployments().inNamespace("default").withName("web-serving-app-deployment")
            .createOrReplace(desiredWebServingDeployment(currentCrd));
}

var currentServingServiceNullable = client.services().inNamespace("default")
            .withName("web-serving-app-svc").get();
var currentServingService = Optional.ofNullable(currentServingServiceNullable);

if(!skipUpdate && (reload || !desiredWebServingService(currentCrd).getSpec().equals(
        currentServingService.map(Service::getSpec).orElse(null)))) {

    System.out.println("Reconciling Service");
    client.services().inNamespace("default").withName("web-serving-app-svc")
            .createOrReplace(desiredWebServingService(currentCrd));
}

If any of the service or deployment differs from the defaults we will replace them with the defaults.

synchronized (changes) {
    changes.wait();
}

Then the aforementioned lock.

So now the only thing is to define the desired configmap, service and deployment.

private static Deployment desiredWebServingDeployment(WebServingResource crd) {
    return new DeploymentBuilder()
            .withNewMetadata()
            .withName("web-serving-app-deployment")
            .withNamespace("default")
            .addToLabels("app", "web-serving-app")
            .withOwnerReferences(createOwnerReference(crd))
            .endMetadata()
            .withNewSpec()
            .withReplicas(1)
            .withNewSelector()
            .addToMatchLabels("app", "web-serving-app")
            .endSelector()
            .withNewTemplate()
            .withNewMetadata()
            .addToLabels("app", "web-serving-app")
            .endMetadata()
            .withNewSpec()
            .addNewContainer()
            .withName("web-serving-app-container")
            .withImage("ghcr.io/dgawlik/webpage-serving:1.0.5")
            .withVolumeMounts(new VolumeMountBuilder()
                    .withName("web-serving-app-config")
                    .withMountPath("/static")
                    .build())
            .addNewPort()
            .withContainerPort(8080)
            .endPort()
            .endContainer()
            .withVolumes(new VolumeBuilder()
                    .withName("web-serving-app-config")
                    .withConfigMap(new ConfigMapVolumeSourceBuilder()
                            .withName("web-serving-app-config")
                            .build())
                    .build())
            .withImagePullSecrets(new LocalObjectReferenceBuilder()
                    .withName("regcred").build())
            .endSpec()
            .endTemplate()
            .endSpec()
            .build();
}

private static Service desiredWebServingService(WebServingResource crd) {
    return new ServiceBuilder()
            .editMetadata()
            .withName("web-serving-app-svc")
            .withOwnerReferences(createOwnerReference(crd))
            .withNamespace(crd.getMetadata().getNamespace())
            .endMetadata()
            .editSpec()
            .addNewPort()
            .withPort(8080)
            .withTargetPort(new IntOrString(8080))
            .endPort()
            .addToSelector("app", "web-serving-app")
            .endSpec()
            .build();
}

private static ConfigMap desiredConfigMap(WebServingResource crd) {
    return new ConfigMapBuilder()
            .withMetadata(
                    new ObjectMetaBuilder()
                            .withName("web-serving-app-config")
                            .withNamespace(crd.getMetadata().getNamespace())
                            .withOwnerReferences(createOwnerReference(crd))
                            .build())
            .withData(Map.of("page1", crd.getSpec().page1(),
                    "page2", crd.getSpec().page2()))
            .build();
}

private static OwnerReference createOwnerReference(WebServingResource crd) {
    return new OwnerReferenceBuilder()
            .withApiVersion(crd.getApiVersion())
            .withKind(crd.getKind())
            .withName(crd.getMetadata().getName())
            .withUid(crd.getMetadata().getUid())
            .withController(true)
            .build();
}

The magic of the OwnerReference is that you mark the resource which is it’s parent. Whenever you delete the parent k8s will delete automatically all the dependant resources.

But you can’t run it yet. You need a docker credentials in kubernetes:

kubectl delete secret regcred

kubectl create secret docker-registry regcred \
  --docker-server=ghcr.io \
  --docker-username=dgawlik \
  --docker-password=$GITHUB_TOKEN

Run this script once. Then we also need to set up the ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-ingress
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-serving-app-svc
                port:
                  number: 8080

The workflow

So first you build the operator project. Then you take target/classes/META-INF/fabric8/webservingresources.com.github.webserving-v1.yml and apply it. From now on the kubernetes is ready to accept your crd. Here it is:

apiVersion: com.github.webserving/v1alpha1
kind: WebServingResource
metadata:
  name: example-ws
  namespace: default
spec:
  page1: |
    <h1>Hola amigos!</h1>
    <p>Buenos dias!</p>
  page2: |
    <h1>Hello my friend</h1>
    <p>Good evening</p>

You apply the crd kubectl apply -f src/main/resources/crd-instance.yaml. And then you run Main of the operator.

Then monitor the pod if it is up. Next just take the ip of the cluster:

minikube ip

And in your browser navigate to /page1 and /page2.

Then try to change the crd and apply it again. After a second you should see the changes.

The end.

Conclusion

A bright observer will notice that the code has some concurrency issues. A lot can happen in between the start and the end of the loop. But there are a lot of cases to consider and tried to keep it simple. You can do it as aftermath.

Like wise for the deployment. Instead of running it in IDE you can build the image the same way as for serving app and write deployment of it. That’s basically demystification of the operator — it is just a pod like every other.

I hope you found it useful.

Thanks for reading.

I almost forgot - here is the repo:

https://github.com/dgawlik/operator-hello-world

以上是用Java编写kooperator的详细内容。更多信息请关注PHP中文网其他相关文章！