用Java編寫kooperator

本教學專門針對具有 Java 背景、想要學習如何快速編寫第一個 kubernetes 運算子的開發人員。為什麼是運營商？有以下幾個優點：

• 顯著減少維護，節省擊鍵次數
• 彈性內建於您建立的任何系統
• 學習的樂趣，​​認真了解 Kubernetes 的具體細節

我會嘗試將理論限制在最低限度，並展示一個萬無一失的食譜如何「烤蛋糕」。我選擇 Java 是因為它比較接近我的工作經驗，而且說實話它比 Go 更容易（但有些人可能不同意）。

讓我們直接跳到它。

理論與背景

沒有人喜歡閱讀冗長的文檔，但讓我們快速了解一下，好嗎？

什麼是 Pod？

Pod 是一組具有共用網路介面（並給定唯一的 IP 位址）和儲存的容器。

什麼是副本集？

副本集控制 Pod 的建立和刪除，以便在每個時刻都有指定範本數量的 Pod。

什麼是部署？

Deployment 擁有副本集並間接擁有 Pod。當您建立部署時，Pod 就會被創建，當您刪除它時，Pod 就會消失。

什麼是服務？

服務是一組 Pod 的單一互聯網端點（它在它們之間平均分配負載）。您可以將其公開為從叢集外部可見。它會自動建立端點切片。

kubernetes 的問題在於它從一開始就被設計為無狀態的。副本集不會追蹤 Pod 的身份，當特定 Pod 消失時，就會建立新的 Pod。有一些用例需要狀態，例如資料庫和快取叢集。有狀態集只能部分緩解這個問題。

這就是為什麼人們開始編寫運算子來減輕維護負擔的原因。我不會深入討論該模式和各種 sdks — 您可以從這裡開始。

控制器和調節

kubernetes 中工作的一切、機器的每個微小齒輪都基於控制循環的簡單概念。因此，此控制循環對於特定資源類型的作用是檢查是什麼以及應該是什麼（如清單中所定義）。如果存在不匹配，它會嘗試執行一些操作來修復該問題。這就是所謂的和解。

運算子的真正意義是相同的概念，但適用於自訂資源。自訂資源是將 kubernetes api 擴展到您定義的某些資源類型的方法。如果您在 kubernetes 中設定了 crd，則可以在此資源上執行所有操作，例如取得、列出、更新、刪除等。實際工作會做什麼？沒錯——我們的營運商。

激勵範例和 Java 應用程式

作為第一次測試技術的典型，您選擇最基本的問題。因為概念特別複雜，所以這個例子中的 hello world 會有點長。無論如何，在大多數來源中，我看到最簡單的用例是設定靜態頁面服務。

所以項目是這樣的：我們將定義代表我們想要服務的兩個頁面的自訂資源。套用資源後，操作員將自動在 Spring Boot 中設定服務應用程序，建立包含頁面內容的配置映射，將組態映射載入到 apps pod 中的磁碟區中，並為該 pod 設定服務。有趣的是，如果我們修改資源，它將動態重新綁定所有內容，並且新頁面變更將立即可見。第二個有趣的事情是，如果我們刪除資源，它將刪除所有內容，使我們的叢集保持乾淨。

提供 Java 應用程式

這將是 Spring Boot 中非常簡單的靜態頁面伺服器。您只需要 spring-boot-starter-web，因此請繼續使用 spring 初始化程序並選擇：

• 行家
• java 21
• 最新穩定版本（對我來說是3.3.4）
• graal 虛擬機器
• 和 Spring Boot 入門網站

應用程式就是這樣：

@SpringBootApplication
@RestController
public class WebpageServingApplication {

    @GetMapping(value = "/{page}", produces = "text/html")
    public String page(@PathVariable String page) throws IOException {
        return Files.readString(Path.of("/static/"+page));
    }

    public static void main(String[] args) {
        SpringApplication.run(WebpageServingApplication.class, args);
    }

}

無論我們作為路徑變數傳遞什麼，都將從 /static 目錄中取得（在我們的範例中為 page1 和 page2）。因此靜態目錄將從配置映射中掛載，但稍後再說。

所以現在我們必須建立一個原生鏡像並將其推送到遠端儲存庫。

提示1

<plugin>
    <groupid>org.graalvm.buildtools</groupid>
    <artifactid>native-maven-plugin</artifactid>
    <configuration>
        <buildargs>
            <buildarg>-Ob</buildarg>
        </buildargs>
    </configuration>
</plugin>

像這樣配置 GraalVM，您將以最低的記憶體消耗（大約 2GB）實現最快的建置。對我來說這是必須的，因為我只有 16GB 內存並且安裝了很多東西。

提示2

<plugin>
    <groupid>org.springframework.boot</groupid>
    <artifactid>spring-boot-maven-plugin</artifactid>
    <configuration>
        <image>
            <publish>true</publish>
            <builder>paketobuildpacks/builder-jammy-full:latest</builder>
            <name>ghcr.io/dgawlik/webpage-serving:1.0.5</name>
            <env>
                <bp_jvm_version>21</bp_jvm_version>
            </env>
        </image>
        <docker>
            <publishregistry>
                <url>https://ghcr.io/dgawlik</url>
                <username>dgawlik</username>
                <password>${env.GITHUB_TOKEN}</password>
            </publishregistry>
        </docker>
    </configuration>
</plugin>

• use paketobuildpacks/builder-jammy-full:latest while you are testing because -tiny and -base won’t have bash installed and you won’t be able to attach to container. Once you are done you can switch.
• publish true will cause building image to push it to repository, so go ahead and switch it to your repo
• BP_JVM_VERSION will be the java version of the builder image, it should be the same as the java of your project. As far as I know the latest java available is 21.

So now you do:

mvn spring-boot:build-image

And that’s it.

Operator with Fabric8

Now the fun starts. First you will need this in your pom:

<dependencies>
   <dependency>
      <groupid>io.fabric8</groupid>
      <artifactid>kubernetes-client</artifactid>
      <version>6.13.4</version>
   </dependency>
   <dependency>
      <groupid>io.fabric8</groupid>
      <artifactid>crd-generator-apt</artifactid>
      <version>6.13.4</version>
      <scope>provided</scope>
   </dependency>
</dependencies>

crd-generator-apt is a plugin that scans a project, detects CRD pojos and generates the manifest.

Since I mentioned it, these resources are:

@Group("com.github.webserving")
@Version("v1alpha1")
@ShortNames("websrv")
public class WebServingResource extends CustomResource<webservingspec webservingstatus> implements Namespaced {
}
</webservingspec>

public record WebServingSpec(String page1, String page2) {
}

public record WebServingStatus (String status) {
}

What is common in all resource manifests in Kubernetes is that most of them has spec and status. So you can see that the spec will consist of two pages pasted in heredoc format. Now, the proper way to handle things would be to manipulate status to reflect whatever operator is doing. If for example it is waiting on deployment to finish it would have status = “Processing”, on everything done it would patch the status to “Ready” and so on. But we will skip that because this is just simple demo.

Good news is that the logic of the operator is all in main class and really short. So step by step here it is:

KubernetesClient client = new KubernetesClientBuilder()
    .withTaskExecutor(executor).build();

var crdClient = client.resources(WebServingResource.class)
    .inNamespace("default");


var handler = new GenericResourceEventHandler(update -> {
   synchronized (changes) {
       changes.notifyAll();
   }
});

crdClient.inform(handler).start();

client.apps().deployments().inNamespace("default")
     .withName("web-serving-app-deployment").inform(handler).start();

client.services().inNamespace("default")
   .withName("web-serving-app-svc").inform(handler).start();

client.configMaps().inNamespace("default")
    .withName("web-serving-app-config").inform(handler).start();

So the heart of the program is of course Fabric8 Kuberenetes client built in first line. It is convenient to customize it with own executor. I used famous virtual threads, so when waiting on blocking io java will suspend the logic and move to main.

How here is a new part. The most basic version would be to run forever the loop and put Thread.sleep(1000) in it or so. But there is more clever way - kubernetes informers. Informer is websocket connection to kubernetes api server and it informs the client each time the subscribed resource changes. There is more to it you can read on the internet for example how to use various caches which fetch updates all at once in batch. But here it just subscribes directly per resource. The handler is a little bit bloated so I wrote a helper class GenericResourceEventHandler.

public class GenericResourceEventHandler<t> implements ResourceEventHandler<t> {

    private final Consumer<t> handler;

    public GenericResourceEventHandler(Consumer<t> handler) {
        this.handler = handler;
    }


    @Override
    public void onAdd(T obj) {
        this.handler.accept(obj);
    }

    @Override
    public void onUpdate(T oldObj, T newObj) {
        this.handler.accept(newObj);
    }

    @Override
    public void onDelete(T obj, boolean deletedFinalStateUnknown) {
        this.handler.accept(null);
    }
}
</t></t></t></t>

Since we only need to wake up the loop in all of the cases then we pass it a generic lambda. The idea for the loop is to wait on lock in the end and then the informer callback releases the lock each time the changes are detected.

Next:

for (; ; ) {

    var crdList = crdClient.list().getItems();
    var crd = Optional.ofNullable(crdList.isEmpty() ? null : crdList.get(0));


    var skipUpdate = false;
    var reload = false;

    if (!crd.isPresent()) {
        System.out.println("No WebServingResource found, reconciling disabled");
        currentCrd = null;
        skipUpdate = true;
    } else if (!crd.get().getSpec().equals(
            Optional.ofNullable(currentCrd)
                    .map(WebServingResource::getSpec).orElse(null))) {
        currentCrd = crd.orElse(null);
        System.out.println("Crd changed, Reconciling ConfigMap");
        reload = true;
    }

If there is no crd then there is nothing to be done. And if the crd changed then we have to reload everything.

var currentConfigMap = client.configMaps().inNamespace("default")
        .withName("web-serving-app-config").get();

if(!skipUpdate && (reload || desiredConfigMap(currentCrd).equals(currentConfigMap))) {
    System.out.println("New configmap, reconciling WebServingResource");
    client.configMaps().inNamespace("default").withName("web-serving-app-config")
            .createOrReplace(desiredConfigMap(currentCrd));
    reload = true;
}

This is for the case that ConfigMap is changed in between the iterations. Since it is mounted in pod then we have to reload the deployment.

var currentServingDeploymentNullable = client.apps().deployments().inNamespace("default")
        .withName("web-serving-app-deployment").get();
var currentServingDeployment = Optional.ofNullable(currentServingDeploymentNullable);

if(!skipUpdate && (reload || !desiredWebServingDeployment(currentCrd).getSpec().equals(
        currentServingDeployment.map(Deployment::getSpec).orElse(null)))) {

    System.out.println("Reconciling Deployment");
    client.apps().deployments().inNamespace("default").withName("web-serving-app-deployment")
            .createOrReplace(desiredWebServingDeployment(currentCrd));
}

var currentServingServiceNullable = client.services().inNamespace("default")
            .withName("web-serving-app-svc").get();
var currentServingService = Optional.ofNullable(currentServingServiceNullable);

if(!skipUpdate && (reload || !desiredWebServingService(currentCrd).getSpec().equals(
        currentServingService.map(Service::getSpec).orElse(null)))) {

    System.out.println("Reconciling Service");
    client.services().inNamespace("default").withName("web-serving-app-svc")
            .createOrReplace(desiredWebServingService(currentCrd));
}

If any of the service or deployment differs from the defaults we will replace them with the defaults.

synchronized (changes) {
    changes.wait();
}

Then the aforementioned lock.

So now the only thing is to define the desired configmap, service and deployment.

private static Deployment desiredWebServingDeployment(WebServingResource crd) {
    return new DeploymentBuilder()
            .withNewMetadata()
            .withName("web-serving-app-deployment")
            .withNamespace("default")
            .addToLabels("app", "web-serving-app")
            .withOwnerReferences(createOwnerReference(crd))
            .endMetadata()
            .withNewSpec()
            .withReplicas(1)
            .withNewSelector()
            .addToMatchLabels("app", "web-serving-app")
            .endSelector()
            .withNewTemplate()
            .withNewMetadata()
            .addToLabels("app", "web-serving-app")
            .endMetadata()
            .withNewSpec()
            .addNewContainer()
            .withName("web-serving-app-container")
            .withImage("ghcr.io/dgawlik/webpage-serving:1.0.5")
            .withVolumeMounts(new VolumeMountBuilder()
                    .withName("web-serving-app-config")
                    .withMountPath("/static")
                    .build())
            .addNewPort()
            .withContainerPort(8080)
            .endPort()
            .endContainer()
            .withVolumes(new VolumeBuilder()
                    .withName("web-serving-app-config")
                    .withConfigMap(new ConfigMapVolumeSourceBuilder()
                            .withName("web-serving-app-config")
                            .build())
                    .build())
            .withImagePullSecrets(new LocalObjectReferenceBuilder()
                    .withName("regcred").build())
            .endSpec()
            .endTemplate()
            .endSpec()
            .build();
}

private static Service desiredWebServingService(WebServingResource crd) {
    return new ServiceBuilder()
            .editMetadata()
            .withName("web-serving-app-svc")
            .withOwnerReferences(createOwnerReference(crd))
            .withNamespace(crd.getMetadata().getNamespace())
            .endMetadata()
            .editSpec()
            .addNewPort()
            .withPort(8080)
            .withTargetPort(new IntOrString(8080))
            .endPort()
            .addToSelector("app", "web-serving-app")
            .endSpec()
            .build();
}

private static ConfigMap desiredConfigMap(WebServingResource crd) {
    return new ConfigMapBuilder()
            .withMetadata(
                    new ObjectMetaBuilder()
                            .withName("web-serving-app-config")
                            .withNamespace(crd.getMetadata().getNamespace())
                            .withOwnerReferences(createOwnerReference(crd))
                            .build())
            .withData(Map.of("page1", crd.getSpec().page1(),
                    "page2", crd.getSpec().page2()))
            .build();
}

private static OwnerReference createOwnerReference(WebServingResource crd) {
    return new OwnerReferenceBuilder()
            .withApiVersion(crd.getApiVersion())
            .withKind(crd.getKind())
            .withName(crd.getMetadata().getName())
            .withUid(crd.getMetadata().getUid())
            .withController(true)
            .build();
}

The magic of the OwnerReference is that you mark the resource which is it’s parent. Whenever you delete the parent k8s will delete automatically all the dependant resources.

But you can’t run it yet. You need a docker credentials in kubernetes:

kubectl delete secret regcred

kubectl create secret docker-registry regcred \
  --docker-server=ghcr.io \
  --docker-username=dgawlik \
  --docker-password=$GITHUB_TOKEN

Run this script once. Then we also need to set up the ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: demo-ingress
spec:
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-serving-app-svc
                port:
                  number: 8080

The workflow

So first you build the operator project. Then you take target/classes/META-INF/fabric8/webservingresources.com.github.webserving-v1.yml and apply it. From now on the kubernetes is ready to accept your crd. Here it is:

apiVersion: com.github.webserving/v1alpha1
kind: WebServingResource
metadata:
  name: example-ws
  namespace: default
spec:
  page1: |
    <h1 id="Hola-amigos">Hola amigos!</h1>
    <p>Buenos dias!</p>
  page2: |
    <h1 id="Hello-my-friend">Hello my friend</h1>
    <p>Good evening</p>

You apply the crd kubectl apply -f src/main/resources/crd-instance.yaml. And then you run Main of the operator.

Then monitor the pod if it is up. Next just take the ip of the cluster:

minikube ip

And in your browser navigate to /page1 and /page2.

Then try to change the crd and apply it again. After a second you should see the changes.

The end.

Conclusion

A bright observer will notice that the code has some concurrency issues. A lot can happen in between the start and the end of the loop. But there are a lot of cases to consider and tried to keep it simple. You can do it as aftermath.

Like wise for the deployment. Instead of running it in IDE you can build the image the same way as for serving app and write deployment of it. That’s basically demystification of the operator — it is just a pod like every other.

I hope you found it useful.

Thanks for reading.

I almost forgot - here is the repo:

https://github.com/dgawlik/operator-hello-world

以上是用Java編寫kooperator的詳細內容。更多資訊請關注PHP中文網其他相關文章！