Analyzing Kubernetes gRPC load balancing (L4 vs L7)

This article is introduced by the go language tutorial column to introduce gRPC load balancing in Kubernetes. I hope it will be helpful to friends in need!

                                                                                                               

Installation environment dependencies

• docker-desktop >= 4.1.1
• kubernetes >= 1.21.5
• go >= 1.17
• protobuf >= 3.17.3
• istioctl >= 1.11.4

Download and install Docker Desktop and start the built-in Kubernetes cluster.

# 安装 Gobrew install go# 安装 Protobufbrew install protobuf# 安装 Istiobrew install istioctl
kubectl config use-context docker-desktop
istioctl install -y

Project address

github.com/jxlwqq/grpc-lb

Pull code:

git clone git@github.com:jxlwqq/grpc-lb.gitcd grpc-lb

Makefile Introduction

Install protoc-gen-go and protoc-gen-grpcBased on proto File, generate *_pb.go and *_grpc.pb.goBuild docker imageDeploy services in the cluster##make kube-deletemake istio-injectFor specific logic, please check the Makefile.

Command	Description
##make init
make protoc
make docker-build
make kube-deploy
Delete service
Inject Istio sidecar

L4 vs L7 Load BalancingThe so-called layer four is load balancing based on IP ports, while layer seven is load balancing based on application layer information such as URLs; Kubernetes The built-in Service load balancing is based on iptables/ipvs and only supports L4. In other words, Service supports the HTTP/1.1 protocol but does not support the HTTP/2 protocol.

Envoy (Istio) is more versatile and supports all HTTP/2 functions requested and responded by gRPC as the underlying routing and load balancing.

Project ArchitectureThis project tests the support of Service and Envoy (Istio) for HTTP/RPC load balancing respectively.

cmd/server/main.go: Server, providing both HTTP and RPC services. The response data is the Pod name where the server container is located (based on Downward API).

• cmd/client-http/main.go: HTTP client, through HTTP mode, calls the server interface cyclically and prints the return value.
• cmd/client-grpc/main.go: The gRPC client, through RPC, calls the server method remotely in a loop and prints the return value.

Testing PrincipleThe server deploys 3 copies in the Kubernetes cluster in the form of Deployment. The Pod names of the 3 copies are different, and client-http and client-grpc will call the server once every second and print the return value. If all three Pod names exist in the return value, it indicates that effective load balancing is being performed; otherwise, it indicates that effective load balancing is not being performed.

Test ServiceBuild the image:

make docker-build # 构建镜像（构建好的镜像，不 push 到远程仓库中）

View the image:

docker images ls

Return:

REPOSITORY            TAG       IMAGE ID       CREATED          SIZE
grpc-lb/client-grpc   latest    95d32ead8d9b   12 seconds ago   16.6MB
grpc-lb/client-http   latest    dbf0341206f6   22 seconds ago   11.5MB
grpc-lb/server        latest    1ef346785b2a   29 seconds ago   18.2MB

Deployment Go to the cluster:

make kube-deploy  # 在集群中部署服务

View Pod:

kubectl get pods

Return:

NAME                           READY   STATUS    RESTARTS   AGE
client-grpc-6c565594f4-tdf75   1/1     Running   0          2m48s
client-http-55d95c744d-f7nx4   1/1     Running   0          2m49s
server-7c4bfd74d-29c69         1/1     Running   0          2m51s
server-7c4bfd74d-4btvw         1/1     Running   0          2m51s
server-7c4bfd74d-fk8zf         1/1     Running   0          2m51s

View the log of client-http Pod:

export CLIENT_HTTP_POD=$(kubectl get pod -l app=client-http -o jsonpath={.items..metadata.name})kubectl logs "${CLIENT_HTTP_POD}"

Return:

#1: server-7c4bfd74d-4btvw#2: server-7c4bfd74d-4btvw#3: server-7c4bfd74d-29c69#4: server-7c4bfd74d-fk8zf#5: server-7c4bfd74d-fk8zf#6: server-7c4bfd74d-29c69#7: server-7c4bfd74d-fk8zf#8: server-7c4bfd74d-4btvw#9: server-7c4bfd74d-fk8zf

View the log of client-grpc Pod:

export CLIENT_GRPC_POD=$(kubectl get pod -l app=client-grpc -o jsonpath={.items..metadata.name})kubectl logs "${CLIENT_GRPC_POD}"

Return:

#1: server-7c4bfd74d-fk8zf#2: server-7c4bfd74d-fk8zf#3: server-7c4bfd74d-fk8zf#4: server-7c4bfd74d-fk8zf#5: server-7c4bfd74d-fk8zf#6: server-7c4bfd74d-fk8zf#7: server-7c4bfd74d-fk8zf#8: server-7c4bfd74d-fk8zf#9: server-7c4bfd74d-fk8zf

It can be seen that the HTTP request is carrying a payload, while the RPC request is carrying an invalid load.

Test Envoy(Istio)We have deployed an Istio in the cluster, but there is no command space for automatic injection, so we perform manual injection here.

Manual injection:

make istio-inject # 注入 Istio 边车

View Pod:

kubectl get pods

Return:

NAME                           READY   STATUS    RESTARTS   AGE
client-grpc-7864f57779-f6blx   2/2     Running   0          17s
client-http-f8964854c-jclkd    2/2     Running   0          21s
server-7846bd6bb4-bcfws        2/2     Running   0          27s
server-7846bd6bb4-fv29s        2/2     Running   0          40s
server-7846bd6bb4-hzqj6        2/2     Running   0          34s

View the log of client-http Pod:

export CLIENT_HTTP_POD=$(kubectl get pod -l app=client-http -o jsonpath={.items..metadata.name})kubectl logs "${CLIENT_HTTP_POD}"

Return:

#1: server-7846bd6bb4-hzqj6#2: server-7846bd6bb4-fv29s#3: server-7846bd6bb4-hzqj6#4: server-7846bd6bb4-hzqj6#5: server-7846bd6bb4-hzqj6#6: server-7846bd6bb4-hzqj6#7: server-7846bd6bb4-hzqj6#8: server-7846bd6bb4-bcfws#9: server-7846bd6bb4-fv29s

View the log of client-grpc Pod:

export CLIENT_GRPC_POD=$(kubectl get pod -l app=client-grpc -o jsonpath={.items..metadata.name})kubectl logs "${CLIENT_GRPC_POD}"

Return:

#1: server-7846bd6bb4-fv29s#2: server-7846bd6bb4-hzqj6#3: server-7846bd6bb4-fv29s#4: server-7846bd6bb4-bcfws#5: server-7846bd6bb4-fv29s#6: server-7846bd6bb4-hzqj6#7: server-7846bd6bb4-fv29s#8: server-7846bd6bb4-bcfws#9: server-7846bd6bb4-fv29s

It can be seen that both HTTP requests and RPC requests are carrying payloads.

Cleanup

make kube-delete
istioctl experimental uninstall --purge

The above is the detailed content of Analyzing Kubernetes gRPC load balancing (L4 vs L7). For more information, please follow other related articles on the PHP Chinese website!