Customize Go Builds on AWS SAM with Dockerfiles and Makefiles

This post continues the series Building APPS with AWS SAM and Go, building upon the first installment. The previous chapter highlighted AWS's limited guidance on structuring scalable Go projects without redundant code.

This article demonstrates techniques for managing build processes using Dockerfiles and Makefiles.

The accompanying code is available here: https://www.php.cn/link/5655cf23be4dda7082c8bb3a8d8f8016. Explore the different Git branches for various use cases.

Let's begin!

The Challenge

After developing a new project structure, I chose Nix for dependency management (languages, tools, libraries). Nix operates by creating a temporary shell with the specified dependencies.

I encountered an error when executing binaries built within a Nix shell:

<code>libc.so.6 not found in /nix/23fj39chsggb09s.libc</code>

This halted Lambda execution. Debugging revealed the root cause: Go sometimes dynamically links C libraries to executables, specifying system paths. The libraries linked to Nix-built executables were:

<code>$ ldd bootstrap 
        linux-vdso.so.1 (0x00007ffff7fc4000)
        libresolv.so.2 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/libresolv.so.2 (0x00007ffff7fac000)
        libpthread.so.0 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/libpthread.so.0 (0x00007ffff7fa7000)
        libc.so.6 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/libc.so.6 (0x00007ffff7c00000)
        /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib/ld-linux-x86-64.so.2 => /nix/store/65h17wjrrlsj2rj540igylrx7fqcd6vq-glibc-2.40-36/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc6000)</code>

Nix's non-standard dependency storage, combined with Lambda's isolated Docker containers, prevented the Lambda from locating these libraries, as they only existed in my local Nix installation. A solution was needed to instruct AWS SAM on how to compile the code and manage library linking.

Deploying Go Projects to AWS

Two deployment methods exist:

Zip Files ?

Compile locally and send the executable to AWS in a .zip file. AWS copies the executable to the Docker container. This offers the fastest cold starts.

Docker Images ?

Provide AWS with instructions to compile within the execution Docker container. This ensures compatibility but results in slower cold starts.

Solutions

I opted for Dockerfiles to continue using Nix, but both methods are presented below.

Zip Files ?

For Zip files, use this project structure (note the Makefile):

<code>.
├── cmd/
│   ├── function1/
│   │   └── function1.go  # contains main()
│   └── function2/
│       └── function2.go  # contains main()
├── internal/
│   └── SHAREDFUNC.go
├── Makefile
├── go.mod
├── go.sum
├── samconfig.toml
└── template.yaml</code>

The Makefile defines build commands for each function using the build-<function_name> pattern (required by AWS SAM):

<code>.PHONY: build

build:
    sam build

build-HelloWorldFunction:
    GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap ./cmd/function1/main.go
    cp ./bootstrap $(ARTIFACTS_DIR)

build-ByeWorldFunction:
    GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap ./cmd/function2/main.go
    cp ./bootstrap $(ARTIFACTS_DIR)</code>

Inform SAM of this process:

<code>  HelloWorldFunction:
    Type: AWS::Serverless::Function 
    Metadata:
      BuildMethod: makefile
    Properties:
      CodeUri: ./
      Handler: bootstrap
      Runtime: provided.al2023
      Architectures:
        - x86_64
      Events:
        CatchAll:
          Type: Api 
          Properties:
            Path: /hello
            Method: GET</code>

BuildMethod: makefile tells SAM to use the Makefile, located where CodeUri specifies.

Docker Images ?

Create a Dockerfile and .dockerignore in the root directory:

<code>.
├── cmd/
│   ├── function1/
│   │   └── function1.go  # contains main()
│   └── function2/
│       └── function2.go  # contains main()
├── internal/
│   └── SHAREDFUNC.go
├── Dockerfile
├── .dockerignore
├── go.mod
├── go.sum
├── samconfig.toml
└── template.yaml</code>

The Dockerfile specifies build steps. ARG ENTRY_POINT specifies the lambda entry point at build time:

<code>FROM public.ecr.aws/docker/library/golang:1.19 as build-image
ARG ENTRY_POINT  # !IMPORTANT
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -tags lambda.norpc -o lambda-handler ${ENTRY_POINT}
FROM public.ecr.aws/lambda/provided:al2023
COPY --from=build-image /src/lambda-handler .
ENTRYPOINT ./lambda-handler</code>

Modify template.yaml:

<code>libc.so.6 not found in /nix/23fj39chsggb09s.libc</code>

Note Metadata and PackageType: Image. DockerBuildArgs passes ENTRY_POINT from the Dockerfile, allowing a single Dockerfile for all lambdas.

Conclusion

This detailed explanation provides a comprehensive approach to managing Go builds within AWS SAM using both Zip files and Docker images. The choice depends on the priorities of build speed versus deployment consistency.

The above is the detailed content of Customize Go Builds on AWS SAM with Dockerfiles and Makefiles. For more information, please follow other related articles on the PHP Chinese website!