How to Install Python Packages from GCP Artifact Registry in Docker file

You have developed a great Python package to provide internal use of the company. You want to publish it so that your colleague can start using it. Because the package is only for internal use, it cannot be published on the Pypi (official Python package registry). Instead, because your company uses GCP, the natural choice is Artifact Registry.

As described in the document, it is very simple to publish the package to the registry.

库

I use POETRY packaging library. Here are some commands you will use:

After posting your bag to Artifact Registry, you can provide it as a dependent item for other projects.

<code class="language-bash">poetry source add --priority=supplemental gcp_registry https://{LOCATION}-python.pkg.dev/{REPO}/{PACKAGE}/
poetry publish --no-interaction --build --repository gcp_registry</code>

Installation package

Installing a package on the local machine, please create a requirements_private.txt file:

Then, use the following command to install packages:

<code>--index-url https://{LOCATION}-python.pkg.dev/{REPO}/{PACKAGE}/simple/
--extra-index-url https://pypi.org/simple
{YOUR_PACKAGE_NAME}</code>

Keyring package processing artifact registry authentication. Make sure your application defaults to the application (ADC) before continuing.

<code class="language-bash">pip install keyring
pip install keyrings.google-artifactregistry-auth
pip install -r /opt/requirements_private.txt</code>

Docker challenge

---

When running an application in Docker, you will face other challenges:

You don't want to copy sensitive information (such as your service account file) to the Docker mirror.

You still need to use Artifact Registry for authentication.
2. The solution is simple, but there is no good document record. It took me a few days to figure out this, so I want to save your time and help you realize it in a few minutes.

Solution

---

Passing Google_Application_CREDENTIALS environment variables during the Docker construction period, pointing to the path of the service account file (rather than the file content itself).

The service account file is secretly installed under the path specified by Google_Application_CREDENTIALS.

1. All operations are performed in the same RUN statement, including installing Keyring bags and private dependencies. This is important because the installation of files exists only in the context.
2. Make sure you have an appropriate authority to read the file.
Dockerfile Example
The following is the appearance of your Dockerfile:

Requirements_private.txt is still the same.

<code class="language-dockerfile">ARG GOOGLE_APPLICATION_CREDENTIALS

COPY requirements_private.txt /opt/requirements_private.txt

RUN --mount=type=secret,id=creds,target=/opt/mykey.json,mode=0444 \
     pip install keyring && \
     pip install keyrings.google-artifactregistry-auth && \
     pip install -r /opt/requirements_private.txt

COPY requirements.txt /opt/requirements.txt
RUN pip install -r /opt/requirements.txt</code>

As you can see, you can have multiple requirements files. In my example, the Requirements.txt file is used to host the package in Pypi public registry. Then your docker_compose.yml file

<code>--index-url https://{LOCATION}-python.pkg.dev/{REPO}/{PACKAGE}/simple/
--extra-index-url https://pypi.org/simple
{YOUR_PACKAGE_NAME}</code>

Then you can run the constructing command:

<code class="language-yaml">services:
  app:
    build:
      context: .
      args:
        - GOOGLE_APPLICATION_CREDENTIALS=/opt/mykey.json
      secrets:
        - creds

secrets:
  creds:
    file: "C:/your/local/host/path/to/google_service_account.json"</code>

I hope this article will help you integrated with Artifact Registry and Docker.

The above is the detailed content of How to Install Python Packages from GCP Artifact Registry in Docker file. For more information, please follow other related articles on the PHP Chinese website!