Maison > Article > base de données > Démarrez rapidement un environnement de développement pour MySQL, PostgreSQL, MongoDB, Redis et Kafka à l'aide de Docker Compose

Démarrez rapidement un environnement de développement pour MySQL, PostgreSQL, MongoDB, Redis et Kafka à l'aide de Docker Compose

Patricia Arquetteoriginal: 2024-10-28 07:40:30424parcourir

Quickly Start Dev Environment For MySQL, PostgreSQL, MongoDB, Redis, and Kafka Using Docker Compose

Voici comment mettre en place rapidement un environnement de développement avec MySQL, PostgreSQL, MongoDB, Redis et Kafka à l'aide de Docker Compose, avec des images bitnami , les variables d'environnement et les outils d'interface utilisateur pour chaque base de données. Nous allons suivre le processus étape par étape :

Pourquoi utiliser des images Bitnami ?

Préconfiguré et optimisé : Les images Bitnami sont préconfigurées avec les meilleures pratiques, ce qui les rend plus faciles à configurer et à optimiser pour les cas d'utilisation courants.
Sécurité : Bitnami met régulièrement à jour ses images pour corriger les vulnérabilités, offrant ainsi une option plus sécurisée par rapport à certaines images gérées par la communauté qui peuvent ne pas être mises à jour aussi fréquemment.
Cohérence entre les environnements : Bitnami garantit que ses images fonctionnent de manière cohérente dans différents environnements, ce qui en fait un bon choix pour les configurations de test, de développement et de production.
Facilité d'utilisation : Ils incluent souvent des scripts et des valeurs par défaut qui simplifient les déploiements, réduisant ainsi le besoin de configuration et d'installation manuelles.
Documentation et assistance : Bitnami fournit une documentation détaillée et parfois une assistance via sa société mère, VMware, ce qui peut être utile pour le dépannage et l'utilisation en entreprise.

Une autre note d'importation concerne les licences, cela peut varier, mais le logiciel bitnami est généralement gratuit, ses conteneurs et packages sont basés sur des logiciels open source et utilisent des licences comme MIT, Apache 2.0 ou GPL... Lire la suite À propos des licences pour les sources ouvertes

Étape 1 : Installer Docker et Docker Compose

Installer Docker : suivez les instructions correspondant à votre système d'exploitation dans la documentation officielle de Docker.
Installer Docker Compose : suivez les instructions du guide d'installation de Docker Compose

Étape 2 : Structure du projet

Créez la structure de projet suivante :

dev-environment/
├── components   # for mounting container volumes
├── scripts/
│   ├── pgadmin
│   │   ├──servers.json   # for pgadmin automatically load postgreDB
│   ├── create-topics.sh  # for creating kafka topics
│   ├── mongo-init.sh     # init script for mongodb
│   ├── mysql-init.sql    # init script for mysql
│   ├── postgres-init.sql # init script for postgre
├── .env
├── docker-compose.yml

Étape 3 : (.env) Fichier

Créez un fichier .env avec le contenu suivant :

# MySQL Configuration
MYSQL_PORT=23306
MYSQL_USERNAME=dev-user
MYSQL_PASSWORD=dev-password
MYSQL_DATABASE=dev_database

# PostgreSQL Configuration
POSTGRES_PORT=25432
POSTGRES_USERNAME=dev-user
POSTGRES_PASSWORD=dev-password
POSTGRES_DATABASE=dev_database

# MongoDB Configuration
MONGO_PORT=27017
MONGO_USERNAME=dev-user
MONGO_PASSWORD=dev-password
MONGO_DATABASE=dev_database

# Redis Configuration
REDIS_PORT=26379
REDIS_PASSWORD=dev-password

# Kafka Configuration
KAFKA_PORT=29092
KAFKA_USERNAME=dev-user
KAFKA_PASSWORD=dev-password

# UI Tools Configuration
PHPMYADMIN_PORT=280
PGADMIN_PORT=281
MONGOEXPRESS_PORT=28081
REDIS_COMMANDER_PORT=28082
KAFKA_UI_PORT=28080

# Data Directory for Volumes
DATA_DIR=./

Étape 4 : (docker-compose.yml) Fichier

Créez le fichier docker-compose.yml :

version: '3.8'
services:
  dev-mysql:
    image: bitnami/mysql:latest
    # This container_name can be used for internal connections between containers (running on the same docker virtual network)
    container_name: dev-mysql
    ports:
      # This mapping means that requests sent to the ${MYSQL_PORT} on the host machine will be forwarded to port 3306 in the dev-mysql container. This setup allows users to access the MySQL database from outside the container, such as from a local machine or another service.
      - '${MYSQL_PORT}:3306'
    environment:
      # Setup environment variables for container
      - MYSQL_ROOT_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_USER=${MYSQL_USERNAME}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_DATABASE=${MYSQL_DATABASE}
    volumes:
      # Syncs msyql data from inside container to host machine, to keep them accross container restarts
      - '${DATA_DIR}/components/mysql/data:/bitnami/mysql/data'
      # Add custom script to init db
      - './scripts/mysql-init.sql:/docker-entrypoint-initdb.d/init.sql'

  phpmyadmin:
    image: phpmyadmin/phpmyadmin:latest
    container_name: dev-phpmyadmin
    # The depends_on option in Docker specifies that a container should be started only after the specified dependent container (e.g., dev-mysql) has been started (but not ensuring that it is ready)
    depends_on:
      - dev-mysql
    ports:
      - '${PHPMYADMIN_PORT}:80'
    environment:
      - PMA_HOST=dev-mysql
      # use internal port for internal connections, not exposed port ${MYSQL_PORT}
      - PMA_PORT=3306
      - PMA_USER=${MYSQL_USERNAME}
      - PMA_PASSWORD=${MYSQL_PASSWORD}

  #=======

  dev-postgresql:
    image: bitnami/postgresql:latest
    container_name: dev-postgresql
    ports:
      - '${POSTGRES_PORT}:5432'
    environment:
      - POSTGRESQL_USERNAME=${POSTGRES_USERNAME}
      - POSTGRESQL_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRESQL_DATABASE=${POSTGRES_DATABASE}
    volumes:
      # This setup will ensure that PostgreSQL data from inside container is synced to host machine, enabling persistence across container restarts.
      - '${DATA_DIR}/components/postgresql/data:/bitnami/postgresql/data'
      # Most relational databases support a special docker-entrypoint-initdb.d folder. This folder is used to initialise the database automatically when the container is first created.
      # We can put .sql or .sh scripts there, and Docker will automatically, here ./scripts/postgres-init.sql from host machine be automatically copied to the Docker container during the build and then run it
      - ./scripts/postgres-init.sql:/docker-entrypoint-initdb.d/init.sql:ro

  pgadmin:
    image: dpage/pgadmin4:latest
    container_name: dev-pgadmin
    depends_on:
      - dev-postgresql
    ports:
      - '${PGADMIN_PORT}:80'
    # user: root used to ensure that the container has full administrative privileges,
    # necessary when performing actions that require elevated permissions, such as mounting volumes (properly read or write to the mounted volumes), executing certain entrypoint commands, or accessing specific directories from host machine
    user: root
    environment:
      # PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD - Sets the default credentials for the pgAdmin user
      - PGADMIN_DEFAULT_EMAIL=admin@dev.com
      - PGADMIN_DEFAULT_PASSWORD=${POSTGRES_PASSWORD}
      # PGADMIN_CONFIG_SERVER_MODE - determines whether pgAdmin runs in server mode (multi-user) or desktop mode (single-user). We’re setting it to false, so we won’t be prompted for login credentials
      - PGADMIN_CONFIG_SERVER_MODE=False
      # PGADMIN_CONFIG_MASTER_PASSWORD_REQUIRED - controls whether a master password is required to access saved server definitions and other sensitive information
      - PGADMIN_CONFIG_MASTER_PASSWORD_REQUIRED=False
    volumes:
      # This setup will ensure that PGAdmin data from inside container is synced to host machine, enabling persistence across container restarts.
      - '${DATA_DIR}/components/pgadmin:/var/lib/pgadmin'
      # This setup to make PGAdmin automatically detect and connect to PostgreSQL when it starts (following the config being set in servers.json)
      - ./scripts/pgadmin/servers.json:/pgadmin4/servers.json:ro

  #=======

  dev-mongodb:
    image: bitnami/mongodb:latest
    container_name: dev-mongodb
    ports:
      - '${MONGO_PORT}:27017'
    environment:
      - MONGO_INITDB_ROOT_USERNAME=${MONGO_USERNAME}
      - MONGO_INITDB_ROOT_PASSWORD=${MONGO_PASSWORD}
      - MONGO_INITDB_DATABASE=${MONGO_DATABASE}
      - MONGODB_ROOT_USER=${MONGO_USERNAME}
      - MONGODB_ROOT_PASSWORD=${MONGO_PASSWORD}
      - MONGODB_DATABASE=${MONGO_DATABASE}
    volumes:
      - '${DATA_DIR}/components/mongodb/data:/bitnami/mongodb'
      # This line maps ./scripts/mongo-init.sh from host machine to /docker-entrypoint-initdb.d/mongo-init.sh inside container with 'ro' mode (read only mode) which means container can't modify the mounted file
      - ./scripts/mongo-init.sh:/docker-entrypoint-initdb.d/mongo-init.sh:ro
      # - ./scripts/mongo-init.sh:/bitnami/scripts/mongo-init.sh:ro

  mongo-express:
    image: mongo-express:latest
    container_name: dev-mongoexpress
    depends_on:
      - dev-mongodb
    ports:
      - '${MONGOEXPRESS_PORT}:8081'
    environment:
      - ME_CONFIG_MONGODB_ENABLE_ADMIN=true
      - ME_CONFIG_MONGODB_ADMINUSERNAME=${MONGO_USERNAME}
      - ME_CONFIG_MONGODB_ADMINPASSWORD=${MONGO_PASSWORD}
      # - ME_CONFIG_MONGODB_SERVER=dev-mongodb
      # - ME_CONFIG_MONGODB_PORT=${MONGO_PORT}
      - ME_CONFIG_MONGODB_URL=mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@dev-mongodb:${MONGO_PORT}/${MONGO_DATABASE}?authSource=admin&ssl=false&directConnection=true
    restart: unless-stopped
    # 'restart: unless-stopped' restarts a container automatically unless it is explicitly stopped by the user.
    # some others: 1. 'no': (Default option if not specified) meaning the container won't automatically restart if it stops or crashes.
    #              2. 'always': The container will restart regardless of the reason it stopped, including if Docker is restarted.
    #              3. 'on-failure': The container will restart only if it exits with a non-zero status indicating an error. (and won't restart if it stops when completing as short running task and return 0 status).

  #=======

  dev-redis:
    image: bitnami/redis:latest
    container_name: dev-redis
    ports:
      - '${REDIS_PORT}:6379'
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD}
    volumes:
      - '${DATA_DIR}/components/redis:/bitnami/redis'
    networks:
      - dev-network

  redis-commander:
    image: rediscommander/redis-commander:latest
    container_name: dev-redis-commander
    depends_on:
      - dev-redis
    ports:
      - '${REDIS_COMMANDER_PORT}:8081'
    environment:
      - REDIS_HOST=dev-redis
      # While exposed port ${REDIS_PORT} being bind to host network, redis-commander still using internal port 6379 (being use internally inside docker virtual network) to connect to redis
      - REDIS_PORT=6379
      - REDIS_PASSWORD=${REDIS_PASSWORD}
    networks:
      - dev-network
    # This networks setup is optional, in case not being set, both redis-commader and redis will both be assigned to default docker network (usually named bridge) and still being able to connect each other

  #=======

  dev-kafka:
    image: 'bitnami/kafka:latest'
    container_name: dev-kafka
    ports:
      - '${KAFKA_PORT}:9094'
    environment:
      # Sets the timezone for the container to "Asia/Shanghai". This ensures that logs and timestamps inside the Kafka container align with the Shanghai timezone.
      - TZ=Asia/Shanghai
      # KAFKA_CFG_NODE_ID=0: Identifies the Kafka node with ID 0. This is crucial for multi-node Kafka clusters to distinguish each node uniquely.
      - KAFKA_CFG_NODE_ID=0
      # KAFKA_CFG_PROCESS_ROLES=controller,broker: Specifies the roles the Kafka node will perform, in this case, both as a controller (managing cluster metadata) and a broker (handling messages).
      - KAFKA_CFG_PROCESS_ROLES=controller,broker
      # KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@541876ab254ff3f10f26905ad234f238:9093: Defines the quorum voters for the Kafka controllers. It indicates that node 0 (the current node) acts as a voter for controller decisions and will be accessible at 9093 on 541876ab254ff3f10f26905ad234f238.
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@541876ab254ff3f10f26905ad234f238:9093
      # The following lists different listeners for Kafka. Each listener binds a protocol to a specific port:
      # PLAINTEXT for client connections (:9092). CONTROLLER for internal controller communication (:9093). EXTERNAL for external client access (:9094).SASL_PLAINTEXT for SASL-authenticated clients (:9095).
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094,SASL_PLAINTEXT://:9095
      # KAFKA_CFG_ADVERTISED_LISTENERS specifies how clients should connect to Kafka externally:
      # PLAINTEXT at dev-kafka:9092 for internal communication. EXTERNAL at 127.0.0.1:${KAFKA_PORT} (host access). SASL_PLAINTEXT for SASL connections (kafka:9095).
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://dev-kafka:9092,EXTERNAL://127.0.0.1:${KAFKA_PORT},SASL_PLAINTEXT://kafka:9095
      # The following maps security protocols to each listener. For example, CONTROLLER uses PLAINTEXT, and EXTERNAL uses SASL_PLAINTEXT.
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT,PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT
      # Indicates that the CONTROLLER role should use the CONTROLLER listener for communications.
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      # Specifies users with relevant passwords that can connect to Kafka using SASL authentication
      - KAFKA_CLIENT_USERS=${KAFKA_USERNAME}
      - KAFKA_CLIENT_PASSWORDS=${KAFKA_PASSWORD}
    volumes:
      - '${DATA_DIR}/components/kafka/data:/bitnami/kafka/data'
      # Maps a local file create-topics.sh from the ./scripts directory to the path /opt/bitnami/kafka/create_topic.sh inside the Kafka container
      # This script can be used to automatically create Kafka topics when the container starts
      - ./scripts/create-topics.sh:/opt/bitnami/kafka/create_topic.sh:ro
    # Following command starts the Kafka server in the background using /opt/bitnami/scripts/kafka/run.sh. then sleep 5 to ensure that the Kafka server is fully up and running.
    # Executes the create_topic.sh script, which is used to create Kafka topics. Uses 'wait' to keep the script running until all background processes (like the Kafka server) finish,
    command: >
      bash -c "
      /opt/bitnami/scripts/kafka/run.sh & sleep 5; /opt/bitnami/kafka/create_topic.sh; wait
      "

  kafka-ui:
    image: provectuslabs/kafka-ui:latest
    container_name: dev-kafka-ui
    ports:
      - '${KAFKA_UI_PORT}:8080'
    environment:
      # Sets the name of the Kafka cluster displayed in the UI as "local."
      - KAFKA_CLUSTERS_0_NAME=local
      # Specifies the address (dev-kafka:9092) for the Kafka broker that the UI should connect to.
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=dev-kafka:9092
      # Uses the provided ${KAFKA_USERNAME} for SASL (Simple Authentication and Security Layer) authentication with the Kafka cluster.
      - KAFKA_CLUSTERS_0_SASL_USER=${KAFKA_USERNAME}
      # Uses the ${KAFKA_PASSWORD} for authentication with the Kafka broker.
      - KAFKA_CLUSTERS_0_SASL_PASSWORD=${KAFKA_PASSWORD}
      # Sets the SASL mechanism as 'PLAIN', which is a simple username-password-based authentication method.
      - KAFKA_CLUSTERS_0_SASL_MECHANISM=PLAIN
      # Configures the communication protocol as SASL_PLAINTEXT, which means it uses SASL for authentication without encryption over plaintext communication.
      - KAFKA_CLUSTERS_0_SECURITY_PROTOCOL=SASL_PLAINTEXT
    depends_on:
      - dev-kafka

networks:
  dev-network:
    driver: bridge

Étape 5 : Scripts

Créez les scripts nécessaires dans le dossier scripts.

pgadmin/servers.json :

    {
    "Servers": {
      "1": {
        "Name": "Local PostgreSQL",
        "Group": "Servers",
        "Host": "dev-postgresql",
        "Port": 5432,
        "MaintenanceDB": "dev_database",
        "Username": "dev-user",
        "Password": "dev-password",
        "SSLMode": "prefer",
        "Favorite": true
      }
    }
  }

create-topics.sh :

# Wait for Kafka to be ready
until /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092; do
  echo "Waiting for Kafka to be ready..."
  sleep 2
done

# Create topics
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic latestMsgToRedis
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic msgToPush
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic offlineMsgToMongoMysql

echo "Topics created."

mongo-init.sh :

# mongosh --: Launches the MongoDB shell, connecting to the default MongoDB instance.
# "$MONGO_INITDB_DATABASE": Specifies the database to connect to (using the value from the environment variable).
# <<EOF: Indicates the start of a multi-line input block. Everything between <<EOF and EOF is treated as MongoDB shell commands to be executed. 
# db.getSiblingDB('admin'): Switches to the admin database, which is the default administrative database in MongoDB. It allows you to perform administrative tasks like user creation, where the user dev-user will be created.
# db.auth('$MONGO_INITDB_ROOT_USERNAME', '$MONGO_INITDB_ROOT_PASSWORD') (commented out): This line, if executed, would authenticate the user with the given credentials against the "admin" database. It’s necessary if the following operations require authentication.
# The user dev-user is created in the admin database with the specified username and password.
# { role: 'root', db: 'admin' }: Allows full access to the admin database.
# { role: 'readWrite', db: '$MONGO_INITDB_DATABASE' }: Grants read and write permissions specifically for dev_database.

mongosh -- "$MONGO_INITDB_DATABASE" <<EOF
db = db.getSiblingDB('admin')
db.auth('$MONGO_INITDB_ROOT_USERNAME', '$MONGO_INITDB_ROOT_PASSWORD')
db.createUser({
  user: "$MONGODB_ROOT_USER",
  pwd: "$MONGODB_ROOT_PASSWORD",
  roles: [
    { role: 'root', db: 'admin' },
    { role: 'root', db: '$MONGO_INITDB_DATABASE' }
  ]
})

db = db.getSiblingDB('$MONGO_INITDB_DATABASE');
db.createCollection('users');
db.users.insertMany([
  { username: 'user1', email: 'user1@example.com' },
  { username: 'user2', email: 'user2@example.com' }
]);
EOF

mysql-init.sql :

-- CREATE TABLE IF NOT EXISTS test (id SERIAL PRIMARY KEY, name VARCHAR(50));

BEGIN;

-- structure setup

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    username VARCHAR(50) NOT NULL,
    email VARCHAR(100) NOT NULL
);

-- data setup

INSERT INTO users (username, email) 
VALUES ('user1', 'user1@example.com');

INSERT INTO users (username, email) 
VALUES ('user2', 'user2@example.com');

COMMIT;

postgres-init.sql :

-- CREATE TABLE IF NOT EXISTS test (id SERIAL PRIMARY KEY, name VARCHAR(50));

BEGIN;

-- structure setup

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    username VARCHAR(50) NOT NULL,
    email VARCHAR(100) NOT NULL
);

-- data setup

INSERT INTO users (username, email) 
VALUES ('user1', 'user1@example.com');

INSERT INTO users (username, email) 
VALUES ('user2', 'user2@example.com');

COMMIT;

Étape 6 : Exécutez Docker Compose

Dans votre terminal, accédez au dossier dev-environment et exécutez :

dev-environment/
├── components   # for mounting container volumes
├── scripts/
│   ├── pgadmin
│   │   ├──servers.json   # for pgadmin automatically load postgreDB
│   ├── create-topics.sh  # for creating kafka topics
│   ├── mongo-init.sh     # init script for mongodb
│   ├── mysql-init.sql    # init script for mysql
│   ├── postgres-init.sql # init script for postgre
├── .env
├── docker-compose.yml

Cette commande démarrera tous les services, chacun avec sa propre configuration de conteneur, de port et d'environnement telle que définie.

Étape 7 : accéder aux bases de données à l'aide des outils d'interface utilisateur

phpMyAdmin : Accès via http://localhost:280
Mongo Express : Accès via http://localhost:28081
pgAdmin 4 : Accès via http://localhost:281
Redis Commander : Accès via http://localhost:28082
Kafka UI : Accès via http://localhost:28080

Chaque outil d'interface utilisateur est déjà configuré pour se connecter à son conteneur de base de données respectif.

Étape 8 : Accès via CLI

Tout d'abord, nous devons charger toutes les variables d'environnement du fichier .env vers la session CLI de travail en cours. Pour ce faire, nous pouvons utiliser la commande suivante :

# MySQL Configuration
MYSQL_PORT=23306
MYSQL_USERNAME=dev-user
MYSQL_PASSWORD=dev-password
MYSQL_DATABASE=dev_database

# PostgreSQL Configuration
POSTGRES_PORT=25432
POSTGRES_USERNAME=dev-user
POSTGRES_PASSWORD=dev-password
POSTGRES_DATABASE=dev_database

# MongoDB Configuration
MONGO_PORT=27017
MONGO_USERNAME=dev-user
MONGO_PASSWORD=dev-password
MONGO_DATABASE=dev_database

# Redis Configuration
REDIS_PORT=26379
REDIS_PASSWORD=dev-password

# Kafka Configuration
KAFKA_PORT=29092
KAFKA_USERNAME=dev-user
KAFKA_PASSWORD=dev-password

# UI Tools Configuration
PHPMYADMIN_PORT=280
PGADMIN_PORT=281
MONGOEXPRESS_PORT=28081
REDIS_COMMANDER_PORT=28082
KAFKA_UI_PORT=28080

# Data Directory for Volumes
DATA_DIR=./

grep -v '^#' .env : filtre les commentaires (lignes commençant par #) du fichier .env.
xargs : convertit chaque ligne en paires clé=valeur.
export : charge les variables dans l'environnement actuel, les rendant disponibles pour une utilisation dans la session.

Accéder à la CLI MySQL

Pour accéder à la base de données MySQL dans le conteneur dev-mysql :

version: '3.8'
services:
  dev-mysql:
    image: bitnami/mysql:latest
    # This container_name can be used for internal connections between containers (running on the same docker virtual network)
    container_name: dev-mysql
    ports:
      # This mapping means that requests sent to the ${MYSQL_PORT} on the host machine will be forwarded to port 3306 in the dev-mysql container. This setup allows users to access the MySQL database from outside the container, such as from a local machine or another service.
      - '${MYSQL_PORT}:3306'
    environment:
      # Setup environment variables for container
      - MYSQL_ROOT_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_USER=${MYSQL_USERNAME}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
      - MYSQL_DATABASE=${MYSQL_DATABASE}
    volumes:
      # Syncs msyql data from inside container to host machine, to keep them accross container restarts
      - '${DATA_DIR}/components/mysql/data:/bitnami/mysql/data'
      # Add custom script to init db
      - './scripts/mysql-init.sql:/docker-entrypoint-initdb.d/init.sql'

  phpmyadmin:
    image: phpmyadmin/phpmyadmin:latest
    container_name: dev-phpmyadmin
    # The depends_on option in Docker specifies that a container should be started only after the specified dependent container (e.g., dev-mysql) has been started (but not ensuring that it is ready)
    depends_on:
      - dev-mysql
    ports:
      - '${PHPMYADMIN_PORT}:80'
    environment:
      - PMA_HOST=dev-mysql
      # use internal port for internal connections, not exposed port ${MYSQL_PORT}
      - PMA_PORT=3306
      - PMA_USER=${MYSQL_USERNAME}
      - PMA_PASSWORD=${MYSQL_PASSWORD}

  #=======

  dev-postgresql:
    image: bitnami/postgresql:latest
    container_name: dev-postgresql
    ports:
      - '${POSTGRES_PORT}:5432'
    environment:
      - POSTGRESQL_USERNAME=${POSTGRES_USERNAME}
      - POSTGRESQL_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRESQL_DATABASE=${POSTGRES_DATABASE}
    volumes:
      # This setup will ensure that PostgreSQL data from inside container is synced to host machine, enabling persistence across container restarts.
      - '${DATA_DIR}/components/postgresql/data:/bitnami/postgresql/data'
      # Most relational databases support a special docker-entrypoint-initdb.d folder. This folder is used to initialise the database automatically when the container is first created.
      # We can put .sql or .sh scripts there, and Docker will automatically, here ./scripts/postgres-init.sql from host machine be automatically copied to the Docker container during the build and then run it
      - ./scripts/postgres-init.sql:/docker-entrypoint-initdb.d/init.sql:ro

  pgadmin:
    image: dpage/pgadmin4:latest
    container_name: dev-pgadmin
    depends_on:
      - dev-postgresql
    ports:
      - '${PGADMIN_PORT}:80'
    # user: root used to ensure that the container has full administrative privileges,
    # necessary when performing actions that require elevated permissions, such as mounting volumes (properly read or write to the mounted volumes), executing certain entrypoint commands, or accessing specific directories from host machine
    user: root
    environment:
      # PGADMIN_DEFAULT_EMAIL and PGADMIN_DEFAULT_PASSWORD - Sets the default credentials for the pgAdmin user
      - PGADMIN_DEFAULT_EMAIL=admin@dev.com
      - PGADMIN_DEFAULT_PASSWORD=${POSTGRES_PASSWORD}
      # PGADMIN_CONFIG_SERVER_MODE - determines whether pgAdmin runs in server mode (multi-user) or desktop mode (single-user). We’re setting it to false, so we won’t be prompted for login credentials
      - PGADMIN_CONFIG_SERVER_MODE=False
      # PGADMIN_CONFIG_MASTER_PASSWORD_REQUIRED - controls whether a master password is required to access saved server definitions and other sensitive information
      - PGADMIN_CONFIG_MASTER_PASSWORD_REQUIRED=False
    volumes:
      # This setup will ensure that PGAdmin data from inside container is synced to host machine, enabling persistence across container restarts.
      - '${DATA_DIR}/components/pgadmin:/var/lib/pgadmin'
      # This setup to make PGAdmin automatically detect and connect to PostgreSQL when it starts (following the config being set in servers.json)
      - ./scripts/pgadmin/servers.json:/pgadmin4/servers.json:ro

  #=======

  dev-mongodb:
    image: bitnami/mongodb:latest
    container_name: dev-mongodb
    ports:
      - '${MONGO_PORT}:27017'
    environment:
      - MONGO_INITDB_ROOT_USERNAME=${MONGO_USERNAME}
      - MONGO_INITDB_ROOT_PASSWORD=${MONGO_PASSWORD}
      - MONGO_INITDB_DATABASE=${MONGO_DATABASE}
      - MONGODB_ROOT_USER=${MONGO_USERNAME}
      - MONGODB_ROOT_PASSWORD=${MONGO_PASSWORD}
      - MONGODB_DATABASE=${MONGO_DATABASE}
    volumes:
      - '${DATA_DIR}/components/mongodb/data:/bitnami/mongodb'
      # This line maps ./scripts/mongo-init.sh from host machine to /docker-entrypoint-initdb.d/mongo-init.sh inside container with 'ro' mode (read only mode) which means container can't modify the mounted file
      - ./scripts/mongo-init.sh:/docker-entrypoint-initdb.d/mongo-init.sh:ro
      # - ./scripts/mongo-init.sh:/bitnami/scripts/mongo-init.sh:ro

  mongo-express:
    image: mongo-express:latest
    container_name: dev-mongoexpress
    depends_on:
      - dev-mongodb
    ports:
      - '${MONGOEXPRESS_PORT}:8081'
    environment:
      - ME_CONFIG_MONGODB_ENABLE_ADMIN=true
      - ME_CONFIG_MONGODB_ADMINUSERNAME=${MONGO_USERNAME}
      - ME_CONFIG_MONGODB_ADMINPASSWORD=${MONGO_PASSWORD}
      # - ME_CONFIG_MONGODB_SERVER=dev-mongodb
      # - ME_CONFIG_MONGODB_PORT=${MONGO_PORT}
      - ME_CONFIG_MONGODB_URL=mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@dev-mongodb:${MONGO_PORT}/${MONGO_DATABASE}?authSource=admin&ssl=false&directConnection=true
    restart: unless-stopped
    # 'restart: unless-stopped' restarts a container automatically unless it is explicitly stopped by the user.
    # some others: 1. 'no': (Default option if not specified) meaning the container won't automatically restart if it stops or crashes.
    #              2. 'always': The container will restart regardless of the reason it stopped, including if Docker is restarted.
    #              3. 'on-failure': The container will restart only if it exits with a non-zero status indicating an error. (and won't restart if it stops when completing as short running task and return 0 status).

  #=======

  dev-redis:
    image: bitnami/redis:latest
    container_name: dev-redis
    ports:
      - '${REDIS_PORT}:6379'
    environment:
      - REDIS_PASSWORD=${REDIS_PASSWORD}
    volumes:
      - '${DATA_DIR}/components/redis:/bitnami/redis'
    networks:
      - dev-network

  redis-commander:
    image: rediscommander/redis-commander:latest
    container_name: dev-redis-commander
    depends_on:
      - dev-redis
    ports:
      - '${REDIS_COMMANDER_PORT}:8081'
    environment:
      - REDIS_HOST=dev-redis
      # While exposed port ${REDIS_PORT} being bind to host network, redis-commander still using internal port 6379 (being use internally inside docker virtual network) to connect to redis
      - REDIS_PORT=6379
      - REDIS_PASSWORD=${REDIS_PASSWORD}
    networks:
      - dev-network
    # This networks setup is optional, in case not being set, both redis-commader and redis will both be assigned to default docker network (usually named bridge) and still being able to connect each other

  #=======

  dev-kafka:
    image: 'bitnami/kafka:latest'
    container_name: dev-kafka
    ports:
      - '${KAFKA_PORT}:9094'
    environment:
      # Sets the timezone for the container to "Asia/Shanghai". This ensures that logs and timestamps inside the Kafka container align with the Shanghai timezone.
      - TZ=Asia/Shanghai
      # KAFKA_CFG_NODE_ID=0: Identifies the Kafka node with ID 0. This is crucial for multi-node Kafka clusters to distinguish each node uniquely.
      - KAFKA_CFG_NODE_ID=0
      # KAFKA_CFG_PROCESS_ROLES=controller,broker: Specifies the roles the Kafka node will perform, in this case, both as a controller (managing cluster metadata) and a broker (handling messages).
      - KAFKA_CFG_PROCESS_ROLES=controller,broker
      # KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@541876ab254ff3f10f26905ad234f238:9093: Defines the quorum voters for the Kafka controllers. It indicates that node 0 (the current node) acts as a voter for controller decisions and will be accessible at 9093 on 541876ab254ff3f10f26905ad234f238.
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@541876ab254ff3f10f26905ad234f238:9093
      # The following lists different listeners for Kafka. Each listener binds a protocol to a specific port:
      # PLAINTEXT for client connections (:9092). CONTROLLER for internal controller communication (:9093). EXTERNAL for external client access (:9094).SASL_PLAINTEXT for SASL-authenticated clients (:9095).
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092,CONTROLLER://:9093,EXTERNAL://:9094,SASL_PLAINTEXT://:9095
      # KAFKA_CFG_ADVERTISED_LISTENERS specifies how clients should connect to Kafka externally:
      # PLAINTEXT at dev-kafka:9092 for internal communication. EXTERNAL at 127.0.0.1:${KAFKA_PORT} (host access). SASL_PLAINTEXT for SASL connections (kafka:9095).
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://dev-kafka:9092,EXTERNAL://127.0.0.1:${KAFKA_PORT},SASL_PLAINTEXT://kafka:9095
      # The following maps security protocols to each listener. For example, CONTROLLER uses PLAINTEXT, and EXTERNAL uses SASL_PLAINTEXT.
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,EXTERNAL:SASL_PLAINTEXT,PLAINTEXT:PLAINTEXT,SASL_PLAINTEXT:SASL_PLAINTEXT
      # Indicates that the CONTROLLER role should use the CONTROLLER listener for communications.
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      # Specifies users with relevant passwords that can connect to Kafka using SASL authentication
      - KAFKA_CLIENT_USERS=${KAFKA_USERNAME}
      - KAFKA_CLIENT_PASSWORDS=${KAFKA_PASSWORD}
    volumes:
      - '${DATA_DIR}/components/kafka/data:/bitnami/kafka/data'
      # Maps a local file create-topics.sh from the ./scripts directory to the path /opt/bitnami/kafka/create_topic.sh inside the Kafka container
      # This script can be used to automatically create Kafka topics when the container starts
      - ./scripts/create-topics.sh:/opt/bitnami/kafka/create_topic.sh:ro
    # Following command starts the Kafka server in the background using /opt/bitnami/scripts/kafka/run.sh. then sleep 5 to ensure that the Kafka server is fully up and running.
    # Executes the create_topic.sh script, which is used to create Kafka topics. Uses 'wait' to keep the script running until all background processes (like the Kafka server) finish,
    command: >
      bash -c "
      /opt/bitnami/scripts/kafka/run.sh & sleep 5; /opt/bitnami/kafka/create_topic.sh; wait
      "

  kafka-ui:
    image: provectuslabs/kafka-ui:latest
    container_name: dev-kafka-ui
    ports:
      - '${KAFKA_UI_PORT}:8080'
    environment:
      # Sets the name of the Kafka cluster displayed in the UI as "local."
      - KAFKA_CLUSTERS_0_NAME=local
      # Specifies the address (dev-kafka:9092) for the Kafka broker that the UI should connect to.
      - KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS=dev-kafka:9092
      # Uses the provided ${KAFKA_USERNAME} for SASL (Simple Authentication and Security Layer) authentication with the Kafka cluster.
      - KAFKA_CLUSTERS_0_SASL_USER=${KAFKA_USERNAME}
      # Uses the ${KAFKA_PASSWORD} for authentication with the Kafka broker.
      - KAFKA_CLUSTERS_0_SASL_PASSWORD=${KAFKA_PASSWORD}
      # Sets the SASL mechanism as 'PLAIN', which is a simple username-password-based authentication method.
      - KAFKA_CLUSTERS_0_SASL_MECHANISM=PLAIN
      # Configures the communication protocol as SASL_PLAINTEXT, which means it uses SASL for authentication without encryption over plaintext communication.
      - KAFKA_CLUSTERS_0_SECURITY_PROTOCOL=SASL_PLAINTEXT
    depends_on:
      - dev-kafka

networks:
  dev-network:
    driver: bridge

Accéder à la CLI PostgreSQL

Pour accéder à la base de données PostgreSQL dans le conteneur dev-postgresql :

    {
    "Servers": {
      "1": {
        "Name": "Local PostgreSQL",
        "Group": "Servers",
        "Host": "dev-postgresql",
        "Port": 5432,
        "MaintenanceDB": "dev_database",
        "Username": "dev-user",
        "Password": "dev-password",
        "SSLMode": "prefer",
        "Favorite": true
      }
    }
  }

Accéder à la CLI MongoDB

Pour accéder au shell MongoDB dans le conteneur dev-mongodb :

# Wait for Kafka to be ready
until /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092; do
  echo "Waiting for Kafka to be ready..."
  sleep 2
done

# Create topics
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic latestMsgToRedis
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic msgToPush
/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 8 --topic offlineMsgToMongoMysql

echo "Topics created."

Accéder à Redis CLI

Pour accéder à la CLI Redis dans le conteneur dev-redis :

# mongosh --: Launches the MongoDB shell, connecting to the default MongoDB instance.
# "$MONGO_INITDB_DATABASE": Specifies the database to connect to (using the value from the environment variable).
# <<EOF: Indicates the start of a multi-line input block. Everything between <<EOF and EOF is treated as MongoDB shell commands to be executed. 
# db.getSiblingDB('admin'): Switches to the admin database, which is the default administrative database in MongoDB. It allows you to perform administrative tasks like user creation, where the user dev-user will be created.
# db.auth('$MONGO_INITDB_ROOT_USERNAME', '$MONGO_INITDB_ROOT_PASSWORD') (commented out): This line, if executed, would authenticate the user with the given credentials against the "admin" database. It’s necessary if the following operations require authentication.
# The user dev-user is created in the admin database with the specified username and password.
# { role: 'root', db: 'admin' }: Allows full access to the admin database.
# { role: 'readWrite', db: '$MONGO_INITDB_DATABASE' }: Grants read and write permissions specifically for dev_database.

mongosh -- "$MONGO_INITDB_DATABASE" <<EOF
db = db.getSiblingDB('admin')
db.auth('$MONGO_INITDB_ROOT_USERNAME', '$MONGO_INITDB_ROOT_PASSWORD')
db.createUser({
  user: "$MONGODB_ROOT_USER",
  pwd: "$MONGODB_ROOT_PASSWORD",
  roles: [
    { role: 'root', db: 'admin' },
    { role: 'root', db: '$MONGO_INITDB_DATABASE' }
  ]
})

db = db.getSiblingDB('$MONGO_INITDB_DATABASE');
db.createCollection('users');
db.users.insertMany([
  { username: 'user1', email: 'user1@example.com' },
  { username: 'user2', email: 'user2@example.com' }
]);
EOF

Accéder à la CLI Kafka

Pour accéder à la CLI Kafka dans le conteneur dev-kafka :

-- CREATE TABLE IF NOT EXISTS test (id SERIAL PRIMARY KEY, name VARCHAR(50));

BEGIN;

-- structure setup

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    username VARCHAR(50) NOT NULL,
    email VARCHAR(100) NOT NULL
);

-- data setup

INSERT INTO users (username, email) 
VALUES ('user1', 'user1@example.com');

INSERT INTO users (username, email) 
VALUES ('user2', 'user2@example.com');

COMMIT;

Résumé

Cette configuration utilise Docker Compose avec des variables d'environnement, des images Bitnami et des mappages de volumes pour créer un environnement de développement reproductible. En utilisant docker-compose up -d, vous pouvez rapidement démarrer ou détruire l'ensemble de l'environnement avec docker-compose down, le rendant ainsi adapté au développement et aux tests locaux.

Ce qui précède est le contenu détaillé de. pour plus d'informations, suivez d'autres articles connexes sur le site Web de PHP en chinois!

sql mysql json express kafka for include Session using this docker database redis mongodb postgresql apache http ui phpMyAdmin Access

Déclaration：

Le contenu de cet article est volontairement contribué par les internautes et les droits d'auteur appartiennent à l'auteur original. Ce site n'assume aucune responsabilité légale correspondante. Si vous trouvez un contenu suspecté de plagiat ou de contrefaçon, veuillez contacter admin@php.cn

Article précédent：Comment corriger l'erreur « Le répertoire mysqld_safe \'/var/run/mysqld\' pour le fichier de socket UNIX n'existe pas\ » ?Article suivant：Comment corriger l'erreur « Le répertoire mysqld_safe \'/var/run/mysqld\' pour le fichier de socket UNIX n'existe pas\ » ?

Articles Liés

Voir plus