在 Golang 中建立 Google Drive 下載器（第 1 部分）

Introduction

In this tutorial, we’ll build a powerful downloader that allows downloading files from Google Drive and other cloud providers. With Golang’s efficient concurrency patterns, you'll be able to manage multiple downloads concurrently, stream large files, and track progress in real-time. Whether you’re downloading a few small files or handling large data sets, this project will showcase how to build a scalable and robust downloader that can easily be extended to support multiple cloud platforms.

If you're looking for a way to simplify and automate downloading large files, this tutorial is perfect for you. By the end, you’ll have a flexible and customizable Go-based downloader to suit your needs.

In a hurry?

If you're just looking to use this downloader with a UI, visit evolveasdev.com for read the full article & Go Downloader's Github. You'll find the docs to get it running fast.

What You’ll Learn

• Go Concurrency Patterns:
  Learn how to use Goroutines, channels and mutexes to handle multiple concurrent file downloads efficiently.

• Streaming Large Downloads:
  Explore how to stream large files while managing memory and system resources effectively.

• Concurrent File Downloads:
  Understand how to download files concurrently, speeding up the process and improving performance.

• Real-Time Progress Updates:
  Implement progress tracking to provide real-time feedback on download status.

• Handling Interruptions and Cancellations:
  Learn how to gracefully cancel one or all ongoing downloads.

Note: This tutorial will only focus on the core downloading logic.

Environment Setup

First before doing anything make sure to properly setup your environment to avoid potential bugs in future.

Prerequisites

• Install Go
• AIR for auto reloading
• Makefile to run complex commands
• Goose for PostgreSQL migrations

Configuring Makefile

Create a makefile at the root of the project with the following.

# Load environment variables from .env file
include ./.env

# To run the application
run: build
    @./bin/go-downloader

# Build the application
build:
    @go build -tags '!dev' -o bin/go-downloader

# Database migration status
db-status:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) status

# Run database migrations
up:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) up

# Roll back the last database migration
down:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) down

# Reset database migrations
reset:
    @GOOSE_DRIVER=postgres GOOSE_DBSTRING=$(DB_URL) goose -dir=$(migrationPath) reset

High-Level Folder Structure Overview

go-downloader/
├── api
├── config
├── migrations
├── service
├── setting
├── store
├── types
├── util
├── .env
├── .air.toml
├── Makefile
├── go.mod
├── go.sum
└── main.go

Setting environment variables

Create a .env file in root or handle environment variables however you like, we'll use joho/godotenv package.

GOOGLE_CLIENT_ID
GOOGLE_CLIENT_SECRET
SESSION_SECRET=something-super-secret
APP_URL=http://localhost:3000
POSTGRES_USER
POSTGRES_PASSWORD
POSTGRES_DB

Creating a Web Server

We'll now start creating the web server that'll handle all the incoming requests.

Heads Up! The main part of this guide begins here. Get ready to dive in!

API Layer

To start, create the following files inside api folder api.go and route.go

route.go file setup

All API routes will be defined in this here. We create a NewRouter struct that takes the env configuration, allowing all routes and handlers to access environment variables.

package api

import (
    "github.com/gofiber/fiber/v2"
    "github.com/nilotpaul/go-downloader/config"
)

type Router struct {
    env       config.EnvConfig
}

func NewRouter(env config.EnvConfig) *Router {
    return &Router{
        env:      env,
    }
}

func (h *Router) RegisterRoutes(r fiber.Router) {
    r.Get("/healthcheck", func(c *fiber.Ctx) error {
        return c.JSON("OK")
    })
}

api.go file setup

Here, we’ll add all the necessary middlewares, such as CORS and logging, before starting the server.

type APIServer struct {
    listenAddr string
    env        config.EnvConfig
}

func NewAPIServer(listenAddr string, env config.EnvConfig) *APIServer {
    return &APIServer{
        listenAddr: listenAddr,
        env:        env,
    }
}

func (s *APIServer) Start() error {
    app := fiber.New(fiber.Config{
        AppName:      "Go Downloader",
    })

    handler := NewRouter()
    handler.RegisterRoutes(app)

    log.Printf("Server started on http://localhost:%s", s.listenAddr)

    return app.Listen(":" + s.listenAddr)
}

Main Entrypoint

This is the main package in main.go file which will act as a entrypoint to the whole.

func main() {
    // Loads all Env vars from .env file.
    env := config.MustLoadEnv()

    log.Fatal(s.Start())
}

This is enough to start up the server and test it.

Start the server

air

that's it.?

Testing

curl http://localhost:3000/healthcheck

The response should be OK with status 200

Creating a Provider Store

We need to implement a scalable solution for adding support for multiple cloud providers if necessary.

Working on provider registry

// Better to keep it in a seperate folder.
// Specific only to OAuth Providers.
type OAuthProvider interface {
    Authenticate(string) error
    GetAccessToken() string
    GetRefreshToken() string
    RefreshToken(*fiber.Ctx, string, bool) (*oauth2.Token, error)
    IsTokenValid() bool
    GetAuthURL(state string) string
    CreateOrUpdateAccount() (string, error)
    CreateSession(c *fiber.Ctx, userID string) error
    UpdateTokens(*GoogleAccount) error
}

type ProviderRegistry struct {
    Providers map[string]OAuthProvider
}

func NewProviderRegistry() *ProviderRegistry {
    return &ProviderRegistry{
        Providers: make(map[string]OAuthProvider),
    }
}

func (r *ProviderRegistry) Register(providerName string, p OAuthProvider) {
    r.Providers[providerName] = p
}

func (r *ProviderRegistry) GetProvider(providerName string) (OAuthProvider, error) {
    p, exists := r.Providers[providerName]
    if !exists {
        return nil, fmt.Errorf("Provider not found")
    }

    return p, nil
}

The ProviderRegistry serves as a central map to hold all our OAuth providers. When we initialize our providers, we'll register them in this map. This allows us to easily access any registered provider's functionalities throughout our service.

You'll see this action later.

Initializing the Provider Store

We'll register our providers based on the environment variables provided.

func InitStore(env config.EnvConfig) *ProviderRegistry {
    r := NewProviderRegistry()

    if len(env.GoogleClientSecret) != 0 || len(env.GoogleClientID) != 0 {
        googleProvider := NewGoogleProvider(googleProviderConfig{
            googleClientID:     env.GoogleClientID,
            googleClientSecret: env.GoogleClientSecret,
            googleRedirectURL:  env.AppURL + "/callback/google",
        }, env)

        r.Register("google", googleProvider)
    }

    return r
}

Read the full article here.

總結

我們已經為 Go 中的 Google Drive Downloader 奠定了基礎，涵蓋了設定專案結構、處理 Google OAuth 等關鍵元件，並為未來的擴充奠定了基礎。一路上，我們觸及了一些重要的主題：

• 使用 Goose 進行資料庫遷移，以確保平穩且可擴展的資料管理。
• 建置供應商註冊表以便將來輕鬆擴展對其他雲端儲存供應商的支援。
• 設計靈活的架構，以便輕鬆處理未來更複雜的邏輯。

這對一篇文章來說已經足夠了，因為事情變得相當長了！我們將在第 2 部分中回來完成我們的工作，我們將在其中處理主要的下載功能。

在那之前，請隨意探索我的 GitHub 中的當前實現，並繼續關注後續步驟。祝您下載愉快！

以上是在 Golang 中建立 Google Drive 下載器（第 1 部分）的詳細內容。更多資訊請關注PHP中文網其他相關文章！