Home >Backend Development >Golang >Files are not submitted to Google Cloud Storage when using http.Request.Context() instead of context.Background()

Files are not submitted to Google Cloud Storage when using http.Request.Context() instead of context.Background()

PHPzforward: 2024-02-09 16:24:17572browse

使用 http.Request.Context() 而不是 context.Background() 时，文件不会提交到谷歌云存储

php小编西瓜在介绍Go语言开发中的一个重要细节时指出，使用http.Request.Context()代替context.Background()时，上传文件不会被自动提交到谷歌云存储。这个小细节对于开发者来说非常重要，因为它可能会导致文件上传失败或数据丢失。了解并正确使用这个方法可以避免不必要的问题，确保文件的安全上传和存储。

问题内容

我是新手，我编写了一个简单的宠物项目来将加密文件上传到谷歌云存储。 问题： 在我调用 addentry 函数的http处理程序中，当我传递 r.context() 而不是 context.background() 时，文件不会上传到谷歌存储。在谷歌的控制台中，我可以看到收到的字节，而且我根本没有收到任何错误，但文件本身不存在。

处理程序代码：

//imports, structs skipped

func processaddentryrequest(w http.responsewriter, r *http.request) {
    //validation, getting params from jwt skipped

    reader, err := r.multipartreader()
    if err != nil {
        http.error(w, "multipart/form-data expected, not found", http.statusbadrequest)
        log.println("could not init multipartreader from request: %w", err)
        return
    }

    gsservice, err := googlestorageservice.new()
    if err != nil {
        http.error(w, "internal server error", http.statusinternalservererror)
        log.println(err)
        return
    }

    bservice, err := backupservice.new(
        backupservice.withcipher(aes.new()),
        backupservice.withstorageservice(gsservice))

    if err != nil {
        http.error(w, "internal server error", http.statusinternalservererror)
        log.println(err)
        return
    }
    var entry *entryrepo.backupentry
    for {
        part, err := reader.nextpart()
        if err == io.eof {
            break
        }
        if err != nil {
            err = fmt.errorf("error reading: %w", err)
            log.println(err)
            http.error(w, "internal server error", http.statusinternalservererror)
            return
        }

        formname := part.formname()
        if formname == "file" {
            entry, err = bservice.addentry(r.context(), backupid, part.filename(), part.header.get("content-type"), part)
            if err != nil {
                errclose := part.close()
                err = fmt.errorf("failed to add entry for backup %s: %w; error while closing multipart.part: %w", backupid, err, errclose)
                log.println(err)
                http.error(w, "internal server error", http.statusinternalservererror)
                return

            }
        }
        err = part.close()
        if err != nil {
            err = fmt.errorf("failed to close part for backup %s: %w", backupid, err)
            log.println(err)
            http.error(w, "internal server error", http.statusinternalservererror)
            return
        }

    }
    response.entry = *entry
    jsondata, err := json.marshal(response)
    if err != nil {
        http.error(w, "internal server error", http.statusinternalservererror)
        log.println("failed to marshal json: %w\n", err)
        return
    }
    w.writeheader(http.statusok)
    w.header().set("content-type", "application/json")
    w.write(jsondata)
}

addentry函数代码：

func (backupservice *backupservice) addentry(ctx context.context, backupid string, entryfullname string,
    entrymimetype string, in io.reader) (*ber.backupentry, error) {
    // validation skipped for readability
    entryid := uuid.new()
    outpath := backupid + "/" + entryid.string()

    entrychan := make(chan *ber.backupentry)
    errchan := make(chan error)
    go func() {
        gstorageclient, err := backupservice.storageservice.getclient(ctx)
        if err != nil {
            errchan <- fmt.errorf("could not add entry: %w", err)
        }
        defer gstorageclient.close()

        out := gstorageclient.bucket(config.gcbucketname()).object(outpath).newwriter(ctx)
        defer out.close()

        mac, size, err := backupservice.cipher.encrypt(ctx, in, out,
            []byte(config.encryptionkey()), []byte(config.hmackey()))
        if err != nil {
            errchan <- fmt.errorf("could not upload encrypted backupentry with id=%s: %w", entryid.string(), err)
        }

        //db operation skipped
        entrychan <- entry
    }()
    select {
    case <-ctx.done():
        dbcleanuperr, storagecleanuperr := backupservice.cleanupentry(entryid.string(), outpath)
        return nil,
            fmt.errorf("could not add entry %s to backup %s: context was cancelled; db cleanup err: %w; gstorage cleanup err: %w",
                entryid.string(), backupid, dbcleanuperr, storagecleanuperr)
    case err := <-errchan:
        dbcleanuperr, storagecleanuperr := backupservice.cleanupentry(entryid.string(), outpath)
        return nil,
            fmt.errorf("could not add entry %s to backup %s: %w; db cleanup err: %w; gstorage cleanup err: %w",
                entryid.string(), backupid, err, dbcleanuperr, storagecleanuperr)
    case entry := <-entrychan:
        return entry, nil
    }
}

我尝试过的：

启用 godebug：http2debug=1 检查调试 http 日志。日志不同，我将它们放在下面。
检查上下文是否在所有级别上都关闭：在http处理程序本身中，在addentry函数中添加encrypt函数。上下文没有关闭，一切似乎都工作正常。
从 r.context() 创建派生上下文，结果与 r.context() 相同
显然，将 context.background() 而不是 r.context() 传递给 addentry 函数，就可以解决问题

当 r.context() 作为参数传递时的调试日志：

ft-api  | 2023/08/02 19:23:46 stdout: 2023/08/02 19:23:46 [2606f10843ab/yuha6i98ak-000004] "post http://0.0.0.0:8080/backup http/1.1" from 172.18.0.1:36052 - 200 570b in 193.850041ms
ft-api  | 2023/08/02 19:23:47 stdout: 2023/08/02 19:23:47 [2606f10843ab/yuha6i98ak-000005] "post http://0.0.0.0:8080/entry http/1.1" from 172.18.0.1:36052 - 200 127b in 171.975584ms
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header ":authority" = "oauth2.googleapis.com"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header ":method" = "post"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header ":path" = "/token"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header ":scheme" = "https"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header "content-type" = "application/x-www-form-urlencoded"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header "content-length" = "861"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header "accept-encoding" = "gzip"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9019: http2: transport encoding header "user-agent" = "go-http-client/2.0"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9968: http2: transport received headers flags=end_headers stream=3 len=42
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9968: http2: transport received data flags=padded stream=3 len=974 data="\x1f\x8b\b\x00\x00\x00\x00\x00\x02\xff\x1d\xd4Ɏ\xa3h\x00\x04\xd0\x7f\xf1\xb9ia&\xeb܌]\xb4\xc9*l\x01\x05\x06.\x88%\xd9\xf7d\xcd\xd6\xfc\xfbxs\f).q\x88\xf7\xf7\x12\xa7)&$z\x86\x06\xf7\x97\x7f.g\f\x94\xdf\xe9\uf13dƸ\xe98=\x9a\r\xe33\xe6fѶ\xa3\xcf\xd7Ϫn-\x04\x9b]\x7f\xaf\xfc|\x95\xec!ha\x97^[\xa4\xad\x0f\xe8\xeb\x03:\x051u\xb1\x12~2\x1e\xf5\xbe\xbftν9\x8f\xe7\xedз\xaa\xac\x87\a\xa3\xfd!\x82ȡӝyb\xe3n\xb8\x91\xf8\x91*\xa1\xc5\xe7\x80\xc1|ڗ}p\x99\xd0d\xf7\xf8\xc7\xc7\"\xda\x1d\xabl\x96\xa4\x98\xf1'\xb2g\xef\xb1=\x10o\x7f\x8dc\x96\x8d\xc6\xfe\xfc\xae\xdc?\xb7f\x0f\xc0\xa2k\xb9^\xd5!\xc38\xe2\a\x84\xca1\x83\xfeg6\xbanjpzw\xcf\xfb\xd3jͯn\x8d\xac\xf2[\xbe۰\xfcx\xbfbp\xbd\xdd\x14\xa1--\xbe" (613 bytes omitted)
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9968: http2: transport received data flags=end_stream|padded stream=3 len=136 data=""
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 h2_bundle.go:9968: http2: transport received ping len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x02"
ft-api  | 2023/08/02 19:23:47 stderr: 2023/08/02 19:23:47 transport.go:3015: http2: transport failed to get client conn for storage.googleapis.com:443: http2: no cached connection was available

当 context.background() 作为参数传递时的调试日志（我从日志中删除了 jwt 令牌值，但它在那里）

ft-api  | 2023/08/02 19:24:51 stdout: 2023/08/02 19:24:51 [2606f10843ab/ukd5p0sXox-000001] "POST http://0.0.0.0:8080/backup HTTP/1.1" from 172.18.0.1:38934 - 200 570B in 465.2635ms
ft-api  | 2023/08/02 19:24:52 stdout: 2023/08/02 19:24:52 [2606f10843ab/ukd5p0sXox-000002] "POST http://0.0.0.0:8080/entry HTTP/1.1" from 172.18.0.1:38934 - 200 127B in 172.084834ms
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport failed to get client conn for oauth2.googleapis.com:443: http2: no cached connection was available
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport creating client conn 0x40003cc780 to 142.250.186.74:443
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header ":authority" = "oauth2.googleapis.com"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header ":method" = "POST"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header ":path" = "/token"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header ":scheme" = "https"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header "content-type" = "application/x-www-form-urlencoded"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header "content-length" = "861"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header "accept-encoding" = "gzip"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9019: http2: Transport encoding header "user-agent" = "Go-http-client/2.0"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received SETTINGS len=18, settings: MAX_CONCURRENT_STREAMS=100, INITIAL_WINDOW_SIZE=1048576, MAX_HEADER_LIST_SIZE=65536
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received WINDOW_UPDATE len=4 (conn) incr=983041
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received SETTINGS flags=ACK len=0
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received HEADERS flags=END_HEADERS stream=1 len=220
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received DATA flags=PADDED stream=1 len=1030 data="\x1f\x8b\b\x00\x00\x00\x00\x00\x02\xff\x1d\xd4Ir\xa3:\x00\x00лx\xddt1\x83z\x179&\x86\x84`#3n(!F3\v̠\xae\x7f\xf7\x9f\xea;\xbcz\x7fO\x98\x90|\x9e\x93eh\xf2\xfe\xf4\xe7t`\x11\xfc&\xbfS\xfe\r\xe7M'|\x9d\x83\x1b'\x06\xdf\x1f\xa2\xfa\xcd]78\xf5\xc4,\xedf\x8f\x18r\xb9+\xf8\x1aņl\xaf\x0f<\\#\x14\xa3\xeb\xce8O\xfb\xec\xa8p\xef#|}&\xeeM\x1e\xc5E\x94\x91-<Е\xbd\xea\x88X\xc8y\xbf<\x94\xfeHL5\xb8\x19&\x1e{?\xe3\x1a\xa3\xb2y\xf3\xd2T:ȿ\x1dX9K[4ԟ\x98a\x0e9\xfe \xea\xe1h\x910\xac\x9a\xf7h\xaf\xba\xa2\xd1\xeak\x1b\xc2\xcb\xea\xe4\x03\x9c\xaf\xfa\xed\xbd\x05\x89\x1f\xbf)\x1d5\xfc\xfevT\xf8\x9d\x9c\x81*4\xe9\xad|P̍(\xdb\xe8l=d\xe3\xc3c\xf5\xf2q\xdd\x05\xf4\xf6\x95\xc7\x10Ŗ\xbe\xa6\xf6\xf9Q|w" (611 bytes omitted)
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received DATA flags=END_STREAM|PADDED stream=1 len=208 data=""
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 h2_bundle.go:9968: http2: Transport received PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport failed to get client conn for storage.googleapis.com:443: http2: no cached connection was available
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport creating client conn 0x400029e000 to 216.58.212.144:443
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header ":authority" = "storage.googleapis.com"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header ":method" = "POST"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header ":path" = "/upload/storage/v1/b/bucketname/o?alt=json&name=a934c415-07bd-4686-8ad0-90ff5ef68c7e%2F6e6b9563-c9d9-484e-a24f-cd2f8bd1fcd8&prettyPrint=false&projection=full&uploadType=multipart"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header ":scheme" = "https"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header "content-type" = "multipart/related; boundary=976d8006696028fea8c40fc53fe0a817a80f334ae41925d1e08c3bb3c917"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header "x-cloud-trace-context" = "cb5a9d07f5007be3d8dfe277e1fb6beb/9294889295348469653;o=0"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header "authorization" = "Bearer <<token value removed>>"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header "x-goog-api-client" = "gl-go/1.20.6 gccl/1.30.1"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header "user-agent" = "gcloud-golang-storage/1.30.1"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:2076: http2: Transport encoding header "accept-encoding" = "gzip"
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received SETTINGS len=18, settings: MAX_CONCURRENT_STREAMS=100, INITIAL_WINDOW_SIZE=1048576, MAX_HEADER_LIST_SIZE=65536
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received WINDOW_UPDATE len=4 (conn) incr=983041
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received SETTINGS flags=ACK len=0
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received HEADERS flags=END_HEADERS stream=1 len=314
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received DATA stream=1 len=944 data="{\"kind\":\"storage#object\",\"id\":\"bucketname/a934c415-07bd-4686-8ad0-90ff5ef68c7e/6e6b9563-c9d9-484e-a24f-cd2f8bd1fcd8/1691004292747760\",\"selfLink\":\"https://www.googleapis.com/storage/v1/b/bucketname/o/a934c415-07bd-4686-8ad0-90ff5ef68c7e%2F6e6b9563" (688 bytes omitted)
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received DATA flags=END_STREAM stream=1 len=0 data=""
ft-api  | 2023/08/02 19:24:52 stderr: 2023/08/02 19:24:52 transport.go:3015: http2: Transport received PING len=8 ping="\x00\x00\x00\x00\x00\x00\x00\x00"

解决方法

通过在 gstorage 对象编写器上显式调用 Close() 方法而不是推迟它来解决该问题。我仍然不完全确定此行为的原因（我知道除非编写器关闭，否则文件不会提交到 gstorage，但不涉及上下文取消，因此它应该已关闭）。

编辑：在我看来，问题可能出在 go 执行 goroutine 内部声明的延迟函数的顺序上。如果有人对此有解释，我们将非常欢迎。

The above is the detailed content of Files are not submitted to Google Cloud Storage when using http.Request.Context() instead of context.Background(). For more information, please follow other related articles on the PHP Chinese website!

php Go语言对象 background http

Statement：

This article is reproduced at:stackoverflow.com. If there is any infringement, please contact admin@php.cn delete

Previous article：Native Windows function IsTpmReady throws "Too many posts to semaphore" errorNext article：Native Windows function IsTpmReady throws "Too many posts to semaphore" error

See more

Files are not submitted to Google Cloud Storage when using http.Request.Context() instead of context.Background()

问题内容

解决方法

Related articles