How can I write to a file in memory and then send it over HTTP without using an intermediate file?

Question content

I need to request, process and format data into csv and then send them via a web service upon request.

Assuming that the requested and processed data is in the data below, I managed to do this using an intermediate temporary file:

package main

import (
    "encoding/csv"
    "io/ioutil"
    "net/http"
    "os"
    "strconv"

    "github.com/go-chi/chi/v5"
)

type data struct {
    name string
    age  int
}

func main() {
    data := []data{
        {"john", 30},
        {"jane", 20},
    }
    tempfilename := "temp.csv"
    // create temporary intermediate file
    file, err := os.create(tempfilename)
    defer file.close()
    if err != nil {
        panic(err)
    }
    w := csv.newwriter(file)
    var csvdata [][]string
    for _, record := range data {
        row := []string{record.name, strconv.itoa(record.age)}
        csvdata = append(csvdata, row)
    }
    w.writeall(csvdata)

    // read temporary intermediate file to send it via http
    filebytes, err := ioutil.readfile(tempfilename)
    if err != nil {
        panic(err)
    }
    // send the file on request
    router := chi.newrouter()
    router.get("/data", func(w http.responsewriter, r *http.request) {
        w.header().set("content-type", "application/octet-stream")
        w.writeheader(http.statusok)
        w.write(filebytes)
    })
    http.listenandserve(":8087", router)
}

Request successful:

PS C:\temp> wget http://localhost:8087/data
--2023-06-13 15:34:00--  http://localhost:8087/data
Resolving localhost (localhost)... ::1, 127.0.0.1
Connecting to localhost (localhost)|::1|:8087... connected.
HTTP request sent, awaiting response... 200 OK
Length: 16 [application/octet-stream]
Saving to: 'data'

data                          100%[=================================================>]      16  --.-KB/s    in 0s

2023-06-13 15:34:00 (523 KB/s) - 'data' saved [16/16]

PS C:\temp> cat data
John,30
Jane,20

So far so good - now I want to delete the intermediate files via

• Use memory "containers" to write csv data
• Write its contents as bytes to the http writer

I still don't quite understand how to use io.* and bufio.* to log in the required type:

• csv.newwriter() requires *os.file and returns *csv.writer
• This *csv.writer content needs to be extracted somehow as byte ...
• ...In order to provide http.responsewriter.write()

What are the general mechanisms (and ones specific to this case) that solve different types of memory file manipulation problems?

Correct answer

##csv.newwriter takes io.writer, so you can http.responsewriter An instance of is passed to it without writing the contents to a file or memory.

This is the demo:

package main

import (
    "encoding/csv"
    "fmt"
    "net/http"
    "strconv"

    "github.com/go-chi/chi/v5"
)

type Data struct {
    Name string
    Age  int
}

func main() {
    router := chi.NewRouter()
    router.Get("/data", func(w http.ResponseWriter, r *http.Request) {
        w.Header().Set("Content-Type", "application/octet-stream")
        w.WriteHeader(http.StatusOK)
        data := []Data{
            {"John", 30},
            {"Jane", 20},
        }
        csvWriter := csv.NewWriter(w)
        var csvData [][]string
        for _, record := range data {
            row := []string{record.Name, strconv.Itoa(record.Age)}
            csvData = append(csvData, row)
        }
        if err := csvWriter.WriteAll(csvData); err != nil {
            // handle the error
            fmt.Println(err)
        }
    })
    http.ListenAndServe(":8087", router)
}

The above is the detailed content of How can I write to a file in memory and then send it over HTTP without using an intermediate file?. For more information, please follow other related articles on the PHP Chinese website!