How to Read and Echo Currently Writing Size of Uploaded File in Server in Realtime Without Blocking Server and Client?

How to read and print the size of an uploaded file being written on the server side in real time without blocking the server and client?

Let us expand on this issue:

In order to get the progress of file upload in real time, we set it from the Blob, File, TypedArray or ArrayBuffer object through fetch() in the POST request The body object.

The current implementation sets the File object as the second argument to fetch() passed to the body object.

Requirements:

As a text/event-stream reads the size of a file being written to the server's file system and echoes it back to the client. Stops when all bytes provided by the var query string parameter in the GET request have been written. File reading currently happens in a separate scripting environment, where GET calls are made to the script that reads the file, and then POST is made to the script that writes the file to the server.

Wait until you complete the file size echoing part before trying to resolve potential issues with handling server-side file writes or file reads to get the current file size.

Currently trying to use php to meet the requirements. But also interested in c, bash, nodejs, python or other languages ​​or methods that can be used to perform the same task.

The client side javascript part is fine. I'm just not well versed in php (one of the most widely used server-side languages ​​in the world) to implement the pattern without including unnecessary parts.

Motivation:

Progress indicator for fetch?

Related:

Fetch with ReadableStream

Question:

Get

PHP Notice:  Undefined index: HTTP_LAST_EVENT_ID in stream.php on line 7

in the terminal.

Also, if you replace

while(file_exists($_GET["filename"]) 
  &amp;&amp; filesize($_GET["filename"]) < intval($_GET["filesize"]))

with

while(true)

it will generate an error at the EventSource.

Without the sleep() call, the correct file size is dispatched to the message event for a file of size 3.3MB, 3321824, 61921, 26214 and 38093 are printed respectively when the same file is uploaded three times. The expected result is to get the file size when writing a file at:

stream_copy_to_stream($input, $file);

instead of the filesize of the uploaded file object. Will fopen() or stream_copy_to_stream() prevent other php processes from accessing stream.php?

Things tried so far:

php Quote from

• Beyond $_POST, $_GET and $_FILE: In JavaScriptPHP Handling Blobs
• Introduction to Server-Sent Events with PHP Examples

php

// 能否合并 `data.php`、`stream.php` 为同一个文件？
// 能否使用 `STREAM_NOTIFY_PROGRESS` 
// "Indicates current progress of the stream transfer 
// in bytes_transferred and possibly bytes_max as well" to read bytes?
// do we need to call `stream_set_blocking` to `false`
// data.php
<?php

  $filename = $_SERVER["HTTP_X_FILENAME"];
  $input = fopen("php://input", "rb");
  $file = fopen($filename, "wb"); 
  stream_copy_to_stream($input, $file);
  fclose($input);
  fclose($file);
  echo "upload of " . $filename . " successful";

?>

// stream.php
<?php

  header("Content-Type: text/event-stream");
  header("Cache-Control: no-cache");
  header("Connection: keep-alive");
  // `PHP Notice:  Undefined index: HTTP_LAST_EVENT_ID in stream.php on line 7` ?
  $lastId = $_SERVER["HTTP_LAST_EVENT_ID"] || 0;
  if (isset($lastId) &amp;&amp; !empty($lastId) &amp;&amp; is_numeric($lastId)) {
      $lastId = intval($lastId);
      $lastId++;
  }
  // else {
  //  $lastId = 0;
  // }

  // while current file size read is less than or equal to 
  // `$_GET["filesize"]` of `$_GET["filename"]`
  // how to loop only when above is `true`
  while (true) {
    $upload = $_GET["filename"];
    // is this the correct function and variable to use
    // to get written bytes of `stream_copy_to_stream($input, $file);`?
    $data = filesize($upload);
    // $data = $_GET["filename"] . " " . $_GET["filesize"];
    if ($data) {
      sendMessage($lastId, $data);
      $lastId++;
    } 
    // else {
    //   close stream 
    // }
    // not necessary here, though without thousands of `message` events
    // will be dispatched
    // sleep(1);
    }

    function sendMessage($id, $data) {
      echo "id: $id\n";
      echo "data: $data\n\n";
      ob_flush();
      flush();
    }
?>

javascript

<!DOCTYPE html>
<html>
<head>
</head>
<body>
<input type="file">
<progress value="0" max="0" step="1"></progress>
<script>

const [url, stream, header] = ["data.php", "stream.php", "x-filename"];

const [input, progress, handleFile] = [
        document.querySelector("input[type=file]")
      , document.querySelector("progress")
      , (event) => {
          const [file] = input.files;
          const [{size:filesize, name:filename}, headers, params] = [
                  file, new Headers(), new URLSearchParams()
                ];
          // set `filename`, `filesize` as search parameters for `stream` URL
          Object.entries({filename, filesize})
          .forEach(([...props]) => params.append.apply(params, props));
          // set header for `POST`
          headers.append(header, filename);
          // reset `progress.value` set `progress.max` to `filesize`
          [progress.value, progress.max] = [0, filesize];
          const [request, source] = [
            new Request(url, {
                  method:"POST", headers:headers, body:file
                })
            // https://stackoverflow.com/a/42330433/
          , new EventSource(`${stream}?${params.toString()}`)
          ];
          source.addEventListener("message", (e) => {
            // update `progress` here,
            // call `.close()` when `e.data === filesize` 
            // `progress.value = e.data`, should be this simple
            console.log(e.data, e.lastEventId);
          }, true);

          source.addEventListener("open", (e) => {
            console.log("fetch upload progress open");
          }, true);

          source.addEventListener("error", (e) => {
            console.error("fetch upload progress error");
          }, true);
          // sanity check for tests, 
          // we don't need `source` when `e.data === filesize`;
          // we could call `.close()` within `message` event handler
          setTimeout(() => source.close(), 30000);
          // we don't need `source' to be in `Promise` chain, 
          // though we could resolve if `e.data === filesize`
          // before `response`, then wait for `.text()`; etc.
          // TODO: if and where to merge or branch `EventSource`,
          // `fetch` to single or two `Promise` chains
          const upload = fetch(request);
          upload
          .then(response => response.text())
          .then(res => console.log(res))
          .catch(err => console.error(err));
        }
];

input.addEventListener("change", handleFile, true);
</script>
</body>
</html>

The above is the detailed content of How to Read and Echo Currently Writing Size of Uploaded File in Server in Realtime Without Blocking Server and Client?. For more information, please follow other related articles on the PHP Chinese website!