Learn how to capture the first frame of a video using javascript

Related learning recommendations: javascript video tutorial

JavaScript Intercept the first frame of the video

1. Background

In the development of corporate materials, in addition to video uploading, you also need to use the first frame or perhaps several frames in the video as the video Cover display.

Problems:

Therefore, the difficulty of intercepting the first frame of the video with JS was born. However, after checking the information, I found that there are only two types of information available on the Internet. The first one iswasm ffmpeg Cooperate with the backend to intercept, the second is to intercept by JS yourself. The advantages and disadvantages are also obvious. The first one has a relatively high cost of cooperation and is not very flexible. The second one can be used under normal conditions, but there will be compatibility issues (see you later in IE) and black screen interception issues.

2. Wasm ffmpeg

The advantages and disadvantages of this method are also obvious. The cooperation cost is relatively high, and it will cause a sharp increase in web memory. However, the number of captured frames for the supported video types is Very flexible; since it involves the server, you can refer to wasm ffmpeg to capture videos.

3. JavaScript front-end interception

If you want to intercept the front-end here, you need to understand the compatibility and response events of the video and canvas tags. And it may not be so friendly to IE.

1. I won’t add too much knowledge about canvas here. Let’s mainly look at the response events of the video tag:

Execution results

According to the sequence, the first thing to be triggered turned out to be the timeupdate event. According to assumptions, the first thing to be executed should beloadedmetadata, the metadata is loaded. Regarding this, there is no clear explanation on MDN, but you can reason about it:

When currentTime is updated, the timeupdate event

# will be triggered.

##Conclusion: Although it is triggered first, the video file has not been loaded yet, and the content of the canvas itself is intercepted. Note: The timeupdate event is triggered 4-66 times per second depending on the system used. Due to the high trigger frequency, too small unit (millisecond level), event response delay and other reasons, it cannot be completely and accurately controlled.

• loadedmetadata As mentioned above, it is triggered after the metadata is loaded, but the data does not include the video file itself. Conclusion: If the video file is large and the loading time is long, the first frame that has been loaded still cannot be captured. Supplement: The URL.createObjectURL() method can basically make it invisible, but it is not safe.

• loadeddata Triggered when the current frame number (the first frame) is loaded, no problem. Conclusion: Available. Supplement: What if the first frame is a black screen and you want to use the next frame? Sorry, the number of remaining frames and the fact that they have not been loaded are not within the scope of its consideration. This event is ignored.

• canplay is triggered when the video can start playing, that is, it is triggered after the number of frames to be loaded is determined based on the number of uploaded video frames (24/25/30/60, etc.) and the playback screen is satisfied. Summary: Because there are more loading events than loadeddata (one more event), it is generally feasible. Supplement: It can be satisfied by controlling currentTime (but it cannot be as accurate as the second frame), which can be regarded as the "current playback frame".

• play It will only be triggered when playback starts, which does not meet the requirement of uploading quick screenshots.

• waiting Triggered when the next picture has been played but the next picture has not been buffered. It is suitable for inserting small advertisements.

2. After understanding the above response events, let us take a look at the next interception operation:

The above results what is it then?

You can see that the first frame of the video has been successfully intercepted. At this stage, the requirements have been basically achieved, but whether the picture is valid? It’s not known yet, so we need to make further judgments.

3. Is the first frame valid?

In fact, sometimes the first frame of the captured picture is not very consistent with expectations due to poor video quality or other factors. There will always be pure black pictures, transparent pictures, white pictures and other invalid pictures. Therefore, we need to identify the validity of the image. So, how to identify the effectiveness of images?

At this time, you need to know a new attribute: Uint8ClampedArray

Uint8ClampedArray (8-bit unsigned integer fixed array ) A typed array represents an array of 8-bit unsigned integers whose values ​​are fixed in the range 0-255; if you specify a value outside the range [0,255], it will be replaced with 0 or 255; if you specify a non-integer, then it will be set to the nearest integer. (array) contents are initialized to 0. Once created, you can reference the elements of the array using object methods, or using standard array indexing syntax (that is, using square brackets).

If you are interested in Uint8ClampedArray, you can further study Uint8ClampedArray asynchronously here.

Did you discover anything? Is 0~255 a common value? It is the hexadecimal value corresponding to the color. Okay, then, the next step is to implement what we have thought about and see if it is such a principle.

The code is implemented, so what about the results? Here I use white pictures, transparent pictures, and black pictures to compare. Is the result obtained consistent with what we imagined: First, let’s take a look at the transparent image:

As you can see, all the results in the array are 0;

White image:

Oops, all are 255, then black should be all 0, don’t worry, let’s take a look

Black picture:

There is an unexpected number, 238, which is a color value that is biased toward 255 white. Why is this? In fact, it is because The white and transparent colors are not excessive, but the black is excessive. This problem will occur when the canvas is drawn, but it can be ignored.

After knowing the color values ​​of these three, the next judgment will be easier. Just add a condition.

Why are it 200 and 0? In fact, you can judge the reasonable range of these two values ​​​​based on the actual situation. The corresponding color value of 200 is #c8c8c8, which is gray, and 0 is transparent color, so it is judged to be an invalid image here. As long as there is no value in the brr array, it means it is an invalid picture.

So what is the actual situation? Here is another actual comparison picture:

As you can see, there are values ​​in brr, and there are a lot of them, so this is the actual picture Valid image.

4. Finally

JavaScript has finished capturing the first frame of the video. If you still want to optimize for invalid images, just use the default image display.

4. Summary

JavaScript intercepts the first frame of the video. The process is complicated and involves a large amount of data loops, which will cause a certain amount of memory growth, but it can indeed solve this problem. , and has been used in enterprise materials. A clever optimization method is used. Only if a value in the brr array is pushed, it will break directly, which can greatly optimize performance. If you have better solutions, please share them with us!

If you want to learn more about programming, please pay attention to the php training column!

The above is the detailed content of Learn how to capture the first frame of a video using javascript. For more information, please follow other related articles on the PHP Chinese website!