Overview and examples of capturing audio and video information using HTML5_html5 tutorial tips

Overview of this article
Capturing audio and video information has long been a difficulty in Web development. For many years, we have relied on browser plug-ins to fulfill this need.
In HTML 5, there are many APIs that can access hardware devices, such as Geolocation API to access GPS devices, Orientation API to access accelerometer devices, WebGL API to access GPU devices, Web Audio API to access audio playback devices, etc. . These APIs are very powerful because developers can directly access the underlying hardware devices by writing JavaScript script code.
This article introduces a new API that allows web applications to have the ability to access the user's camera and microphone devices by using the navigatior.getUserMedia() method.

History of technology development for capturing media data
In the past few years, the need to access client local devices in web applications began to emerge, so the W3C organization decided to organize A DAP (Device APIS POLICY) working group to develop a unified standard for the realization of this requirement.
Let’s take a look at what happened in 2011:

Capturing media data in HTML page files
The first standard to be developed by the DAP working group is how Capture media data in HTML pages of web applications. They decided to overload the input element of type file () and add a new attribute value to the accept attribute.
If the developer wants to implement the function of users taking pictures through the camera, they can write the code as shown below.

Copy code

The code is as follows:

The code for recording video data and audio data is similar:

Copy code

The code is as follows:

In these codes, just use the file control (input element of type file) to complete the photo taking or the ability to record media data. However, because these codes still lack the ability to implement some related requirements (such as rendering captured video data in canvas elements, or applying WEBGL filters to captured video data), they have not been widely used by developers. application.
Supported browsers:
Android 3.0 browser
Chrome for Android (0.16)
Firefox Mobile 10.0
device element
If you use the file control, capture the media data and process it Processing capabilities are very limited, so a new standard emerged that could support any device. This standard uses the device element.
Opera browser is the first browser to capture video data through the device element. Almost on the same day, the WhatWG organization decided to use the navigator.getUserMedia() method to capture media data. A week later, Opera launched a new browser that supports the navigator.getUserMedia() method. Later, Microsoft Tools launched the IE 9 browser that supports this method. The usage of
device element is as follows.

Copy code

The code is as follows:

<script> <br />function update(stream) { <br />document. querySelector('video').src = stream.url; <br />} <br /></script>

Supported browsers:
Unfortunately, no official version of the browser supports the device element so far.
WEBRTC
Recently, due to the emergence of WebRTC (Web Real Time Communication: Web real-time communication) API, media data capture technology has made great progress. Google, Opera, Mozilla and other companies are working hard to implement it in their browsers.
WebRTC API is an API closely related to the getUserMedia method, which provides the ability to access the client's local camera or microphone device.
Supported browsers:
So far, in the Chrome 18 version of the browser, WebRTC can be used after setting it in the chrome://flags page. In the Chrome 21 version of the browser, this API is used by default. , no more settings required. The WebRTC API is supported by default in browsers above Opera 12 and Firefox 17.
Using the getUserMedia method
By using the getUserMedia method, we can directly access the client's local camera device and microphone device without relying on plug-ins.
Detecting browser support
You can use the method shown below to detect whether the browser supports the getUserMedia method.

Copy code

The code is as follows:

function hasGetUserMedia() {
/ /Please note: The prefix
return is not used in Opera browser!!(navigator.getUserMedia || navigator.webkitGetUserMedia ||
navigator.mozGetUserMedia || navigator.msGetUserMedia);
}
if (hasGetUserMedia()) {
alert('Your browser supports the getUserMedia method');
}
else {
alert('Your browser does not support the getUserMedia method');
}

Get permission to access the device
In order to access the client camera device and microphone device, we first need to obtain permission. The first parameter of the getUserMedia method is an object specifying the media type. For example, when you want to access the camera device, the first parameter should be {video:true}. In order to access the camera device and microphone device at the same time, you need to use the {video:true,audio:true} parameters. The code is as follows:

Copy code

The code is as follows:

In this code, the use of video elements is combined. Please note that we do not use the src attribute value of the video element, but specify a URL address that refers to the media file for the video element, and convert the LocalMediaStream object representing the video data obtained from the camera into a Blob URL.
In this code, the autoplay attribute is also used for the video element. If this attribute is not used, the video element will stay at the first frame obtained.
Please note: In the Chrome browser, if you only use {audio:true}, a BUG will occur. In the Opera browser, the audio element cannot be used either.
If you want multiple browsers to support the getUserMedia method at the same time, please use the code as shown below:

Copy the code

The code is as follows:

window.URL = window.URL || window.webkitURL;
navigator.getUserMedia = navigator.getUserMedia || navigator.webkitGetUserMedia ||
navigator. mozGetUserMedia || navigator.msGetUserMedia;
var video = document.getElementById('video');
if (navigator.getUserMedia) {
navigator.getUserMedia({audio: true, video: true}, function (stream) {
video.src = window.URL.createObjectURL(stream);
}, onFailSoHard);
}
else {
alert('Your browser does not support getUserMedia Method');
}

Security
In some browsers, when the getUserMedia method is called, a prompt window is displayed asking the user whether to allow or deny access to their camera or microphone.
Take photos
In the Canvas API, you can use the ctx.drawImage(video,0,0) method to output a certain frame in the video element to the canvas element. Of course, since we have output the captured image information from the user's camera to the video element, of course, we can also output the image information to the canvas element through the video element, that is, to realize the real-time photo taking function. The code is as follows.

Copy code

The code is as follows:

> ;


var video = document.getElementById('video');
var canvas = document.getElementById('canvas');
var ctx = canvas.getContext('2d');
var localMediaStream = null;
function snapshot() {
if (localMediaStream) {
ctx.drawImage(video, 0, 0);
document.getElementById('img').src = canvas.toDataURL( 'image/png');
}
}
video.addEventListener('click', snapshot, false);
//Do not use vendor prefix
navigator.getUserMedia({video : true}, function(stream) {
video.src = window.URL.createObjectURL(stream);
localMediaStream = stream;
}, onFailSoHard);

Apply CSS Filters
So far, CSS filters can be used in Chrome 18 and above browsers.
Through the use of CSS filters, we can add various image filter effects to the video captured in the video element.

Copy code

The code is as follows:

<script> <br />var idx = 0; <br />var filters = ['grayscale', 'sepia', 'blur', 'brightness', 'contrast', 'hue-rotate', <br />'hue-rotate2', ' hue-rotate3', 'saturate', 'invert', '']; <br />function changeFilter(e) { <br />var el = e.target; <br />el.className = ''; <br />var effect = filters[idx % filters.length]; // loop through filters. <br />if (effect) { <br />el.classList.add(effect); <br />} <br />} <br />document. getElementById('video').addEventListener('click', changeFilter, false); <br /></script>