Problems and solutions encountered with HTML5 audio and video on mobile terminals

Recently, we are studying the use of video to replace animation, and the use of video to replace sprite animation. We call this kind of video interactive video. This article mainly introduces relevant information to you about mobile HTML5 audio and video problems and solutions, as a reference for you, and I hope it can help you.

Traditional sprite animation:

1. The disk space is large and the download is slow, especially online playback, it will be even slower

2. There are too many files. When playing online, too many http requests will lead to slow response or abnormal behavior.

Therefore, it is urgent to develop a set of technologies to use Video instead of sprite animation. We call this kind of video interactive video

The problem with traditional video:

1. Traditional video can only be played in a square area

2. Traditional videos are played in windows on the iPad. On iPhone, they can only be played in full screen.

3. Traditional videos, when played, It will definitely appear on the front end

Interactive video has the following characteristics:

1. On the iPhone, full screen playback is not required , can be played in an area

2. Interactive video can appear under ordinary graphic objects

3. Interactive video can have a mask, so You can remove the background of the video and integrate the video with ordinary graphic objects

Summary:If the video is simply played, we will set it as a traditional video. For videos that need to be used for specific purposes, we set them as interactive videos.

The research has initially yielded results. By the way, I will summarize the practical problems encountered in audio and video during the development of mobile H5 in the past few years and give my own solutions.

Look at the final actual effect: Compatible with PC (>IE9), iPhone, iPad, Android 5.0

It solves the problems of manual, automatic, windowing, etc. on iPhone, and can basically be used in actual production

The right side is the original Video mp4 file

The video on the left replaces the animation, and then supports the background mask effect, which can reveal the base image and support a series of interactive operations

H5 audio audio

Every time an audio object is passed through new Audio, you can see that a new thread will be generated on IOS. This is disgusting

Solution: new Audio An object, by replacing different audio addresses, achieves the purpose of not opening more threads

The support on Android is not strong

Solution: Problems on low versions of Android No solution. Generally, in mixed development, the underlying interface processing can be adjusted. For example, phonegap

cannot play automatically on iPhone

Solution: play automatically on iPhone, yes A process that was done when IOS was designed seems to be to prevent automatic theft of traffic.

To put it simply, it needs to be simulated by the user to manually trigger it, so we need to call this piece of code at the beginning:

This is from my project, I just deducted it directly

//修复ios 浏览器不能自动播放音频的问题 在加载时创建新的audio 用的时候更换src即可
Xut.fix = Xut.fix||{};
if (Xut.plat.isBrowser && Xut.plat.isIOS) {
    var isAudio = false
    var fixaudio = function() {
        if (!isAudio) {
            isAudio = true;
            Xut.fix.audio = new Audio();
            document.removeEventListener('touchstart', fixaudio, false);
        }
    };
    document.addEventListener('touchstart', fixaudio, false);
}

If you bind such a code to the body: create an audio object through manual triggering , and then save it in the global

When using it, you can directly replace the audio object as follows

//如果为ios browser 用Xut.fix.audio 指定src 初始化见app.js
if (Xut.fix.audio) {
    audio 
=  Xut.fix.audio;
    audio.src = url;
} else {
    audio = new Audio(url);
}
audio.autoplay = true;
audio.play();

. To put it simply, it must be automatic Only the objects created by the user can be played.

H5 video audio

Video tags may be rarely used on the mobile side. Android support is terrible. It only improved in 5.0 by visual inspection

An old problem on iPhone, it cannot play automatically (to save data, save your sister!!!), and the default is full-screen control playback

For a long time, I ignored this video For processing, Android uses the bottom layer, and iPhone directly uses VideoJS, with built-in flash and h5 switching. Flash also has support issues

A while ago, the boss had a request. We have too many application animations, all of which are combination animations of the sprite route. The size of an app ranges from hundreds of megabytes to hundreds of megabytes

So there is an urgent need for a solution to compress images

The final solution is to use video instead of animation, because video compression technology has been developed for many years. Already very mature. Current video compression technology can easily compress 720P high-definition movies to 10M/minute, or 160K/second. It is at least dozens of times smaller than the file size of the image sequence. At the same time, most devices support hardware decompression of videos. In this way, the CPU consumption of video playback is very low, the battery consumption is also very low, and the playback speed is still fast. Even full-screen playback at 25 frames can be easily achieved.

After the plan is finalized, there are several problems that need to be solved

1. The entire video, including certain objects in the video, can respond to the user's clicks, slides and other operations
2. On the iPhone, it can be played in a window
3. The background can be filtered out, so that it can be used like a PNG image

The final effect is also shown in the starting gif animation:

Video replaces animation, and supports the background mask effect, which can reveal the base image

It also solves the problem of manual, automatic and not full screen

iphone windowing

Solution:

Combined processing through canvas + video tag

Principle: Get the original image frame of the video, through canavs is drawn to the page

Here I attach the source code directly. The code is generally written, but several key points are highlighted

http://stackoverflow.com/questions/3699552/html5-inline -video-on-iphone-vs-ipad-browser

Video instead of animation

This is a bit troublesome. You need to interact and drag the canvas to control the image. I haven't finished writing them all yet, and general company needs won't have this. Here's a brief description. It's also processed by canvas + video, but it needs a cached canvas container for preprocessing. Through preprocessing, get By changing the RBG value of each pixel of each picture, the background can be filtered out, so that it can be used like a PNG image. I will write it later and publish it~~

Related recommendations ：

Introduction to relevant examples of html5 audio

Detailed introduction to HTML5 audio and video

Ask for advice on installing the HTML5 audio playback plug-in in WordPress

The above is the detailed content of Problems and solutions encountered with HTML5 audio and video on mobile terminals. For more information, please follow other related articles on the PHP Chinese website!