Detailed explanation of HTML5 mobile audio and video problems and solutions

Recently, we are studying the use of video to replace animation, and the use of video to replace sprite animation. We call this kind of video interactive video.

Traditional sprite animation:

• The disk space is large and the download is slow, especially when played online, it will be even slower

• There are too many files. When playing online, too many http requests will lead to slow response or abnormal behavior.

Therefore, it is urgent to develop a set of technologies to use Video instead of sprite animation. We call this kind of video interactive video

The problem with traditional video:

• Traditional video can only be played in a square area

• Traditional videos are played in windows on the iPad. On iPhone, they can only be played in full screen.

• Traditional videos will definitely appear when playing. At the front end

• 
  

##interactive video has the following characteristics:

• Under the iPhone, there is no need to play in full screen, it can be played in an area

• Interactive video can appear under ordinary graphics

  Objects

• Interactive videos can have masks, which can remove the background of the video and blend the video with ordinary graphic objects

Summary: For videos that are simply played, we will set them to traditional videos. For videos that need to be used for specific purposes, we set them as interactive videos.

The research has initially yielded results. By the way, I will summarize the practical problems encountered in

audio and video during the years of mobile H5 development and give my own solutions

Look at the final actual effect: compatible with PC (>IE9), iPhone, iPad, Android 5.0

It solves the problems of manual, automatic, windowing, etc. on iPhone, and can basically be used Actual production

The right side is the original video mp4 file

The video on the left replaces the animation, and then supports the background mask effect, which can reveal the base map and support a series of interactive operations

H5 audio audio

• # Each time an audio object is passed through new Audio, you can see it on IOS A new thread will be generated, which is disgusting

Solution:

new Audio object, by replacing different audio addresses, to achieve The purpose of not opening more threads

• The support on Android is not strong

Solution:

The problem on lower versions of Android is not solved. Generally, hybrid development can be handled by adjusting the underlying

interface. For example, phonegap

• cannot be automatically used on iPhone Play

Solution:

Automatic playback on iPhone is a process done when IOS was designed. It seems to be to prevent automatic theft of traffic

To put it simply, you need to simulate the user to trigger it manually

So we need to call this piece of code at the beginning:

This is from my project, I just deducted it directly

//修复ios 浏览器不能自动播放音频的问题 在加载时创建新的audio 用的时候更换src即可
Xut.fix = Xut.fix||{};
if (Xut.plat.isBrowser && Xut.plat.isIOS) {
    var isAudio = false
    var fixaudio = function() {
        if (!isAudio) {
            isAudio = true;
            Xut.fix.audio = new Audio();
            document.removeEventListener('touchstart', fixaudio, false);
        }
    };
    document.addEventListener('touchstart', fixaudio, false);
}

If such a code is bound to the body: create an audio object through manual triggering, and then save it in the global

When used, it is as follows

//如果为ios browser 用Xut.fix.audio 指定src 初始化见app.js
if (Xut.fix.audio) {
    audio 
=
 Xut.fix.audio;
    audio.src = url;
} else {
    audio = new Audio(url);
}
audio.autoplay = true;
audio.play();

Just replace the audio object directly. To put it simply, the object created by the user must be triggered automatically to play

H5 video audio

video Tags may be rarely used on mobile devices, and Android support is terrible. It is visually improved with version 5.0

An old problem on iPhone, it cannot play automatically (save data, save your sister!!!), and the default is Full-screen control playback

For a long time, I ignored this video processing. Android uses the bottom layer, and iPhone uses VideoJS directly. It has built-in flash and h5 switching, and flash also has support issues

before Recently, the boss had a request. We have too many application animations, all of which are combination animations of sprite routes. One app can range from hundreds of megabytes to hundreds of megabytes

, so we urgently need a solution to compress

pictures.

The final solution is to use video instead of animation, because video compression technology has been developed for many years and has become very mature. Current video compression technology can easily compress 7Detailed explanation of HTML5 mobile audio and video problems and solutions0P

high-definition movies to 10M/minute, or 160K/second. It is at least dozens of times smaller than the file size of the image sequence. At the same time, most devices support hardware decompression of videos. In this way, the CPU consumption of video playback is very low, the battery consumption is also very low, and the playback speed is still fast. Even full-screen playback of Detailed explanation of HTML5 mobile audio and video problems and solutions5 frames can be easily achieved.

After the plan is finalized, several problems need to be solved.

1. The entire video, including certain objects in the video, can respond to the user’s clicks and slides. Class operations

2. On the iPhone, it can be played in a window

3. The background can be filtered out, so it can be used like a PNG image

The final actual effect is also shown in the starting gif animation:

The video replaces the animation, and then supports the background mask effect, which can reveal the base image

Also solved the problem of manual, automatic and not full screen

iphone windowing

Solution:

Passcanvas + video tag combined processing

Principle: Get the original frame of the video and draw it to the page through canavs

Video instead of animation

This is a bit troublesome. It requires interaction and dragging the canvas to control the image. I haven't finished writing it all yet, and general company needs will not have this

Here is a brief description, it is also processed by canvas + video, but a cached canvas container is required for preprocessing

Through preprocessing, get By changing the RBG value of each pixel of each image, the background can be filtered out,

so that it can be used like a PNG image

The above is the detailed content of Detailed explanation of HTML5 mobile audio and video problems and solutions. For more information, please follow other related articles on the PHP Chinese website!