Research and implementation of high-speed downloading in Sogou market and loading APP pages on the web page_html/css_WEB-ITnose

Github blog post address, the update here may not be very timely.

I haven’t blogged for a long time, what a big mistake. It just so happened that I had just completed the high-speed download function of Sogou Market recently, so I used it to fill this hole.

It is said that in order to increase the number of users, improve user activity and cooperate with promotion, what is even more deceiving is that other markets also have these functions, etc., and require a bundled download function. WTF.

Of course, complaints are complaints, the task still needs to be completed.

The specific requirements are: When the user browses the WAP site on the mobile phone, 1. When entering the application details page, open the details page in this application (Sogou Market) 2. When clicking on the WAP side for high-speed download, if If this application has been installed, call this application to download, otherwise download the bundle package of this application. After the installation is completed, download the application that the user wanted to download before when this application is opened. How to implement these two functions? The following step-by-step analysis:

1. Functional requirements

As described before.

Generally speaking, to complete the previous two functions, the application generally needs to be installed and running in the background (the simplest understanding is that the application has been opened and has not been killed by other programs), so this By default, the application is considered installed in this case; otherwise, it is considered not installed (this is the case unless otherwise specified below).

2. Competitive product analysis

Of course, since competing products have these functions, let’s use doctrine first and study how they do it. Through Chrome packet capture analysis, the final analysis is as follows.

Baidu Mobile Assistant http://127.0.0.1:6259/

Sogou Mobile Assistant http://127.0.0.1:12388/

Wandoujia Custom Agreement wandoujia ://detail/app/cn.buding.martin

The solutions of Baidu Mobile Assistant and Sogou Mobile Assistant are basically the same. use a mobile phone local service address. 127.0.0.1 (loopback address), the port address is different , Wandoujia uses a custom protocol, and the browser will automatically distribute it to activities registered with the wandoujia protocol.

Open application details When opening the corresponding details page of the browser, Baidu Mobile Assistant and Sogou Mobile Assistant access the local loopback address at the same time, while Wandoujia calls the custom protocol by System call related applications (usually Wandoujia).

Principle of high-speed download Wandoujia does not implement calling the client to download when the application is installed. When clicking download, it asks whether to download the bundle or directly download the application you want to download. (See the attached picture at the end of the article for the description copy, the copy is inductive). Select Download Bundle to download the Wandoujia installation package with a specific file name. After the installation is completed, scan a specific directory (usually download and the download directory of common browsers). If there is a file that meets the rules, extract the relevant resource ID, and then download the bundled APP.

1. When the application is not installed. Baidu Mobile and Sogou Mobile Assistant click on the high-speed download (the names are also misleading) to directly download an APP. The content of all APPs is the same, but the APP names are different, similar to app_bundled id_xx.apk. For example, Sogou Mobile Assistant is bundled with WeChat. The bundle id for MobileTools_8271386494777466339_71.apk, bundled with other packages is different. Other processes are the same as Wandoujia. If there are multiple apks that meet the conditions, Sogou Mobile Assistant will download the latest information, and Baidu Mobile Assistant will not study it.
2. If the mobile APP is already installed. Wandoujia has not been implemented, but Baidu Mobile Assistant and Sogou Mobile Assistant directly request a loopback address (127.0.0.1:port/actionpath?parameter) from the browser web page. Port is the port address, Baidu Mobile Assistant is 6259, and Sogou Mobile Assistant to 12388. Actionpath is the operation that needs to be performed. Different operations have different values. Parameter is the parameter of the related operation. Here, the APP data that needs to be bundled and downloaded is passed. The mobile APP receives relevant data through the HTTP server in the application and then downloads the application and returns the corresponding data.

3. The final solution

Launch the details page, using both HTTP server and custom protocol, and the custom protocol does not need to be in the background all the time Running a Service that hosts an HTTP server will save power.

The follow-up High-speed download, Wandoujia is no longer available. Since we talked about customizing the protocol to save power, can we consider it? Combined with the needs, let's take a look. If the application is not installed, download the bundle package directly. There is no problem in either case; if the application is installed, call up the client to download. Can it be done with a custom protocol? How do I know if the app is installed? Does the browser have a corresponding API? Obviously not, even if there is, it may only be partially available, but it must support all browsers. The only way is to access the HTTP server and set a timeout. If there is no response within a certain period of time, the application will be considered not installed and the corresponding bundle will be downloaded. Moreover, through subsequent expansion of the HTTP server, two-way interaction with the Web can be carried out through the web server. For example, Baidu Mobile Assistant can obtain location information through the web. Custom protocols obviously cannot do this. In addition to saving electricity of course.

The adjustment of high-speed downloading has been solved. If the application is not installed, after the application is installed, how do you know which application the user needs to download before? That is Identification of bundled applications. The initial plan of Sogou Market is to enter a file in the application (under the asset folder) to store the ID number of the bundled application. However, it can be used for a small number of applications (it was used initially to test the bundling effect). Currently, it is definitely not possible for a large number of applications. It requires a large number of packages and a lot of storage space.

Refer to the solution of competing products, it is indeed perfect. However, considering that most of the current mobile phone security or cleaning software will prompt you to delete the installation package after the application is successfully installed. At the same time, the download directory of the browser may be Change. Can the entire SD card be scanned here to find files that meet the conditions? Definitely not, it would be too time consuming.

How to know the bundled application ID when the application is not installed or not running in the background. The above two solutions are not perfect, how to solve it. After discussing with the backend Yanfei, when downloading the apk at high speed, the webpage records the bundled app id and packagename downloaded by the mobile phone as well as the unique id (uuid) of the device, and then when opening the market, requests the corresponding interface and passes in the uuid. Obtain the data to download the corresponding application. The idea was great, but later it was discovered that it had no practical value, and uuid did not have a reasonable solution for the time being. The key is that what is generated at both ends must be unique, mac, ip? How to generate?

After dinner and discussion with Guochang, I considered whether local code could be called to perform related tasks? But after thinking about it carefully, I found that this is usually the interaction between WebView and js. This is simply not possible in the current usage scenarios. This Webview must be within your own application.

Considering many factors, we finally decided to refer to the solutions of Baidu Mobile Assistant and Sogou Mobile Assistant, despite their flaws. Regarding the custom protocol mode, you can refer to this article. Android custom protocol and HTML automatically try to call the local APP when loading, and Android registers and listens to the custom protocol. This article will not go into too much detail.

The final solution is to run an HttpServer through Service in the background on the mobile phone and listen to port 12307.

Launch the client

1. High-speed download

When clicking high-speed download on the web page, access http ://127.0.0.1:12307/appdown?downid=1&packagename=com.sogou.map

If successful, the returned json string is in the form of

{ "status":1, "message" :"OK" }

A status of 0 indicates failure, and a status of 1 indicates success. message is a specific description.

If the phone does not respond within a certain period of time, the default application is not installed and the bundled APK package is downloaded.

If the mobile terminal responds, the web page does not do any processing. The mobile terminal obtains the corresponding downid and packagename, and first determines whether the application has been installed. If installed, it will prompt that the application has been installed. If not, then Prompt the user that the application is being downloaded, and then add it to the download list.

2. Open the details page

http://127.0.0.1:12307/godetail?downid=1&packagename=com.sogou.map&backtohome=ture when backtohome is true , press the back button on the mobile phone to return to the homepage of the application, and return to the browser when it is false. (Please refer to Wandoujia, such as (http://www.wandoujia.com/apps/com.tencent.mm) [http://www.wandoujia.com/apps/com.tencent.mm] must be accessed via mobile phone. User-Agent is different from PC access)

Installation package application bundled

How to identify which application is bundled?

Refer to the implementation method of competing products later, by downloading files with different file names (file names have rules), after the user installs the application and opens it, read the download folder to determine whether there is an apk that meets the rules. If it exists, extract the corresponding ID and download it accordingly. As long as the server provides an interface to allow the browser to download the same file, it will display it as a different file name.

For example, the WeChat details address of Sogou Mobile Assistant is http://wap.sogou.com/app/apkdetail.jsp?ppid=34&cid=49&docid=8271386494777466339&e=1394, and the address for high-speed downloading of the installation package is http:// download.zhushou.sogou.com/zhushou/android/MobileTools.apk?dn=MobileTools_8271386494777466339_71.apk The downloaded apk name is MobileTools_8271386494777466339_71.apk. The bundled application ID is 8271386494777466339, and the application is WeChat. This can be achieved by setting the Content-Disposition field of the headers of the HTTP return data to attachment; filename="MobileTools_8271386494777466339_71.apk". Generally speaking, mainstream browsers support this attribute.

Therefore, it is finally determined that the naming method of the APK is SogouAppmall_binding_downid_9.apk. If the application is judged to be in this format (can be matched through regular matching), the downid will be extracted (downid must be an integer). When the application is not installed, click High-speed Download to download the apk named according to the above rules to the folder. When the user installs and opens the application, scan the apk in the commonly used download folders (common /download, /downloads, and browser download folders, etc.). If there is one that meets the rules, the id in a certain file will be extracted. After completion, download and delete the installation package (to prevent future interference). Considering that the same folder may have multiple installation packages that comply with the rules, the two most recent ones that meet the rules will be selected in chronological order. The other folders will be ignored and will be processed next time (generally this The situation is also relatively rare). Of course, there are other details that need to be dealt with, which will not be discussed in detail here.

4. Performance Optimization and Outlook

Program optimization is something that ambitious siege masters must do, so of course, it must be optimized.

In order to save power, the following two strategies are specified.

1. When the network changes, if there is no network, the service will be closed. If there is a network, the service will be opened. Turn the service on and off respectively when unlocking and locking the screen.

2. Having an internet connection and having the screen turned on are necessary conditions to trigger the service to start. It can largely reduce the power consumption problem caused by network changes when the application is locked.

The above two strategies can be used in combination, and there may be some problems. For example, sometimes the conditions are met, but the service is not turned on (which may be caused by frequent network switching).

The above conditions are set because subsequent operations can only be completed with an Internet connection. Secondly, when the screen is locked, ordinary users cannot trigger related processes unless the program automatically triggers it (instant messaging software For example, receiving information, or automatically downloading, but this is too rogue) needs to run in the background when the screen is locked.

The HTTP server is the open source Nanohttpd on Github. In addition to high-speed downloads, the http server can also be used in many aspects, such as transmitting data to computers, interacting with applications and web pages, and so on.

5. Final implementation

You can visit http://m.app.sogou.com/ to view the final effect.

The general idea is the above description. I hope it can inspire everyone and serve as a starting point.

It is not online yet and is expected to wait until November 5th.

6. Related tools and appendices

(1) Chrome developer tools device mode (data capture)

(2) Demo addresses for WeChat downloads in three markets : Baidu Mobile Assistant, Sogou Mobile Assistant, Wandoujia

At that time (the page screenshot on 2014-10-27 is as follows)

(3) Common browsers APK download path (# means the line change is a comment, one directory per line, traversing subdirectories is currently not supported)

#this is the binding apk file path#this is  a comment#some system default, Chrome, Opera, Opera mini, Sogou Broswer, ES File Explorer,Dolphin Browserdownload#meizu browser, some system defaultdownload/apkdownloads#uc browserUCDownloads#QQ BroswerQQBrowser/安装包QQBrowser/下载qqbrowser/download#QQ Broswer HDQQBrowser#baidu broswerbaidu/flyflow/downloads#baidu appbaidu/searchbox/downloads#Maxthon BroswerMxBrowser/DownloadsTTDownload/installapkApplicationThunderDownload#liebao broswerkbrowser_fast/download/App#360 Broswer360Browser/download#360 Express Broswer360ExpressBrowser/download#2345 broswerDownload/2345浏览器下载/安装包#hao123hao123/down/apkDolphinBrowserCN/downloadUCDLFilesQCDownloadLXDOWNLOAD/DOWNLOADapc/ApcBrowser/downloads#YueDong Broswer,Ignore,The apk file name is changed by the broswer,the same with 4G Explorer(do not support header's Content-Disposition attribute)#ydBrowser/download#4G-explorer/apks

Hahahaha, Perfect solution to all problems. It can be expanded later.