Detailed explanation of web cross-device ultrasonic communication solution based on HTML5

Detailed explanation of web cross-device ultrasonic communication solution based on HTML5:

Foreword: Chirp The iPhone launched the prelude to audio file transfer. We no longer need MMS, Bluetooth pairing, or IM to transfer data. It shares data through the chirping of birds, which is simple and interesting, and can quickly achieve one-to-many sharing.

##        

In addition, Alipay has tried to promote "sound wave payment", using the mobile phone to send ultrasonic waves to the terminal as a transaction password to further complete the transaction. However, Alipay's technology is currently only used for payment, and it requires specific vending machines that can accept sonic payments before it can be used.

" advanced sound for game and interactive apps -webaudio api ##" Web's web with HTML5 with HTML5 Audio API implements an ultrasonic interconnection solution based on Web. In this way, there is no need to install any client, nor does Bluetooth or # With the support of ##NFC, you can connect two devices that use browsers to access the Internet and transfer pictures, music, videos and other files. This idea is cool and helps us take a big step forward in the functionality and application level of Web (Maybe you have this idea. Unthinkable, unheard of), let’s take a look at the specific principles and implementation process. ……………………………………✄……………………………………

Gorgeous separation Line

       

The phone in your pocket is a cool, beautiful tool with many functions. But when it wants to talk to other devices, such as a TV or laptop, the user experience drops significantly.

Bill BuxtonThere is a very infectious speech on this topic, describing the three stages of high-tech evolution:    

1,

Equipment work: functional integrity and stability##       2,

Device smoothness: good user experience##3.

Many devices working together

We have officially entered the

2

stage with the release of iPhone , but connecting devices is a pain. There are many ways to achieve this: Bluetooth, BluetoothLE, WiFi direct, via localWiFi Network discovery and more. This article tackles the problem from a completely unexpected angle: using ultrasound to broadcast and send data between adjacent devices. Most importantly, this approach uses the Web Audio API, making connection with pure Web possible. Demo video

            The Tower of Babel of Equipment

                        ##Airplay   and Chromecast is a great way to solve connectivity issues with devices within the same ecosystem (e.g. Apple or Google) , but common problems are still difficult to solve.

Because there are many possible technical approaches, it may happen that there is no common method among the devices you want to connect to. Even if two devices have Bluetooth, one may need to be configured, but the other does not, or supports a different version of the standard. Now this is especially common with Bluetooth, many devices support Bluetooth 4.0 (aka BTLE) on hardware, but many devices don't This latest protocol is not supported for various reasons. ##          

This situation is even worse on

Web because of the low level device connectionAPIIt is not provided due to security sandbox reasons. And since the development of Web is very slow, it is difficult to imagine that this situation can be solved in a short time.

         

Transmitting data in interesting ways                           Electric Imp The provided

Blinkup

is an interesting way to communicate across terminals. It uses a series of flashes of light to light up a smartphone and the Imp - a device that looks like a small SD card and has a light sensor. transfer data between.       Dial-up modems do something similar. They encode and decode digital data on analog telephone lines. Remember those annoying connection noises? The dial-up modem will turn on the speaker to let the user know that a handshake is taking place. If you forget these, you can review them here. Even in today's analog phones, the sound you hear when you press a number key on a keypad is equivalent to the frequency that the phone system uses for analog-to-digital conversion. This conversion uses Dual Tone Multi-frequency signal (DTMF).

Your phone and many other devices around you have speakers and microphones. Both pieces of hardware can use sound to send and receive data, similar to what a modem does on a phone line. Even better, if the operating system supports sending and receiving at a high enough frequency, we can create a silent data channel.       Transmitting data using sound

##         I should have noticed Encoding data through sound is not new. Audio Watermark

The idea is to encode the signature into the music, which the listener cannot recognize through human means, but can be understood by another device. This is a smart way to prevent piracy.

# Most ordinary speakers can produce the sound of # 44.1khz ## (This leads to the highest frequency of 22KHz - According to Shannon Sampling Theorem). This allows us to encode data not just into sound, but also to produce audio that adults cannot hear. However, children and animals may still hear it :)

##     Technically, it should be noted that microphones are sometimes not compatible with speakers, especially on mobile phones , because they are often optimized for people talking on the phone, using a lower sampling rate to make the sound better. In other cases, even if the hardware is compatible, the firmware needs to run at a lower sampling rate to ensure power supply. If this is the case, the device will not be able to receive sound waves, and sound-based connections will only work in one direction.

SONICNET.JS, an implementation of Web Audio

#      

To illustrate these concepts, I built a JavaScript library for sending and receiving via sound data. My approach does not require learning complicated audio watermarking technology, and it is even simpler than DTMF. Basically, you specify how often a range is used, and the set of letters that can be transmitted. The spectrum is divided into ranges corresponding to the specified start and end letters, with each letter / code corresponding to a portion of the entire frequency range.

       

The sending end converts each character that needs to be sent into the midpoint value of the corresponding frequency range, and sends the frequency within a time interval. The receiving end will perform a discontinuous Fourier transform on the signal (Yu Jie's note: My undergraduate major is information and communication engineering, friends who have studied digital signal processing should know this very well), and look for peaks within the specified frequency range. After finding a peak within a signal interval, it converts the frequency back into characters. This is basically a Single Tone Multi-Frequency Signal (STMF) scheme.

         

There is a question of sending timing: on the sending end, how long should it take to send a character; on the receiving end, how long should it take to receive it? We used a simple scheme to avoid adjacent repeated characters.

       

I designed a API## similar to socket #Used for acoustic communication. The client code looks like this: ##

ssocket = new SonicSocket({alphabet: '0123456789'});
function onButton() {
 ssocket.send('31415');
}

Service The end looks like this:

##

sserver = new SonicServer({alphabet: '0123456789'});
sserver.on('message', function(message) {
  //Expect message to be '31415'.
 console.log(message);
});
sserver.start();

##                  

JS

Library support

github.

      Of course, using it requires an implementation of Web Audio (mostly using OscillatorNode## on the sending side #, use AnalyserNode at the receiving end). I've tried Macbook on Chrome to Chrome, as well as Transfer case from Mac Chrome to Android Chrome.

      I wrote a pair of examples to illustrate this idea. They appear in the video demonstrated at the beginning of this article. The first example allows you to send emoticons from one device to another. It uses a very small character set, including only 6 characters, each corresponding to 1 expressions. You can choose one of the 6 emoticons, and the corresponding character will be sent through the sonic network and received and displayed at the other end.

# Sonicnet.js The more realistic application is this chat Apply , generate a unique 5 digit token and use it to communicate between 2 devices Create connection. This is done with the help of a pairing server, which establishes a proxy connection between the two devices via WebSocket. Once the connection is established, the conversation itself is sent over WebSocket. Server-side code is hosted on nodejitsu.

          Conclusion and expectations

##               Web Audio APIIt has evolved to make such applications possible, which is a major progress. I'm obsessed with sonicnet.js implementation on the Internet of Things. This is pure Web technology that can be used to pair devices. The combination of ubiquitous browsers and audio hardware will be a huge win, and even at the hardware level, Webplatforms don't need to wait for Bluetooth or other near-field connectivity technologies at all Mature.

      If this article has piqued your interest, you can try to use sonicnet.js to write an application. As I mentioned before, receiving high frequency audio is not available on all devices due to hardware/firmware limitations, so I'd love to know which ones are available and which ones are not. Can't. My expectation is that most phones should only be able to send, while most laptops should be able to receive and send at the same time. If you tried the emoticon demo on your device, please fill out this form. At the time of writing this article, the Chrome Beta version on the Android platform does not support real-time input, so transferring data from mobile phone to laptop is the only option.

The above is the detailed content of Detailed explanation of web cross-device ultrasonic communication solution based on HTML5. For more information, please follow other related articles on the PHP Chinese website!