A hurdle that cannot be overcome in underwater visual intelligence research: wading optics

Water-related Optics mainly studies the material interaction mechanism between light and water and the cross-media propagation mechanism of light, and solves various problems related to the intelligent acquisition of water-related optical data, information transmission and intelligent signal processing. , exploring the science of the application of optics in the water-related field is an important discipline support for underwater security in the Vicinagearth Security (VS) system.

1 Introduction

Wading is related to water, and generally refers to water bodies including oceans, rivers, lakes, clouds, rain, fog, snow and ice, etc., such as As shown in Figure 1. Compared with underwater optics, ocean optics is more fully considered. The research object of wading optics covers all water bodies that are part or whole of the light propagation path. By exploring its optical properties in liquid, gas, and solid states, and the light in water, The propagation mechanism in cross-media solves various problems related to the intelligent acquisition of optical data, information transmission and intelligent signal processing in water-related fields. It is Vicinagearth Security (Vicinage originates from the old French/Latin visnage/ The vicinus ('neighbor'), VS) system is an important support for underwater security and is of great significance to the defense, protection, production, safety and rescue of my country's territorial waters.

"Wading Optics" is further developed into a cross-domain scenario based on the single scenarios of "Underwater Optics" and "Ocean Optics", by measuring light propagated in water bodies and across media. Phase, intensity, frequency, polarization and other physical quantities, obtain image, temperature, vibration, pressure, magnetic field and other parameter information in water and cross-media environments, and develop optical detection, sensing, measurement, imaging, and communication in the water-related field. and intelligent signal processing and other technologies.

Figure 1 Wading optics

Currently the development of wading optics is facing Bottleneck issues such as high light absorption and strong scattering by water bodies make its development status far behind actual needs. Therefore, the field of water-related optics urgently needs more attention.

In order to promote the exchange of my country's underwater optics technology and the application of industry, academia and research, Li Xuelong's leadership team proactively laid out the strategic area of ​​wading optics in the new era and first proposed "underwater optics". In May 2016, On March 10, the country’s first “Underwater Optics” Summit Forum was initiated and held in Xi’an. Subsequently, the second consecutive session was held in Xi'an on June 22, 2018, developing "underwater optics" into a redefined "ocean optics". The forum was officially renamed the "National Ocean Optics Summit Forum" and initiated the establishment of the "China Ocean Optics Summit Forum". Ocean Optics Special Committee of the Optical Engineering Society".

As of the time of writing, the forum has been successfully held for five times. The fifth forum (2022) attracted more than 30,000 online attention and participation. National Ocean Optics The summit forum has become one of the most important and most watched optical conferences in my country. In terms of promoting industry-university-research application, Li Xuelong established the Qingdao National Laboratory of Marine Science and Technology (currently) and the unit's marine optics joint laboratory in 2016. In the same year, he proposed and took the lead in preparing for the establishment of Shaanxi Provincial Key Laboratory of Ocean Optics, my country’s first provincial and ministerial-level key laboratory for wading optics. The laboratory was approved to be established in 2018. Li Xuelong served as the first director and led the team to complete the full-ocean deep high-definition optical imaging and image processing system, which won the first prize of the Science and Technology Progress Award of the Chinese Optical Engineering Society.

As the research and development of marine science and technology continues to deepen, human understanding of the ocean and the level of technical equipment have also continued to improve. "Ocean optics" has evolved from the traditional study of ocean optical properties to light in the ocean. The science of detecting ocean propagation laws and using optical technology has further developed into a comprehensive science that focuses on the study of deep-sea science, technology and equipment, builds deep-sea bases, detects deep-sea space, and develops deep-sea resources.

Faced with problems such as the vast space of the deep sea, complex hydrological characteristics, and difficulty in perceiving information, Li Xuelong established the Key Laboratory of Intelligent Interaction and Application of the Ministry of Industry and Information Technology at Northwestern Polytechnical University in 2020, fully considering the relationship between water and air and other media. , the close connection between optical equipment and algorithms has further developed "ocean optics" into "wading optics", expanding the research object from a single field to multiple water bodies such as oceans, rivers, lakes, clouds, rain, fog, snow and ice, as well as with Other fields related to water bodies revolve around a series of scientific issues such as "the material interaction mechanism between light and water and the cross-media propagation mechanism of light", "dynamic target detection in complex environments", "high confidence data resolution under redundant heterogeneity" , led the team to overcome problems such as the difficulty in modeling the degradation mechanism, the imperfect observation equipment system, and the difficulty in analyzing scene target data, and completed the research and development of a series of domestic ocean observation technologies and equipment. In 2022, we established a wading optics laboratory and led the team to obtain support from the "Underwater XX Guidance" national key project. The development of wading optics has taken another solid step.

Figure 2 Wading optical frame

2 Light Mechanism of material interaction with water

The optical properties of water are the macroscopic manifestation of the material interaction between light and water, and are an important basis for studying water-related optics. The inherent optical properties of water are the optical parameters of the natural water itself, which are independent of the ambient light field. Commonly used intrinsic optical parameters of water include spectral absorption coefficient, spectral scattering coefficient, spectral attenuation coefficient, volume scattering function, backscattering coefficient, forward scattering coefficient, beam attenuation coefficient, etc. The apparent optical properties of the water body are the characteristics of the water body due to the action of the light field. They are determined by the time and spatial distribution of the light field in the water and the inherent optical properties of the water body, and can change with changes in the light field.

Figure 3 Attenuation of different wavelengths in the visible spectrum under different water quality

The linear effect of water on light refers to the absorption, scattering and refraction of light during its transmission in water-related areas. "A setting sun spreads over the water, half of the river is rustling and half of the river is red" vividly explains that light will be scattered and refracted when it enters the water body, and reflects the dispersion characteristics of light.

The nonlinear effect of water on light refers to the material interaction between light and water. When the light intensity is less than the breakdown threshold in the water, the interaction between light and water will Nonlinear processes such as stimulated Raman scattering, vibrational scattering and Brillouin scattering are produced. When the light intensity exceeds the breakdown threshold of the water body, multi-photon excitation, reverse Bremsstrahlung absorption and electron collision avalanche ionization will cause the water body to break down and produce plasma radiation. Studying the nonlinear process in the material interaction mechanism between laser and water is of great significance in the laser industrial fields such as underwater laser cutting, welding, and cladding, and in the field of laser clinical medicine.

3 Wading optical information acquisition

Information can reflect the characteristics and essence of things in nature, and humans can obtain and identify the differences in nature information to understand and transform the world. Wading optical data acquisition mainly accurately measures and describes the substances and their physical parameters in the wading environment, which is an effective way to understand the wading environment. At present, the main ways to obtain water-related optical data include optical sensing technology, spectral measurement technology and optical imaging detection technology.

3.1 Optical sensing technology

Optical sensing technology is based on optical principles, sensing environmental information through optical technology, and then digitally collecting and analyzing it through a data acquisition system Adjustment mainly includes optical remote sensing technology and optical in-situ sensing technology.

3.2 Spectral measurement technology

Spectrum can be used to study and identify the structure, composition and state of water and substances in the water. Spectral measurement technology has greatly improved the accuracy of water-related measurements. sensitivity and resolution.

1) Laser-induced spectral breakdown technology

Laser-induced spectroscopic breakdown technology is a spectroscopic technology based on the action of laser on matter to generate transient plasma and analyze the sample based on the characteristic emission spectrum of atoms and ions in the plasma. It can achieve In-situ, real-time, continuous, non-contact detection of substances.

2) Laser Raman spectroscopy technology

Laser Raman spectroscopy technology is an in-situ, real-time, Non-destructive, multi-material simultaneous detection optical sensor technology has the ability to quantitatively detect the composition of target objects in water-related environments, and can realize long-term in-situ monitoring of acid ion concentrations in seawater. It is useful for understanding seafloor hydrothermal activity areas, earthquake source areas, and Seafloor sediments will be of great significance.

3.3 Optical imaging detection technology

Wading optical imaging detection technology is the most intuitive detection technology that reflects the water environment in wading optical data acquisition. The low resolution of underwater acoustic imaging, slow acquisition and processing speed, and inability to perform high-resolution imaging in real time have restricted its progress in underwater imaging. Underwater optical imaging technology can use visual imaging equipment to directly obtain image or video information to achieve the collection and analysis of underwater targets.

1. Distance gating imaging technology

The working principle of wading distance gating imaging technology is through time Control to remove the background noise introduced by scattered light that does not contain the target signal, and ensure that the signal light reflected by the target arrives just within the gating working time. Li Xuelong's team developed a distance-gated imaging prototype based on this principle, achieving imaging at 6 times the attenuation distance.

2. Polarization imaging technology

Water-wading polarization imaging technology compares the differences in polarization information of scattered light fields and uniqueness, analyze the changing trend of the polarization characteristics of the target and the background in the image, and invert the intensity changes of the target information light and background scattered light, which can effectively suppress the backscattered light and achieve clear water-related optical imaging.

3. Carrier modulation imaging technology

Carrier modulation imaging technology uses a high-frequency microwave subcarrier signal to laser The emitted light pulse is modulated, and after being backscattered by the water body, the scattered light is filtered out at the receiving end through a band-pass filter with the modulation frequency as the center frequency, thereby suppressing the scattered low-frequency components. Li Xuelong's team developed a high-energy microwave frequency modulated laser and cooperated in the development of a microwave frequency modulated lidar system, which has the ability to improve the signal-to-noise ratio and increase the underwater detection range. It can effectively solve the backscatter problem and achieve The intelligent interaction between the environment and the device improves the underwater detection distance.

4. Correlation imaging technology

#Correlation imaging is a method that uses the second-order coherence of the light field to achieve Imaging technology, as a non-local imaging technology, uses a single-pixel intensity detector to collect the target light intensity signal and reconstructs the image combined with the projected light field. At the same time, this imaging method can incorporate the environment model and deep learning neural network into the imaging algorithm. It can realize intelligent computational imaging under low-light conditions and solve the problem of weak anti-interference ability of traditional underwater imaging. Li Xuelong's team developed an underwater correlation imaging system, and combined with intelligent scientific algorithms, has achieved high-definition reconstruction of images under different turbidities.

Figure 4 Schematic diagram of wading correlation imaging

5 .Compressed sensing imaging technology

Compressed sensing theory is a brand-new signal sampling theory, as shown in Figure 5, if the signal is compressible, or the signal is in a certain If the transformation base is sparse, the compression process and the sampling process can be completed simultaneously, and the information extraction can be completed during the sampling process.

Figure 5 Mathematical expression of compressed sensing

Li Xuelong’s team studied a fast computational microscopy imaging method based on deep learning. Deep learning is used to reduce the amount of optical microscopy imaging data collection, and compressed sensing is used to improve the resolution and signal-to-noise ratio of optical microscopy imaging. Then, using computational reconstruction mode, we can obtain multi-dimensional high-altitude time-resolved information of the sample that cannot or is difficult to obtain directly with traditional microscopy techniques. Deep learning technology represented by data drive and compressed sensing technology represented by physical model drive have improved the unpredictability of actual imaging physical processes and the complexity of solving high-dimensional ill-conditioned inverse problems.

#6. Spectral imaging technology

Spectral imaging technology combines spectral measurement with imaging technology to create an image Each pixel can extract multi-channel spectral features, thereby achieving multi-space point, multi-channel precision measurement and multi-modal recognition. Based on technologies such as wide spectrum, high score, and snapshot, Li Xuelong's team proposed key technologies such as wide spectrum differential continuous fine spectrum, reference active correction, and nonlinear prediction, which changed the status quo of using chemical analysis as the single standard and provided solutions for complex seawater quality analysis. It provides a new standard and is the first of its kind in the world.

4 Wading optical information transmission

After the wading optical equipment completes information collection, it needs to transmit real-time information to the back-end for processing. The entire process includes two key technologies: underwater wireless optical communication and wading optical image information processing. Underwater Wireless Optical Communication (UWOC) uses light beams as information carriers to realize real-time transmission of large amounts of information such as images and videos underwater. Compared with underwater acoustic communication and underwater electromagnetic wave communication, the UWOC system has a smaller size, lower design cost, and stronger concealment. With the help of UWOC technology, an all-optical communication network integrating air, space, ground and sea can be built, as shown in Figure 6.

Figure 6 Air, space, ground and sea integrated optical communication network

Currently , UWOC’s main research directions include underwater signal transceiver device design, underwater channel modeling, and underwater channel signal modulation and demodulation. However, UWOC cannot be applied in actual long-distance, strong turbulence, and high-rate wireless communication processes. In the future, technologies such as signal modulation and demodulation, turbulence compensation, and stable tracking and aiming empowered by intelligent science will play an indispensable role in underwater optical communication systems. In addition, in the future, underwater optical communication can also be combined with underwater acoustic communication, underwater electromagnetic wave communication and other methods to overcome the shortcomings of existing technologies such as short communication distance and poor stability, and ultimately improve the communication chain in complex underwater optical transmission scenarios. effectiveness and reliability of the road.

5 Wading optical information processing

Wading optical image is an important information carrier for wading optical information detection and contains a large amount of information , how to intelligently process optical images, quickly and accurately restore, enhance, and extract effective information in images is the focus of water-related optical image information processing. Water-related optical image information processing has important application value in water-related weak and dark target detection and identification, underwater security, water-related ecological monitoring, water-related equipment detection, and water-related military reconnaissance.

5.1 Water-related image restoration and enhancement technology

Water-related image restoration starts from the principle of water-related optical imaging. First, the degradation model of the water-related image is established, and then through the first It uses empirical information and premise assumptions to estimate the interference factors that affect image clarity, and uses the inversion degradation process to eliminate the impact of interference factors, thereby improving image clarity.

Figure 7 Wading image restoration technology

Wading image enhancement is a type of image enhancement that changes the image A non-physical model method that improves visual quality and contrast by using pixel values.

5.2 Water wading image quality evaluation

Figure 8 Wading image enhancement technology

Wading image quality evaluation is a comprehensive image quality evaluation standard targeting the underwater image degradation mechanism . At present, underwater image quality evaluation methods usually calculate weighted scores from several measurement angles, and the weights are often determined by experience. Therefore, underwater image quality evaluation scores are often far away from human subjective feelings. From the perspectives of visual saliency, cognitive psychology, and information measurement, it is necessary to construct an underwater image quality that is more in line with human subjective feelings. Evaluation methods are a research direction worth exploring in the future.

5.3 Cognitive calculation of wading environment

##Figure 9 Multi-detection modal recognition Cognitive computing

Cognitive computing of wading environment provides a good foundation for the development and utilization of wading water resources. It is an important tool for revealing the diversity of wading biodiversity and exploring underwater topography and mineral resources. One of the key technologies of resources.

6 Water-related optical application scenarios: Underwater security

Driven by the international situation and national needs, on-site security (Vicinagearth Security , VS) came into being. On-site security refers to a diversified, cross-domain, three-dimensional, collaborative, and intelligent technology system that meets the needs of defense, protection, production, safety, and rescue in the near-ground space. Specific application scenarios include low-altitude security, underwater security and cross-domain security. Underwater security is one of the cores of local security. It mainly covers national security and defense in underwater space, including seabed monitoring, detection, communication, concealment, guidance, etc., and covers industrial production, social economy, and scientific research. Protection, production, safety, and rescue in education and other aspects are of great significance to national defense security, social stability, and economic development.

Figure 10 Ground-based security space scope

6.1 Water-related environment and resource monitoring

(1) Undersea Observation Network

The undersea observation system places the observation instrument on the seafloor, and the instrument completes in-situ detection. And the data is transmitted through the network to achieve all-weather, comprehensive, long-term continuous, real-time observation. The observation scope includes the depth of the earth under the sea, the seabed interface, sea water bodies and the sea surface. The seabed observation system can use wading optical technology to conduct comprehensive development and research on the ocean. It is the third type of earth science observation system established by humans on the seabed after ground and sea surface observations and airborne remote sensing. It will comprehensively deepen human understanding of the ocean. understanding.

##Figure 11 Seabed Observation Network

(2) Deep sea camera

#In order to obtain the real seabed environment, the deep sea camera system is indispensable in the construction of underwater security . The depth and breadth of deep-sea exploration represent the country’s scientific and technological development level and national defense strength. As an optical visual data acquisition technology, deep-sea cameras can be widely mounted on deep-sea vehicles such as manned submersibles, underwater robots, and landers. It effectively expands the detection range and information volume, and avoids the embarrassment of "blind men and elephants" in deep-sea exploration. It is the key to deep-sea exploration. It is a necessary means for resource surveying, deep-sea mineral development, marine ecological observation and observation of deep-sea biological and chemical activities.

Figure 12 Deep-sea camera (a) sea pupil, (b) deep-sea panoramic camera, (c) underwater captured by deep-sea camera 8152 meters lionfish eating

Li Xuelong's team developed my country's first full-ocean deep high-definition camera "Haitu". The "full-ocean deep high-definition optical imaging and image processing system" completed by the team won the first prize of the 2019 China Optical Engineering Society Science and Technology Progress Award prize. It solves the problem of obtaining high-definition visual data in a deep-sea high-pressure environment, and breaks through key technologies such as full-sea deep dry cabin sealing, underwater optical aberration correction, color restoration, and underwater image enhancement.

The camera is suitable for water depths from 0 to 11,000 meters, with an underwater field of view of 60°, a resolution of 1920×1080, and an underwater weight of 10kg. The relevant technical indicators have reached the international advanced level.

In March 2017, the "Haitui" full-ocean depth high-definition camera followed the "Exploration 1" to complete the scientific research mission of the Mariana Trench. As the main camera, it has been deployed four times. It dived to a depth of 7,000 meters and dived to a depth of 10,000 meters three times, with a maximum depth of 10,909 meters. A total of 12 hours of high-definition video was collected. It was the first time in the history of my country's deep-sea scientific research that it completed the acquisition of high-definition video of the entire ocean depth. For the first time, a lionfish at a depth of 8,152 meters was recorded, which was the maximum depth at which fish were observed to survive internationally at that time. It provided important original data for multidisciplinary research on marine life and physical oceans in the Mariana Trench abyss.

The "Sea Pupil II" full-ocean depth high-definition camera developed subsequently was carried out on the Mariana Trench scientific research mission again in September 2018 along with the "Discovery 1" TS09 voyage. During the period, 10 dives were completed, 4 of which dived to a depth of 10,000 meters. 140 hours of effective high-definition video was collected, with a total data volume of 233GB. Many precious ocean observation data were obtained, and many gaps in the field of marine scientific research were filled.

In addition, marine pasture monitoring, marine oil and gas exploration, water-related pipe network monitoring, marine photovoltaics, etc. are also important application scenarios.

6.2 Water-related detection and communication

The ocean is an important strategic resource that countries around the world compete for. Comprehensively mastering the basic data of my country’s territorial waters is the basis for safeguarding national maritime rights and interests. All-weather water surveillance is a means of underwater surveillance and security defense. The development of water-related detection and communication technology will help our country improve its ability to deal with complex situations and improve its ability to safeguard maritime rights. Underwater lidar detection, underwater optical concealment, laser anti-submarine and anti-mine, underwater photoelectric countermeasures, laser anti-submarine communication, underwater optical guidance, and wading safety rescue are the main application scenarios.

6.3 Wading Laser Industry

In underwater security, especially the development and utilization of rivers, lakes, and marine resources are inseparable from the construction of various underwater projects. For example, a series of water-related projects include building port terminals, repairing ships, building oil well platforms, laying and maintaining pipelines, etc. With the in-depth research and development of laser welding equipment in various countries, high-power lasers used in underwater laser welding have generally appeared.

In addition, in order to extend the service life of industrial structural components in seawater environments and reduce construction costs, underwater in-situ repair technology is usually used to repair and repair damaged and aging industrial structural components. maintain. Underwater laser cladding technology is an effective solution with controllable heat input, high efficiency, good stability, little influence from water pressure, wide range of welding materials, low heat input, fast cooling speed, small heat affected zone, and low residual stress. Etc.

7 Conclusion and Outlook

With the gradual improvement of the water-related optical discipline system, the world situation will face a huge turning point, and the ocean has become a Struggle for strategic resources. The improvement of wading detection technology will greatly release marine resources, further improve productivity, human production and lifestyle will enter a new stage of development, and the acquisition of production materials will produce transformative development.

Marine life is an extremely important carbon sink and carbon aggregate on the earth. As the scale of marine construction continues to expand and the technological level continues to improve, the ecological capacity of our country's sea areas will continue to increase. On the one hand, it can obtain a large amount of production materials and living materials, which provides an important guarantee for the sustainable and stable development of our country. On the other hand, the role of carbon sink and carbon transfer is getting stronger and stronger, and its contribution to my country's "carbon neutrality" and "carbon peak" will become more and more obvious.

With the continuous improvement of related technologies for wading optics, managing the ocean requires the support of related information technologies such as the Internet of Things and multi-modal cognitive computing. The Internet of Things technology provides important information for the acquisition and transmission of wading optical data. Technical means and multi-modal cognitive computing provide strong support for comprehensive and efficient intelligent processing of water-related optical information, realize the mining of water-related optical big data, efficient information transmission and intelligent signal processing, and improve the informatization of related technologies in the water-related field. and intelligence to provide reliable technical support for the construction of a maritime power.

The above is the detailed content of A hurdle that cannot be overcome in underwater visual intelligence research: wading optics. For more information, please follow other related articles on the PHP Chinese website!