After autonomous driving hardware is pre-embedded, how do car companies realize data closed loop?

Hardware pre-embedding is a new model adopted by car OEMs in the trend of autonomous driving. That is, by pre-embedding hardware in the car, subsequent cars only need to continuously upgrade the software, users can get a better user experience while reducing the pressure on OEMs to put their models on the market. There is a necessary connection between hardware pre-embedding and OTA remote upgrade. After talking about hardware pre-embedding, we will talk about the principle of OTA.

# Before that, let’s first discuss a topic that neither hardware pre-embedding nor OTA can get rid of – how is the data closed loop achieved during research and development by car companies?

As for hardware pre-embedding and OTA, when you go out and turn left, you can see the content sorted out before the stern.

Getting back to the topic, back to the topic of this issue: After the automatic driving hardware is pre-embedded, how do car companies realize data closed loop?

Only 5 steps to achieve data closed loop

NI proposes a 5-step method for data collection/processing to create a closed-loop data loop:

• Road test
• Big data management
• Scenario virtualization
• Data open-loop reinfusion
• Data Closed loop

From data collection to data closed loop, just 5 steps

1. Road test - Data Record System AD

Road test is an important source of collecting real scene data, whether it is Both in-vehicle communication and vehicle-to-road communication require processing large amounts of data. Automatic test systems generally need to be equipped with data acquisition synchronization and storage equipment.

What types of sensors are needed for self-driving cars?

Self-driving cars are autonomous vehicles that integrate sensing, decision-making, and control functions. Among them, the sensing system replaces the human driver’s functions of sight, hearing, touch, etc., and integrates cameras Massive traffic environment data collected by sensors such as radar and radar can accurately identify various traffic elements and provide support for autonomous vehicle decision-making systems.

1. Camera

Cameras are divided into two categories according to video collection methods: digital cameras and analog cameras.

Digital cameras are generally used in vehicles, which can convert analog video signals generated by video capture equipment into digital signals, and then store them in the computer.

Analog cameras can only convert the captured video signals into digital mode through a specific video capture card, and then compress them before they can be converted to a computer for use.

For example, the landline phone we use is an analog signal, which easily produces noise during the call (current sound or inaudible clear) situation. In order to maintain good call quality, our mobile phones digitize the analog signals of the phone, so the call quality between mobile phones is very clear. Using the same principle, using a digital camera can effectively reduce the noise in the image and improve the imaging effect.

Summary in one sentence: Analog video signals can have infinitely many different values ​​within a certain time range. The digital video signal is formed by sampling, quantizing and encoding based on the analog signal. Analog signals are prone to signal noise and interference and have been gradually replaced by digital signals.

## Picture source: Bosch official website

2. Radar

Radar can actively detect the surrounding environment and is less affected by the external environment than visual sensors. It is one of the important sensors for autonomous vehicles. Radar emits electromagnetic waves to the target and receives the echo, thereby obtaining data such as target distance, orientation, and distance change rate. According to the electromagnetic wave band, radar can be subdivided into three categories: lidar, millimeter wave radar and ultrasonic radar.

NI provides a modular data acquisition solution based on the PXI platform, which can meet the needs of multiple cameras, vehicle Ethernet, CAN/CANFD, GPS, etc. Data collection can also meet the data collection needs of continuous stages. For example, the ADAS domain controller is basically in the stage of research and development and iteration. You can use the databypass collection method to use NI test equipment as a middleware to collect data back from the middle, so that there will be no Destroy the data link from the sensor to the ECU. In addition, NI's solution supports data storage capacities of 15TB~200TB.

# In order to ensure the accuracy of data collection, the vehicle used to record the data needs to be equipped with sensors and measurement technology that meet its specific level of intelligent driving. The main sensor types include cameras, millimeter wave radar and lidar. At the heart of the system is the fusion controller, which acquires data from all sensors and calculates a model of the current environment in real time, which is then used to control the drive, steering and braking systems.

High-performance data acquisition systems that meet the needs of various sensors, ECUs and communication data recording need to be specially designed for continuous operation and provide highly reliable RAID storage devices that can measure cameras, millimeter waves Radar, lidar raw data, with test fusion controller, also includes additional reference camera video data and inertial navigation system data that provide the precise position of the vehicle and vehicle Ethernet, CAN, CANFD, FlexRay and many other bus data.

2. Big data management - DIAdem

In terms of process, big data management is divided into There are three parts for fleet management, data center and terminal data usage. In the process of transmitting data from the vehicle to the data center, backhaul methods such as data migration are commonly used, and the data volume can reach 10-100TB/each vehicle/day. In turn, the data center can also monitor the vehicle in real time, which is mainly done through the vehicle's 4G or 5G network.

For vehicles at different levels, the amount of data that needs to be collected also increases exponentially. Taking the currently common L3 stage as an example, with the introduction of 4K ultra-high-definition cameras, 128-line lidar and other sensors, the amount of data recorded by the data collection system for 8 hours a day is as high as 30TB.

The collection system needs to provide high-bandwidth and high-capacity data storage, and it also needs to consider how to transmit the data to the data center in a simple way. For example, data is transmitted to a data center with a capacity of PB through a dedicated data upload machine.

In addition to these cloud-based tools, here is another data mining artifact-DIAdem. It can be deployed in the cloud to implement corresponding data visualization and data mining for road test data, multi-source data images, point clouds and some bus data. DIAdem can provide 200 data plug-ins, which are compatible with various types of data formats from test manufacturers, such as ASC, MDF4, TTL, MAT, etc.

3. Scene virtualization - monoDrive

After collecting data through road test , it is necessary to clean and classify the data, select scenarios, and finally combine real road tests and virtual simulation tests. monoDrive is such a tool that can achieve high-fidelity sensor physical modeling and scene semantic segmentation. It also supports cloud simulation functions and can deploy a large number of test cases to the cloud to accelerate the progress of simulation. Another important advantage of monoDrive is the extended functionality of automatic generation of realto virtual scenes.

It is worth mentioning that during the process of scene reconstruction or sensor modeling, NI can label real sensor data and virtual sensor data accordingly.

#A tool that converts real vehicle data into simulation scenarios. Based on the accumulation of large amounts of real vehicle data, a data-driven R&D model is implemented.

##▲ ‍‍ Collect scene conversion effects

The tool chain supports the arrangement and combination of huge amounts of dynamic data and static data, solving the problem of large deviations between case design and actual conditions, allowing autonomous driving Simulation simulation is closer to the real world.

▲ ‍‍Semantic Scene Transformation Toolchain

4. Data open-loop recirculation - system architecture based on PXI platform

What this step needs to do is to convert the original data Backfeeding into the ADAS perception software, part of it is directly feeding back into the software stack to do some software backfeeding, mainly for the testing part of some models. Another part is directly fed back to the real ECU, which can more realistically reproduce some of the situations we discovered during the road test.

The composition of NI system architecture: user data center, Replay PC based on Linux platform, NI PXI platform

#In order to form a data closed loop and fully utilize the value of original data, more and more car companies are building such a clustered data feedback system. When many car companies define the AEB function, they usually require that the AEB function must not trigger falsely for 100,000 kilometers or 200,000 kilometers.

Give me an example——

#If you run 80,000 kilometers during the road test, something suddenly happens. If a false trigger occurs, then you need to go back and modify the corresponding software. After modifying the software, do I still need to run another 100,000 kilometers? Obviously doing it all over again would waste a lot of time and experience.

Faced with such a challenge, what are the effective methods?

Wang Shuai explained: “If something goes wrong during the first road test at 80,000 kilometers, we can ensure that all the previous mileage data is recorded. Then once the problem is discovered, we will iteratively modify the software. After modifying the software, we will use the original data collected to do a regression test to test whether the software can function normally on the original data after modification. operation. This reduces our investment in road testing, thus speeding up development."

5. Closed-loop testing - SIL and HIL

Closed-loop testing usually includes software-in-the-loop (SIL) and hardware-in-the-loop (HIL). In terms of software-in-the-loop, NI's solution is to use the monoDrive platform to provide an environment that can be deployed in batches on the cloud. In terms of hardware-in-the-loop, NI's solution is to use the PXI platform and NI's advantages in cameras, vehicle buses, and data synchronization to implement a closed-loop system with data injection functions for multiple types of sensors.

The above is the detailed content of After autonomous driving hardware is pre-embedded, how do car companies realize data closed loop?. For more information, please follow other related articles on the PHP Chinese website!