Apple patent reveals ideas for various AR gaming entertainment activities in self-driving cars

Automobile manufacturers are actively developing and applying augmented reality (AR) technology to cars. In fact, if you pay attention to the patent sharing on Yingwei.com, you will find that Apple has been exploring this field and has applied for a number of related patents

Recently, the US Patent and Trademark Office announced an Apple invention called "Visual content overlay system", which is related to AR cars. In short, this is an AR visual content overlay system that can overlay various digital elements on the transparent windshield of a car, while also blurring and replacing the view of other objects

Through the AR visual content overlay system, the driver's situational awareness can be improved, driving efficiency improved, driving fun increased, while reducing the risk of the vehicle deviating from the current lane and reducing the possibility of accidents

Apple’s AR system can not only be used in self-driving cars, but can also provide occupants with various AR-based entertainment activities, such as games

In Figure 1 a car is shown. The vehicle is labeled 110, is located in an external environment 100, and is equipped with a vehicle navigation system (VNS) 140. Apple notes that, in one instance, VNS 140 may automatically control one or more vehicle control elements 150 such that vehicle 110 can autonomously navigate along one or more driving routes in environment 100

In one embodiment, VNS 140 displays information to one or more occupants through one or more interfaces 130 . At least one interface 130 may include a transparent surface and may display information to the occupant in the form of an augmented reality (AR) graphics overlay

The vehicle 110 is composed of one or more sets of external sensors 116 and one or more sets of internal sensors 118 for detecting various data information

Figure 2A-B shows a vehicle equipped with a transparent surface to provide an augmented reality display

In one embodiment, the AR display elements may provide information about vehicle speed in three-dimensional form, including vehicle speed and local speed limits, route instructions, sign translation, etc.

The AR visual content overlay system can be selectively generated based on personal data, driving routes and environmental conditions, as shown in Figure 3A-B

For example, by monitoring the environmental conditions obtained by the sensor 340, the augmented reality display system can determine the current environmental conditions. If it is determined that the current environmental conditions are heavy rain, the augmented reality system can compare the determined environmental conditions with the speed limit adjustment database and related environmental conditions to determine a 5 mph reduction in the local speed limit

As shown in Figure 3A, the speed limit can be adjusted from 352 to 362 to reflect monitoring of local environmental conditions

At the same time, the system can highlight specific content based on the driving route and blur irrelevant content. As shown in FIG. 3B , based on the determined content 372 associated with the current driving route navigated by the vehicle 300 , such as destinations of the driving route, checkpoints of the driving route, etc., the augmented reality display system included in the vehicle 300 may generate highlights. Specific content, while blurring irrelevant content 374, limiting the perceived content of road signs 370 to relevant content 372

In one embodiment, the AR visual content overlay system can control content display based on the occupant's personal data. For example, it may be determined that the occupant has sufficient experience with the vehicle navigating along a particular travel route, so it may be determined that the occupant 610 does not require a visual indication that the vehicle 600 is navigating along a particular travel route

For example, it may be determined that the passenger has insufficient experience navigating vehicle 600 on a specific driving route, and therefore it may be determined that passenger 610 requires AR route navigation

In one embodiment, the display of visual content may be controlled based on the occupant's stress level, cognitive load, etc. For example, upon determining that occupant 610 has high cognitive load, the augmented reality display system may remove elements 662, 664, thereby reducing the cognitive load of occupant 610

When it is determined that the occupant 610 has a high stress level, the augmented reality display system may provide specific content. For example, it may display elements 662, 664 related to vehicle control, provide the occupant 610 with an indication of the navigation actions being performed during vehicle navigation, and explain to the occupant 610 the assurance that the vehicle is navigating along the route of travel to the destination.

In one embodiment, the AR visual content overlay system can simulate environmental objects. As shown in Figures 7A-B, the system can simulate the presence of "speed bumps" at specific locations 790 on the road, thereby enhancing the occupant's perception of the speed bumps. Occupants sometimes do not necessarily notice the presence of speed bumps due to attention issues and the fact that speed bumps are sometimes not obvious or low-profile

At the same time, the system can also enhance the driver's awareness of the surrounding situation by highlighting lane boundaries, thereby reducing the risk of the vehicle deviating from the current driving lane. As shown in the figure, when the light level in the environment 721 is lower than the threshold, the augmented reality display system can adjust the color, brightness and other parameters of the elements 781 and 783 to help the occupant 710 more easily perceive the lane boundary

The augmented reality display system can dynamically adjust the color, brightness, pattern, animation sequence, etc. of individual elements based on the relative distance between the vehicle and the lane boundary to indicate that the vehicle is approaching the lane boundary

In one embodiment, the AR visual content overlay system can determine and identify areas of the surrounding environment and provide warnings to the occupants. As shown in FIG. 8 , an augmented reality display system included in vehicle 810 may identify region 820 based on detecting structure 822 in environment 800 and determining that structure 822 is associated with one or more specific regions.

For example, the augmented reality display system may identify that the detected structure 822 is a structure of a school, and may determine, in part, that the structure 822 is associated with a school area 820 in the environment 800 based on determining that the structure 822 is a school. connection, and can generate relevant prompt information accordingly

In one embodiment, the AR visual content overlay system can provide various AR-based entertainment activities for the occupants, such as games or film and television programs. As shown in Figures 9 and 10, based on receiving commands and as a result of occupant 1010 interacting with a user interface contained in vehicle 1000, video content may be played

The augmented reality display system can generate display elements and present video content on a transparent surface that can be perceived by the occupants

In another example, an augmented reality display system can generate various display elements as part of an interactive game and provide input based on the occupant's movements, gestures, and interactions with one or more user interfaces

Related Patents

: Apple Patent | Visual content overlay system Apple submitted a patent application called "Visual Content Overlay System" in April 2023, and it was recently published by the U.S. Patent and Trademark Office

The above is the detailed content of Apple patent reveals ideas for various AR gaming entertainment activities in self-driving cars. For more information, please follow other related articles on the PHP Chinese website!