Infrared μLED lighting lens structure method to achieve any given direction in Microsoft AR/VR patent

According to a report from Yingwei.com on January 2, 2024, eye tracking technology for XR headsets is developing rapidly. This technology can track eye movements and determine the direction of a user's gaze by analyzing the direction of the user's eyes and the infrared light signals reflected from the eyes. For example, a common technology is to install one or more infrared light sources on a head-mounted display to emit infrared light toward the user's eyes from different directions. The sensor detects the infrared light (flicker) reflected from the user's eyes and is used to determine the XYZ position and gaze direction of the user's eyes.

In order to improve the accuracy and processing power of infrared eye tracking systems, it is usually necessary to place several different infrared light sources around the user's eyes. These light sources produce different flicker effects on the cornea. By analyzing these flickering effects, the system can more accurately track the user's eye movements. This method can improve the tracking accuracy of the system and be able to handle more complex eye movement behaviors.

Unfortunately, traditional XR systems have some limitations when using infrared light sources for positioning. Typically, traditional systems place the infrared light source on the edge of the headset or other structure where the display lens and screen are mounted. However, this peripheral position is not always the best option, so it is necessary to add more light sources or use larger, more powerful light sources. If the light source could be placed closer or better relative to the user's eye iris, no additional light source would be needed.

In order to solve the problem of visual obstruction from the user's perspective in the mixed reality environment, the XR system tried a new method, which is to place the infrared light source away from the edge of the display and close to the eyes in the user's field of view. However, there are problems with this existing system because the size of the infrared light source is usually in the 1.0 mm to 4.0 mm diameter/width range, which is very visible in the user's field of view. Therefore, in order to reduce the impact on the user's viewing angle, most conventional systems only position the infrared light source at a sub-optimal location at the peripheral edge of the lens/display.

In response to this problem, Microsoft filed a patent application called "Microled based invisible illumination for eye tracking", which provides a solution. According to Microsoft, this invention allows infrared light sources to be directly illuminated in the viewing area of ​​the head-mounted display lens in the optimal direction without interfering with the user's perception of the mixed reality environment. This solution is also capable of locating the user's eyes/iris and corresponding camera sensors to power eye tracking technology.

The company has proposed a method to create illumination lens structures using infrared μLED light sources smaller than 100 μm. This μLED is embedded into the lens of the headset, making its presence invisible to the user. Therefore, during use, this μLED will not cause any obstruction or interference to the presentation of the mixed reality environment.

Now let's pay attention to Figure 1. The configuration of the head-mounted device is based on the camera 130 or other sensors capturing the reflection of light for eye tracking. During use, the light source emits a flash of light around the user's eyes. Once the light is emitted, it reflects back from the user's eyes (specifically the iris) and is detected by the camera.

By sensing the intensity and timing of light intensity relative to the light source, the headset light processing module is able to accurately detect the position and direction of the user's eyes and iris.

The system can differentiate between the user's pupils and irises by performing additional processing on the image captured by the camera/sensor. This processed image can help the system determine the position of the user's eyes as well as the direction and gaze of the user's eyes relative to the projected hologram or other object. Using the position information of the user's eyes, the system can also position where needed and re-project the hologram to support the presentation of mixed reality environments.

FIG. 2 shows a representation of a user's eye 200 with an infrared light source 210 emitting infrared light 220 toward the user's eye 200 . Infrared light is reflected back as specular and diffuse reflections. Figure 2 also shows how a camera 250 or other sensor may be positioned to detect one or more reflections.

The system can determine the relative positioning of the user's eyes/iris through the position of the light source, the time the light source emits light, the position of the camera, and the measured intensity and time of the detected light reflections from the user's eyes. This is because light reflects differently in different parts of the user's eye, for example it reflects differently in the pupil and iris areas of the cornea than in the sclera. The detection and measurement of this difference is based on whether the reflection is specular or diffuse.

In order for the light source to be optimally positioned, such as close to the user's cornea, the light source needs to be positioned at the appropriate location. Unfortunately, traditional infrared light sources are too large (e.g. 1-4 mm) to be positioned within the user's field of view without obstructing the user's view of the environment through the lens.

To help solve these problems, Microsoft proposes the use of lighting lenses configured with infrared μLEDs. With this configuration, the light source can be optimally placed close to the user's eyes without having to consider existing limitations imposed by the physical form factor of the headset mounting structure.

Figure 3 shows an infrared μLED lens structure 300 with 16 infrared μLEDs arranged in a grid. The infrared μLED is placed over a transparent substrate 310, forming a circuit along conductive traces 320 between the anode and cathode terminals. For example, when the circuit is powered by the headset's power supply, which can be electrically connected to the anode and cathode terminals and controlled by a lighting control unit, the infrared μLED will activate and emit infrared light.

The luminous wavelength of infrared μLED is between 790 μm ~ 1mm. In preferred embodiments, the infrared μLED emits light at a wavelength of approximately 800-900 μm, even more preferably at a wavelength of approximately 850 μm.

Figures 4A-4D show different lighting lens structures with different numbers of infrared μLEDs. Specifically, the distribution pattern of the infrared uLEDs is somewhat circular/elliptical in Figure 4A, but somewhat rectangular in Figure 4C, and somewhat diamond-shaped in Figure 4D. On the other hand, the embodiment shown in Figure 4B includes two different patterns, a circular/oval outer pattern and a triangular inner pattern.

In addition, the lighting lens structure 400A is composed of 6 infrared uLEDs, while the lighting lens structures 400C and 400D are each composed of 8 infrared uLEDs. Illumination lens structure 400B contains 19 infrared uLEDs.

Different quantities and patterns of infrared uLEDs can be changed to suit different needs and preferences.

In Figure 5, the head display 500 is configured with a transparent lighting lens structure 510, and the lens structure 510 has a plurality of infrared μLEDs 540. As shown in the figure, the infrared μLEDs 540 are distributed in a circular shape, 8 in each eye and lens area. Different IR μLEDs 540 can be selected to be connected in a single circuit or in two or more different circuits. The electrical traces forming the circuit are not shown.

During use, the light emitted from the infrared uLED will be directed at least partially toward the user's eyes, and the light will be reflected back and detected by the device camera 530.

Figure 6 shows an embodiment of a pair of glasses 600 having a first transparent illuminating lens structure 610 including four infrared uLEDs and a second transparent illuminating lens structure 620 including eight infrared uLEDs.

This example is used to illustrate that there does not necessarily have to be matching/symmetrical infrared uLED distribution on both sides of the headset. Regardless of the number and distribution of infrared uLEDs, it should be understood that during use, light emitted from the infrared uLEDs will be directed at least partially toward the user's eyes, and the light may be reflected back and detected by the device camera 630.

In a related embodiment, the head-mounted display system 700 shown in FIG. 7 includes a transparent illumination lens structure 710 with a sun visor. In this example, one side of the visor has several infrared uLEDs forming a circular pattern. The other side of the visor has four infrared uLEDs arranged in a square pattern.

Regarding all the previous examples, the infrared uLED does not appear proportional. In fact, IR uLEDs are so small (

Also because infrared LEDs are so small, they can be used to illuminate the user's eyes with infrared light while being positioned in the lens the user is passing through without obstructing the user's field of view through the lens.

The conductive traces are very thin, with a width less than 50 μm, or even less than 25 μm, so they are not visually noticeable and basically invisible when used close to the user's eyes. This configuration is particularly advantageous in enabling the trace to be positioned inside the illumination lens structure. Even if it is positioned directly in front of the user's eyes, it will not hinder the user's view of the environment through the lens during use.

Figure 8 visualizes the manufacturing process used to manufacture the illumination lens structure of the invention.

As shown, the fabrication process includes obtaining a wafer 810 containing one or more infrared μLEDs. For example, wafer 810 may be an epitaxial wafer formed by an epitaxial growth or deposition process.

The process 800 shown in Figure 8 also includes obtaining a substrate 820 onto which the infrared uLED is transferred. As shown, a transfer process 830 is performed to transfer one or more infrared uLEDs to conductive traces 825 that are already on a substrate 820 and form one or more different circuits 827 on the substrate.

The size of an IR μLED removed from and placed on a substrate is limited to

The maximum size of infrared uLED can be less than 75 μm, less than 50 μm, or even less than 20 μm. In one embodiment, the maximum dimension of the infrared uLED is approximately 10 μm.

The width of the trace is also limited, and the thickness cannot be greater than

Figure 9 illustrates a flow diagram 900 configured for performing eye tracking, wherein the head-mounted display includes an illumination lens containing a plurality of infrared μLEDs, and each of the plurality of infrared μLEDs has a

System components control the illumination of infrared uLEDs, emitting infrared light from one or more infrared uLEDs in an illumination lens toward the user's eyes.

Next, the headset is further configured to detect and process flickers of infrared light reflected from the user's eyes during use of the headset, and determine the positioning of the user's eyes based on the detected and processed flickers.

Attention now turns to Figure 10, which illustrates a flow chart 1000 for fabricating an illumination lens structure using multiple infrared uLEDs.

This includes applying multiple traces to a transparent backplane for obtaining a transparent substrate. A plurality of the traces are electrically conductive and form at least one electrical circuit between the anode terminal and the cathode terminal.

Then, an IR μLED wafer is obtained that contains multiple IR μLEDs or material that can be individually extracted into discrete IR μLEDs with a maximum size of

Related Patents: Microsoft Patent | Microled based invisible illumination for eye tracking

The Microsoft patent application titled "Microled based invisible illumination for eye tracking" was originally submitted in January 2023 and was recently published by the US Patent and Trademark Office.

It should be noted that, generally speaking, after a U.S. patent application is reviewed, it will be automatically published 18 months from the filing date or priority date, or it will be published within 18 months from the filing date at the request of the applicant. Note that publication of a patent application does not mean that the patent is approved. After a patent application is filed, the USPTO requires actual review, which can take anywhere from 1 to 3 years.

The above is the detailed content of Infrared μLED lighting lens structure method to achieve any given direction in Microsoft AR/VR patent. For more information, please follow other related articles on the PHP Chinese website!