


Touch has never been so real! Two Chinese Ph.D.s from the University of Southern California innovate the 'tactile perception' algorithm
The development of electronic technology allows us to enjoy an "audio-visual feast" anytime and anywhere, and human hearing and vision have been completely liberated.
#In recent years, adding "haptics" to devices has gradually become a new research hotspot, especially with the blessing of the concept of "metaverse". The high sense of touch will undoubtedly greatly enhance the realism of the virtual world.
Current tactile sensing technology mainly simulates and renders touch through a "data-driven" model. The model first records the user's interaction with the real texture. The signal is then input to the texture generation part, and the touch sensation is "played back" to the user in the form of vibration.
# Some recent methods are mostly based on user interactive motion and high-frequency vibration signals to model texture features, such as friction and microscopic surface features.
Although data-driven greatly improves the realism of simulation, there are still many limitations.
#For example, there are "countless types" of textures in the world. If each texture was recorded, the manpower and material resources required would be unimaginable. It also cannot meet the needs of some niche users.
Human beings are very sensitive to touch. Different people have different feelings for the same object. The data-driven approach cannot fundamentally eliminate the problem of touch. Texture recording to texture rendering perceptual mismatch issue.
Recently, three doctoral students at the University of Southern California Viterbi School of Engineering proposed a new "preference-driven" model framework that uses humans to resolve texture details. The ability to adjust the generated virtual perception can ultimately achieve quite realistic tactile perception. The paper was published in IEEE Transactions on Haptics.
Paper link : https://ieeexplore.ieee.org/document/9772285
The preference-driven model will first give the user a realistic touch texture, and then the model will use Dozens of variables randomly generate three virtual textures, from which the user can then choose the one that feels most similar to the real object.
With continuous trial and error and feedback, the model will continuously optimize the distribution of variables through search, making the generated texture closer to the user's preference. This method has significant advantages over directly recording and playing back textures, as there is always a gap between what the computer reads and how humans actually feel.
This process is actually similar to "Party A and Party B". As the perceiver (Party A), if we feel that the touch feels wrong, we will Go back and let the algorithm (Party B) modify and regenerate until the generated effect is satisfactory.
This is actually very reasonable, because different people will have different feelings when touching the same object, but the signal released by the computer is the same, so It is necessary to customize the touch according to each person!
The entire system consists of two modules. The first is a deep convolutional generative adversarial network (DCGAN), which is used to map the vectors of the latent space to the texture model. In the UPenn Haptic Texture Toolkit (HaTT) for training.
The second module is a comparison-based evolutionary algorithm: from a set of generated texture models, the covariance matrix adaptive evolution strategy (CMA-ES) will create a new texture model based on user preference feedback. evolution.
To simulate real textures, the researchers first ask users to touch real textures using custom tools, and then use a haptic device to touch a set of virtual texture candidates, where haptic Feedback is transmitted via Haptuator connected to the device's stylus.
The only thing the user needs to do is to select the virtual texture that is closest to the real texture and use a simple slider interface to adjust the amount of friction. Because friction is critical to texture feel, it can also vary from person to person.
Then all virtual textures will be updated according to the evolution strategy according to the user's selection, and then the user will select and adjust again.
Repeat the above process until the user finds a virtual texture that they believe is close to the real texture and saves it, or a closer virtual texture cannot be found .
The researchers divided the evaluation process into two phases, each with a separate group of participants.
#In the first stage, five participants generated and searched virtual textures for 5 real textures respectively.
The second stage is to evaluate the gap between the final saved preference-driven texture (VTp) and its corresponding real texture (RT).
The evaluation method mainly uses adjective rating to evaluate perceptual dimensions including roughness, hardness and smoothness.
#And compare the similarities between VTp, RT and data-driven textures (VTd).
#The experimental results also show that following the evolutionary process, users can effectively find a virtual texture model that is more realistic than the data-driven model.
In addition, more than 80% of participants believed that the virtual texture ratings generated by the preference-driven model were better than those generated by the data-driven model.
Haptic devices are becoming more and more popular in video games, fashion design, and surgical simulations, and even at home, we are starting to see users using Laptops are as popular as those tactile devices.
#For example, adding touch to first-person video games will greatly enhance the player’s sense of reality.
The author of the paper stated that when we interact with the environment through tools, tactile feedback is just one form, one kind of sensory feedback, and audio is another sensory feedback, both are very important.
#In addition to games, the results of this work will be particularly useful for virtual textures used in dental or surgical training, which need to be very accurate.
"Surgical training is absolutely a huge field that requires very realistic textures and tactile feedback; decoration design also requires a high degree of precision in texture during development Simulate it on the ground and then manufacture it."
Everything from video games to fashion design is integrating haptic technology, and existing virtual texture databases can be improved with this user-preference approach.
Texture search models also allow users to extract virtual textures from databases, such as the University of Pennsylvania's Tactile Texture Toolkit, which can be refined until they are obtained desired result.
#Once this technology is combined with the texture search model, you can use virtual textures that have been recorded by others before, and then optimize the textures based on strategies.
#The author imagines that in the future, models may not even need real textures.
The sense of some common things in our lives is very intuitive, and we are hard-wired to fine-tune our senses by looking at photos without having to Reference to real textures.
For example, when we see a table, we can imagine how it will feel once we touch the table, using our prior knowledge of the surface. With experience knowledge, you can provide visual feedback to users and allow them to select matching content.
The first author of the article, Shihan Lu, is currently a doctoral candidate at the School of Computer Science at the University of Southern California. He has previously done sound-related work in immersive technology. , which makes virtual textures more immersive by introducing matching sounds when tools interact with them.
The second author of the article, Mianlun Zheng (Zheng Mianlun), is a doctoral candidate in the School of Computer Science at the University of Southern California. He graduated from Wuhan University with bachelor's and master's degrees. .
##
The above is the detailed content of Touch has never been so real! Two Chinese Ph.D.s from the University of Southern California innovate the 'tactile perception' algorithm. For more information, please follow other related articles on the PHP Chinese website!

In John Rawls' seminal 1971 book The Theory of Justice, he proposed a thought experiment that we should take as the core of today's AI design and use decision-making: the veil of ignorance. This philosophy provides a simple tool for understanding equity and also provides a blueprint for leaders to use this understanding to design and implement AI equitably. Imagine that you are making rules for a new society. But there is a premise: you don’t know in advance what role you will play in this society. You may end up being rich or poor, healthy or disabled, belonging to a majority or marginal minority. Operating under this "veil of ignorance" prevents rule makers from making decisions that benefit themselves. On the contrary, people will be more motivated to formulate public

Numerous companies specialize in robotic process automation (RPA), offering bots to automate repetitive tasks—UiPath, Automation Anywhere, Blue Prism, and others. Meanwhile, process mining, orchestration, and intelligent document processing speciali

The future of AI is moving beyond simple word prediction and conversational simulation; AI agents are emerging, capable of independent action and task completion. This shift is already evident in tools like Anthropic's Claude. AI Agents: Research a

Rapid technological advancements necessitate a forward-looking perspective on the future of work. What happens when AI transcends mere productivity enhancement and begins shaping our societal structures? Topher McDougal's upcoming book, Gaia Wakes:

Product classification, often involving complex codes like "HS 8471.30" from systems such as the Harmonized System (HS), is crucial for international trade and domestic sales. These codes ensure correct tax application, impacting every inv

The future of energy consumption in data centers and climate technology investment This article explores the surge in energy consumption in AI-driven data centers and its impact on climate change, and analyzes innovative solutions and policy recommendations to address this challenge. Challenges of energy demand: Large and ultra-large-scale data centers consume huge power, comparable to the sum of hundreds of thousands of ordinary North American families, and emerging AI ultra-large-scale centers consume dozens of times more power than this. In the first eight months of 2024, Microsoft, Meta, Google and Amazon have invested approximately US$125 billion in the construction and operation of AI data centers (JP Morgan, 2024) (Table 1). Growing energy demand is both a challenge and an opportunity. According to Canary Media, the looming electricity

Generative AI is revolutionizing film and television production. Luma's Ray 2 model, as well as Runway's Gen-4, OpenAI's Sora, Google's Veo and other new models, are improving the quality of generated videos at an unprecedented speed. These models can easily create complex special effects and realistic scenes, even short video clips and camera-perceived motion effects have been achieved. While the manipulation and consistency of these tools still need to be improved, the speed of progress is amazing. Generative video is becoming an independent medium. Some models are good at animation production, while others are good at live-action images. It is worth noting that Adobe's Firefly and Moonvalley's Ma

ChatGPT user experience declines: is it a model degradation or user expectations? Recently, a large number of ChatGPT paid users have complained about their performance degradation, which has attracted widespread attention. Users reported slower responses to models, shorter answers, lack of help, and even more hallucinations. Some users expressed dissatisfaction on social media, pointing out that ChatGPT has become “too flattering” and tends to verify user views rather than provide critical feedback. This not only affects the user experience, but also brings actual losses to corporate customers, such as reduced productivity and waste of computing resources. Evidence of performance degradation Many users have reported significant degradation in ChatGPT performance, especially in older models such as GPT-4 (which will soon be discontinued from service at the end of this month). this


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Atom editor mac version download
The most popular open source editor

VSCode Windows 64-bit Download
A free and powerful IDE editor launched by Microsoft

Zend Studio 13.0.1
Powerful PHP integrated development environment

SublimeText3 English version
Recommended: Win version, supports code prompts!

Notepad++7.3.1
Easy-to-use and free code editor
