search
HomeTechnology peripheralsAIComputer vision technology is about to undergo a major transformation

Will computer vision reinvent itself again?

Ryad Benosman, a professor of ophthalmology at the University of Pittsburgh and an adjunct professor at CMU’s Robotics Institute, thinks so. As one of the founders of event-based vision technology, Benosman expects neuromorphic vision—computer vision based on event-based cameras—to be the next direction in computer vision.

"Computer vision has been reinvented many, many times," Benosman said. “I’ve seen it reinvented at least twice.”

Benosman cited the shift from image processing with a bit of photogrammetry to geometry-based methods in the 1990s, and then today’s rapid advances in machine learning . Despite these changes, modern computer vision technology is still primarily based on image sensors—cameras that produce images similar to those seen by the human eye.

According to Benosman, the image sensing paradigm will hinder innovation in alternative technologies until it is no longer useful. The development of high-performance processors (e.g., GPUs) delays the need to find alternative solutions, thus prolonging this impact.

"Why do we use images for computer vision? That's the million-dollar question," he said. "We have no reason to use images - it's just because of historical momentum. Even before there were cameras, images had momentum." Image cameras have been around since the 1500s, with artists using room-sized devices to trace images of people or scenery outside a room onto canvas. Over the years, the paintings were replaced with film to record the images. Innovations such as digital photography eventually made it easy for image cameras to become the basis of modern computer vision technology.

However, Benosman believes that computer vision technology based on image cameras is extremely inefficient. His analogy was the defense system of a medieval castle: guards positioned around the walls looked around for approaching enemies. The drummer beat steadily, and with every beat, each guard shouted out what they saw. How easy is it to overhear a guard spotting an enemy at the edge of a distant forest amid all the commotion?

The 21st century hardware equivalent of a drum beat is an electronic clock signal, and the guards are pixels. A large amount of data is created and must be checked every clock cycle, which results in a large amount of redundant information and thus requires a lot of unnecessary calculations.

"People are burning so much energy, it's taking up Castle's entire computing power to protect themselves," Benosman said. If an interesting event is discovered - represented by the enemy in this analogy - "you have to move around collecting useless information, people are screaming everywhere, so there is a lot of bandwidth... Now imagine you have a complex castle. All These people all have to be heard."

Enter Neuromorphic Vision. The basic idea is inspired by the way biological systems work, which is to detect changes in scene dynamics rather than continuously analyzing the entire scene. In our castle analogy, this means keeping the guards quiet until they see something of interest, then calling out their location to raise the alarm. In electronic form, this means letting individual pixels determine whether they see something relevant.

"Pixels can decide for themselves what message they should send," Benosman said.

"Instead of getting system information, they can look for meaningful information - features. That's what makes the difference."

Prophesee vs. DVS sensor evaluation kit developed in collaboration with Sony. Benosman is the co-founder of Prophesee.

Computer vision technology is about to undergo a major transformationThis event-based approach can save significant power and reduce latency compared to fixed-frequency system acquisition.

"You want something more adaptive, and that's what the relative changes [event-based vision] gives you - adaptive acquisition frequency," he said. "When you look at amplitude changes, if something is moving very fast, we're going to get a lot of samples. If something's not changing, you're going to get almost zero, so you're adjusting your acquisition frequency based on the dynamics of the scene . That's what it brings. That's why it's a good design."

Benosman entered the field of neuromorphic vision in 2000, convinced that advanced computer vision would never work because images were not The correct way.

“The biggest shift is to say that we can do vision without grayscale and without images, which was heretical in the late 2000s — totally heretical,” he said.

The technique Benosman proposed—the basis for today’s event-based sensing—was so different that papers submitted to the most important IEEE computer vision journal at the time were rejected without review. In fact, it wasn’t until the development of the Dynamic Vision Sensor (DVS) in 2008 that the technology began to gain momentum.

Neuroscience Inspiration

Neuromorphic technologies are technologies inspired by biological systems, including the ultimate computer: the brain and its neurons, or computational elements. The problem is that no one fully understands how neurons work. While we know that neurons respond to incoming electrical signals called spikes, until recently, researchers described neurons as rather hasty, assuming that only the number of spikes mattered. This hypothesis persisted for decades, but recent work has proven that the timing of these spikes is absolutely critical and that the brain is structured to create delays in these spikes to encode information.

Today’s spiking neural networks simulate the spikes seen in the brain and are simplified versions of the real thing—usually binary representations of the spikes. "I receive a 1, I wake up, I calculate, I sleep," Benosman explained. The reality is much more complex. When a spike arrives, the neuron starts integrating the value of the spike over time; neurons also leak, meaning the results are dynamic. Additionally, there are approximately 50 different types of neurons with 50 different integration profiles.

Current electronic versions lack integrated dynamic paths, connectivity between neurons, and different weights and delays. "The problem is that to make a product that works, you can't [imitate] all the complexity because we don't understand it," he said. "If we had a good theory of the brain, we would solve it. The problem is, we just don't know."

Bensoman runs a unique lab dedicated to understanding the mathematics behind cortical computation, aiming to before creating new mathematical models and replicating them into silicon devices. This involves direct monitoring of spikes from real retinas.

Currently, Bensoman opposes faithfully replicating biological neurons, calling the approach outdated.

"The idea of ​​replicating neurons in silicon came about because people looked at transistors and saw a mechanism that looked like a real neuron, so there was some thought behind it in the beginning," he said. "We don't have cells; we have silicon. You need to adapt your computing substrate, not the other way around... If I know what I'm computing and I have the chip, I can optimize this equation and do it at the lowest cost, lowest power consumption, The lowest latency to run it."

Processing Power

The realization that exact replicas of neurons are not needed, and the development of DVS cameras, are the driving forces behind today's vision systems. While systems are already commercially available, progress is needed before fully human-like vision can be used commercially.

Benosman said the original DVS cameras had "large, thick pixels" because the components surrounding the photodiodes themselves greatly reduced the fill factor. While investments in developing these cameras have accelerated the technology, Benosman made it clear that today's incident cameras are simply improvements on original research equipment developed back in 2000. The most advanced DVS cameras from Sony, Samsung and Omnivision have tiny pixels that incorporate advanced technologies like 3D stacking and reduce noise. Benosman's concern is whether the types of sensors used today can successfully scale.

"The problem is, once you increase the number of pixels, you get a lot of data because you're still very fast," he said. "You could probably still process it in real time, but you'd get too much relative change from too many pixels. That's killing everyone right now because they see the potential, but they don't have the right processors to support it. ”

Computer vision technology is about to undergo a major transformation

This Prophesee customer application example shows the difference between the image camera (upper left corner of each box) and DVS sensor output.

General purpose neuromorphic processors lag behind their DVS camera counterparts. Efforts by some of the industry’s biggest players (IBM Truenorth, Intel Loihi) are still ongoing. The right processor and the right sensor will be an unbeatable combination, Benosman said.

“[Today’s DVS] sensors are extremely fast, have ultra-low bandwidth, and have high dynamic range so you can see indoors and outdoors,” Benosman said. "This is the future. Is it going to take off? Absolutely." "Whoever can put the processor in there and deliver the full stack is going to win because it's going to be unbeatable," he added road.

The above is the detailed content of Computer vision technology is about to undergo a major transformation. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
2023年机器学习的十大概念和技术2023年机器学习的十大概念和技术Apr 04, 2023 pm 12:30 PM

机器学习是一个不断发展的学科,一直在创造新的想法和技术。本文罗列了2023年机器学习的十大概念和技术。 本文罗列了2023年机器学习的十大概念和技术。2023年机器学习的十大概念和技术是一个教计算机从数据中学习的过程,无需明确的编程。机器学习是一个不断发展的学科,一直在创造新的想法和技术。为了保持领先,数据科学家应该关注其中一些网站,以跟上最新的发展。这将有助于了解机器学习中的技术如何在实践中使用,并为自己的业务或工作领域中的可能应用提供想法。2023年机器学习的十大概念和技术:1. 深度神经网

人工智能自动获取知识和技能,实现自我完善的过程是什么人工智能自动获取知识和技能,实现自我完善的过程是什么Aug 24, 2022 am 11:57 AM

实现自我完善的过程是“机器学习”。机器学习是人工智能核心,是使计算机具有智能的根本途径;它使计算机能模拟人的学习行为,自动地通过学习来获取知识和技能,不断改善性能,实现自我完善。机器学习主要研究三方面问题:1、学习机理,人类获取知识、技能和抽象概念的天赋能力;2、学习方法,对生物学习机理进行简化的基础上,用计算的方法进行再现;3、学习系统,能够在一定程度上实现机器学习的系统。

超参数优化比较之网格搜索、随机搜索和贝叶斯优化超参数优化比较之网格搜索、随机搜索和贝叶斯优化Apr 04, 2023 pm 12:05 PM

本文将详细介绍用来提高机器学习效果的最常见的超参数优化方法。 译者 | 朱先忠​审校 | 孙淑娟​简介​通常,在尝试改进机器学习模型时,人们首先想到的解决方案是添加更多的训练数据。额外的数据通常是有帮助(在某些情况下除外)的,但生成高质量的数据可能非常昂贵。通过使用现有数据获得最佳模型性能,超参数优化可以节省我们的时间和资源。​顾名思义,超参数优化是为机器学习模型确定最佳超参数组合以满足优化函数(即,给定研究中的数据集,最大化模型的性能)的过程。换句话说,每个模型都会提供多个有关选项的调整“按钮

得益于OpenAI技术,微软必应的搜索流量超过谷歌得益于OpenAI技术,微软必应的搜索流量超过谷歌Mar 31, 2023 pm 10:38 PM

截至3月20日的数据显示,自微软2月7日推出其人工智能版本以来,必应搜索引擎的页面访问量增加了15.8%,而Alphabet旗下的谷歌搜索引擎则下降了近1%。 3月23日消息,外媒报道称,分析公司Similarweb的数据显示,在整合了OpenAI的技术后,微软旗下的必应在页面访问量方面实现了更多的增长。​​​​截至3月20日的数据显示,自微软2月7日推出其人工智能版本以来,必应搜索引擎的页面访问量增加了15.8%,而Alphabet旗下的谷歌搜索引擎则下降了近1%。这些数据是微软在与谷歌争夺生

荣耀的人工智能助手叫什么名字荣耀的人工智能助手叫什么名字Sep 06, 2022 pm 03:31 PM

荣耀的人工智能助手叫“YOYO”,也即悠悠;YOYO除了能够实现语音操控等基本功能之外,还拥有智慧视觉、智慧识屏、情景智能、智慧搜索等功能,可以在系统设置页面中的智慧助手里进行相关的设置。

人工智能在教育领域的应用主要有哪些人工智能在教育领域的应用主要有哪些Dec 14, 2020 pm 05:08 PM

人工智能在教育领域的应用主要有个性化学习、虚拟导师、教育机器人和场景式教育。人工智能在教育领域的应用目前还处于早期探索阶段,但是潜力却是巨大的。

30行Python代码就可以调用ChatGPT API总结论文的主要内容30行Python代码就可以调用ChatGPT API总结论文的主要内容Apr 04, 2023 pm 12:05 PM

阅读论文可以说是我们的日常工作之一,论文的数量太多,我们如何快速阅读归纳呢?自从ChatGPT出现以后,有很多阅读论文的服务可以使用。其实使用ChatGPT API非常简单,我们只用30行python代码就可以在本地搭建一个自己的应用。 阅读论文可以说是我们的日常工作之一,论文的数量太多,我们如何快速阅读归纳呢?自从ChatGPT出现以后,有很多阅读论文的服务可以使用。其实使用ChatGPT API非常简单,我们只用30行python代码就可以在本地搭建一个自己的应用。使用 Python 和 C

人工智能在生活中的应用有哪些人工智能在生活中的应用有哪些Jul 20, 2022 pm 04:47 PM

人工智能在生活中的应用有:1、虚拟个人助理,使用者可通过声控、文字输入的方式,来完成一些日常生活的小事;2、语音评测,利用云计算技术,将自动口语评测服务放在云端,并开放API接口供客户远程使用;3、无人汽车,主要依靠车内的以计算机系统为主的智能驾驶仪来实现无人驾驶的目标;4、天气预测,通过手机GPRS系统,定位到用户所处的位置,在利用算法,对覆盖全国的雷达图进行数据分析并预测。

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
1 months agoBy尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment

SAP NetWeaver Server Adapter for Eclipse

SAP NetWeaver Server Adapter for Eclipse

Integrate Eclipse with SAP NetWeaver application server.

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function