When it comes to AI infrastructure, should you choose edge or cloud?

Nowadays, edge computing has been a hot topic. Hailed as the most exciting technological shift in recent years, there’s a lot of talk about its transformative power!

With increasingly powerful AI/ML algorithms redefining “intelligence” and the availability of cheaper, more powerful “edge” devices, the hype is largely real. But if you think about the history of edge computing, it's much earlier than recent interest would lead us to believe.

In fact, computing and intelligence originally started at the edge, when high-bandwidth network connectivity was almost non-existent for most applications. Even in the late 1990s, critical measurement equipment deployed remotely in factories or fields often had dedicated computing power to process incoming sensor data. However, the algorithms in these devices are only rudimentary in terms of "intelligence": primarily signal processing or data conversion.

Cloud-based computing began to gain traction in the late 2000s as network capabilities improved and connectivity increased. At the same time, powerful artificial intelligence algorithms are gaining traction as a means of unlocking meaningful information from large amounts of structured and unstructured data. In just a decade, cloud AI has become the go-to choice for AI applications.

However, the shift to the cloud also brings several issues: data upload and download costs, network reliability and data security, etc. At the same time, edge computing’s trade-offs between processing power and cost or floor space are diminishing with the rise of affordable yet powerful edge devices. It seems we are now back to considering “edge computing” as a viable and attractive option for building smart applications.

As the debate rages over which is better, edge AI or cloud AI, anyone familiar with both frameworks is likely to respond “It depends.” !". The reason is that edge and cloud infrastructure are not competing, but complementary frameworks.

Both have experienced tremendous development and refinement over the past few years, particularly as a foundation for artificial intelligence development and deployment. As with any technology choice, the choice really comes down to the specific application: goals, value drivers, economics, and any constraints on power consumption, size, and connectivity. Therefore, it’s important to understand the pros and cons of cloud and edge AI before trying to set up the right infrastructure.

Cloud-based AI is an attractive option when looking for flexibility, scalability, and ease of deployment. Today, most cloud service providers offer powerful frameworks for training and deployment of AI models and offer pay-as-you-go packages with little upfront commitment or investment. The cloud offers virtually unlimited computing and storage options, making it particularly suitable for large AI models.

But for real-time applications that require continuous evaluation of sensor or image data, this can become an unwieldy option because they must transmit data back and forth, incurring huge costs. This data transfer also makes the cloud largely unsuitable for low-latency applications that require closed-loop control or instant action.

On the other hand, edge AI is a reasonable choice for real-time data analysis for automatic alarms or closed-loop control. While edge infrastructure does require an upfront investment in edge hardware, operating costs are much lower compared to the cloud.

Today, there are a variety of edge AI hardware options available, including NPUs (Neural Processing Units), TPUs (Tensor Processing Units), and SOCs (Systems on a Chip) with dedicated AI accelerators and SoMs (systems on modules). Low-cost and low-power hardware for artificial intelligence is an active area of ​​research and has the potential to provide better options. On the other hand, AI-based consumer applications must deal with a fairly diverse range of edge devices (phones, tablets, PCs, etc.), making edge deployment a potentially daunting prospect.

As a result, edge infrastructure may not be conducive to rapid prototyping or easy to scale. While the concept of federated learning, distributed training of AI models allows for training and deployment at the edge, the cloud remains the logical choice for training large models that require sufficient computing power.

But the solution doesn’t have to be an either/or choice. As applications transition to more microservices-based architectures, they can be broken down into smaller functions or microservices with their own specific deployment frameworks. So instead of having to choose between cloud and edge, the focus can be on using both optimally for specific applications.

For example, an application might start as a rapid prototype on the cloud. As it evolves, functions that require low latency and real-time decision-making can be moved to the edge, while those that require scale and flexibility can remain in the cloud. Model training or retraining can be managed centrally in the cloud, while some federated learning at the edge can improve accuracy locally. Likewise, sensitive data can be processed at the edge and more general data can be moved to the cloud.

Organizations, developers, and practitioners would be better off thinking of cloud and edge not as distinct alternatives, but as a continuum from edge to cloud, with many different infrastructure options in between. This includes different types of edges – operational edge, network edge, mobile endpoints, etc., as well as different types of distributed processing on the network – private clouds, public clouds, cloudlets, fog computing, etc.

While complexity can be a challenge, finding the right combination of technologies begins to offer organizations a unique opportunity to maximize the value of AI while minimizing costs and risks. ​

The above is the detailed content of When it comes to AI infrastructure, should you choose edge or cloud?. For more information, please follow other related articles on the PHP Chinese website!