


1. The evolution of underlying computing power of GNNs recommendation system
In the past 20 years, computing forms have continued to evolve. Before 2010, cloud computing was particularly popular, while other computing forms were relatively weak. With the rapid development of hardware computing power and the introduction of end-side chips, edge computing has become particularly important. The current two major computing forms have shaped the development of AI in two polarized directions. On the one hand, under the cloud computing architecture, we can use ultra-large-scale cluster capabilities to train large-scale AI models, such as Foundation Model or some generative models. On the other hand, with the development of edge computing, we can also deploy AI models to the terminal side to provide more lightweight services, such as performing various recognition tasks on the terminal side. At the same time, with the development of the metaverse, the calculations of many models will be placed on the end side. Therefore, the core issue that these two computing forms want to reconcile is the balance between computing and transmission, followed by the polarized development of artificial intelligence.
2. Personalization of end-side GNNs recommendation system
These two calculations What opportunities does form bring to GNNs recommendation systems?
Duanyun’s perspective can be compared to the perspective of a global picture and a localized subgraph. In the recommendation system of GNNs, the global subgraph is a global subgraph that is continuously gathered from many node-level subgraphs. Its advantage is that the data is complete and it can provide a relatively comprehensive relationship between nodes. This kind of inductive bias may be more universal. It summarizes the rules of various nodes and extracts the inductive bias, so it has strong generalization ability. The localized subgraph is not necessarily particularly complete, but its advantage is that it can accurately describe the evolution of an individual's behavior on the subgraph and provide personalized node relationship establishment. Therefore, the relationship between the terminal and the cloud is a bit like a global subgraph and a localized subgraph. Cloud computing can provide powerful centralized computing power to provide services, while the terminal can provide some data personalized services
We can combine the advantages of the global graph and localized subgraphs to To better improve the performance of the model, a study published in WSDM2022 this year explored this. It proposes an Ada-GNN (Adapting to Local Patterns for improving Graph Neural Networks) model, which has an entire graph modeling for the global graph, and also builds some local models using subgraphs to do some adaptation. The essence of such adaptation is to allow the model that combines the global model and the local model to perceive the rules of the local graph in a more refined manner and improve personalized learning performance.
Now we use a specific example to explain why we should pay attention to subgraphs. In the e-commerce recommendation system, there is a group of digital enthusiasts who can describe the relationship between digital products, such as mobile phones, Pads, cameras and mobile phone peripheral products. Once he clicked on one of the cameras, an inductive bias was induced. An inductive bias map induced by the group contribution map may encourage us to recommend this kind of mobile phone, but if we return to the individual perspective, if he is a photography enthusiast and pays special attention to photography products, this will sometimes result The paradox shown below. Is the inductive bias induced by the group contribution map too strong for certain groups, especially this tail group? This is what we often call the Matthew effect.
In general, the existing polarized computing forms can reshape our modeling of GNNs recommendation systems. Traditional recommendation systems recall products or items from a candidate pool, perceive the relationship between them through GNNs modeling, and then rank and recommend users. However, due to the support of edge computing, we can deploy personalization models on the end side to perceive more fine-grained personalization by learning on subgraphs. Of course, this new recommendation system architecture for device-cloud collaboration has an assumption, that is, the computing power and power consumption of the device are feasible. But the actual situation is that the computing power overhead of a small model is not large. If you compress it to one or two megabytes and put the computing overhead on an existing smartphone, it will not actually consume more computing power than a game APP. and large electrical energy. Therefore, with the further development of edge computing and the improvement of end device performance, it provides greater possibilities for more GNNs modeling on the end side
If we want to put the GNNs model on the terminal, we must consider the computing power and storage capacity of the terminal. We also mentioned model compression earlier. If you want the GNNs model to be more effective on the device side, if you put a relatively large GNNs model on it, you must perform model compression. The traditional methods of model compression, pruning and quantization, can be used on existing GNNs models, but they will cause performance losses in recommendation systems. In this scenario, we cannot sacrifice performance in order to build a device-side model, so although pruning and quantization are useful, they have limited effect.
Another useful model compression method is distillation. Although it may only be reduced by several times, the cost is similar. A recent study published in KDD is about the distillation of GNNs. In GNNs, distillation of graphical data modeling faces a challenge that distance measures are easily defined in logit space, but in latent feature space, especially layer-by-layer distance measures between teacher GNNs and student GNNs. In this regard, this research on KDD provides a solution to achieve learnable design by learning a metric through adversarial generation
In GNNs recommendation system, in addition to the previous The model compression technique mentioned, split deployment is a specific and very useful technique. It is closely related to the model architecture of GNNs recommendation system, because the bottom layer of GNNs is the Item Embedding of the product, and after several layers of MLP non-linear transformation, the aggregation strategy of GNNs is used
Once a model is trained, it has a natural advantage. The base layer is all shared, and only the GNNs layer can be customized. For personalization here, we can split the model into two parts and put the public part of the model in the cloud. Because the computing power is sufficient, the personalized part can be deployed on the terminal. In this way, we only need to store the GNN of the intermediate kernel in the terminal. In actual recommendation systems, this approach can greatly save the storage overhead of the entire model. We have practiced in Alibaba's scenarios. The model after split deployment may reach the KB level. Then through further simple bit quantization model, the model can be made very small, and there is almost no particularly large overhead when placed on the terminal. Of course, this is a split method based on experience. One of Huawei's recent work published on KDD is automatic model splitting, which can sense the performance of terminal equipment and automatically split this model. Of course, if applied to GNNs, some reshaping may be required
When deploying the model in some serious distribution transfer scenarios, our Pre-trained models are relatively old before they are deployed on the device. This is because the frequency of actual graph data flowing back to the cloud for training is relatively slow, and sometimes it may take a week.
The main bottleneck here isResource constraints, although you may not necessarily encounter this bottleneck in research, in practice you will encounter the problem of outdated end-side models. As the domain changes, the data changes, the model is no longer applicable, and performance will decline. At this time, online personalization of the GNNs model is needed, but personalization on the end will face the challenge of end-side computing power and storage overhead.
Another challenge is data sparseness. Because the end data only has individual nodes, its data sparsity is also a big challenge. Recent research has a relatively efficient approach, which is Parameter-Efficient Transfer. Applying some model patches between layers can be compared to the residual network, but you only need to learn the patches when learning. Through a flag mechanism, it can be turned on when in use and turned off when not in use. When turned off, it can be degraded to the original basic model, which is both safe and efficient.
This is a more practical and efficient approach, published on KDD2021, which can achieve online personalization of GNNs models. The most important thing is that we discovered from such a practice that by sensing the subgraph information of this local model, the overall performance can indeed be steadily improved. It also alleviates the Matthew effect.
In the recommendation system, tail users still face the problem of the Matthew effect on graph data. However, if we adopt a divide-and-conquer modeling approach and personalize subgraphs, we can improve the recommendation experience for users with sparse behaviors. Especially for the tail crowd, the performance improvement will be more significant
3. Implementation of terminal-cloud collaborative GNNs recommendation system
In the GNNs recommendation system, there is a GNNs model for cloud-side services and a small model of GNNs for the client side. There are three implementation forms of GNNs recommendation system services. The first is session recommendation, which is a common batch session recommendation in recommendation systems to save costs. That is, batch recommendations are made at one time and require users to browse many products before the recommendation will be triggered again. The second is to recommend only one at a time in extreme cases. The third type is the end-to-end personalized model we mentioned. Each of these three recommendation system methods has its own advantages. When user interests change slowly, we only need the cloud side to perceive it accurately, so it is enough for the cloud side model to do session recommendation. When user interests change more and more diversely, personalized recommendation of end-side subgraphs can relatively improve recommendation performance.
In situations where user behavior suddenly becomes very sparse, recommendations rely more on common sense reasoning. In order to coordinate these three recommendation behaviors, a meta coordinator - Meta Controller can be established to coordinate the GNNs recommendation system
Construct a three-way coexistence end-cloud One challenge of collaborative recommendation systems is the construction of data sets, because we don’t know how to manage these models and how to make decisions. So this is just a counterfactual reasoning mechanism. Although we do not have such a data set, we do have a single-channel data set, and we construct some proxy models through evaluation to evaluate their causal effects. If the causal effect is relatively large, then the benefits of making such a decision will be relatively large, and pseudo labels, that is, counterfactual data sets, can be constructed. The specific steps are as follows:
There are three models D0, D1, and D2 in a single channel. By learning the causal model of an agent, their causal effects are estimated. Construct a decision label and construct a counterfactual dataset to train the meta-coordinator. Finally, we can prove that this meta-coordinator has a stable performance improvement compared to each single-channel model. It has significant advantages over random heuristics. We can construct a recommendation system for device-cloud collaboration in this way.
4. Security Issues of the End-Side GNNs Recommendation System
Finally, let’s discuss the security issues of the end-side GNNs recommendation system. Once the device-cloud collaborative GNNs recommendation system is opened for use, it will inevitably face problems in the open environment. Because the model needs to be personalized for learning, there will be some risks of attacks, such as escape attacks, poisoning attacks, backdoor attacks, etc., which may ultimately cause the recommendation system to face huge security risks
The underlying computing power drives the direction of the current-end cloud collaborative GNNs recommendation system, but it is still in the early stages of development and there are some potential problems, such as security issues. At the same time, in personalized model building There is still a lot of room for improvement in the field of modeling.
5. Question and Answer Session
Q1: When building a graph model on the terminal, will the traffic of subgraphs be distributed too much?
A1: The sub-picture is not distributed, it is actually aggregated. The first point is that sub-pictures are distributed in an accompanying manner. For example, when we want to recommend products, it will naturally carry attribute information of the products. The issuance of companion style here has the same level of overhead as attributes. In fact, the overhead is not very high. Because it does not deliver the entire big picture, but only some neighbor subgraphs. At most, the second-order neighbor subgraphs are still very small. The second point is that some sub-graphs on the end are automatically constructed based on some co-occurrence and clicks based on feedback from user behavior, so it is a form of double-end aggregation and the overall cost is not particularly large.
The above is the detailed content of GNNs technology applied to recommendation systems and its practical applications. For more information, please follow other related articles on the PHP Chinese website!

如何使用Go语言和Redis实现推荐系统推荐系统是现代互联网平台中重要的一环,它帮助用户发现和获取感兴趣的信息。而Go语言和Redis是两个非常流行的工具,它们在实现推荐系统的过程中能够发挥重要作用。本文将介绍如何使用Go语言和Redis来实现一个简单的推荐系统,并提供具体的代码示例。Redis是一个开源的内存数据库,它提供了键值对的存储接口,并支持多种数据

随着互联网技术的不断发展和普及,推荐系统作为一种重要的信息过滤技术,越来越受到广泛的应用和关注。在实现推荐系统算法方面,Java作为一种快速、可靠的编程语言,已被广泛应用。本文将介绍利用Java实现的推荐系统算法和应用,并着重介绍三种常见的推荐系统算法:基于用户的协同过滤算法、基于物品的协同过滤算法和基于内容的推荐算法。基于用户的协同过滤算法基于用户的协同过

随着互联网应用的普及,微服务架构已成为目前比较流行的一种架构方式。其中,微服务架构的关键就是将应用拆分为不同的服务,通过RPC方式进行通信,实现松散耦合的服务架构。在本文中,我们将结合实际案例,介绍如何使用go-micro构建一款微服务推荐系统。一、什么是微服务推荐系统微服务推荐系统是一种基于微服务架构的推荐系统,它将推荐系统中的不同模块(如特征工程、分类

一、场景介绍首先来介绍一下本文涉及的场景——“有好货”场景。它的位置是在淘宝首页的四宫格,分为一跳精选页和二跳承接页。承接页主要有两种形式,一种是图文的承接页,另一种是短视频的承接页。这个场景的目标主要是为用户提供满意的好货,带动GMV的增长,从而进一步撬动达人的供给。二、流行度偏差是什么,为什么接下来进入本文的重点,流行度偏差。流行度偏差是什么?为什么会产生流行度偏差?1、流行度偏差是什么流行度偏差有很多别名,比如马太效应、信息茧房,直观来讲它是高爆品的狂欢,越热门的商品,越容易曝光。这会导致

随着云计算技术的不断发展和普及,云上搜索和推荐系统也越来越得到了人们的青睐。而针对这一需求,Go语言也提供了很好的解决方案。在Go语言中,我们可以利用其高速的并发处理能力和丰富的标准库实现一个高效的云上搜索和推荐系统。下面将介绍Go语言如何实现这样的系统。一、云上搜索首先,我们需要对搜索的姿势和原理进行了解。搜索姿势指的是搜索引擎根据用户输入的关键字匹配页面

一、问题背景:冷启动建模的必要性和重要性作为一个内容平台,云音乐每天都会有大量的新内容上线。虽然相较于短视频等其他平台,云音乐平台的新内容数量相对较少,但实际数量可能远远超出大家的想象。同时,音乐内容与短视频、新闻、商品推荐又有着显著的不同。音乐的生命周期跨度极长,通常会以年为单位。有些歌曲可能在沉寂几个月、几年之后爆发,经典歌曲甚至可能经过十几年仍然有着极强的生命力。因此,对于音乐平台的推荐系统来说,发掘冷门、长尾的优质内容,并把它们推荐给合适的用户,相比其他类目的推荐显得更加重要冷门、长尾的

随着互联网的迅速发展,推荐系统变得越来越重要。推荐系统是一种用于预测用户感兴趣的物品的算法。在互联网应用程序中,推荐系统可以提供个性化建议和推荐,从而提高用户满意度和转化率。PHP是一种被广泛应用于Web开发的编程语言。本文将探讨PHP中的推荐系统和协同过滤技术。推荐系统的原理推荐系统依赖于机器学习算法和数据分析,它通过对用户历史行为进行分析,预测

作者 | 汪昊审校 | 孙淑娟推荐系统是目前互联网行业最火爆的技术之一。在过去的十年中,互联网行业诞生了数以百万计的推荐系统模型迭代版本。尽管针对不同场景进行优化的推荐系统模型非常之多,但是经典的模型非常少。矩阵分解是推荐系统领域勃兴早期,在 Netflix 大赛中展露头角的推荐系统算法,也是过去十年中最为成功的推荐系统算法。尽管到 2023 年的今天,推荐系统领域早已是深度学习的天下,矩阵分解仍然广泛应用于各大公司研发过程中,并且仍然有许多科研人员在从事相关算法的研究工作。矩阵分解算法最为经典


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

AI Hentai Generator
Generate AI Hentai for free.

Hot Article

Hot Tools

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function

Dreamweaver CS6
Visual web development tools

WebStorm Mac version
Useful JavaScript development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

DVWA
Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software
