To prevent and control “fake grass planting”, Xiaohongshu’s technical team has done these major things

This article mainly focuses on the best practice area of ​​community and e-commerce risk control in Xiaohongshu - brush volume management.

As a content community, the number of monthly active users of Xiaohongshu has been growing steadily since it exceeded 200 million in October 2021. Based on the continuous accumulation of UGC content assets, Xiaohongshu is becoming a diverse lifestyle gathering place for more and more users. All these have made Xiaohongshu a target for illegal production.

From the user perspective, security needs to solve two problems on the content platform: Ensure that user information is safe on the Internet, and ensure that the information users obtain on the Internet is authentic.

Volume brushing, as a common cheating method in the community and transactions, will cause the information obtained by users to be untrue. Whether it is community traffic such as note reading, likes, and comments, or e-commerce data such as product views and sales, obtained through brushing, it will undermine our content security and affect users' trust in the platform.

1. Enter the black industry to understand the realization of brush volume

1.1 What is brush volume?

In a basic understanding, volume brushing can be understood as data fraud, such as using cheating methods to obtain false likes, collections and other data, in order to obtain higher commercial value. In the e-commerce scenario, brushing is more concentrated at the merchant level, such as hiring brushing users and using fake logistics to structure transactions and fabricate high GMV.

In order to deeply understand the nature of brush volume, we visited a group of companies specializing in the brush volume industry chain. The following is an introduction to three of them and their implementation methods:

1.1.1 Group Control Brushing Company

This company provides flushing services, mainly based on group control devices of machines to brush the volume, including jailbroken machines and normal iPhones. They took advantage of Apple's security mechanisms, such as the image packaging of Apple mobile phones: the client containing the user login status in the entire phone is packaged and stored into a file, and then these files are stored in a sandbox environment. Based on this method, a single mobile phone is used to restore batch sandbox environments and accounts to achieve good group control effects.

1.1.2 Account raising and diversion company

This is an e-commerce company that sells high imitation clothes through diversion. The main method is to register an account through a virtual hall card and then maintain the account. When the account used is blacklisted, it needs to open a nearby offline second-hand mobile phone store to replace the blacklisted phone at a lower price (5-10 yuan). After getting the new equipment, hire a group of real people, register an account and continue to publish and operate content. Finally, do brand promotion and get paid. It is understood that one account can attract hundreds of thousands of GMVs. Considering the number of accounts, the company's income is considerable.

##1.1.3 Crowdsourced brushing company

Crowdsourcing black products usually have professional organizations that look for tasks on major platforms and recruit people to do part-time jobs. Part-time workers can get paid by completing tasks using their own accounts.

1.2 Redefinition of brush volume

Does brushing likes and likes mean brush volume? Yes, but not completely. We need to understand brush volume from a higher perspective.

In the community environment of Xiaohongshu, we divide users into two categories: content publishers and content consumers. Content publishers are responsible for producing and providing quality content to consumers. Content consumers, in contrast, pay time costs, opportunity costs, information costs, emotional costs, and the value of the information contained in the content they view. In other words, publishers and consumers are actually engaging in a type of transaction, and the content platform is a market economy platform. When there is a market, there must be currency in circulation. In the content community, currency is units that can be measured such as readings, likes, collections, and comments.

##Under this analogy,

washing is equivalent to the act of making counterfeit currency

, the release of false content is similar to fake goods and will lead to inflation in the overall market. Although these tokens exist in the secondary market (community), they will eventually be traded with the primary market (e-commerce, advertising) to realize traffic monetization. For Xiaohongshu, washing is equivalent to counterfeit currency, and it needs to be managed sparing no effort.

2. Challenges faced by risk control

Since it has been classified as counterfeit currency, why does the wash volume still exist for a long time? Here we must mention several challenges facing anti-cheating: It is difficult to identify, difficult to govern, and difficult to consolidate.

2.1 Difficulty in identification

The difficulty in identification can be summarized into three parts: Uncontrollable terminal, AI versus AI, real person versus real person.

2.1.1 The terminal is uncontrollable

In the community brushing scene, the terminal is actually uncontrollable, and it is difficult for adversaries to judge whether the client is A normal user is still a black product. Theoretically speaking, if the communication method is not trustworthy, there is no way to make it trustworthy.

We once tried to reverse engineer an interface protocol attack tool, disassembled the code inside, and sent a request to the server disguised as our client by cracking the signature algorithm. This is by far the easiest one, the other methods are more difficult to control.

（*Protocol attack）

2.1.2 AI versus AI

Although risk control is constantly improving and AI computing has been used on a large scale to fight against it, our opponents are also constantly breaking through and upgrading.

Take the black and gray group control method as an example. Utilize the vulnerability of third-party authorization to generate batches of third-party accounts on the public platform, and use the third-party accounts to complete the registration of Xiaohongshu accounts. Then use the UI's automatic script clicking tool to control clicks and keystrokes, and operate on your mobile phone to gain likes and followers in batches. The current popular method is to use 4G mobile phone/EC20 chips to conduct hybrid attacks to avoid our identification of black IPs.

#The promotion of AI intelligence in risk control is slow. The reason is that AI algorithms are good at solving first-order problems, such as whether a picture contains a payment QR code. However, in actual scenarios, what we need to judge is whether the chat content is risky. From a human perspective, there is no risk in sending payment QR codes between friends who are in frequent contact, but sending payment QR codes between strangers who have just chatted is a high-risk behavior. The risk here depends on the aggregation of these two levels of information, payment QR code and stranger chat. Compared with humans, it is difficult for AI to automatically judge the aggregate recognition of these two dimensions, so it requires a lot of human intervention. 2.1.3 Real-person vs. real-person

Recruit people through online group chats, offline small tasks, and even through formal recruitment channels. These people are mixed with real users and there is real normal behavior. At a certain point, they will charge money to increase their volume, which makes the entire identification extremely complicated.

2.2 Difficulty in governance

On the Xiaohongshu platform, the content community is divided into K (bloggers), B (brands/merchants), MCN organizations ( intermediary), C-side (ordinary users). The question that has troubled us for a long time is who is doing this behavior for profit, is it the K-side, B-side, MCN side or C-side? And is this person's behavior at the moment good or bad? While managing, how to avoid accidental injuries?

The real situation is that

every end may participate in brushing

. After the brand releases its marketing needs, MCN accepts the task and distributes it to K on major platforms, allowing K to publish relevant notes to attract users, and finally achieve the purpose of marketing promotion. In this process, MCN has the motivation to increase the volume in order to complete the exposure requirements on the brand side and gain profits. Similarly, in order to complete the tasks issued by MCN and make up for his own lack of traffic, K also has the motivation to increase his traffic. Different roles on the brand side, such as the operations side, may also brush up the volume in order to complete the company's tasks. Some competitors, in order to frame their peers, will also gather C-end users to brush their notebooks, resulting in risk control penalties for bloggers and notebooks.

What’s even more troublesome is that a large number of real people will be mixed in the entire group, making it difficult to accurately distinguish brushing behavior. Even if you can tell, how can you tell whether the person's current behavior is normal? There will be issues of recall and accidental damage. It is necessary to ensure a complete recall without accidentally damaging a batch of users, which will affect the smooth operation of the business. At the same time, the cold start of the

platform is also a problem that needs to be solved. For example, why do new small merchants increase their sales? This is because in the early days, small merchants were inferior to large merchants in terms of logistics, product quality and price control. Even if a user accidentally searches for a product from a new merchant, the sales volume is 0, and the buyer does not trust it, and in the end he still loses the opportunity to make a transaction. This is also the demand for brush volume from small businesses.

2.3 Consolidation is Difficult

Any safe operation will eventually fall into the problem of input-output ratio. In scenarios where content is flushed, it is difficult to measure the ROI of risk control investments because there is no direct revenue calculation.

The second difficulty is measuring the effectiveness of prevention and control. We talked about CTR earlier, but is the CTR value enough to judge the effect of risk control? the answer is negative. The results of prevention and control will involve multiple dimensions. Among the brushing cases we manage, many of the fraudulent brushings come from real-person crowdsourcing. Should the effectiveness of prevention and control be judged based on the user dimension or the number of abnormal behaviors? The management of brush volume is a confrontation between people. After one method of risk control and governance, opponents will use new means and technologies to launch the next round of attacks. Therefore, the effect of prevention and control is constantly fluctuating. We Continuous confrontation is required.

3. Xiaohongshu’s governance practices

Xiaohongshu is a platform for sincere sharing and friendly interaction. For us, what is more important than technical capabilities is the attitude of the platform. This also means that our attitude towards volume washing behavior is to severely crack down on zero tolerance.

Within the Security Technology Department, we have proposed three solution directions, which are: Engine-based risk control infrastructure, process-based risk control countermeasures, and intelligent risk control identification.

3.1 Engineizing risk control infrastructure

According to domestic statistics, there are 1.5 million black industry employees, far more than our risk control technical team. How to fight on a small scale What about the entire industry chain? The solution we proposed is quick to iterate and easy to trial and error, and the risk control infrastructure is engine-based. It connects the engineering engine with the entire upstream and downstream links to improve the overall risk control efficiency. The risk control engine here not only refers to deploying strategies on an independent node, but also includes information access across the entire link, and uses upstream and downstream information to identify risks.

Taking the near-line platform as an example, we have highly abstracted the existing quasi-real-time tasks, allowing users to quickly go online by assembling tasks in a short time, thus improving efficiency. The engine capability of this platform can combine multiple data sources and historical data in the background to provide risk control identification results in relatively real-time. It also combines behavioral sequence analysis, graph analysis and algorithm models to improve the effectiveness of risk control capabilities.

The near-line platform makes up for the limitations of single rule identification, integrates with the original real-time risk control system, and is an effective supplement to other prevention and control methods. The near-line platform supports the ability to comprehensively process data and events, and also supports multi-source data access. It has flexible data processing capabilities. More importantly, it provides functions that can support algorithms and strategies and students’ customized data processing logic. Free up manpower and improve accuracy while engineering processes.

The following figure shows the overall system and architecture of the near-line system:

# #Activate from device->Account registration and login->Notes/mall browsing->Interaction->Content publishing/purchasing goods, etc. At present, we have covered user behavior in all scenarios. After obtaining business data, we will make real-time requests and quasi-real-time streams The formal, and offline data are connected to the risk control engine to realize multi-scenario-level joint prevention and control. The obtained identity characteristics, network environment, timing characteristics, cumulative factors, etc. are processed and mined, and then input to the decision-making analysis layer. In addition, the identification capabilities are migrated, and newly identified risk points are precipitated and reused in other risk scenarios.

In addition to the risk control engine,

the general business gateway side (edith2.0) has also been unified to improve the access efficiency and stability of new services as a whole and achieve minute-level access to services .

3.2 Streamline risk control and confrontation

The biggest difficulty in business risk control is confrontation, no matter what the cheating becomes In all forms, the only thing that remains unchanged and always exists is confrontation. A standardized process may not guarantee that we will get the optimal solution immediately, but it can help us analyze and expose problems, establish a forward iteration link with feedback, and improve our efficiency.

The operational accumulation capability driven by intelligence technology has gradually formed a feasible process system in Xiaohongshu’s risk control team and is continuously operating. The entire process can be divided into five steps:

Risk perception, capacity building, risk identification, risk treatment, and effect assessment.

Risk perception: Rely on the indicator system to discover risks faster, change passive firefighting into active defense, and drive capacity building and subsequent processes.

Capacity building: Establish rapid response capabilities for confrontation and quickly access a flexibly configurable risk control system.

Risk identification: Improve identification accuracy, establish a feature system, mine cheating features, and provide demand feedback on missing capabilities.

Risk disposal: Choose more effective ways to reduce the cost of confrontation, intercept behaviors, limit the flow of notes, and deal with users, and can form effective feedback on strategy iterations.

Effectiveness evaluation: Also relies on the indicator system to evaluate changes in risk levels, such as cheating and leakage, black product service prices, account prices, etc., and can also provide feedback on manslaughter and leakage.

After discovering a new intelligence or sensing a new cheating attack, operate according to this process, and analyze and determine from the process what went wrong: Is it because the intelligence is not discovered enough, or is it The transformation of intelligence into identification is not fast enough, or the identification, processing and recall are insufficient.

Take the purchase volume analysis in intelligence operations as an example. After discovering illegal services through intelligence or price fluctuations, purchase tests are conducted, and then the techniques are analyzed and appropriate strategies or models are selected for identification and disposal. As for the effects, The evaluation requires secondary purchasing or continuous monitoring of the price of black products. The current process will not end until the feedback on the identification and disposal has been implemented and the effect evaluation reaches the standard.

3.3 Intelligent risk control identification

In terms of the level of confrontation and the intensity of prevention and control, the intelligent identification of risk control can be roughly divided into three stages: the first stage, Low-cost machine techniques for black production, are analyzed and judged based on the characteristics of the behavioral subjects; in the second stage, in the high-frequency confrontation with black production, are based on the characteristics of the group Gang mining; in the third stage, in the identification of real-person brushing, based on the association of the brushing community, graph-based algorithm is used to discover the subject relationship. At present, these three stages have been implemented and practiced in Xiaohongshu.

3.3.1 Behavior Subject Characteristics Analysis

Traffic data as a form of expression, its basic component is each interactive behavior of the user , risk control is also analyzed and identified based on behavioral characteristics. In the process of identification, the quality of behavioral characteristics is directly related to the identification and recall of black products, so the construction of data characteristics is particularly important. "A sharpened knife will never waste time cutting firewood." At this stage, we established a set of feature systemsbased on users, devices, etc. of behavioral subjects, laying the foundation for intelligent identification of risk control.

The purpose of black production is to complete the volume brushing task at the lowest cost. At this stage, black production often uses machine means to carry out automatic or semi-automatic batch brushing, not only Low cost and high efficiency. Our basic assumption is that when performing machine brushing, there must be clear and explainable abnormal characteristics on the machine account or device. Therefore, how to analyze and identify abnormal characteristics is a proposition that needs to be solved in the first stage of risk control. In terms of identification and division of labor, we implement the end-cloud prevention and control model. End-defense designs reasonable device reporting and detection methods to complete preliminary prevention and control of devices. Cloud defense is based on user characteristics, device characteristics and behavioral characteristics. Conduct abnormal analysis and identification of behaviors.

Take the protocol interface request as an example. The black product directly requests the back-end interface by cracking the signature algorithm of the platform. It can complete the brushing without the need of real equipment. It is the lowest cost and most efficient brushing commonly used by black products. means. The lack of real devices also results in no real front-end management and device reporting. Even if the device information is forged, the fields and device environment of the device reporting information cannot be effectively faked to be the same as normal devices.

In addition to identifying and preventing environmental anomalies through verification of relevant characteristic parameters, we also established supervised identification relying on a comprehensive characteristic system and precipitated black production samples. Model. In the entire model, feature engineering is an important link to ensure model recall, and the processing of model results is the guarantee of accuracy.

Feature engineering: In feature engineering, we divide it into two parts and proceed sequentially.

• Feature selection: After continuous "artificial confrontation" in the early stage, we have accumulated rich experience in feature confrontation, and selected comprehensive features and labels from the perspective of black production brushing techniques, such as device kernel version , farm tags, change machine fields, registration duration, batch registration tags, etc.
• Feature processing: During feature processing, discrete features and continuous features are standardized separately and then LabelEncoder is performed. Different splicing methods are tried on the characterization results to convert scattered feature data into multi-dimensional model input.

Result processing: In terms of design, we trained the regression model as an infer, that is, the input value is a continuous score, which is more efficient in result processing than the classification model. flexible.

• Sample distribution: By analyzing the model result distribution of real brush samples, the "appropriate" threshold is semi-automatically calculated as the standard for distinguishing black products from real behavior.
• Operation evaluation: Before the model goes online, risk control operations will evaluate the accuracy of the end-to-end results. Only when the accuracy of the evaluation reaches the standard can the model be put online for identification.

3.3.2 Group Feature Mining

As mentioned before, confrontation always exists. When black manufacturers find that low-cost machine techniques cannot bypass risk control verification, they will constantly try to use real machines, fake device parameters, use cheating tools and other countermeasures, and upgrade the techniques to eliminate low-cost techniques. In the constant confrontation, the analysis of subject characteristics in the first stage appears relatively passive and cannot carry out active prevention and control.

From the perspective of risk control, our basic assumption at this stage is: no matter how black products are forged or bypassed, the nature of the gang cannot be changed. There must be similarities and differences in the data characteristics. Gather . Therefore, at this stage, using gangs as the identification target to mine group characteristics and using structured data feature systems is the second line of defense that needs to be established.

Take the constantly changing methods of machine brushing as an example. The black production methods gradually transformed from protocol interfaces to modified machine group control, and evolved into automated click tools. In order to proactively prevent and control, we use the equipment clustering model to identify and recall , and have completed effective containment of the above machine techniques.

The device clustering model is characterized by basic device information (such as device model, app version, system kernel, etc.), and performs feature engineering and similarity calculation on the basic device information of users who interact with the same note or the same blogger. , use clustering algorithm to delineate device groups with high similarity. After identification, on the one hand, black devices are recorded and subsequent behavior interception is performed; on the other hand, the device tags of the black device group are output to expand the device black database. This model is an unsupervised model and can well adapt to the characteristics of black products that frequently change cheating methods. It can also supplement online strategies and expand recall.

The following is the model flow chart and cluster identification case:

In the entire model, we rely on Feature Engineering and Cluster Screening High recall and high accuracy clustering of black production equipment:

• Feature Engineering: In the first stage, we established a set of Comprehensive and highly reusable feature system. At this stage, we analyze the characteristics of the gang's equipment and select appropriate 20 to 30-dimensional equipment features as input to ensure model recall from the data source.
• Cluster screening: In the continuous risk control confrontation, we have accumulated enough device tags and device black libraries. In the screening of the output results of the clustering model, this part of black devices is used as Seeds are used to calculate black production concentrations in clusters to screen out accurate and high-recall device clusters.

3.3.3 Graph-based subject discovery

As mentioned above, the method of brushing volume will gradually evolve into real people versus real people. When black producers find that machine methods cannot bypass risk control, they will choose to recruit crowdsourcing or organize mutual brushing, and try to use real people's behavior to brush up the volume. Although they are all actions of real people, their essence is that they lack real will and are "counterfeit coins." Not only can the analysis of the main characteristics of real people’s behavior fail to detect abnormalities, but also group characteristics cannot be found on the equipment.

Due to the particularity of the volume washing behavior, our reasonable assumption is that no matter how we change the method, the ultimate direction of the volume washing behavior is the main body of the buying volume. As long as there is subject correlation, Behaviors can be identified and recalled through data features. At this stage, Xiaohongshu relies on unique communities, e-commerce behavior graphs, and volume and order brushing graphs to actively discover and mine a group of behavioral subjects. It uses entity association and graph algorithms to identify real cheating users or those who are raising accounts. Users' recall can effectively find "fish that slipped through the net".

*Figure 1: Association diagram of notes liked by normal users

(red The dots are notes, the blue dots are users)

##*Figure 2: Association of notes liked by cheating users Figure

(The red dots are notes, the blue dots are users)

Figure 1 : Among the notes liked by 7 normal users, only one note overlaps, that is, the note with a red dot in the middle.

Figure 2: The notes liked by 9 cheating users are the same batch of notes, the height of the liked notes Overlapping and not liked other notes other than this batch of notes.

The above picture is the behavior correlation diagram between users and notes (red is the note, blue is the user). Figure 1 shows the behavior correlation between normal users liking normal notes. You can find the behavior The user's group relevance is low and they will not agree with a large number of notes. However, there is occasional overlap caused by the system recommending notes of the same type due to their "hobbies". In Figure 2, the behavioral user likes a batch of cheating notes, which has strong community relevance. It is a typical behavior correlation diagram of cheating users liking cheating notes.

Take "driving" mutual brushing as an example, that is, after the black industry receives the task of brushing the volume, it acts as an organizer and middleman, issuing tasks to people who often participate in mutual brushing (such as mutual likes), and assigning tasks to designated The target is refreshed. Black products will not only publish one task, nor will behavioral users only do one task. There is a relatively fixed "circle" between the two. For this type of cheating, we used the label propagation model to recall community users in batches.

The tag propagation model is based on the risk control map to diffuse seed users. The seed users come from the brush users who have been identified and precipitated by risk control. The associated entities are buying notes and buying blogs. Mainly, etc., propagated through the point-edge correlation relationship in the risk control graph. Real people are used to brush the volume, and the equipment and accounts are limited. In order to achieve the purpose of brushing the volume, behavioral users will enable small accounts to repeatedly accept multiple tasks. The purpose of tag propagation is to spread the brushing tags to other brushing users in the community through the correct path to complete the identification and recall of brushing.

• Propagation path: User tags will be propagated through strong paths such as devices and third-party accounts, as well as through associated paths with purchasers such as notes, authors, etc.
• User tags: Since the users who are brushing the volume are real people, the labels given to users through label propagation often cannot be directly intercepted. Therefore, based on the basic assumption of real people brushing the volume, we The tags under the buying subjects were clustered and analyzed to identify the buying subjects and wash users.

4. Thinking outside of technology to solve problems

The above are mainly based on technology to solve problems, but in actual work, we also need to think outside of technology and from a more macro perspective. From the perspective of helping business and improving security. Based on this purpose, while delving deeply into technology, we have developed a set of practical and effective methodologies:

Start by eliminating the impact of trouble, then solve the people who cause trouble, and finally eliminate the motivations that cause trouble.

Three Points to Solve Trouble

Eliminate Trouble Impact

After identifying the cheating objects and data through technical means, actual management and disposal actions will be carried out to block the false traffic from the platform.

solve those who cause trouble

Attack people in this industry chain and every link, including account transactions, platform data transactions, transaction persons, etc. . Here we use a joint operation between risk control and legal departments to deal with the troublemakers. From June to August this year, six MCNs and some authors who engaged in volume brushing were severely punished and lawsuits were launched against them.

In the e-commerce scenario, merchants who brush orders mainly do so to attract traffic and achieve higher GMV. The identified merchant accounts will be handed over to the operation students for professional judgment, and will be handed over to the rule side for punishment according to the severity of the bad behavior; users with bad behavior will be placed on a blacklist, and their ordering behavior will be restricted by risk control .

Address the motivations for causing trouble

The purpose of the volume buyers in the community scenario is to realize his commercial value through cheating, and what we have to do is to reduce the commercial value of these users or institutions. Risk Control Joint Ecology and Operation Department, will punish cheating traffic in accordance with platform rules, laws and regulations, etc. After continuous crackdowns, the cost of black accounts has become higher, the price of brushing services continues to rise, the unit price of likes and collections has increased by more than 300%, and the willingness of users to cheat is gradually decreasing.

Since 2022, Xiaohongshu has cleaned up 3.1 billion cheating like behaviors. It can be seen that the platform’s attitude towards the matter of brushing up the volume has always been very resolute.

(Author of this article: Xiaohongshu Security Technology Department Lu Xun Shiying Wang Ma Ye Feng Shi Hao Lao Pi)

The above is the detailed content of To prevent and control “fake grass planting”, Xiaohongshu’s technical team has done these major things. For more information, please follow other related articles on the PHP Chinese website!