search
HomeTechnology peripheralsAIUsing small tricks to excavate the diffusion model, the generated images are almost replicas of the original training data, and privacy is about to be exposed.

Denoising diffusion models are an emerging class of generative neural networks that generate images from a training distribution through an iterative denoising process. This type of diffusion model produces higher quality samples and is easier to scale and control than previous methods such as GANs and VAEs. As a result, after rapid development, they can already produce high-resolution images, and there is great public interest in large models such as the DALL-E 2.

The beauty of generative diffusion models lies in their ability to synthesize new images that are, superficially, unlike anything in the training set. In fact, large-scale training efforts in the past have not found that overfitting will be a problem. Researchers in privacy-sensitive fields have even suggested that diffusion models can be used to protect privacy by generating synthetic examples to generate real images. This series of work is conducted under the assumption that the diffusion model does not memorize and regenerate training data. Doing so would violate all privacy guarantees and breed many problems with model generalization and digital forgery.

In this article, researchers from Google, DeepMind and other institutions have proven that the SOTA diffusion model can indeed remember and regenerate a single training example.

Using small tricks to excavate the diffusion model, the generated images are almost replicas of the original training data, and privacy is about to be exposed.

## Paper address: https://arxiv.org/pdf/2301.13188v1.pdf

First, the research proposes and implements a new definition of memory in image models. Then, the study designed a two-stage data extraction attack, using standard methods to generate images and label some images. The study applied the method to Stable Diffusion and Imagen, resulting in the extraction of more than 100 nearly identical copies of training images, which ranged from personally identifiable photos to trademarked logos (Figure 1).

Using small tricks to excavate the diffusion model, the generated images are almost replicas of the original training data, and privacy is about to be exposed.

In order to better understand how memory works and why, researchers trained hundreds of people on CIFAR10 Diffusion model to analyze the impact of model accuracy, hyperparameters, augmentation, and deduplication on privacy. Diffusion models are the least private form of image models evaluated in the study, leaking twice as much training data as GANs. Worse, research also finds that existing privacy-enhancing technologies fail to provide acceptable privacy-utility trade-offs. Overall, this paper highlights the tension that exists between increasingly powerful generative models and data privacy, and raises questions about how diffusion models work and how they can be deployed appropriately.

Why do you want to do this research?

There are two motivations behind understanding how diffusion models remember and regenerate training data.

The first is to understand the privacy risks. Regenerating diffusion models that scrape data from the Internet may pose similar privacy and copyright risks as language models. For example, it has been pointed out that memorizing and regenerating copyrighted text and source code present potential indicators of infringement. Likewise, copying an image created by a professional artist is called digital forgery, and there is a debate in the art world.

The second is to understand generalization. In addition to data privacy, understanding how and why diffusion models remember training data helps understand their ability to generalize. For example, a common question with large-scale generative models is whether their impressive results result from true generation or are the result of directly copying and remixing the training data. By studying memory, it is possible to provide a concrete empirical description of the rate at which generative models perform such data replication.

Extract data from SOTA diffusion model

Extract data from Stable Diffusion

Now Extract training data from Stable Diffusion, the largest and most popular open source diffusion model.

This extraction applies the method of previous work to the image, including two steps:

1. Diffusion model using standard sampling method And generate multiple examples using the known prompts from the previous section.

2. Perform inference to separate the new generation model from the memorized training model.

To evaluate the effectiveness of the intrusion, the study selected 350,000 most repeated examples from the training dataset and generated 500 candidate images for each prompt (generating 175 million in total image).

First, the study sorts all these generated images to determine which ones are generated by memorizing the training data. Each of these generated images is then compared to the training images under Definition 1 in the paper, and each image is annotated as extracted or not extracted. The study found that 94 images were extracted. To ensure that these images did not just fit some arbitrary definition, the study also manually annotated the first 1,000 generated images through visual analysis, either with or without memory. And another 13 (total 109 images) were found to be almost copies of the training examples, even though they did not meet the study L_2 norm definition. Figure 3 shows a subset of the extracted images, which are reproduced with near-perfect pixel accuracy.

Using small tricks to excavate the diffusion model, the generated images are almost replicas of the original training data, and privacy is about to be exposed.

The experiment also gives the calculation curve given an annotated ordered image set, Evaluate the number of extracted images versus the false positive rate of the intrusion. The intrusion is exceptionally accurate: out of 175 million generated images, 50 memory images with 0 false positives could be identified, and all memory images could be extracted with over 50% accuracy. Figure 4 contains precision-recall curves for both definitions of memory.

Using small tricks to excavate the diffusion model, the generated images are almost replicas of the original training data, and privacy is about to be exposed.

Extract data from image

Although Stable Diffusion is currently the best choice among publicly available diffusion models, some non-public models have achieved stronger performance using larger models and datasets. Previous research has found that larger models are more likely to remember training data, so this study looked at Imagen, a 2 billion parameter text-to-image diffusion model.

Surprisingly, research has found that hacking non-distributed images in Imagen is more effective than in Stable Diffusion. On Imagen, the study tried to extract 500 images with the highest out-of-distribution (OOD) score. Imagen memorizes and copies 3 of the images (which are unique in the training data set). In contrast, when the study applied the same method to Stable Diffusion, it failed to identify any memories even after trying to extract the 10,000 most outlier samples. Therefore, Imagen is less private than Stable Diffusion on both copied and non-copied images. This may be due to the fact that Imagen uses a larger model than Stable Diffusion and therefore remembers more images. Additionally, Imagen trains with more iterations on smaller datasets, which can also help improve memory levels.

The above is the detailed content of Using small tricks to excavate the diffusion model, the generated images are almost replicas of the original training data, and privacy is about to be exposed.. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:51CTO.COM. If there is any infringement, please contact admin@php.cn delete
What is Graph of Thought in Prompt EngineeringWhat is Graph of Thought in Prompt EngineeringApr 13, 2025 am 11:53 AM

Introduction In prompt engineering, “Graph of Thought” refers to a novel approach that uses graph theory to structure and guide AI’s reasoning process. Unlike traditional methods, which often involve linear s

Optimize Your Organisation's Email Marketing with GenAI AgentsOptimize Your Organisation's Email Marketing with GenAI AgentsApr 13, 2025 am 11:44 AM

Introduction Congratulations! You run a successful business. Through your web pages, social media campaigns, webinars, conferences, free resources, and other sources, you collect 5000 email IDs daily. The next obvious step is

Real-Time App Performance Monitoring with Apache PinotReal-Time App Performance Monitoring with Apache PinotApr 13, 2025 am 11:40 AM

Introduction In today’s fast-paced software development environment, ensuring optimal application performance is crucial. Monitoring real-time metrics such as response times, error rates, and resource utilization can help main

ChatGPT Hits 1 Billion Users? 'Doubled In Just Weeks' Says OpenAI CEOChatGPT Hits 1 Billion Users? 'Doubled In Just Weeks' Says OpenAI CEOApr 13, 2025 am 11:23 AM

“How many users do you have?” he prodded. “I think the last time we said was 500 million weekly actives, and it is growing very rapidly,” replied Altman. “You told me that it like doubled in just a few weeks,” Anderson continued. “I said that priv

Pixtral-12B: Mistral AI's First Multimodal Model - Analytics VidhyaPixtral-12B: Mistral AI's First Multimodal Model - Analytics VidhyaApr 13, 2025 am 11:20 AM

Introduction Mistral has released its very first multimodal model, namely the Pixtral-12B-2409. This model is built upon Mistral’s 12 Billion parameter, Nemo 12B. What sets this model apart? It can now take both images and tex

Agentic Frameworks for Generative AI Applications - Analytics VidhyaAgentic Frameworks for Generative AI Applications - Analytics VidhyaApr 13, 2025 am 11:13 AM

Imagine having an AI-powered assistant that not only responds to your queries but also autonomously gathers information, executes tasks, and even handles multiple types of data—text, images, and code. Sounds futuristic? In this a

Applications of Generative AI in the Financial SectorApplications of Generative AI in the Financial SectorApr 13, 2025 am 11:12 AM

Introduction The finance industry is the cornerstone of any country’s development, as it drives economic growth by facilitating efficient transactions and credit availability. The ease with which transactions occur and credit

Guide to Online Learning and Passive-Aggressive AlgorithmsGuide to Online Learning and Passive-Aggressive AlgorithmsApr 13, 2025 am 11:09 AM

Introduction Data is being generated at an unprecedented rate from sources such as social media, financial transactions, and e-commerce platforms. Handling this continuous stream of information is a challenge, but it offers an

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
WWE 2K25: How To Unlock Everything In MyRise
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

SecLists

SecLists

SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.