This model obsessed former Tesla AI Director Andrej Karpathy for three days!

Recently, an AI researcher and digital artist named Xander Steenbrugge uploaded a very shocking video "Journey Across Time".

The great evolution of life on earth, starting from the primitive ocean, to ancient lizards, dinosaurs, mammals, to monkeys, orangutans, apes, homo sapiens... and finally science fiction appeared in the future world.

And Steenbrugge also commented excitedly: We are crossing a threshold where generative artificial intelligence is no longer just about novel aesthetics, but has evolved into an amazing tool that can build Powerful, human-centered narrative. Steenbrugge said the video used 36 consecutive phrases. To find the best possible order, he tried over a thousand different hints and seeds, and applied many "hint engineering" techniques to the code to figure out what worked and what didn't.

Former Tesla AI Senior Director Andrej Karpathy was so shocked after reading it that he couldn't help but give it a try.

"Surreal steampunk neural network machine, in the shape of a brain, placed on a base covered with neurons made of gears", while typing this text Later, his masterpiece was produced.

2 minute video (~1 hour rendered on A100), generated by smooth interpolation between models with random noise input.

This model, called Stable Diffusion, uses a method of "interpolation" between the meanings of two sentences. The insertion is a semantic gap rather than a visual one, so it dramatically changes the way the story is told.

And this is just the beginning of the digital content creation revolution driven by generative artificial intelligence.

‍Stable Diffusion: A Milestone for Open Source Models

Since the beginning of 2021, artificial intelligence that can generate images from text descriptions has been developing rapidly. At that time, OpenAI demonstrated impressive results with DALL-E 1 and CLIP.

In 2022, OpenAI released the impressive DALL-E 2, Google showed off Imagen and Parti, Midjourney launched a public beta, and Craiyon created AI images everywhere kind of social media.

And recently, Stability AI has released a brand new model-Stable Diffusion.

Unlike DALL-E 2, however, Stable Diffusion can generate a variety of famous figures banned by OpenAI.

Although systems such as Midjourney, Pixelz.ai, etc. can also do it, the quality of their generation is far from comparable to Stable Diffusion. , and none of them are open source.

Now we invite our old friend Musk to perform a second-second transformation into "Black Widow" (Scarlett Johansson).

Apparently, Stability AI, as the lead developer, hopes that more than one company or team will be able to train variants of Stable Diffusion.

For example, you are a researcher who does not have massive GPU computing power. Don't worry, Stable Diffusion will soon be able to run on a graphics card with only 5.1GB VRAM.

For another example, if you are a fan of MacBook equipped with Apple M1 chip, Stable Diffusion can also run. However, the time for image generation will change from seconds to minutes.

It seems that multimodal models are heading down the path that large language models have taken before: moving away from a single vendor and making numerous alternatives widely available through open source.

In addition, Runway is already studying how to implement text-to-video editing based on Stable Diffusion.

In a word, let the tennis court become a beach;

No matter the lightning and thunder, or the wind and sunshine;

Whether it’s on the moon or on Mars;

##can’t stop me from playing tennis...

Using Compute to Accelerate Basic AI Open Source

After the testing phase, Stable Diffusion will be free and the code and trained models will be released as open source. There will also be a hosted version with a web interface for users to test the system.

Stable Diffusion is the result of a collaboration between researchers at Stability AI, RunwayML, LMU Munich, EleutherAI and LAION. EleutherAI is famous for its open source language models GPT-J-6B and GPT-NeoX-20B, etc.

The non-profit organization LAION (Open Network for Large-Scale Artificial Intelligence) provided the open-source LAION 5B dataset for training data, which the team filtered based on human feedback during the initial testing phase to Create the final LAION-Aesthetics training dataset.

Patrick Esser of Runway and Robin Rombach of LMU Munich led the project, whose research in the CompVis group at Heidelberg University laid the foundation for the project. At Heidelberg University they created the widely used VQGAN and Latent Diffusion. These two models, coupled with the results of OpenAI and Google Brain, enable Stable Diffusion.

Stability AI was founded in 2020, and the investor behind it is mathematician and computer scientist Emad Mostaque. He has worked as an analyst at various hedge funds.

With Stability AI and his private wealth, Mostaque hopes to cultivate an open source AI research community. His startup has previously supported the creation of the “LAION 5B” data set. To train Stable Diffusion's models, Stability AI provided servers with 4,000 Nvidia A100 GPUs.

"No one else has decision-making power except our 75 employees - not billionaires, large funds, or governments. We are completely independent." Mostaque said. "We plan to use our computing to accelerate the open source of basic artificial intelligence."

Netizens are going crazy

In addition to the two videos at the beginning, some netizens also use Stable Diffusion generates an aging animation.

He said changing a word in a long prompt during production had a more subtle effect than in a short prompt. Additionally, using descriptions such as old, middle-aged, or young often works better than stating specific ages such as 10, 40, or 80.

In addition, there are various dreamy still pictures.

「Lover」

「Lion」

"Dark Soul" written by Leyendecker and Maxence

"Cyber ​​Peking Opera"

Easter Egg

Feel the tortured mental journey of Andrej Karpathy, Tesla’s former artificial intelligence and autonomous driving vision director.

After watching the video made by the master, Karpathy was very excited.

#The results achieved through a wave of operations will not feel out of place even in a small kindergarten class. Desperate Karpathy said that he wanted to hire a "prompt engineer" to create for him. FinallyKarpathy finally came to his senses and created new works that he was very satisfied with - steampunk artificial neural network machine and blueberry pasta.

The above is the detailed content of This model obsessed former Tesla AI Director Andrej Karpathy for three days!. For more information, please follow other related articles on the PHP Chinese website!