Home > Article > Technology peripherals > July’s most popular AI research list is released, with Ma Yi’s latest “Standard Model” ranked ninth
The list of the most popular AI research in July is out!
This list compiled by Reddit netizen @bycloudai is ranked among the top ten AI research in July 2022 based on Twitter likes, retweets and Github stars, including DeepMind, Google, Well-known institutions such as MIT CSAIL.
Let’s take a look at who is on the list~
Author: Mary Phuong, Marcus Hutter
Institution: DeepMind
Abstract: This paper is intended to be a stand-alone, mathematically accurate overview of the Transformer architecture and algorithm. It covers what Transformers are, how they are trained, their uses, their key architectural components and a preview of the most prominent models.
Authors: Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q Tran, Yi Tay, Donald Metzler
Institutions: Google, MIT CSAIL
Abstract: Recent advances in Transformer-based large language models (LLMs) have driven significant performance improvements on many tasks. However, while performance improves, model size also increases dramatically, which may lead to complex inference processes and increased costs. In practice, however, large language models produce a series of iterations consisting of varying degrees of difficulty.
In this work, we introduce Confident Adaptive Language Model-ing (CALM), a framework for dynamically allocating varying amounts of computer input and generation duration.
Early exit decoding involves several issues we address here, such as: (1) what confidence measure to use; (2) linking sequence-level constraints to exit decisions for local tokens; (3) backtracking Hidden representation lost due to early exit of the previous token. Through theoretical analysis and experiments on three different text generation tasks, we demonstrate the efficacy of our framework in reducing computation – potentially speeding up up to 3x while maintaining high performance.
Author: Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, etc.
Organization: Anthropic
Abstract: This paper investigates whether language models can assess the validity of their own claims and predict which questions they will be able to answer correctly. We first show that when larger models are provided in the correct format, they calibrate well to a variety of multiple choice and true/false questions. Therefore, we can self-evaluate the open sampling task by asking the model to first propose an answer and then evaluate the probability P(True) that its answer is correct.
We find P(True) exciting in its performance, calibration, and scaling across a variety of tasks. The performance of the self-assessment improves further when we allow the model to consider many of its own samples before predicting the validity of a particular possibility. Next, we investigate whether we can train a model to predict P(IK), the probability of "I know the answer to the question", without reference to any specific suggested answer.
Authors: Chien-Yao Wang, Alexey Bochkovskiy, Hong-Yuan Mark Liao
Institution: Institute of Information Science, Academia Sinica
Author: David Dohan, Winnie Xu, Aitor Lewkowycz et al
Institution: Google
Author: Zuzeng Lin, Ailin Huang, Zhewei Huang et al
Institution: Wuhan University, Megvii Technology
Author: Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, etc.
Institution: DeepMind
Author: Phillip Rust, Jonas F. Lotz, Emanuele Bugliarello, etc.
Institution: University of Copenhagen, Johns Hopkins University, Uppsala University
##Top9: On the Principles of Parsimony and Self-Consistency for the Emergence of IntelligenceAuthors: Ma Yi, Cao Ying, Shen XiangyangInstitution: University of California, Berkeley, Guangdong-Hong Kong-Macao Greater Bay Area Digital Economy Research InstituteThis paper is a research review on the emergence and development of artificial intelligence published by Professor Ma Yi, computer scientist Dr. Shen Xiangyang, and neuroscientist Professor Cao Ying. It can be called an outline of the development of AI in the past 70 years. do. Top10:Scaling Laws vs Model Architectures:How does Inductive Bias Influence ScalingAuthor: Yi Tay, Mostafa Dehghani, Samira AbnarInstitution: Google, DeepMind
After reading the papers of the Top 10 experts, let’s talk about some interesting details of this list. As we all know, Twitter likes can be generated by robots. The author’s use of the number of likes as a key indicator for the list is indeed open to question. In addition, the previously highly popular "Infinite Visual Generation Model NUWA-Infinity" only ranked 12th in terms of number of likes on Twitter, but the number of Github stars has exceeded 2.4k. Since NUWA Infinity released its first version as early as November 2021, this list only counts the number of likes for the second version after that. Therefore it only ranks 12th.
The above is the detailed content of July’s most popular AI research list is released, with Ma Yi’s latest “Standard Model” ranked ninth. For more information, please follow other related articles on the PHP Chinese website!