770 million parameters, exceeding 540 billion PaLM! UW Google proposes "step-by-step distillation", which only requires 80% of training data | ACL 2023

Large language models excel in performance and are able to solve new tasks with zero- or few-shot hints. However, in actual application deployment, LLM is not very practical because its memory utilization efficiency is low and it requires a large amount of computing resources. For example, running a language model service with 175 billion parameters requires at least 350GB of video memory. Most of the current most advanced language models have more than 500 billion parameters. Many research teams do not have enough resources to run them, and they cannot meet the low-latency performance in real applications.

There are also some studies using manually labeled data or distillation using LLM-generated labels to train smaller, task-specific models, but fine-tuning and distillation require a large amount of training data to achieve comparable performance to LLM.

In order to solve the problem of resource requirements for large models, the University of Washington and Google collaborated to propose a new distillation mechanism called "Distilling Step-by-Step". Through step-by-step distillation, the size of the distilled model is smaller than the original model, but the performance is better, and less training data is required during the fine-tuning and distillation process

Please click the following link to view the paper: https://arxiv.org/abs/2305.02301

The distribution distillation mechanism extracts the prediction reason from LLM (rationale ) as additional supervisory information for training small models within a multi-task framework.

After experiments on 4 NLP benchmarks, we found:

1. Compared with fine-tuning and distillation, this mechanism uses less Training samples achieve better performance;

Compared with few-sample prompt LLM, this mechanism uses smaller size models to achieve better performance

3. At the same time, it reduces the model size and The data volume can also achieve better performance than LLM.

In the experiment, the 770M T5 model after fine-tuning was better than the 540B PaLM model with few sample hints in the benchmark test using only 80% of the available data, while the T5 model with the same standard fine-tuning even used 100% Data sets are also difficult to match.

Distillation method

The key idea of ​​distribution distillation is to gradually extract information-rich prediction reasons described in natural language, that is, intermediate reasoning steps, to explain the connection between the input problem and the model output , and use these data to train small models more efficiently

Distribution distillation mainly includes two stages:

1. From Extraction principle (rationale) in LLMResearchers use the few-sample Chain of Thinking (CoT) prompt to extract the prediction intermediate steps from LLM.

After determining the target task, first prepare several samples in the LLM input prompt. Each example consists of a triplet, including input, principle and output

After inputting prompts, LLM can imitate the triplet demonstration to generate other new problems. Prediction principle, for example, in the common sense question and answer task, given

input question:

Sammy wants to go to a place where crowds gather. Where will he choose? The options are: (a) densely populated area, (b) race track, (c) desert, (d) apartment, (e) roadblock

(Sammy wanted to go to where the people are. Where might he go? Answer Choices: (a) populated areas, (b) race track, (c) desert, (d) apartment, (e) roadblock）

After gradual refinement, LLM You can give the correct answer to the question "(a) Densely populated area" and provide a reason for answering the question "The answer must be a place with many people. Among the above choices, only densely populated areas have many people." After gradual refinement, LLM was able to conclude that the correct answer is "(a) densely populated area" and provided the reason for answering the question "The answer must be a place with many people. Among the above choices, only densely populated areas have many people." people."

By providing CoT examples paired with rationales in prompts, the contextual learning capability allows LLM to generate appropriate answer reasons for unencountered question types

2. Training Mini Model

By constructing the training process as a multi-task problem, the reasons for prediction can be extracted and incorporated into the training small model

In addition to the standard label prediction task In addition, the researchers also used a new reason generation task to train a small model, so that the model can learn to generate intermediate reasoning steps for prediction, and guide the model to better predict the result label.

Distinguish label prediction and reason generation tasks by adding the task prefixes "label" and "rationale" to the input prompt.

Experimental results

In the experiment, the researchers selected the PaLM model with 540 billion parameters as the LLM baseline, and used the T5 model as the task-related downstream small model.

In this study, we conducted experiments on four benchmark datasets, namely e-SNLI and ANLI for natural language reasoning, CQA for common sense question answering, and SVAMP for Application problems in arithmetic and mathematics. We conducted experiments on these three different NLP tasks

Less training data

The stepwise distillation method performs better than Standard fine-tuning is better and requires less training data

On the e-SNLI dataset, better performance than standard fine-tuning is achieved when using 12.5% ​​of the full dataset, on ANLI, Only 75%, 25% and 20% of the training data are required on CQA and SVAMP respectively.

Compared to standard fine-tuning using a 220M T5 model on manually labeled datasets of varying sizes, distribution distillation is superior when using fewer training examples across all datasets. Distribution compared to LLM prompted by few-shot CoT Distillation results in a model that is much smaller in size but performs better.

On the e-SNLI data set, using the 220M T5 model achieves better performance than the 540B PaLM; on ANLI, using the 770M T5 model achieves better performance than the 540B PaLM. Model size is only 1/700Smaller model, less data

While reducing model size and training data , we successfully achieved performance exceeding few-shot PaLM

In ANLI, outperforming 540B PaLM using a 770M T5 model, while using only 80% of the full dataset

It has been observed that standard fine-tuning cannot reach the performance level of PaLM even with the complete 100% data set, indicating that stepwise distillation can simultaneously reduce the model size and the amount of training data, thereby achieving Performance beyond LLM

The above is the detailed content of 770 million parameters, exceeding 540 billion PaLM! UW Google proposes "step-by-step distillation", which only requires 80% of training data | ACL 2023. For more information, please follow other related articles on the PHP Chinese website!