微调和很少的射击学习之间有什么区别？
我试图理解微调和的概念， 学习。
我了解进行微调的必要性。它本质上是将预训练的模型调整为特定的下游任务。但是，最近，我看到了很多博客文章，上面说明了零照片学习，一声学习和很少的学习。
- 它们与微调有何不同？在我看来，很少有学习是对微调的专业化。我在这里想念什么？
有人可以帮我吗？
I am trying to understand the concept of fine-tuning and few-shot learning.
I understand the need for fine-tuning. It is essentially tuning a pre-trained model to a specific downstream task. However, recently I have seen a plethora of blog posts stating zero-shot learning, one-shot learning and few-shot learning.
- How are they different from fine-tuning? It appears to me that few-shot learning is a specialization of fine-tuning. What am I missing here?
Can anyone please help me?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
微调 - 当您已经进行了经过培训的模型来执行所需的任务时，但是在另一个数据集上，您可以使用预训练的权重并在目标（通常较小）数据集（通常具有较小的学习率）上进行训练。
很少有射击学习 - 当您想使用极少数样品训练任何任务时的模型时。例如，您有一个在不同但相关的任务上培训的模型，并且（选项）将其修改并使用少量示例训练目标任务。
例如：
微调 - 培训模型以进行意图分类，然后在其他数据集上进行微调。
很少的镜头学习 - 在大型文本数据集上培训语言模型，并修改（通常是最后一层），以通过对小标签数据集进行培训来对意图进行分类。
可能还有更多的方法可以进行几次射击学习。对于另外1个例子，训练模型以对某些课程非常小的图像进行分类（零射门为0，一击为1个）训练样本数量。在推论中，正确对这些稀有类别（在训练中很少见）进行分类成为很少射击学习的目的。
Fine tuning - When you already have a model trained to perform the task you want but on a different dataset, you initialise using the pre-trained weights and train it on target (usually smaller) dataset (usually with a smaller learning rate).
Few shot learning - When you want to train a model on any task using very few samples. e.g., you have a model trained on different but related task and you (optionally) modify it and train for target task using small number of examples.
For example:
Fine tuning - Training a model for intent classification and then fine tuning it on a different dataset.
Few shot learning - Training a language model on large text dataset and modifying it (usually last (few) layer) to classify intents by training on small labelled dataset.
There could be many more ways to do few shot learning. For 1 more example, training a model to classify images where some classes have very small (or 0 for zero shot and 1 for one shot) number of training samples. Here in inference, classifying these rare classes (rare in training) correctly becomes the aim of few shot learning.
我想在这里添加几点。微调和一次性镜头学习都是执行转移学习的技术。转移学习正在利用经过某些任务训练的模型来执行其他事情。
简而言之，转移学习是从一个任务中利用知识来提高另一个相关任务的绩效的总体概念。微调，一次性学习和少数学习是转移学习中用于实现此目标的特定技术，每个技术都采用自己的方法来利用从源任务或有限的标记数据中固有的知识，以提高在目标任务。
There are couple of points I would like to add here. Both fine tuning and one-shot, few shot learning are techniques to perform transfer learning. Transfer learning is leveraging a model trained on certain task to perform something else.
In short, transfer learning is the overarching concept of leveraging knowledge from one task to improve performance on another related task. Fine-tuning, one-shot learning, and few-shot learning are specific techniques used within transfer learning to achieve this goal, each with its own approach to leveraging pre-existing knowledge from a source task or limited labeled data to improve performance on a target task.
构建LLM的第一阶段是
预读其中llm在大型，多样化的数据集上训练，以发展对语言的广泛理解。预训练的目的是学习可以应用于各种任务和领域的通用语言模式，关系和表示。第二阶段是
微调。预处理后，可以在较窄的数据集（预处理数据集）上微调该模型，该数据集对特定任务或域更为特定。微调涉及调整模型的权重以更好地适合特定的任务或域，同时仍利用训练期间学到的一般知识和模式。预读就像在开始上学时学习阅读和写作一样。您将学习字母，语法规则和基本词汇。微调就像专门研究特定的主题或流派，例如科幻或医学写作。您基于对语言的一般知识（预处理的模型，仍然具有这些一般技能）并适应以满足您的特定需求（微调模型）。
几乎没有学习就像学习新方言或口音一样。您已经知道语言的基础知识（验证模型），并且您专门研究了特定的主题或流派（微调模型）。现在，您接触了一些新方言或口音的示例，并且很快就会挑选细微差别，并适应您的语言技能，以与母语人士有效沟通。First stage of building a LLM is
pretrainingwhereLLMis trained on a large, diverse dataset to develop a broad understanding of language. The goal of pretraining is to learn general language patterns, relationships, and representations that can be applied to various tasks and domains.Second stage is
Fine-tuning. After pretraining, the model can be fine-tuned on a narrower dataset (Pretraining datasets are massive) that is more specific to particular tasks or domains. Fine-tuning involves adjusting the model's weights to better fit the specific task or domain, while still leveraging the general knowledge and patterns learned during pretraining.Pretrainingis like learning to read and write when you start to go to school. You learn the alphabet, grammar rules, and basic vocabulary.Fine-tuning is like specializing in a particular subject or genre, such as science fiction or medical writing. You build upon your general knowledge of language (pretrained model, you still have those general skills) and adapt it to fit your specific needs (fine-tuned model).
Few-shot learningis like learning a new dialect or accent. You already know the basics of language (pretrained model), and you've specialized in a particular subject or genre (fine-tuned model). Now, you're exposed to a few examples of the new dialect or accent, and you quickly pick up on the nuances and adapt your language skills to communicate effectively with native speakers.