Notes on using examples in prompts (X-shot prompting)
2024-09-27
There are different approaches for providing instructions or examples to a model to perform a task. In particular, one method is to give the LLM some examples. As usual, the terminology is complex, but the ideas are simple.
Let's break down each one and compare them:
Zero-shot Prompting
Zero-shot prompting is when a model is asked to perform a task without being given any examples. Instead, the task is described using only the natural language prompt, and the model has to rely solely on its pre-trained knowledge.
Example prompt for zero shot
Translate the following sentence to French: 'I am happy' → 'Je suis content.'
One-shot Prompting
In one-shot prompting, the model is provided with a single example along with the prompt to help it understand the task better. This can help the model generate more relevant responses by learning from that one instance.
Example prompt for one shot
Translate the following sentence to French: 'I am happy' → 'Je suis content.'
Now translate this sentence: 'I am hungry.'
Few-shot Prompting
Few-shot prompting involves providing the model with a small number (typically 2–5) of task-specific examples before asking it to perform the task. This approach helps the model better understand the structure or context required.
Example prompt for few shots
Translate the following sentence to French: 'I am happy' → 'Je suis content.'
Translate the following sentence to French: 'She is tall' → 'Elle est grande.'
Now translate this sentence: 'I am hungry.'
How they compare
| Aspect | Zero-shot Prompting | One-shot Prompting | Few-shot Prompting |
|---|---|---|---|
| Number of Examples | None | One | A few (usually 2–5) |
| Task Understanding | Relies entirely on the model's pre-trained knowledge and inference | Has minimal help from a single example | Gains better understanding with multiple examples |
| Model Generalization | Requires strong generalization ability since no specific task info is provided | Slightly easier for the model since it sees an example | More guidance is provided, improving performance on the task |
| Ease for Model | Harder for the model to perform well since no guidance is given | Easier than zero-shot, but still somewhat challenging | Easiest among the three; the model has better context from examples |
| Potential Errors | More prone to errors or confusion due to lack of task clarity | Can still misunderstand if the task is complex | Fewer errors, as multiple examples help disambiguate the task |
| Use Case | Useful when there are no available examples or when testing model's generalized capabilities | Best when limited task-specific data is available | Effective when a few high-quality examples can be provided |