Prompt Engineering

The Prompt Report: A Systematic Survey of Prompting Techniques
This 76-page paper on Prompting Techniques has become quite popular.
A nice read for your weekend. – “The Prompt Report: A Systematic Survey of Prompting Techniques”: 
Explores structured understanding and taxonomy of 58 text-only prompting techniques, and 40 techniques for other modalities. 
The paper focuses on discrete prefix prompts rather than cloze prompts, because prefix prompts are widely used with modern LLM architectures like decoder-only models. It excludes soft prompts and techniques using gradient-based updates. 
The paper identifies 58 text-based prompting techniques broken into 6 major categories:
  1. 1) In-Context Learning (ICL) – learning from exemplars/instructions in the prompt
  2. 2) Zero-Shot – prompting without exemplars
  3. 3) Thought Generation – prompting the LLM to articulate reasoning
  4. 4) Decomposition – breaking down complex problems
  5. 5) Ensembling – using multiple prompts and aggregating outputs
  6. 6) Self-Criticism – having the LLM critique its own outputs 

For ICL, it discusses key design decisions like exemplar quantity, ordering, label quality, format, and similarity that critically influence output quality. It also covers ICL techniques like K-Nearest Neighbor exemplar selection. 

Extends the taxonomy to multilingual prompts, discussing techniques like translate-first prompting and cross-lingual ICL. It also covers multimodal prompts spanning image, audio, video, segmentation, and 3D modalities. 

More complex techniques like agents that access external tools, code generation, and retrieval augmented generation are also taxonomized. Evaluation techniques using LLMs are discussed. 

Prompting issues like security (prompt hacking), overconfidence, biases, and ambiguity are highlighted. Two case studies – benchmarking techniques on MMLU and an entrapment detection prompt engineering exercise – are presented.








Leave a Reply

Your email address will not be published. Required fields are marked *