PEFT is a method designed to fine-tune large language models without updating all the parameters of the model. The main idea is to adapt the model to new tasks or domains by modifying only a small subset of parameters, rather than the entire model. LoRA is a specific technique under the umbrella of PEFT. It involves adding low-rank adaptation matrices to the weights of the language model. Instead of updating the entire weight matrix during fine-tuning, LoRA introduces low-rank matrices that approximate the necessary updates. These matrices are much smaller than the original weight matrices, thus requiring fewer parameters to be updated.