Low Rank Adaptations. What are they, implementation and are they useful in smaller models?
Low Rank Adaptations (LoRA) are a hot topic in deep learning, specially for huge models like GPT-3 or StableDiffusion. In summary, LoRA allows for efficient ...