LoRA Explained

YOUR LINK HERE:

http://youtube.com/watch?v=Bq9zqTJDsjg

A parameter efficient fine tuning technique that makes use of a low rank adapter to (1) reduce storage required per task by decreasing the number of trainable parameters added to the network per task (2) remove inference latency ensuring the stored parameters are applied to the existing network architecture instead of adding more • RESOURCES • [1 📚] Paper that introduces LoRA: https://arxiv.org/pdf/2106.09685 • [2 📚] Paper that introduced the additive adapter: https://arxiv.org/abs/2312.12148 • [3 📚] Other PEFT techniques: https://arxiv.org/pdf/2312.12148 • [4 📚] My video on Parameter Efficient Fine Tuning (PEFT): • LLM (Parameter Efficient) Fine Tuning... • [5 📚] Neural networks tend to be more complex than required, hence we can make use of a low rank adapter without losing performance accuracy: https://arxiv.org/pdf/1804.08838 • ABOUT ME • ⭕ Subscribe: https://www.youtube.com/c/CodeEmporiu... • 📚 Medium Blog: / dataemporium • 💻 Github: https://github.com/ajhalthor • 👔 LinkedIn: / ajay-halthor-477974bb • PLAYLISTS FROM MY CHANNEL • ⭕ Deep Learning 101: • Deep Learning 101 • ⭕ Natural Language Processing 101: • Natural Language Processing 101 • ⭕ Reinforcement Learning 101: • Reinforcement Learning 101 • Natural Language Processing 101: • Natural Language Processing 101 • ⭕ Transformers from Scratch: • Natural Language Processing 101 • ⭕ ChatGPT Playlist: • ChatGPT • MATH COURSES (7 day free trial) • 📕 Mathematics for Machine Learning: https://imp.i384100.net/MathML • 📕 Calculus: https://imp.i384100.net/Calculus • 📕 Statistics for Data Science: https://imp.i384100.net/AdvancedStati... • 📕 Bayesian Statistics: https://imp.i384100.net/BayesianStati... • 📕 Linear Algebra: https://imp.i384100.net/LinearAlgebra • 📕 Probability: https://imp.i384100.net/Probability • OTHER RELATED COURSES (7 day free trial) • 📕 ⭐ Deep Learning Specialization: https://imp.i384100.net/Deep-Learning • 📕 Python for Everybody: https://imp.i384100.net/python • 📕 MLOps Course: https://imp.i384100.net/MLOps • 📕 Natural Language Processing (NLP): https://imp.i384100.net/NLP • 📕 Machine Learning in Production: https://imp.i384100.net/MLProduction • 📕 Data Science Specialization: https://imp.i384100.net/DataScience • 📕 Tensorflow: https://imp.i384100.net/Tensorflow • CHAPTERS • 0:00 Introduction • 1:49 Pass 1: Low Rank Matrices • 8:00 Quiz 1 • 8:52 Pass 2: Adapters • 16:38 Quiz 2 • 17:47 Pass 3: Low Rank Adapters • 26:37 Quiz 3 • 27:54 Summary

#############################

New on site