I recently carried out Instruction-tuning on Llama2-7B with Parameter-Efficient Fine-Tuning methods like QLoRA. Here is what I learned.
Now that Llama 3 is out, I went straight to instruction-tuning it to see the kind of performance I can get. Along the way, I tried a new method for alignment called ORPO.
This is the first time I’ve worked on audio data for deep learning - here’s what I learned.
I’ve been coding applications around LLMs for the last ~6 months. I’ve used all sorts of frameworks and libraries to do so - from the OpenAI SDK to HuggingFace, LangChain, LangGraph, AutoGen, CrewAI - you name it, I’ve tried it out. Some of them majorly suck, while others are okay. Here’s what my experience has been like.
Transformers have been all the rage in NLP for the last few years, and there’s even been talk about whether LLMs have solved all the NLP problems, but S4 and Mamba models seem ready to take the world by storm.
I’m currently attending the Summer School by Climate Change AI, here’s my thoughts on what I learn there. I’ll periodically update this entry as I go through more weeks of the school, that lasts through August.