I’ve been coding applications around LLMs for the last ~6 months. I’ve used all sorts of frameworks and libraries to do so - from the OpenAI SDK to HuggingFace, LangChain, LangGraph, AutoGen, CrewAI - you name it, I’ve tried it out. Some of them majorly suck, while others are okay. Here’s what my experience has been like.
Note: I also recently gave a lightning talk on this at PyData London, you can check out the slides for that talk here: Slides
I first started my journey with LLMs almost two years ago, while I was writing my Master’s thesis on Abstractive Summarisation after getting really interested in Language Models through my NLP class. The GPT3.5-backed LLM explosion had not happened completely yet, and so we were still focusing on individual NLP tasks with particular networks instead of just using the newest LLM to fine-tune or few-shot a certain task. I used LSTMs and Transformers with Pointer-Generator mechanism (think sort of basic RAG over the input data to decide when to copy and when to generate new tokens). A few months later, right about when I was submitting my thesis, GPT3.5 was released and we slowly and then all at once ushered into an era of LLMs subsuming most NLP research. I personally don’t think this is a good thing - simple tasks might require a kitchen knife, not all of them need to be solved with a swiss-army knife.
In the ensuing chaos, a somewhat unstable ecosystem of tools has built itself around the major LLMs (which mostly just means OpenAI for all intents and purposes, but Anthropic, Mistral, Cohere et. al. seem to be in the running for their own small pieces of the pie).
These are the popular frameworks: