[GUEST ACCESS MODE: Data is scrambled or limited to provide examples. Make requests using your API key to unlock full data. Check https://lunarcrush.ai/auth for authentication information.]

[@akshay_pachaar](/creator/twitter/akshay_pachaar)
"You're in an ML Engineer interview at MistralAI. The interviewer asks: "We need an LLM that excels across code math & creative writing. How do you achieve multi-domain performance" You: "I'll increase the number of attention heads." Interview over. Here's what you missed:"  
[X Link](https://x.com/akshay_pachaar/status/1977714947477184858) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-13T12:36Z 231.7K followers, 145.1K engagements


"Temperature in LLMs clearly explained Temperature is a key sampling parameter in LLM inference. Today I'll show you what it means and how it actually works. Let's start by prompting OpenAI GPT-3.5 with a low temperature value twice. We observe that it produces identical responses from the LLM. Check this out👇 Now let's prompt it with a high temperature value. This time it produces gibberish output. Check the output below. What is going on here 🤔 Let's dive in.👇 Text-generating LLMs are like classification models with an output layer spanning the entire vocabulary. However instead of"  
[X Link](https://x.com/akshay_pachaar/status/1974817768852574515) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-05T12:43Z 231.7K followers, 86.5K engagements


"Agents without memory aren't agents at all. We often assume LLMs remember things they feel human after all. But the truth is: LLMs are stateless. If you want your agent to recall anything (past chats preferences behaviors) you have to build memory into it. But how to do that Let's understand this step-by-step Agent memory comes in two scopes: X Short-term Handles current conversations. Maintains message history context and state across a session. X Long-term Spans multiple sessions. Remembers preferences past actions and user-specific facts. But theres more. Just like humans long-term memory"  
[X Link](https://x.com/akshay_pachaar/status/1976636283637096612) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-10T13:09Z 231.7K followers, 63.9K engagements


"Did Stanford just kill LLM fine-tuning This new paper from Stanford called Agentic Context Engineering (ACE) proves something wild: you can make models smarter without changing a single weight. Here's how it works: Instead of retraining the model ACE evolves the context itself. The model writes its own prompt reflects on what worked and what didn't then rewrites it. Over and over. It becomes a self-improving system. Think of it like the model keeping a living notebook where every failure becomes a lesson and every success becomes a rule. The results are impressive: - XXXX% better than"  
[X Link](https://x.com/akshay_pachaar/status/1977068456622043635) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-11T17:47Z 231.7K followers, 66.6K engagements


"If anyone needs a video guide to Karpathy's nanochat check out Stanford's CS336 It covers: - Tokenization - Resource Accounting - Pretraining - Finetuning (SFT/RLHF) - Overview of Key Architectures - Working with GPUs - Kernels and Tritons - Parallelism - Scaling Laws - Inference - Evaluation - Alignment Everything you need to prepare for a job at Frontier AI Labs. I'm taking this course and will share my learnings here on X. Link to the playlist in the next tweet"  
[X Link](https://x.com/akshay_pachaar/status/1978140647283687679) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-14T16:47Z 231.7K followers, 116.1K engagements


"I just built an open NotebookLM clone Here's what it can do for you: - Process multi-modal data - Scrape websites and YouTube videos - Create a unified knowledge base - Lets you do RAG over it - Remember every conversation - Generate a podcast 🎙 The idea here is not to reinvent the wheel but to understand how one of the most powerful tools for learning and research actually works by building it step-by-step So by the end of this video you'll learn how to: Process multimodal data (text audio video URLs and YouTube videos) into a format ready for LLMs Store everything in a vector database for"  
[X Link](https://x.com/akshay_pachaar/status/1979166276946723241) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-17T12:43Z 231.7K followers, 69.7K engagements


"Google just dropped a new LLM You can run it locally on just XXX GB RAM. Let's fine-tune this on our own data (100% locally):"  
[X Link](https://x.com/akshay_pachaar/status/1956334618715800005) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-08-15T12:38Z 231.7K followers, 2M engagements


"LLM fine-tuning techniques I'd learn if I were to customize them: Bookmark this. X. LoRA X. QLoRA X. Prefix Tuning X. Adapter Tuning X. Instruction Tuning X. P-Tuning X. BitFit X. Soft Prompts X. RLHF XX. RLAIF XX. DPO (Direct Preference Optimization) XX. GRPO (Group Relative Policy Optimization) XX. RLAIF (RL with AI Feedback) XX. Multi-Task Fine-Tuning XX. Federated Fine-Tuning My favourite is GRPO for building reasoning models. What about you I've shared my full tutorial on GRPO in the replies"  
[X Link](https://x.com/akshay_pachaar/status/1975912824192053324) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-08T13:15Z 231.7K followers, 81.9K engagements


"7 LLM generation parameters explained visually:"  
[X Link](https://x.com/akshay_pachaar/status/1976996023890530506) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-11T12:59Z 231.7K followers, 61.5K engagements


"@zep_ai @milvusio @AssemblyAI @firecrawl_dev If you found it insightful reshare with your network. Find me @akshay_pachaar ✔ For more insights and tutorials on LLMs AI Agents and Machine Learning"  
[X Link](https://x.com/akshay_pachaar/status/1979167526425649585) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-17T12:48Z 231.7K followers, 5621 engagements


"8 RAG architectures for AI engineers:"  
[X Link](https://x.com/akshay_pachaar/status/1969379375582945474) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-09-20T12:33Z 231.6K followers, 140.4K engagements


"Attention heads capture patterns not domain expertise. More heads = richer representations in one pass. More experts = dedicated sub-networks for different knowledge types. The real answer: Mixture of Experts (MoE). Let's understand how MoE differs from standard Transformers:"  
[X Link](https://x.com/akshay_pachaar/status/1977714978334605716) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-13T12:36Z 231.7K followers, 7756 engagements


"Challenge 1) Notice this pattern at the start of training: - The model selects "Expert 2" - The expert gets a bit better - It may get selected again - The expert learns more - It gets selected again - It learns more - And so on Many experts go under-trained"  
[X Link](https://x.com/akshay_pachaar/status/1977715059389513734) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-13T12:36Z 231.6K followers, 2772 engagements


"We solve this in two steps: - Add noise to the feed-forward output of the router so that other experts can get higher logits. - Set all but top K logits to -infinity. After softmax these scores become zero. This way other experts also get the opportunity to train"  
[X Link](https://x.com/akshay_pachaar/status/1977715079379575241) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-13T12:36Z 231.6K followers, 2427 engagements


"MoEs have more parameters to load. However a fraction of them are activated since we only select some experts. This leads to faster inference. Mixtral 8x7B by @MistralAI is one famous LLM that is based on MoE. Here's the visual again that compares Transformers and MoE"  
[X Link](https://x.com/akshay_pachaar/status/1977715113835761883) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-13T12:36Z 231.6K followers, 5265 engagements


"4 LlamaCPP (the OG) @ggerganov 's LlamaCPP enables LLM inference with minimal setup and state-of-the-art performance. Here's DeepSeek R-1 running on a Mac Studio 🪄"  
[X Link](https://x.com/akshay_pachaar/status/1887850312079442391) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-02-07T13:06Z 231.7K followers, 8814 engagements


"Traditional RAG vs. Agentic RAG clearly explained (with visuals):"  
[X Link](https://x.com/akshay_pachaar/status/1955245388241440925) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-08-12T12:30Z 231.7K followers, 685.2K engagements


"A simple technique makes RAG up to 40x faster & 32x memory efficient - Perplexity uses it in its search index - Google uses it in Vertex RAG engine - Azure uses it in its search pipeline Let's understand how to use it in a RAG system (with code):"  
[X Link](https://x.com/akshay_pachaar/status/1958510665217532012) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-08-21T12:45Z 231.7K followers, 520.9K engagements


"8 key skills to become a full-stack AI Engineer:"  
[X Link](https://x.com/akshay_pachaar/status/1964680446446567822) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-09-07T13:21Z 231.7K followers, 494.2K engagements


"A layered overview of key Agentic AI concepts. Let's understand it step-by-step: X LLMs (the foundation layer) At the core you have LLMs like GPT DeepSeek etc. Core ideas: - Tokenization & inference: how text is processed by the model - Prompt engineering: designing inputs for better outputs - LLM APIs: programmatic interfaces to interact with models This is the engine that powers everything else. X AI Agents (built on LLMs) Agents wrap around LLMs to enable autonomous action. Key responsibilities: - Tool usage & function calling: connecting LLMs to external APIs/tools - Agent reasoning:"  
[X Link](https://x.com/akshay_pachaar/status/1975540193584370048) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-07T12:34Z 231.7K followers, 51K engagements


"Link to the GitHub repo:"  
[X Link](https://x.com/akshay_pachaar/status/1977351089256272342) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-12T12:30Z 231.7K followers, 5122 engagements


"Generative vs. discriminative models in ML: (a popular ML interview question)"  
[X Link](https://x.com/akshay_pachaar/status/1978439238376112311) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-15T12:34Z 231.7K followers, 33K engagements


"Tech stack: - @zep_ai for memory - @milvusio as vectorDB - @AssemblyAI for STT - @firecrawl_dev for web-scraping - Kokoro 82M for podcast generation (100% open-source) You can find all the code here:"  
[X Link](https://x.com/akshay_pachaar/status/1979167144207126816) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-17T12:46Z 231.7K followers, 4992 engagements


"@tricalt @zep_ai @milvusio @AssemblyAI @firecrawl_dev 💯 The code is fully modular. You can easily add what you want. I used my favourite tool :)"  
[X Link](https://x.com/akshay_pachaar/status/1979170517799096752) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-17T12:59Z 231.7K followers, XX engagements


"@_avichawla This is super helpful Thanks for sharing Avi"  
[X Link](https://x.com/akshay_pachaar/status/1979436426220703765) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-18T06:36Z 231.7K followers, XXX engagements


"@alex_mikhalev How far do you think that would be Karpathy in his recent interview said AGI is at least a decade away"  
[X Link](https://x.com/akshay_pachaar/status/1979778792269955522) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-19T05:17Z 231.7K followers, XX engagements


"Train gpt-oss using RL on just 15GB VRAM:"  
[X Link](https://x.com/akshay_pachaar/status/1974527757775421530) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-04T17:31Z 231.7K followers, 67.9K engagements


"You're in a Research Scientist interview at OpenAI. The interviewer asks: "How would you expand the context length of an LLM from 2K to 128K tokens" You: "I will fine-tune the model on longer docs with 128K context" Interview over. Here's what you missed:"  
[X Link](https://x.com/akshay_pachaar/status/1975178590447874144) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-06T12:37Z 231.7K followers, 153.3K engagements


"Context Engineering Template for AI Agents A complete system for comprehensive context engineering. Includes documentation examples rules and patterns. XXX% open-source"  
[X Link](https://x.com/akshay_pachaar/status/1977351076660785336) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-12T12:30Z 231.7K followers, 43.9K engagements


"Transformer vs. Mixture of Experts (MoEs):"  
[X Link](https://x.com/akshay_pachaar/status/1977996863367467053) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-14T07:16Z 231.7K followers, 83.6K engagements


"Building your first AI Agent: a clear path Bookmark this. Everyone wants to build agents but most get stuck between theory and chaos. Here's a process Ive used to ship agents that actually work not just prototypes. X Start with a smaller problem Forget universal agents. Pick a clear solvable task. Examples: Automatically book a flight Summarise long research papers Monitor price drops for a product and notify via email Small scope faster iteration easy to debug. X Use a reliable LLM Start with GPT Claude or Ollama (if self-hosting). Your focus should be wiring not fine-tuning. X Build an"  
[X Link](https://x.com/akshay_pachaar/status/1978083834043199612) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-14T13:01Z 231.7K followers, 15.5K engagements


"100% open-source tech stack: - @crewAIInc for building MCP-ready agents - @zep_ai Graphiti to add human-like memory - @Cometml Opik for observability and tracing Find the code here:"  
[X Link](https://x.com/akshay_pachaar/status/1978084188726206801) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-14T13:03Z 231.7K followers, 4217 engagements


"I'm just hosting it on Lightning AI so that's no hard requirement. Zep also has an open-source self-hosting version that you can use. (Graphiti) Most of the code is modular and you can easily replace it with another tool that you prefer. The podcast generation part is XXX% open-source and local"  
[X Link](https://x.com/akshay_pachaar/status/1979429259203444962) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-18T06:08Z 231.7K followers, XXX engagements


"Finally we have a definition for AGI A recent paper breaks down what it actually means to reach human-level AI and the results are surprising. Here's what matters: AGI isn't about passing a single test or excelling at one task. It's about matching the cognitive versatility and skill of a well-educated adult across XX fundamental areas. The paper breaks down human intelligence into XX cognitive domains to create an AGI Score (0-100%). Each domain gets equal weight: - General Knowledge - Reading & Writing - Mathematical Ability - On-the-Spot Reasoning - Working Memory - Long-Term Memory Storage"  
[X Link](https://x.com/akshay_pachaar/status/1979528949266039266) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-18T12:44Z 231.7K followers, 29.2K engagements


"uv is the best thing that has happened to Python devs And you won't find a better cheatsheet than this:"  
[X Link](https://x.com/akshay_pachaar/status/1979598428511568243) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-18T17:20Z 231.7K followers, 93.2K engagements


"if you're looking for a comprehensive guide to LLM finetuning check this a free 115-page book on arxiv covering: fundamentals of LLM peft (lora qlora dora hft) alignment methods (ppo dpo grpo) mixture of experts (MoE) 7-stage fine-tuning pipeline multimodal finetuning & challenges industrial frameworks (hf sagemaker openai) everything you need to know in one place download link in the replies"  
[X Link](https://x.com/akshay_pachaar/status/1979919605570408553) [@akshay_pachaar](/creator/x/akshay_pachaar) 2025-10-19T14:36Z 231.7K followers, 36.8K engagements