#  @lateinteraction Omar Khattab
Omar Khattab posts on X about llm, ai, open ai, databricks the most. They currently have [------] followers and [---] posts still getting attention that total [---------] engagements in the last [--] hours.
### Engagements: [---------] [#](/creator/twitter::1605274291569799168/interactions)
### Mentions: [--] [#](/creator/twitter::1605274291569799168/posts_active)
### Followers: [------] [#](/creator/twitter::1605274291569799168/followers)
### CreatorRank: [-------] [#](/creator/twitter::1605274291569799168/influencer_rank)
### Social Influence
**Social category influence**
[technology brands](/list/technology-brands) [social networks](/list/social-networks) [stocks](/list/stocks) [finance](/list/finance) [currencies](/list/currencies) [countries](/list/countries) [celebrities](/list/celebrities) [events](/list/events) [cryptocurrencies](/list/cryptocurrencies) [fashion brands](/list/fashion-brands)
**Social topic influence**
[llm](/topic/llm) #91, [ai](/topic/ai), [open ai](/topic/open-ai), [databricks](/topic/databricks), [if you](/topic/if-you), [$googl](/topic/$googl), [context window](/topic/context-window) #23, [in the](/topic/in-the), [level](/topic/level), [rl](/topic/rl)
**Top accounts mentioned or mentioned by**
[@dspyoss](/creator/undefined) [@databricks](/creator/undefined) [@maximerivest](/creator/undefined) [@bclavie](/creator/undefined) [@raw_works](/creator/undefined) [@dosco](/creator/undefined) [@tomdoerr](/creator/undefined) [@dbreunig](/creator/undefined) [@a1zhang](/creator/undefined) [@teknium](/creator/undefined) [@jchencxh](/creator/undefined) [@chrisgpotts](/creator/undefined) [@shoggoth1313](/creator/undefined) [@cshorten30](/creator/undefined) [@kristahopsalong](/creator/undefined) [@isaacbmiller1](/creator/undefined) [@goobyesq](/creator/undefined) [@deliprao](/creator/undefined) [@arnavthebigman](/creator/undefined) [@hanchunglee](/creator/undefined)
**Top assets mentioned**
[Alphabet Inc Class A (GOOGL)](/topic/$googl)
### Top Social Posts
Top posts by engagements in the last [--] hours
"Want to learn more about retrieval both key concepts and cutting edge models & tools Watch our recent conversation at the LangChain Retrieval Webinar w/ @hwchase17 @jobergum @charles_irl I discuss the retrieval model and the programming model from 29:00"
[X Link](https://x.com/lateinteraction/status/1662217467790499841) 2023-05-26T22:01Z [----] followers, [--] engagements
"@marktenenholtz I dont say this often but I dont think langchain was ever meant to provide this. It remains great for I dont want to think about this give me an easy thing that runs on my data. If you wanna build novel stuff youre looking for a research framework like our own DSP"
[X Link](https://x.com/lateinteraction/status/1683662560405647365) 2023-07-25T02:16Z [----] followers, [----] engagements
"@marktenenholtz This is not a failure of langchain. People are just projecting expectations never espoused by the library to my knowledge. You cant build new activation functions or new optimizers using HuggingFace APIs. Thats not what they do. You do that in PyTorch"
[X Link](https://x.com/lateinteraction/status/1683663153241133056) 2023-07-25T02:19Z [----] followers, [---] engagements
"@Nils_Reimers @macavaney @jobergum Some slice of MS MARCO might actually be decebt at this. The lexical retrieval (aka Bing index circa 2016) was at the level of full documents and had access to lots of (unreleased) metadata. Its just that the corpus is not topically realistic and the queries are too easy/head"
[X Link](https://x.com/lateinteraction/status/1689105848545714176) 2023-08-09T02:46Z [----] followers, [---] engagements
"Exactly. Folks often ask "ColBERT is great but wouldn't the embeddings take a lot of space" Nope. ColBERTv2 with PLAID (see QT) can use as little as *20 bytes* per vector. Same size as [--] floats. Can fit [---------] passages in 200GB and search them in [---] milliseconds on CPU"
[X Link](https://x.com/lateinteraction/status/1690870655758245888) 2023-08-13T23:39Z [----] followers, 49.9K engagements
"@mystellial It's a very minimal framework (not a lot of code but very rich ideas) so multi-language support may not be too hard. Down the road would happily help with a fork in any language that has enough meta-programming tricks to make things work cleanly"
[X Link](https://x.com/lateinteraction/status/1694816021503221992) 2023-08-24T20:56Z [----] followers, [--] engagements
"@gkossakowski Thank you @gkossakowski No limitation in there but GPT-4 is slow and expensive. We can do really well with GPT-3.5 (or even just local open models). That said GPT-4 can probably enable somewhat fancier / more ambitious pipelines"
[X Link](https://x.com/lateinteraction/status/1694827350267982085) 2023-08-24T21:41Z 16K followers, [---] engagements
"@imaurer Oh Ian do you think it should be emphasized more"
[X Link](https://x.com/lateinteraction/status/1694836667746590766) 2023-08-24T22:18Z [----] followers, [---] engagements
"@ofermend @MetaAI @OpenAI @anyscalecompute @databricks @MSFTDeepSpeed Yes such a week But misses like the most important release of late last week :-) https://x.com/lateinteraction/status/1694748401374490946 🚨Announcing the framework for solving advanced tasks w/ LMs. Express *any* pipeline as clean Pythonic control flow. Just ask DSPy to your modular code into auto-tuned chains of prompts or finetunes for GPT Llama and/or T5.🧵 https://t.co/QColxp8atV https://t.co/PbwRPrctSp https://x.com/lateinteraction/status/1694748401374490946 🚨Announcing the framework for solving advanced tasks w/"
[X Link](https://x.com/lateinteraction/status/1696051441645875395) 2023-08-28T06:45Z 16.3K followers, [---] engagements
""We need an AI compiler" DSPy exists right here: Like PyTorch but for foundation model programming. You write your own control flow. The compiler optimizes prompting/finetuning for the declarative steps of your pipelineaccording to your metrics"
[X Link](https://x.com/lateinteraction/status/1697081890425983472) 2023-08-31T03:00Z [----] followers, 28.7K engagements
"Really enjoyed visiting my undergraduate alma mater Carnegie Mellon Qatar. I presented "ColBERT: Effective and Efficient Search with Late Interaction" to the CS faculty and a room full of the particularly sharp and engaged CMUQ ugrads. Thanks for hosting me @CarnegieMellonQ"
[X Link](https://x.com/lateinteraction/status/1697369626194125209) 2023-08-31T22:03Z 24.9K followers, [----] engagements
"@annargrs @yoavgo @Francis_YAO_ This is the only reason I submit these to non-ACL confs. This is a common challenge now with foundation models/frameworks. I dont need to advertise a paperjust the latest artifacts. But the key goal is to help researchers on active projects so waiting months is infeasible"
[X Link](https://x.com/lateinteraction/status/1699071483115606314) 2023-09-05T14:46Z [----] followers, [---] engagements
"@annargrs @yoavgo @Francis_YAO_ Obviously this type of research poses a different type of challenge for anonymity in general. Id be fine with an artifact-first track that has different rules or expectations but this is an increasingly essential and impactful type of NLP research IMO"
[X Link](https://x.com/lateinteraction/status/1699072078216093736) 2023-09-05T14:48Z [----] followers, [--] engagements
"@sherjilozair @yacineMTB @attharave This realistic understanding is important. But it doesnt change what you should do in terms of the work"
[X Link](https://x.com/lateinteraction/status/1699465442899734614) 2023-09-06T16:51Z [----] followers, [---] engagements
"@khoomeik Its actually deeper than that smh. RAG is an excellent paper but it didnt invent conditioning an LM on some retrieved context This is a much older idea The papers novel explorations are about joint training and marginalization etc which arent even applicable to langchain"
[X Link](https://x.com/lateinteraction/status/1700226146330112083) 2023-09-08T19:14Z [----] followers, [---] engagements
"@marktenenholtz A lot of people use DPR-style models:"
[X Link](https://x.com/lateinteraction/status/1702016995565429118) 2023-09-13T17:50Z [----] followers, [----] engagements
"@marktenenholtz Which actually includes OpenAIs own embedding models in a sense"
[X Link](https://x.com/lateinteraction/status/1702017099781259522) 2023-09-13T17:51Z [----] followers, [----] engagements
"@douwekiela This seems awesome We need to add this to the DSPy compiler so we explore a larger search space"
[X Link](https://x.com/lateinteraction/status/1702801218815607125) 2023-09-15T21:47Z 16.3K followers, [---] engagements
"@WenhuChen @b_arbaretier Oh Wenhu let's chat soon. We're adding a general-purpose ProgramOfThought module like a nn layer in pytorch which can be optimized to fit any multi-stage pipeline and any task/metric. Would be cool if you check it out and see if it matches how you'd generalize PoT yourself"
[X Link](https://x.com/lateinteraction/status/1703104113989529841) 2023-09-16T17:50Z [----] followers, [---] engagements
"@generatorman_ai @datagenproc Yeah for instance colbertv2 can find the passage for this. CTRL-F for Tom Cruise in there"
[X Link](https://x.com/lateinteraction/status/1705617556810371072) 2023-09-23T16:18Z [----] followers, [--] engagements
"@omarsar0 Prompt engineering is needed the same way hyperparameters are important. The architecture design matters a lot more than either. And both can be automatically optimized: https://github.com/stanfordnlp/dspy https://github.com/stanfordnlp/dspy"
[X Link](https://x.com/lateinteraction/status/1705636135958135088) 2023-09-23T17:31Z 16K followers, [----] engagements
"@gazorp5 @jkronand Thanks for engaging. If youd like to read more start in this section:"
[X Link](https://x.com/lateinteraction/status/1705885175643873287) 2023-09-24T10:01Z [----] followers, [----] engagements
"@MystiqCaleid @jkronand It works extremely well with agentsway better than existing stuff. Will put out some threads on that"
[X Link](https://x.com/lateinteraction/status/1705920869145755869) 2023-09-24T12:23Z 16.3K followers, [----] engagements
"Right LLMs are fuzzy devices and everyone writing prompts is coding in assembly. You could be writing well-defined modules instead and asking a compiler to build and optimize the messy parts: With many 🧩 dropping recently a more complete picture is emerging of LLMs not as a chatbot but the kernel process of a new Operating System. E.g. today it orchestrates: - Input & Output across modalities (text audio vision) - Code interpreter ability to write & run https://t.co/2HsyslOG2F With many 🧩 dropping recently a more complete picture is emerging of LLMs not as a chatbot but the kernel process"
[X Link](https://x.com/lateinteraction/status/1707446059398791668) 2023-09-28T17:23Z 16.3K followers, 93.1K engagements
"@karpathy A more mature analogy with all the pieces functional and powerful results here:"
[X Link](https://x.com/lateinteraction/status/1707446366992269596) 2023-09-28T17:25Z [----] followers, 16.4K engagements
"DSPy and ColBERT are interesting academic experiments imo. Each is a multi-paper repo that has one coherent artifact combining our latest research together. We typically release the features as open sourcehence get users/feedbackwell before writing a paper on the new ideas"
[X Link](https://x.com/lateinteraction/status/1710134813410402411) 2023-10-06T03:28Z [----] followers, 26.5K engagements
"Join us in [--] minutes if you want to learn more about DSPy and how it works. Chatting about DSPy LlamaIndex LLMs etc. DSPy (@lateinteraction @TomJoshi2) as an LLM framework has some really cool novel ideas: ✅ Emphasis on programming over prompting: compose declarative modules instead of hand-crafting prompts ✅ Let the compiler optimize for you: given these modules the compiler can auto-tune DSPy (@lateinteraction @TomJoshi2) as an LLM framework has some really cool novel ideas: ✅ Emphasis on programming over prompting: compose declarative modules instead of hand-crafting prompts ✅ Let the"
[X Link](https://x.com/lateinteraction/status/1710412406839804352) 2023-10-06T21:51Z 16.3K followers, 12.6K engagements
"A cool thread yesterday used GPT4 ($50) a 500-word ReAct prompt and [---] lines of code to finetune Llama2-7B to get 26% HotPotQA EM. Let's use [--] lines of DSPywithout any hand-written prompts or any calls to OpenAI ($0)to teach a 9x smaller T5 (770M) model to get 39% EM 🧵"
[X Link](https://x.com/lateinteraction/status/1712135660797317577) 2023-10-11T15:58Z [----] followers, 148.6K engagements
"And ahem use DSPy: It's easy and powerful but also controllable and transparent. You can control every step. Many (other) papers can be implemented in [---] lines of DSPy without any hand-writen prompts with much smaller LMs and with higher quality"
[X Link](https://x.com/lateinteraction/status/1712135684918853980) 2023-10-11T15:58Z [----] followers, [----] engagements
"What should we do next"
[X Link](https://x.com/lateinteraction/status/1712628264408346749) 2023-10-13T00:36Z [----] followers, [----] engagements
"@deliprao @ChrisGPotts I think I need more Twitter (X) threads to graduate @deliprao"
[X Link](https://x.com/lateinteraction/status/1712631493468770494) 2023-10-13T00:49Z [----] followers, [---] engagements
"DSPy is all about *compiling* LM programs. Why "compile"❓ Traditional compilers translate programs to machine code for _specific hardware_ A compiler for LM programs** translates code with LM calls into prompt or finetune chains for _specific LMs_ ** i.e. only DSPy so far"
[X Link](https://x.com/lateinteraction/status/1716967956050247798) 2023-10-25T00:00Z [----] followers, 20.2K engagements
"This is a great recent example of that where we compile a short program into a high-quality prompt chain for Llama2-13b and then compile that into a finetune chain for T5-770M. With really high quality starting from minimal data:"
[X Link](https://x.com/lateinteraction/status/1716968456925684156) 2023-10-25T00:02Z [----] followers, [----] engagements
"@eggie5 There are PRs and public branches all over the place like this one:"
[X Link](https://x.com/lateinteraction/status/1719471706337628204) 2023-10-31T21:49Z [----] followers, [--] engagements
"@atroyn Using representations (or tools in general) to find context and learning to use (and re-encode) contextfor tasksis a key idea thats here to stay. Its a good idea (direct tool use & context manipulation) embedded in a necessary hack (deep learning) not the other way around"
[X Link](https://x.com/lateinteraction/status/1721938611237732487) 2023-11-07T17:12Z [----] followers, [---] engagements
"Q: What does compilation mean for DSPy programs Usually we have CPUs & GPUs. You compile your high-level code so it uses them efficiently. We now have LPUslanguage processing units DSPy compiles your code into control flow with instructions (prompts) that maximize quality"
[X Link](https://x.com/lateinteraction/status/1727388355992998206) 2023-11-22T18:07Z [----] followers, [----] engagements
"Aargh I promise I do"
[X Link](https://x.com/lateinteraction/status/1729698638211727411) 2023-11-29T03:07Z [----] followers, 14.7K engagements
"How come no one told me about this paper RobustQA extends our LoTTE IR benchmark to a large labeled out-of-domain *QA* dataset ColBERTv2 outperforms DPR BM25 Contriever and expensive cross-encoders by [--] [--] [--] and [--] HIT@20 resp. Atlas is strong on downstream QA"
[X Link](https://x.com/lateinteraction/status/1729933458569621675) 2023-11-29T18:40Z [----] followers, 17.8K engagements
"@haldaume3 Yes Learning weights (or soft prompts) is often better Any DSPy module can be compiled into prompts *or* finetunes Prompts are easier to bootstrap from scratch (e.g. zero shot) then compiled into cheaper high-quality finetunes. Only in DSPy Example:"
[X Link](https://x.com/lateinteraction/status/1731137854401921345) 2023-12-03T02:26Z [----] followers, [---] engagements
"This year there were a few cool proposals that create and save "skills" during learning. I find it fascinating that Ashwin Paranjape @XiangLisaLi2 @siddkaramcheti and I called this concept "skills" back in July [----]. It's a good idea and "skill" is indeed a natural name"
[X Link](https://x.com/lateinteraction/status/1731896308389028337) 2023-12-05T04:40Z [----] followers, 13.3K engagements
"@pascalefung Hey Pascale I agree with this. But Im making a distinction between systems and models. Without defining the context of a system (what fact corpus are generations derived from) and its structure (how does it use the LM) it doesnt make sense to speak of accuracy/hallucination"
[X Link](https://x.com/lateinteraction/status/1733520129038983217) 2023-12-09T16:13Z [----] followers, [---] engagements
"The incredible @hrishioa builds what might be one of the richest LM programs. The principle is simple: dont rely on the LM as a user-facing system use it as a building block the way *you* would solve a task. (RAG multi-hop QA self-refine etc are just special cases of this)"
[X Link](https://x.com/lateinteraction/status/1734967999671533639) 2023-12-13T16:06Z [----] followers, 36.2K engagements
"@PierreRunavot @hrishioa Can I interest you in. Nvm Im sure you know about it 😆"
[X Link](https://x.com/lateinteraction/status/1735028577651847374) 2023-12-13T20:07Z [----] followers, [--] engagements
"🚨Announcing a powerful construct by @ShangyinT* @slimshetty_* @arnav_thebigman* Your LM isn't following complex instructions Stop prompting Add a one-liner assertion in your program: up to 35% gains w auto-backtracking & self-refinement🧵"
[X Link](https://x.com/lateinteraction/status/1735326551393161563) 2023-12-14T15:51Z [----] followers, 74.5K engagements
"@Thewimo The VLLM module is here: Are you saying it's not imported in the right places though"
[X Link](https://x.com/lateinteraction/status/1736454998253621451) 2023-12-17T18:35Z [----] followers, [---] engagements
"@Thewimo ah good catch yes VLLM is on main but not on pip yet pip install -e /path/to/local/version will do or if you need it I can upgrade pypi now. Should I"
[X Link](https://x.com/anyuser/status/1736457192335962278) 2023-12-17T18:43Z [--] followers, [--] engagements
"@roydanroy @ThomasW423 @QuanquanGu Heres my definition. Hallucination is indeed *not* well-defined for a statistical model without additional assumptions. Its definited for system in which we specify or assume a number of things:"
[X Link](https://x.com/lateinteraction/status/1736579390124003399) 2023-12-18T02:49Z [----] followers, [---] engagements
"@roydanroy @ThomasW423 @QuanquanGu A *system* is said to hallucinate when it generates a statement not derived correctly from supplied cited or assumed facts. This extrapolates summarization i.e. to open-ended generation conditioned on a large and hopefully explicit corpus"
[X Link](https://x.com/lateinteraction/status/1736582161774604673) 2023-12-18T03:00Z [----] followers, [--] engagements
"@roydanroy @ThomasW423 @QuanquanGu It's also a causal claim: it's not saying hallucination is when the system is wrong it's saying hallucination is when the system extrapolates past what the corpus can support. In particular reasonable deduction leading to incorrect statements is not hallucination"
[X Link](https://x.com/lateinteraction/status/1736582682333905007) 2023-12-18T03:02Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala Here's a thread with the full analogy. It's a bit broader than DNN frameworks though they're a key piece"
[X Link](https://x.com/lateinteraction/status/1736587894142513405) 2023-12-18T03:23Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala Right now most of the emphasis is *incorrectly* placed on aligning the LM itself. That's silly Why do you want to align an (uninitialized) layer in your architecture You should align the full userfacing system instead not its parts A DNN abstraction places the focus right"
[X Link](https://x.com/lateinteraction/status/1736594603271442485) 2023-12-18T03:49Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala Also: What's the alternative DSPy solves problems no other framework is attempting to resolve: stacking prompting patterns and bootstrapping techniques for arbitrary pipelines. LangChain/LlamaIndex/AutoGen are higher-level (for app dev) but should adopt DSPy like HF torch"
[X Link](https://x.com/lateinteraction/status/1736595884962681138) 2023-12-18T03:54Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala If I take this seriously DSPy's adoption is larger *within its target audience* certainly in my sphere at least. In the "LM development stack" DSPy is much lower-level than frameworks for app devs. Much the same way that app devs don't need to use JAX/torch/CUDA"
[X Link](https://x.com/lateinteraction/status/1736610982934491600) 2023-12-18T04:54Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala If you're trying to build and tune a new LM pipeline for a new task LangChain/LlamaIndex/AutoGen aren't useful perhaps even are counter-productive. I know several dozen in this space most using or learning to use DSPy. Their only other option is prompt hacking with strings"
[X Link](https://x.com/lateinteraction/status/1736612930098282602) 2023-12-18T05:02Z [----] followers, [--] engagements
"@shoggoth1313 Definitely valuable But you perhaps betray a limited understanding of this space & its timeline LangChain is a great example of taking off for incidental reasons and then creating a bad reputation for LM frameworks. (LlamaIndex is different; it *is* useful) Hype begets hype"
[X Link](https://x.com/lateinteraction/status/1736619782009708819) 2023-12-18T05:29Z [----] followers, [--] engagements
"@shoggoth1313 Both have built organic communities of new people excited to play with AI and that's their value. A gold rush into a one more tech stack. LangChain is a gateway to the space. LlamaIndex is very conveniant out of the box; pretty useful. DSPy solves new technical problems"
[X Link](https://x.com/lateinteraction/status/1736620706828607514) 2023-12-18T05:33Z [----] followers, [---] engagements
"@shoggoth1313 Hey overall this was quite fun. Definitely lots to think about and you helped me articulate a few things better in short form. Hope it was useful for you as well and good luck with your stealth"
[X Link](https://x.com/lateinteraction/status/1736620983761699133) 2023-12-18T05:34Z [----] followers, [--] engagements
"Progress on dense retrievers is saturating. The best retrievers in [----] will apply new forms of late interaction i.e. scalable attention-like scoring for multi-vector embeddings. A🧵on late interaction how it works efficiently and why/where it's been shown to improve quality"
[X Link](https://x.com/lateinteraction/status/1736804963760976092) 2023-12-18T17:45Z 14K followers, 236.6K engagements
"Say you have 1M documents. With infinite GPUs what would your retriever look like Maybe a cross-encoder Finetune a large LM to take querydoc pairs. Run it 1M times to get a score for all docs. Expressive Given a query the LM can pay attention to every detail in the doc"
[X Link](https://x.com/lateinteraction/status/1736804965942013978) 2023-12-18T17:45Z 14K followers, [----] engagements
"It was a last-ditch run on a Sunday night (3 Nov'19) after complex scoring failed. I spent weeks looking for a "bug". ColBERT w cheap scoring rivaled BERT-large cross-encoders with 10000x more FLOPs Called it ColBERT as a pun: late show / late interaction log scale latency:"
[X Link](https://x.com/lateinteraction/status/1736804976406802703) 2023-12-18T17:45Z [----] followers, [----] engagements
"There's a little-known trick that was essential for ColBERT's results: Query Augmentation. ColBERT appends MASK tokens to the query encoder to allow BERT to create more query vectors that aren't there Is it the earliest form of a scratchpad / chain of thought From Nov 2019"
[X Link](https://x.com/anyuser/status/1736804978667434048) 2023-12-18T17:45Z [--] followers, [----] engagements
"OK but how can ColBERT search 100M docs in [---] milliseconds Late interaction is pruning-capable: it only needs to "touch" 0.1% of the documents to find the top-K. This is by design: it's composed of monotonic functions (Max/Sum) which enable some neat algorithmic tricks"
[X Link](https://x.com/lateinteraction/status/1736804980840182079) 2023-12-18T17:45Z 14K followers, [----] engagements
"We can decompose late interaction into dozens of tiny nearest-neighbor searches at the token level. We'll only fetch & score docs in which at least one token in close to (at least one token in) the query. Otherwise we can prove the score will be too small and we can skip it"
[X Link](https://x.com/lateinteraction/status/1736804982949917032) 2023-12-18T17:45Z 14K followers, [----] engagements
"@jxmnop Are you saying this doesn't exist; is it different from 4-5 things like pause tokens This may be the earliest of the general form afaik:"
[X Link](https://x.com/lateinteraction/status/1737502293325742475) 2023-12-20T15:56Z [----] followers, [----] engagements
"A🧵on beating the hardware lottery for retrieval: the internals of the late interaction stack. ColBERT introduced a quirky multi-vector retrieval architecture. It does wonders for quality. But how can it search 100M docs in [---] sec on CPU Or store [--] billion embeddings in 20GB"
[X Link](https://x.com/lateinteraction/status/1737578879454425202) 2023-12-20T21:01Z [----] followers, 76.8K engagements
"OK I'll pause the🧵here. In the next thread I'll cover: 1) Why does late interaction work so well in *dozens* of eval/analysis papers We'll dig into evals with ColBERT-QA all the way to UDAPDR. 2) What's next for late interaction And for neural retrieval in general"
[X Link](https://x.com/lateinteraction/status/1737578912488767819) 2023-12-20T21:01Z [----] followers, [---] engagements
"Next Thread:"
[X Link](https://x.com/lateinteraction/status/1737579699487052066) 2023-12-20T21:04Z [----] followers, [---] engagements
"@omarsar0 @abacaj more innovation around how to prompt theze models"
[X Link](https://x.com/lateinteraction/status/1737621883909243036) 2023-12-20T23:51Z [----] followers, [---] engagements
"Someone just reminded me not everyone knows where the find the ColBERT code free Google Colab notebook to play with it list of papers etc"
[X Link](https://x.com/lateinteraction/status/1737642493779583261) 2023-12-21T01:13Z [----] followers, [---] engagements
"Someone just reminded me not everyone knows where the find the ColBERT code free Google Colab notebook to play with it list of papers etc"
[X Link](https://x.com/lateinteraction/status/1737642591800393752) 2023-12-21T01:14Z [----] followers, [---] engagements
"There's an important missing perspective in the "GPT-4 is still unmatched" conversation: It's a process (of good engineering at scale) not some secret sauce. To understand let's go back to 2000s/2010s when the gap between "open" IR and closed Google Search grew very large. 🧵 The recent releases of many GPT-3.5 class AIs (Grok Mixtral Gemini Pro) are oddly unilluminating about the future of frontier AI Its been a year & no one has beat GPT-4. Will they Is there some magic there Does it indicate a limit to LLMs Will GPT-4.5 be another huge jump The recent releases of many GPT-3.5 class AIs"
[X Link](https://x.com/lateinteraction/status/1737884271564701902) 2023-12-21T17:14Z [----] followers, 135.8K engagements
"Note: I have no inside OpenAI info & I'm uninterested in individual LMs. Expressive power lies in the *program* wielding the LM As commercial search matured the IR field went from a very active and hot area of research circa [----] to a much less active one 2004ish through 2018"
[X Link](https://x.com/lateinteraction/status/1737884274253304268) 2023-12-21T17:14Z [----] followers, [----] engagements
"Setting this up well takes timeand takes a team willing to be objective/honest and optimize for the right things with a long-term vision. Everyone rushing to "catch up" won't invest in this. Most other efforts are like "what's the right data mix" as if this is a one-off thing"
[X Link](https://x.com/lateinteraction/status/1737884826458554465) 2023-12-21T17:16Z [----] followers, 10.6K engagements
"Addendum 1: You could half-jokingly call this process collective human-policy RL. Addendum 2: OpenAI may apply this process to GPT-3.5 but it's small and cost-optimized. Size is a work multiplier. Others "match" GPT-3.5 differently: they focus on scale & forgo most cost cutting"
[X Link](https://x.com/lateinteraction/status/1737901060164059172) 2023-12-21T18:21Z [----] followers, [----] engagements
"Addendum 3: Another way to put it is this. The community is looking for the silver bullet that will allow GPT-4 to "emerge". But GPT-4 doesn't emerge. It's built very carefully feature by feature or unit test by unit test. Some research is key (ICL) but otherwise engineering"
[X Link](https://x.com/lateinteraction/status/1737909649339498630) 2023-12-21T18:55Z [----] followers, [----] engagements
"This can get around the key limitations of autoregressive LMs control grounding transparency planning and efficiency. And it can simultaneously leverage their main power: that well-scoped *automated hypothesis testing* over the space of prompts is extremely data-efficient"
[X Link](https://x.com/lateinteraction/status/1738257299335188901) 2023-12-22T17:56Z [----] followers, [----] engagements
"👀 I'm being called out. But I agree. DSPy *will* be so huge pretty soon @mbusigin. I'm spending winter break revamping docs/examples and doing much-needed refactoring. The abstraction is incredibly elegant as many have noted. The DX (developer experience) will soon match. I think DSPy would be so huge if it had applied any thought to the DX at all I think DSPy would be so huge if it had applied any thought to the DX at all"
[X Link](https://x.com/lateinteraction/status/1738275430011146363) 2023-12-22T19:08Z [----] followers, 54.3K engagements
"@McaleerStephen Looked into DSPy"
[X Link](https://x.com/lateinteraction/status/1738308965178765438) 2023-12-22T21:22Z [----] followers, [---] engagements
"@hwchase17 @HanchungLee @mbusigin @LangChain 💯 yes what would be useful Han"
[X Link](https://x.com/lateinteraction/status/1738349637906600274) 2023-12-23T00:03Z [----] followers, [---] engagements
"@panabee Indeed it could be very different. But it depends on the strategy from my thread: find all common failure modes create "unit tests" for them extensively and collectively work to pass them. The open community isnt doing anything like this yet. Thats the bottleneck"
[X Link](https://x.com/lateinteraction/status/1738682896867860827) 2023-12-23T22:08Z [----] followers, [--] engagements
"@srchvrs Thanks for the insightful posts Leo I doubt we have a fundamentally distinct understanding here. Im speaking to people who think OpenAI has magical *model* ideas or that Google had magical *IR modeling* ideas. Both have excellent engineering with tens of small key pieces"
[X Link](https://x.com/anyuser/status/1739874113345507497) 2023-12-27T05:01Z [--] followers, [---] engagements
"@olgias @jobergum @bclavie @huggingface @vespaengine has a few more things"
[X Link](https://x.com/lateinteraction/status/1740001957799137379) 2023-12-27T13:29Z [----] followers, [--] engagements
"@Shroominic @akatzzzzz @jxnlco @Teknium1 @arnav_thebigman Let me know if you wanna explore this further @Shroominic. Im not 100% clear on the details of your tweet"
[X Link](https://x.com/lateinteraction/status/1740027603925586093) 2023-12-27T15:11Z [----] followers, [--] engagements
"@mayhewsw @deliprao Honestly fwiw I thought about tweeting with ELMo originally credit where its due"
[X Link](https://x.com/anyuser/status/1740175957091385424) 2023-12-28T01:00Z [--] followers, [--] engagements
"@paul_cal So the original NYT prompt is to demonstrate that the LM saw some NYT text at training time (which is perhaps already known more or less) and that theres a chance it reproduces that text on demand (perhaps not for long though and not in latest versions). Underwhelming"
[X Link](https://x.com/lateinteraction/status/1740874178470023549) 2023-12-29T23:15Z [----] followers, [--] engagements
"I'd been planning to post the third thread in the late interaction / ColBERT sequence. But posting now (or around Jan 01) may mean a lot of people miss it. What do people think Obviously it's a biased sample but curious"
[X Link](https://x.com/lateinteraction/status/1741272921241423936) 2023-12-31T01:39Z [----] followers, [----] engagements
"@trading_indian Haha smart but Twitter really doesnt work that way. Old stuff (i.e. over [--] hours) is basically unboostable"
[X Link](https://x.com/anyuser/status/1741274804576309519) 2023-12-31T01:47Z [--] followers, [--] engagements
"@arnaudai @ChetanyaRastogi @lancehasson Think long term How long ago was it when there were people in ML/NLP who thought you really ought to tune your weights by hand for your classifier because human intuition is unmatched Code in a high-level language. Let compiling/optimization handle low-level instructions"
[X Link](https://x.com/lateinteraction/status/1741569806837711217) 2023-12-31T21:19Z [----] followers, [---] engagements
"@MimansaJ Markup modification/parsing sounds like the one thing good LLMs can do exceptionally well so far Or is that not true"
[X Link](https://x.com/anyuser/status/1742280486704312352) 2024-01-02T20:23Z [--] followers, [--] engagements
"Is there a way to try ColBERT as a complete noob I literally started learning Python a few months ago. Yes Run this free Colab notebook in your browser: @lateinteraction Is there a complete noob - I literally started learning python a few months ago - to try this out @lateinteraction Is there a complete noob - I literally started learning python a few months ago - to try this out"
[X Link](https://x.com/lateinteraction/status/1742390511623938472) 2024-01-03T03:40Z [----] followers, 10.3K engagements
"Fun question. I don't know much about DPO but I can share IR intuition. IR pairwise loss functions (a b) are known to work much better than pointwise loss (a=1 b=0). Pairwise encourages healthy margins without pushing the model to treat a as perfect or b as the worst. @CFGeek my mental model of DPO is "push model away from dispreferred and towards preferred" and SFT is "pull the model towards the gold answer". if this is just like DPO where dispreferred is the current policy i dont have a good intuition for why this would be better than SFT. wdyt @CFGeek my mental model of DPO is "push model"
[X Link](https://x.com/lateinteraction/status/1742429788609105935) 2024-01-03T06:16Z [----] followers, [----] engagements
"@H1iReimer I agree 100%. But I think there's some nuance. If NVIDIA stops selling Titan V GPUs does that mean Systems paper that report efficiency results on Titan V are no longer reproducible Only if the paper is poorly done and is basically testing Titan V not fundamental patterns"
[X Link](https://x.com/anyuser/status/1742925164898394508) 2024-01-04T15:05Z [--] followers, [---] engagements
"@marktenenholtz Just to clarify now ColBERT = ColBERTv2 = PLAID. We stick to the latest i.e. PLAID + some more recent improvements. For all of Common Crawl (wow) I'd do a distributed PLAID. That is build [---] ColBERT indexes on [---] machines and route by query. There are a few ways to split"
[X Link](https://x.com/lateinteraction/status/1743073673827991596) 2024-01-05T00:55Z [----] followers, [----] engagements
"@andrew_n_carr @Teknium1 Um who believes that"
[X Link](https://x.com/lateinteraction/status/1743312389649658355) 2024-01-05T16:43Z [----] followers, [---] engagements
"@vivekkalyansk @TheXeophon @EugeneVinitsky Im missing any/all context but surprised not to see the right() explanation. LMs barely see text in 2D. A good LM can 100% do tic-tac-toe if its linearized/serialized (very) well"
[X Link](https://x.com/anyuser/status/1743478829136695628) 2024-01-06T03:45Z [--] followers, [--] engagements
"@420_gunna @bclavie I have IR course material on youtube [--] videos starting with this one from stanford CS224U Spring [----]. The material covers things from BM25 to ColBERT"
[X Link](https://x.com/lateinteraction/status/1743610947711713281) 2024-01-06T12:30Z [----] followers, [----] engagements
"@n0riskn0r3ward "Our company merged with Tesla yesterday. Don't say embarrassing things about our Tesla.""
[X Link](https://x.com/anyuser/status/1744164160274100551) 2024-01-08T01:08Z [--] followers, [---] engagements
"@Prestigious_AI @Mentor_Nolan Working on it this week One challenge is that we dont have an Anthropic key its not open to all yet() afaik"
[X Link](https://x.com/lateinteraction/status/1745292768392729069) 2024-01-11T03:53Z [----] followers, [--] engagements
"@labloke11 @TheGrizztronic Hey Id love to help you grok this. I gotta run to a meeting in [--] minutes but before this let me clarify that DSPy is not a drop-in replacement for your current workflow. You have to redesign your workflow with DSPy lessons in mind. You have to decide on a metric to optimize"
[X Link](https://x.com/lateinteraction/status/1745844271373242369) 2024-01-12T16:24Z [----] followers, [---] engagements
"Anecdotal example of ColBERT in the wild. Dense retrieval (OpenAI ada-002) doesn't find anything about whether "@paulg is vegan". Returns a random passage about him. ColBERT notices that "Paul liked a chicken McSpicy". That's the sort of thing that makes or breaks a RAG system. query: "was Paul vegan" text splitter: langchain's recursive character text splitter with chunk length of [---] and chunk overlap of [--]. outcome: ColBERT wins this rounds https://t.co/DlybPCkgT9 query: "was Paul vegan" text splitter: langchain's recursive character text splitter with chunk length of [---] and chunk overlap"
[X Link](https://x.com/lateinteraction/status/1746589606638993633) 2024-01-14T17:46Z [----] followers, 38.7K engagements
"@deliprao @Samhanknr Thank you for tagging me on this Delip I have a lot of thoughts about this Im torn between sharing them and saving time to speed up ongoing work so I can show more and tell less. Overall Id love to share my current intuition on this but cant tell if its just too abstract"
[X Link](https://x.com/lateinteraction/status/1747060551203848431) 2024-01-16T00:57Z [----] followers, [---] engagements
"@deliprao @Samhanknr I dont think gradient descent or a single pass through a DNN or passing embeddings around without concrete form / text / provenience can ever solve a few essential problems. But if RAG means retrieve then generate by direct prompting thats obviously not enough either"
[X Link](https://x.com/lateinteraction/status/1747065009891680670) 2024-01-16T01:15Z [----] followers, [---] engagements
"Paraphrasing 09:00 New users always notice how DSPy *forces* you to contend with "what is it that I am trying to optimize for". Working with ppl this leads to much better systems. LMs are cool at demos but to build reliable systems you must define & iterate on your metrics Hey everyone I am BEYOND EXCITED to publish our interview with Omar Khattab (@lateinteraction) from @stanfordnlp 🔥 Omar delivers remarkably clear explanations of DSPy and ColBERT as well as views on the state of AI I hope you find this useful 🎙 https://t.co/6kmlcItT4v https://t.co/38jOP0pW8p Hey everyone I am BEYOND"
[X Link](https://x.com/lateinteraction/status/1747071790646706225) 2024-01-16T01:42Z [----] followers, [----] engagements
"Obligatory ICLR tweet: DSPy accepted as Spotlight. [--]. This could be a notable start of a pattern at ML confsas our metareviewer says "refreshing to see this unconventional research style (developing programming models) at ICLR" [--]. Time for a crisp thread on DSPy & what's new"
[X Link](https://x.com/lateinteraction/status/1747319355367739411) 2024-01-16T18:06Z [----] followers, 59.3K engagements
"We spent a long time thinking about the many LLM frameworks and how they differ. Article below with @heathercmiller's CMU Composable Systems Lab (@Lambda_freak Haoze He) is the best place to understand this emerging **stack**. Agents optimizers chains schemas prompts A new thing Ive been up to lately along with Peter Zhong Haoze He @lateinteraction @ChrisGPotts & @matei_zaharia A Guide to LLM Abstractions https://t.co/oJfCY1tfsU its one thing to call the OpenAI APIs from a webapp its entirely another to build crazy rich https://t.co/yuWhLbFSHR A new thing Ive been up to lately along with"
[X Link](https://x.com/lateinteraction/status/1747394891058180185) 2024-01-16T23:06Z [----] followers, 27K engagements
"You should read our new guide to LLM abstractions a stack with [--] layers To randomly help this tweet reach ppl see how happy DSPy power users feel when they get state-of-the-art scores using DSPy optimizersso happy in fact they make fancy slack emojis. cc: @KarelDoostrlnck We spent a long time thinking about the many LLM frameworks and how they differ. Article below with @heathercmiller's CMU Composable Systems Lab (@Lambda_freak Haoze He) is the best place to understand this emerging **stack**. Agents optimizers chains schemas prompts https://t.co/KlwatjMH5E We spent a long time thinking"
[X Link](https://x.com/lateinteraction/status/1747421103050674658) 2024-01-17T00:50Z [----] followers, [----] engagements
"@regevbs Good luck Let me know if you face issues. We'd love to merge something like this visualizer in the original ColBERT or maybe in RAGatouille by @bclavie if there's interest :D"
[X Link](https://x.com/lateinteraction/status/1747682119571370370) 2024-01-17T18:07Z [----] followers, [--] engagements
"This will be simple but effective on tasks where we need detailed feedback from the metric. Consider this quick-n-dirty metric for generating "correct" "faithful" "engaging" tweets under [---] characters and without hashtags. Seeing these & their values can improve optimization"
[X Link](https://x.com/lateinteraction/status/1748421059513352631) 2024-01-19T19:03Z [----] followers, [----] engagements
"@stalkermustang Haha. I think were talking about different things Igor. I meant: if anyones first PR is to create a (draft incomplete messy) list of first PRs for others then Id happily merge and tweak it. But that may be too far. Thats what I meant by recursive here"
[X Link](https://x.com/lateinteraction/status/1748692604362056040) 2024-01-20T13:03Z [----] followers, [--] engagements
"ChatGPT is going bananas at large scale I'm not even kidding this is really what it responded with https://t.co/7J7yIcmpF7 I'm not even kidding this is really what it responded with https://t.co/7J7yIcmpF7"
[X Link](https://x.com/lateinteraction/status/1748746533456269753) 2024-01-20T16:37Z [----] followers, [----] engagements
"Generic agents will eventually work but not soon. Raw LMs and Agents are both text in - text out which is way too opaque for iterative development. LMs self-distract. You need to program them (maintain control flow in code) & deal with exceptions by actually debugging. 1/2 Are LLM agents just a gimmick Is anyone using them at scale Are LLM agents just a gimmick Is anyone using them at scale"
[X Link](https://x.com/lateinteraction/status/1748759882269098251) 2024-01-20T17:30Z [----] followers, 34.3K engagements
"@ukituki No it doesn't. Structured output just means your output will parse that's a very very low bar. In [----] in the Demonstrate-Search-Predict paper (which is different from the [----] DSPy paper) we wrote about self-distraction. This example is parsed correctly but content is wrong"
[X Link](https://x.com/lateinteraction/status/1748764235050680640) 2024-01-20T17:47Z [----] followers, [---] engagements
"@ecardenas300 @weaviate_io Thank you @ecardenas300 @CShorten30 and @weaviate_io team"
[X Link](https://x.com/lateinteraction/status/1749118577784000682) 2024-01-21T17:15Z [----] followers, [--] engagements
"I really enjoy telling people: hey we just released a new optimizer. Just replace that line and re-compile. Your DSPy program will likely score much higher. Much nicer than: there's a new string that you should insert in your 6-prompt pipeline & keep tweaking until it works. Quite a few happy academic users for new DSPy optimizers this weekend. Fewer painstaking prompts higher scores Turns out if you want to help grad students you need to reach them directly. Grad students: DSPy is a research library la PyTorch (not HF). Its for you as well. Quite a few happy academic users for new DSPy"
[X Link](https://x.com/lateinteraction/status/1749457757538025974) 2024-01-22T15:43Z [----] followers, 12.3K engagements
"It's even nicer IMO that optimizers can update weights not just write better instructions and few-shot examples. It reminds me of my first exposure to Adam. I replaced SGD with Adam and suddenly my tiny DNN for a deep learning course was actually competitive"
[X Link](https://x.com/lateinteraction/status/1749457759400235117) 2024-01-22T15:43Z [----] followers, [---] engagements
"Many DSPy users are not sure what the optimizers do given a pipeline that calls the LM a few times to solve a task. So I added this to the mini FAQ on the README. It's quick-n-dirty and pretty general so feedback welcome. Also new optimizers very much welcome"
[X Link](https://x.com/lateinteraction/status/1749506345060196675) 2024-01-22T18:56Z [----] followers, 15K engagements
"Prompt optimization solves the cold start problem"
[X Link](https://x.com/lateinteraction/status/1750007396716486983) 2024-01-24T04:07Z [----] followers, [----] engagements
"I tried five extremely basic questions it got each one worse than the one before. Before I discuss more am I using the right Bard 🔥Breaking News from Arena Google's Bard has just made a stunning leap surpassing GPT-4 to the SECOND SPOT on the leaderboard Big congrats to @Google for the remarkable achievement The race is heating up like never before Super excited to see what's next for Bard + Gemini https://t.co/QPtsqZdJhC 🔥Breaking News from Arena Google's Bard has just made a stunning leap surpassing GPT-4 to the SECOND SPOT on the leaderboard Big congrats to @Google for the remarkable"
[X Link](https://x.com/lateinteraction/status/1751023782054048113) 2024-01-26T23:26Z [----] followers, 70.2K engagements
"@simonw @marktenenholtz Lots of amazing pointers by @marktenenholtz and @imaurer. The paper's main figure or just this (really solid IMO) thread can also be useful: Progress on dense retrievers is saturating. The best retrievers in [----] will apply new forms of late interaction i.e. scalable attention-like scoring for multi-vector embeddings. A🧵on late interaction how it works efficiently and why/where it's been shown to improve quality https://t.co/2XG33TtM9R Progress on dense retrievers is saturating. The best retrievers in [----] will apply new forms of late interaction i.e. scalable"
[X Link](https://x.com/lateinteraction/status/1751424020338311200) 2024-01-28T01:56Z [----] followers, [---] engagements
"@akatzzzzz @mathemagic1an Why dont we help you add it Which models do you need it for"
[X Link](https://x.com/lateinteraction/status/1751727165656428857) 2024-01-28T22:01Z [----] followers, [--] engagements
"@swyx @CShorten30 @jxnlco @gordic_aleksa Theres an inline notation for prompts and theres a verbose version where you can write instructions. You start with inline notation to get the right program (control flow) and then compile. You only do more where iterative development tells you its strictly needed"
[X Link](https://x.com/lateinteraction/status/1752441696653332934) 2024-01-30T21:20Z [----] followers, [---] engagements
"I'm glad that a lot more people understand the key ideas behind ColBERT and DSPy now. My only remaining goal is to make sure people can also say them correctly; both are quite tricky😆 * Col-BAIR (it's "the late" interaction retriever get it) * Dee-Ess-Pie (like num-pie)"
[X Link](https://x.com/lateinteraction/status/1752762010306281524) 2024-01-31T18:33Z [----] followers, 33.8K engagements
"@srush_nlp The issue with one vector isn't capacity it's learnability. When SGD tries to "fix" a vector in one way it breaks it in other ways. That's why tiny ColBERT vectors outperform massive single vector. And also why ColBERT's sum-max is way better than sum-sum with identical dims"
[X Link](https://x.com/lateinteraction/status/1752804946251247700) 2024-01-31T21:23Z [----] followers, [----] engagements
"There's a recent surge of interest in quicker replication of ColBERTv2 training. To make this easier I uploaded the examples file (64-way) and initial checkpoint (colbert v1.9) to HF hub. Here's the full code to launch training after pip install colbert-ai. (links below)"
[X Link](https://x.com/lateinteraction/status/1753428544259346935) 2024-02-02T14:41Z [----] followers, 18.6K engagements
"And more here should include open-ended evaluation. Its always inspiring to see the kinds of eval campaigns that TREC held in early 2000s. This is why having a solid background in IR (or information access broadly) is one of the biggest assets in working with modern LM systems"
[X Link](https://x.com/lateinteraction/status/1753658949352755672) 2024-02-03T05:57Z [----] followers, [----] engagements
"@markopolojarvi This is a surprisingly common take and I dont get it. I suspect its just bad tooling (do you use conda) or you work with long tail unpopular packages a lot. In my world pip and conda install almost always just work. The only exception is when cuda is involved. Thats it"
[X Link](https://x.com/lateinteraction/status/1753670511706714441) 2024-02-03T06:43Z [----] followers, 31.4K engagements
"@markopolojarvi I mean you and I both know thats not true. Whatever packages are like in Ruby Java or for that matter C and C++ isnt much better than Python at all. And unless PHP got magically better over the past [--] years (its been a while) its worse than all of these. Idk about JS"
[X Link](https://x.com/lateinteraction/status/1753672505137811853) 2024-02-03T06:51Z [----] followers, 10.5K engagements
"@420_gunna Read IR papers and take IR courses I have some super simple material that are admittedly 2-3 years old but theyre still solid. Five videos starting with this one:"
[X Link](https://x.com/lateinteraction/status/1753676254627430830) 2024-02-03T07:06Z [----] followers, [--] engagements
"There are so many things I want to focus on in DSPy that we badly need to outsource the "integrations layer". It's really a time sink to manage all these LM and vector store integrations. Who has a uniform layer to enough of these in a uniform interface that people like"
[X Link](https://x.com/lateinteraction/status/1753852503584804865) 2024-02-03T18:46Z [----] followers, 40.9K engagements
"@HanchungLee @LiteLLM @withmartian @PortkeyAI LangChain is especially great at this but these constraints are the problem: Requirements: * I don't want to get a lot of things we didn't ask for. Just shallow clients for LM and retrieval models / vector stores. * For LMs it must support requesting n [--] completions. It's odd how this isn't in many clients. Requirements: * I don't want to get a lot of things we didn't ask for. Just shallow clients for LM and retrieval models / vector stores. * For LMs it must support requesting n [--] completions. It's odd how this isn't in many clients"
[X Link](https://x.com/lateinteraction/status/1753876116241350880) 2024-02-03T20:20Z [----] followers, [---] engagements
"@_AliArmani_ Hi Ali yea theres vespa @vespaengine. Or you can index your documents in colab and do the search locally or on a cheap CPU server. RAGatouille is the right library for this kind of usage for ColBERT"
[X Link](https://x.com/lateinteraction/status/1754162927438008415) 2024-02-04T15:20Z [----] followers, [--] engagements
"@HamelHusain @wasimlorgat @thesephist This is a whole new paradigm but it needs a fundamental shift in how people view LMs. If you view them as systems or agents that you need to chat with or ask nicely rather than optimize it will be hard to do good ML / software engineering with them like LM programming"
[X Link](https://x.com/lateinteraction/status/1754225801950843190) 2024-02-04T19:29Z [----] followers, [---] engagements
"Im seeing so many should I switch from framework X to DSPy Well weve written extensivelymainly CMU composable systems labs @Lambda_freak and @HaozeHe_Hectoron the emerging LM stack. Different frameworks tackle fundamentally different concerns. Im taking this from Jason Liu https://t.co/PwgaXL8Vef via @YouTube @jxnlco @ConnorShorten So please help me understand how much is DSPy doing what Instructor does and how much something different Im taking this from Jason Liu https://t.co/PwgaXL8Vef via @YouTube @jxnlco @ConnorShorten So please help me understand how much is DSPy doing what Instructor"
[X Link](https://x.com/lateinteraction/status/1754278570439090302) 2024-02-04T22:59Z [----] followers, 33.6K engagements
"@SearchDataEng @Lambda_freak @HaozeHe_Hector Yeah we actually have no competition"
[X Link](https://x.com/lateinteraction/status/1754291783583113507) 2024-02-04T23:52Z [----] followers, [---] engagements
"A nice strong result by @fearnworks from the DSPy discord server. Also: we just made this server (originally created by the community; thanks @dronathon) our official Discord for DSPy. Feel free to join. Link below"
[X Link](https://x.com/lateinteraction/status/1754304294990573953) 2024-02-05T00:41Z [----] followers, [----] engagements
"We now have an official Discord server for DSPy. Feel free to join to ask questions or just discuss cool ideas with the DSPy community. There are so many powerful ideas right at the edges of DSPy and I might start tweeting more of them. This feels exactly the same as when in-context learning was discovered and yet very few people realized just how big of a paradigm shift that was poised to create. There are so many powerful ideas right at the edges of DSPy and I might start tweeting more of them. This feels exactly the same as when in-context learning was discovered and yet very few people"
[X Link](https://x.com/lateinteraction/status/1754927365174788188) 2024-02-06T17:57Z [----] followers, 13.4K engagements
"@bclavie Congrats Ben 🙌 Extremely insightful blog post Ill be looking forward to future posts"
[X Link](https://x.com/lateinteraction/status/1754929615192391763) 2024-02-06T18:06Z [----] followers, [---] engagements
"gpt-3.5 achieves [----] average Jaccard score with [--] shot and [----] after compilation Trying to combine DSPy Pydantic types and JSON Schemas https://t.co/l54jOQvvf2 Trying to combine DSPy Pydantic types and JSON Schemas https://t.co/l54jOQvvf2"
[X Link](https://x.com/lateinteraction/status/1755018013236166694) 2024-02-06T23:57Z [----] followers, 20.9K engagements
"New major release in DSPy With a whole lot of new results on a bunch of tasks. Language Model Assertions work by @arnav_thebigman @ShangyinT @slimshetty_. New: DSPy will now teach your LM to follow complex constraints (assertions) and will handle retries on top of that Theres huge interest now in *programming* with LMs but its unclear how to actually enforce constraints like make sure the output is engaging & has no hallucination. Just ask the LM nicely We built **DSPy LM Assertions** so you have far better controlup to 164% gains https://t.co/23NAOsljGz Theres huge interest now in"
[X Link](https://x.com/lateinteraction/status/1755337965751198042) 2024-02-07T21:09Z [----] followers, 58.6K engagements
"@deter3 @hwchase17 @KevinAFischer Honestly there's no need to say things like that @deter3. LangChain and DSPy are solving very different problems. Thanks to the way @hwchase17 does things LangChain has taught a vast number of developers about LMs/AI far more quickly than it seemed to be happening otherwise"
[X Link](https://x.com/lateinteraction/status/1755348067904348536) 2024-02-07T21:49Z [----] followers, [---] engagements
"From the DSPy discord by @fearnworks. New major release in DSPy With a whole lot of new results on a bunch of tasks. Language Model Assertions work by @arnav_thebigman @ShangyinT @slimshetty_. New: DSPy will now teach your LM to follow complex constraints (assertions) and will handle retries on top of that New major release in DSPy With a whole lot of new results on a bunch of tasks. Language Model Assertions work by @arnav_thebigman @ShangyinT @slimshetty_. New: DSPy will now teach your LM to follow complex constraints (assertions) and will handle retries on top of that"
[X Link](https://x.com/lateinteraction/status/1755396617736511623) 2024-02-08T01:02Z 13K followers, 12.9K engagements
"Well we finally have a backronym for DSPy. now stands for eclarative elf-improving Language rograms (in pthon). What's the impact of this Nothing. You still pip install dspy-ai and work. It's just nice to be able to answer that exceedingly common question"
[X Link](https://x.com/lateinteraction/status/1755594951344439389) 2024-02-08T14:10Z [----] followers, 15.2K engagements
"This is an AMAZING tutorial video on using the latest DSPy optimizers *and* on building advanced DSPy metrics with AI feedback. @CShorten30 improved his RAG program over Weaviate FAQ by 28% from 2.71/4 to 3.46/4. I'll postpone another announcement to tomorrow; watch *this* Hello world DSPy I am SUPER excited to share a new video walking through the end-to-end of how to use DSPy to optimize the CIFAR-10 for LLM programs RAG with FAQs 🛠 This tutorial contains *4 major parts*: (1) library installation settings and creating a dataset with https://t.co/5UxH0Ye8uJ Hello world DSPy I am SUPER"
[X Link](https://x.com/lateinteraction/status/1757070211067592944) 2024-02-12T15:52Z [----] followers, 25.1K engagements
"Incomplete list of DSPy internals experts. Follow them all Optimization: @kristahopsalong @michaelryan207 @KarelDoostrlnck @JoshPurtell Modules & Assertions: @arnav_thebigman @ShangyinT @slimshetty_ Abstractions: @Lambda_freak @HaozeHe_Hector @krypticmouse @cyrusofeden"
[X Link](https://x.com/lateinteraction/status/1757445323444863342) 2024-02-13T16:43Z [----] followers, 14.5K engagements
"Seeing discussions about using DSPy for synthetic data generation and for building agents. These are two big usecases but remember: the real power of DSPy for that is that you can *optimize* (1) the properties of the data that will be synthesized and (2) the agent behavior"
[X Link](https://x.com/lateinteraction/status/1758032914091217388) 2024-02-15T07:38Z [----] followers, [----] engagements
"One of the most frequently asked questions this week: "I optimized a DSPy program. How can I re-use it in the future" compiled_program.save("path.json") loaded_program = YourModule() loaded_program.load("path.json") (it's in the tutorials but gotta make a cheatsheet)"
[X Link](https://x.com/lateinteraction/status/1758040448265077061) 2024-02-15T08:08Z [----] followers, [----] engagements
"New release of [--] rich (and free to run) tutorials on DSPy Assertions for teaching complex properties to LMs Example outputs (h/t @arnav_thebigman). Top: Prompting GPT-3.5 to generate engaging tweets Bottom: Using dspy.Assert to suggest engaging & hallucination-free tweets Lots of interest in using DSPy Assertions since the release [--] weeks ago Here are [--] notebooks on how to use DSPy Assertions for complex tasks with multiple nuanced constraints like JSON formatting or context faithfulness and engagement levels - judged by LMs. https://t.co/YJcNZn8ILj Lots of interest in using DSPy Assertions"
[X Link](https://x.com/lateinteraction/status/1758559254297141472) 2024-02-16T18:29Z [----] followers, 12.3K engagements
"@arnav_thebigman Read the thread by Arnav and check out the three Google Colab notebooks. They're are all pre-cached so you can run them for free in your browser or locally. Many folks on the DSPy discord have been exploring assertions hopefully these three cool tasks/examples make it easier"
[X Link](https://x.com/lateinteraction/status/1758559779428196574) 2024-02-16T18:31Z [----] followers, [---] engagements
"This is a very cool release of a ColBERTv2-style model by @JinaAI_ The model has several advantages like natively working with long text and using a more modern base than BERT-base. I expect @JinaAI_ team has even more up their sleeves for data mixtures in future releases.👀 Introducing jina-colbert-v1-en. It takes late interactions & token-level embeddings of ColBERTv2 and has better zero-shot performance on many tasks (in and out-of-domain). Now on @huggingface under Apache [---] licence https://t.co/snVGgI753H Introducing jina-colbert-v1-en. It takes late interactions & token-level"
[X Link](https://x.com/lateinteraction/status/1758565865455485175) 2024-02-16T18:55Z [----] followers, [----] engagements
"Now that DSPy is popular some nuance. DSPy is *not* a prompting framework. It proposes a new paradigm: a framework for expressing & optimizing Language Programs. Think neural networks but with LM "layers" i.e. abstract modules that *learn* prompts/finetunes given a metric"
[X Link](https://x.com/lateinteraction/status/1758734906648375633) 2024-02-17T06:07Z [----] followers, 43.8K engagements
"I always thought these remarks are not very serious but since theyre common it could be instructive to comment. Tradeoffs. Context (concatenation) & Retrieval pose fundamentally different tradeoffs in recall precision and latency/cost. You cant choose between RAM and disk. With the context size of 10M tokens de we still need vector databases With the context size of 10M tokens de we still need vector databases"
[X Link](https://x.com/lateinteraction/status/1758742584032428113) 2024-02-17T06:38Z [----] followers, 24.8K engagements
"@madmaxbr5 Very well said. Also: At some level late interaction is (very cheap) attention. Being able to scale up lower-fidelity attention is hence not new. Getting perfect recall over 10M tokens is not new Getting good general-purpose attn with 10M tokens with low cost would be new"
[X Link](https://x.com/lateinteraction/status/1758761516780937322) 2024-02-17T07:53Z [----] followers, [---] engagements
"The problem isn't with what the function space can represent -- the problem is the learning process. It's SGD. From [----] to [----] I was actually quite convinced that RNNs could be trained to learn any program. After all they're Turing-complete (or at least some of them are) and they learn a highly compressed model of the input:output mapping they're trained on (rather than mere pointwise From [----] to [----] I was actually quite convinced that RNNs could be trained to learn any program. After all they're Turing-complete (or at least some of them are) and they learn a highly compressed model of"
[X Link](https://x.com/lateinteraction/status/1759238078869410163) 2024-02-18T15:26Z [----] followers, 10.2K engagements
"@GregKamradt Probably nothing good though happy to be proven wrong. Most wont release their slack dumps. Those who would probably wont have managed to create labels around them. Laziness around good data is nearly ubiquitous. But can be a new project over a public discord/gh issues crawl"
[X Link](https://x.com/lateinteraction/status/1760447731741704292) 2024-02-21T23:33Z [----] followers, [---] engagements
"Just saw this interview at The Register with @rickbattlephoto on his recent paper on "The Unreasonable Effectiveness of Eccentric Automatic Prompts". Highlights his & Teja Gollapudi's experience w DSPy at VMware. It has some curious observations on LM behavior. Star Treck👀"
[X Link](https://x.com/lateinteraction/status/1760698375715086346) 2024-02-22T16:09Z [----] followers, 20.4K engagements
"Automatic DSPy optimizers by @kristahopsalong and @michaelryan207 were used by this VMware research team and are now in the news (The Register) before they're even released in a preprint. Follow Michael and Krista for lots of cool stuff soon Just saw this interview at The Register with @rickbattlephoto on his recent paper on "The Unreasonable Effectiveness of Eccentric Automatic Prompts". Highlights his & Teja Gollapudi's experience w DSPy at VMware. It has some curious observations on LM behavior. Star Treck👀 https://t.co/t1FBP9vRAn Just saw this interview at The Register with"
[X Link](https://x.com/lateinteraction/status/1760703758160515412) 2024-02-22T16:31Z [----] followers, [----] engagements
"@prabhatweet @ArizePhoenix Hmm we did merge a big update to dspy today I dont see how it affects this part of the code but maybe it does. Maybe you should update your notebook run for now to pip install dspy-ai==2.2.0 and it will work while we look into this (also cc @thomasahle fyi)"
[X Link](https://x.com/lateinteraction/status/1762003975992414331) 2024-02-26T06:37Z [----] followers, [--] engagements
"Looks like both @krypticmouse and @ndzfs have been independently developing synthetic dataset generators for DSPy optimizers. So today we got two PRs for this with very different approaches Community feedback welcome; maybe we can merge the two into one robust version"
[X Link](https://x.com/lateinteraction/status/1763996362784583795) 2024-03-02T18:34Z [----] followers, 23.9K engagements
"ICYMI from Friday. Our best DSPy prompt optimizer yet for complex language programs (though more coming soon) With great animated descriptions of how it works step by step and a free (cached) google Colab example: Got a pipeline with **multiple prompts** like a DSPy program What's the right way to jointly optimize these prompts Introducing MIPRO a Multi-prompt Instruction Proposal Optimizer. We integrated MIPRO into DSPy. It can deliver +11% gains over existing DSPy optimizers 🧵👇 https://t.co/hFfqSsELyT Got a pipeline with **multiple prompts** like a DSPy program What's the right way to"
[X Link](https://x.com/lateinteraction/status/1767227142285426966) 2024-03-11T16:32Z [----] followers, 10.1K engagements
"AGI is nonsense but I like working on API Artificial Programmable Intelligence"
[X Link](https://x.com/lateinteraction/status/1770268708390486119) 2024-03-20T01:58Z [----] followers, 17.1K engagements
"Oh wow this paper from VMwares @rickbattlephoto comparing manual prompts against prompts produced by DSPy is now highlighted in The Register IEEE Spectrum and Business Insider. On The Unreasonable Effectiveness of Eccentric Automatic Prompts https://www.businessinsider.com/prompt-engineer-ai-careers-tech-fad-2024-3amp https://www.businessinsider.com/prompt-engineer-ai-careers-tech-fad-2024-3amp"
[X Link](https://x.com/lateinteraction/status/1771762326649110843) 2024-03-24T04:53Z [----] followers, [----] engagements
""Who validates LM-based metrics" is a good question. Luckily it's *not* recursive. Your language program may generate say free-form text (hard to evaluate). But your metrics/assertions are just classifiers which makes them much easier to optimize given a few labels. Getting reliable outputs from LLMs is tricky. Use LLMs to validate LLM outputs they say but who validates the validators My collaborators & I are interested in feedback on our ideas to solve this. If you have a spare hour in the next few days please DM/email me 🙏 thx Getting reliable outputs from LLMs is tricky. Use LLMs to"
[X Link](https://x.com/lateinteraction/status/1772294745299001757) 2024-03-25T16:09Z 16.9K followers, 13.3K engagements
"Seems like the very Erik Meijer himself left Meta to. build a new DSPy Well we welcome some competition when he's done Been strange being the *only* programming model that compiles high-level code to optimized LM parameters i.e. instructions examples and weight updates. Folks I left Meta yesterday. This time there were no layoffs or anything thrilling involved. In fact I am more bullish than ever about Meta with the companys increased focus on AI. But given the incredible competitive pressure in the field there is really no advantage to be Folks I left Meta yesterday. This time there were no"
[X Link](https://x.com/lateinteraction/status/1772758620511310112) 2024-03-26T22:52Z [----] followers, 37.8K engagements
"Any baseball fans around An example in the HoVer dataset asks systems to verify this claim: The MVP of a game that Red Flaherty umpired was elected to the Baseball Hall of Fame. I know nothing about baseball but this seems to be oddly phrased especially "game". 1/2"
[X Link](https://x.com/lateinteraction/status/1774244506143207710) 2024-03-31T01:17Z [----] followers, [----] engagements
"I now make a joke about API (artificial programmable intelligence) every time someone asks me after a talk if DSPy is my path to AGI. Its funny because I absolutely mean it. AGI is nonsense but I like working on API Artificial Programmable Intelligence. AGI is nonsense but I like working on API Artificial Programmable Intelligence"
[X Link](https://x.com/lateinteraction/status/1774532627586064496) 2024-03-31T20:22Z 10K followers, [----] engagements
"👀 Replits new code repair LLM uses DSPy for a few-shot prompt pipeline to synthesize diffs Pretty cool https://t.co/dSC2wS6pZ1 Replits new code repair LLM uses DSPy for a few-shot prompt pipeline to synthesize diffs Pretty cool https://t.co/dSC2wS6pZ1"
[X Link](https://x.com/lateinteraction/status/1775557645560172890) 2024-04-03T16:15Z 11K followers, 17.4K engagements
"@ChiefScientist @databricks Thank you Alexy"
[X Link](https://x.com/lateinteraction/status/1775608863741403452) 2024-04-03T19:38Z 10.1K followers, [---] engagements
"Due to interesting circumstances a 52-hour Amtrak may be unavoidable. One can enjoy this especially in a little roomette right"
[X Link](https://x.com/lateinteraction/status/1777041552227131515) 2024-04-07T18:31Z 10.3K followers, 12.9K engagements
"@Lambda_freak thank you Peter I have VRZ phew"
[X Link](https://x.com/lateinteraction/status/1777061424352928171) 2024-04-07T19:50Z 10.3K followers, [---] engagements
"I worry about a bubble burst once people realize that no AGI is nearno reliably generalist LLMs or agents. Might seem less ambitious but it's far wiser to recognize: LLMs mainly create opportunities for making *general* progress for building AIs that solve *specific* tasks"
[X Link](https://x.com/lateinteraction/status/1778844352334508140) 2024-04-12T17:55Z 11.2K followers, 73.8K engagements
"@BrandoHablando Yes we support inference with TGI VLLM and SGlang. I think the first and third are a bit better tested imo than VLLM with DSPy. One nice thing about VLLM is that some people managed to use it with dspy.OpenAI because VLLM supports that API. Tagged some people on GitHub"
[X Link](https://x.com/lateinteraction/status/1779654069583536175) 2024-04-14T23:32Z 10.6K followers, [---] engagements
"@letsrebelagain @jeremyphoward @enginoid @HamelHusain I dont know where [--] minutes came from tbh. I remember OpenAI was having trouble with timeouts around the day Hamel posted this. A few hundred API calls with DSPys parallelism and gpt-3.5s latency should finish in a few minutesat least if you have a decent rate limit"
[X Link](https://x.com/lateinteraction/status/1779716664894091759) 2024-04-15T03:41Z 10.6K followers, [---] engagements
"@HamelHusain @letsrebelagain @jeremyphoward @enginoid I agree. I think this file doesnt normally take that long tbh. We saw github issues in mid Feb about openAI timeouts. Your run was probably just stuck Anyway the sequence weve generally recommended is on the README. The first and all runnables are free and take 2-3 mins"
[X Link](https://x.com/lateinteraction/status/1779720963258544406) 2024-04-15T03:58Z 10.6K followers, [---] engagements
"True but not because people cant afford full SGD. Its because better more modular learning paradigms are emerging. Balancing fine-tuning & coarse-tuning is becoming essential. Systems now must efficiently learn *high-level* behavioral changes and *compositions* of skills. My easy prediction: We are moving to a macro-learning level where SGD will be seen as some technology for GPU-rich heavy players and most of the model design will be model merging + LoRa-like + prompt. A bit like engineering assembles pieces made by aluminum foundries. My easy prediction: We are moving to a macro-learning"
[X Link](https://x.com/lateinteraction/status/1783857815683285335) 2024-04-26T13:57Z 11.1K followers, [----] engagements
"There goes my plan to teach the first course entirely on Programming with LMs Joking aside this is an exciting course topic from @davidschlangen at the University of Potsdam. And apparently then @ChrisGPotts's 224U is no longer the only NLP course that heavily features DSPy. Another class I'm teaching this semester is "Programming w/ LLMs". This sidesteps the whole chatbot / assistant / "an AI" theme and looks at LLMs as function approximators -- where weirdly the function needs to be "found" first. (Yes DSPy will feature heavily.) https://t.co/yqObFXVg5l Another class I'm teaching this"
[X Link](https://x.com/lateinteraction/status/1784647418539327870) 2024-04-28T18:14Z 11.1K followers, 25.4K engagements
"Sufficient practice can often approximate system [--] goal-oriented planning as system [--] reflexive responses like when one internalizes how to drive. But for any degree of internalized system [--] capabilities reliable composition of maximal skills needs explicit system [--] structure"
[X Link](https://x.com/lateinteraction/status/1785153623179293151) 2024-04-30T03:46Z 13.2K followers, 10.8K engagements
"@n0riskn0r3ward @dottxtai Masking out some tokens will not affect the ordering of other tokens. Merging token probabilities within equivalence class (like case insensitive decoding for example) can affect the relative ordering indeed though a challenge is that equivalence classes are often not syntactic"
[X Link](https://x.com/lateinteraction/status/1785690170140094694) 2024-05-01T15:18Z 16.6K followers, [--] engagements
"There's often many "earliest" papers that have discovered or proposed something. We should think of it as a Pareto frontier: how early it was vs. how precise the proposal was at matching the eventual consensus. You'll find that this works out. Either way T5 was super early. Undoubtedly it is Raffel et al (2020) the T5 paper which came earlier. You can see the beginnings of supervised instruction/prompting there (see red highlights) and it was the first ever paper to show you can use formulate any task as a text to text task and train multiple https://t.co/zvbOnwjM3T Undoubtedly it is Raffel"
[X Link](https://x.com/lateinteraction/status/1785692713880625214) 2024-05-01T15:28Z 19.3K followers, 19.1K engagements
"@deliprao 100% it was probably too early in fact. That's always a problem"
[X Link](https://x.com/lateinteraction/status/1785693686258659573) 2024-05-01T15:32Z 11.2K followers, [---] engagements
"@deliprao Both in the sense that it doesn't work well enough yet and people don't grok it well enough yet"
[X Link](https://x.com/lateinteraction/status/1785693813564227817) 2024-05-01T15:32Z 11.2K followers, [---] engagements
"@bclavie @huggingface lol the way they plot that increase makes the graph look like it was a zero before but it had been consistently [------] for several months"
[X Link](https://x.com/lateinteraction/status/1786385357250134235) 2024-05-03T13:20Z 11.2K followers, [---] engagements
"Think of DNNs but replace: * Learnable layers - learnable LM calls. * Dimensions - Natural language types e.g. queries answers reasoning chains * Passing tensors - passing these text objects * CNNs/RNNs/etc - CoT/ReAct/etc * Differentiable loss - arbitrary metric"
[X Link](https://x.com/lateinteraction/status/1788241006007366105) 2024-05-08T16:14Z 11.4K followers, [----] engagements
"The Bitter Lesson is just an artifact of lacking good high-level ML abstractions. Scaling deep learning helps predictably. But after every paradigm shift the best systems include modular specializations that basically look the same every time and should have been reuseable"
[X Link](https://x.com/lateinteraction/status/1789407192690422239) 2024-05-11T21:28Z 11.4K followers, 18.8K engagements
"Crudely ML has [--] components: learning (where SGD shines) and composition (algorithms expressing problem structure). The latter won't go anywhere for almost any problem at a sufficiently high level of complexity. It's programming. We need learning-within-algorithms abstractions"
[X Link](https://x.com/lateinteraction/status/1789412287775752283) 2024-05-11T21:48Z 11.4K followers, [----] engagements
"@ludgerpaehler Let's say SLURM But also let's say you get to choose/inform this. Say it's a small enough lab that maybe you can have "schedule over slack""
[X Link](https://x.com/lateinteraction/status/1790875079556874414) 2024-05-15T22:41Z 11.5K followers, [---] engagements
"API achieved internally"
[X Link](https://x.com/lateinteraction/status/1790940150417047952) 2024-05-16T02:59Z 11.5K followers, 15.6K engagements
"Folks ask how this will affect DSPy & ColBERT. Wellextremely positively Both grew dramatically in [----] while I was busy w faculty interviews & a broken (but now almost recovered) left hand. Looking forward you should expect *a lot* more research and OSS in both directions I'm excited to share that I will be joining MIT EECS as an assistant professor in Fall [----] I'll be recruiting PhD students from the December [----] application pool. Indicate interest if you'd like to work with me on NLP IR or ML Systems Stay tuned for more about my new lab. I'm excited to share that I will be joining MIT"
[X Link](https://x.com/lateinteraction/status/1797679155209032014) 2024-06-03T17:18Z 12.9K followers, 29K engagements
"Despite all the theorems a surprising fraction of grad students believe in free lunch"
[X Link](https://x.com/lateinteraction/status/1799239923129876947) 2024-06-08T00:40Z 13K followers, 28.2K engagements
"What are people doing with ColBERTv2 Apparently it receives 6-7 million downloads per month which is top-25 among all [------] models on HF"
[X Link](https://x.com/lateinteraction/status/1800208322005700638) 2024-06-10T16:48Z 13.2K followers, 32.7K engagements
"I'm joining @hanlintang's 12:30pm keynote tomorrow at the @Data_AI_Summit to talk about DSPy. See you all there Sign up for this technical keynote with @hanlintang our CTO of Neural Networks and @lateinteraction Creator of DSPy. Theyll discuss the latest innovations that help enterprises build & deploy GenAI applications. Tomorrow at 12:30pm save your seat https://t.co/DwWzAYsbHK https://t.co/YVOnXmr5oB Sign up for this technical keynote with @hanlintang our CTO of Neural Networks and @lateinteraction Creator of DSPy. Theyll discuss the latest innovations that help enterprises build & deploy"
[X Link](https://x.com/lateinteraction/status/1800740263499121070) 2024-06-12T04:01Z 13.3K followers, 12.1K engagements
"Super cool blogpost from @SnowflakeDB's Alejandro Herrera on using DSPy for higher-quality cost-effective RAG. "We demonstrate that by algorithmically tuning a @MistralAI Mixtral 8x7B pipeline with DSPy and @SnowflakeDB Cortex we can outperform a larger Llama370B pipeline while delivering 5X cost savings." https://t.co/2gX2US232Y "We demonstrate that by algorithmically tuning a @MistralAI Mixtral 8x7B pipeline with DSPy and @SnowflakeDB Cortex we can outperform a larger Llama370B pipeline while delivering 5X cost savings." https://t.co/2gX2US232Y"
[X Link](https://x.com/lateinteraction/status/1801657111459598426) 2024-06-14T16:45Z 13.2K followers, 22.9K engagements
"three of the largest optimizer updates completed and on their way into DSPy"
[X Link](https://x.com/lateinteraction/status/1802418209041285353) 2024-06-16T19:09Z 13.3K followers, 34.7K engagements
"🚨Announcing the largest study focused on *how* to optimize the prompts within LM programs a key DSPy challenge. Should we use LMs to Craft instructions Self-generate examples Handle credit assignment Specify a Bayesian model By @kristahopsalong* @michaelryan207* &team🧵"
[X Link](https://x.com/lateinteraction/status/1803093936619270464) 2024-06-18T15:54Z 13.7K followers, 152.4K engagements
"one day I will figure out why RL people (and everyone else) use RL for the problem the solution space the specific implementations the area and some underlying philosophy"
[X Link](https://x.com/lateinteraction/status/1803586300068634632) 2024-06-20T00:30Z 13.4K followers, [----] engagements
"@tom_doerr Would merge"
[X Link](https://x.com/lateinteraction/status/1803862503887352258) 2024-06-20T18:48Z 13.3K followers, [---] engagements
"@OwainEvans_UK Awesome Thank you"
[X Link](https://x.com/lateinteraction/status/1804566470892081210) 2024-06-22T17:25Z 13.3K followers, [---] engagements
"@OfirPress Thats a key approach for sure My understanding is that GPT-3.5 (davinci-02 onwards) has always been finetuned at a large scale on such trajectories since at least [----] if the WebGPT work was actually applied at all inside OpenAI. https://arxiv.org/abs/2112.09332 https://arxiv.org/abs/2112.09332"
[X Link](https://x.com/lateinteraction/status/1810313153634615664) 2024-07-08T14:01Z 13.7K followers, [----] engagements
"Nice stuff. Posting with permission"
[X Link](https://x.com/lateinteraction/status/1810734964725665909) 2024-07-09T17:57Z 13.7K followers, 13K engagements
"@dosco We wish we had your contributions in DSPy but I know you're building other awesome impactful things :D"
[X Link](https://x.com/lateinteraction/status/1810749767489835262) 2024-07-09T18:56Z 13.7K followers, [--] engagements
"Exactly. What's surprising even in hindsight is that the difference between working extremely well (81.3% for ColPali) and not working at all (58.8% for BiPali) is the "Col" part of ColPali i.e. ColBERT late interaction. VLMs & vector representations alone don't do the trick It's fascinating how a small 3B model like ColPALI can disrupt the PDF extraction industry overnight It's fascinating how a small 3B model like ColPALI can disrupt the PDF extraction industry overnight"
[X Link](https://x.com/lateinteraction/status/1813140776869658833) 2024-07-16T09:17Z 13.9K followers, 65.9K engagements
Limited data mode. Full metrics available with subscription: lunarcrush.com/pricing
@lateinteraction Omar Khattab
Omar Khattab posts on X about llm, ai, open ai, databricks the most. They currently have [------] followers and [---] posts still getting attention that total [---------] engagements in the last [--] hours.
Engagements: [---------] #
Mentions: [--] #
Followers: [------] #
CreatorRank: [-------] #
Social Influence
Social category influence technology brands social networks stocks finance currencies countries celebrities events cryptocurrencies fashion brands
Social topic influence llm #91, ai, open ai, databricks, if you, $googl, context window #23, in the, level, rl
Top accounts mentioned or mentioned by @dspyoss @databricks @maximerivest @bclavie @raw_works @dosco @tomdoerr @dbreunig @a1zhang @teknium @jchencxh @chrisgpotts @shoggoth1313 @cshorten30 @kristahopsalong @isaacbmiller1 @goobyesq @deliprao @arnavthebigman @hanchunglee
Top assets mentioned Alphabet Inc Class A (GOOGL)
Top Social Posts
Top posts by engagements in the last [--] hours
"Want to learn more about retrieval both key concepts and cutting edge models & tools Watch our recent conversation at the LangChain Retrieval Webinar w/ @hwchase17 @jobergum @charles_irl I discuss the retrieval model and the programming model from 29:00"
X Link 2023-05-26T22:01Z [----] followers, [--] engagements
"@marktenenholtz I dont say this often but I dont think langchain was ever meant to provide this. It remains great for I dont want to think about this give me an easy thing that runs on my data. If you wanna build novel stuff youre looking for a research framework like our own DSP"
X Link 2023-07-25T02:16Z [----] followers, [----] engagements
"@marktenenholtz This is not a failure of langchain. People are just projecting expectations never espoused by the library to my knowledge. You cant build new activation functions or new optimizers using HuggingFace APIs. Thats not what they do. You do that in PyTorch"
X Link 2023-07-25T02:19Z [----] followers, [---] engagements
"@Nils_Reimers @macavaney @jobergum Some slice of MS MARCO might actually be decebt at this. The lexical retrieval (aka Bing index circa 2016) was at the level of full documents and had access to lots of (unreleased) metadata. Its just that the corpus is not topically realistic and the queries are too easy/head"
X Link 2023-08-09T02:46Z [----] followers, [---] engagements
"Exactly. Folks often ask "ColBERT is great but wouldn't the embeddings take a lot of space" Nope. ColBERTv2 with PLAID (see QT) can use as little as 20 bytes per vector. Same size as [--] floats. Can fit [---------] passages in 200GB and search them in [---] milliseconds on CPU"
X Link 2023-08-13T23:39Z [----] followers, 49.9K engagements
"@mystellial It's a very minimal framework (not a lot of code but very rich ideas) so multi-language support may not be too hard. Down the road would happily help with a fork in any language that has enough meta-programming tricks to make things work cleanly"
X Link 2023-08-24T20:56Z [----] followers, [--] engagements
"@gkossakowski Thank you @gkossakowski No limitation in there but GPT-4 is slow and expensive. We can do really well with GPT-3.5 (or even just local open models). That said GPT-4 can probably enable somewhat fancier / more ambitious pipelines"
X Link 2023-08-24T21:41Z 16K followers, [---] engagements
"@imaurer Oh Ian do you think it should be emphasized more"
X Link 2023-08-24T22:18Z [----] followers, [---] engagements
"@ofermend @MetaAI @OpenAI @anyscalecompute @databricks @MSFTDeepSpeed Yes such a week But misses like the most important release of late last week :-) https://x.com/lateinteraction/status/1694748401374490946 🚨Announcing the framework for solving advanced tasks w/ LMs. Express any pipeline as clean Pythonic control flow. Just ask DSPy to your modular code into auto-tuned chains of prompts or finetunes for GPT Llama and/or T5.🧵 https://t.co/QColxp8atV https://t.co/PbwRPrctSp https://x.com/lateinteraction/status/1694748401374490946 🚨Announcing the framework for solving advanced tasks w/"
X Link 2023-08-28T06:45Z 16.3K followers, [---] engagements
""We need an AI compiler" DSPy exists right here: Like PyTorch but for foundation model programming. You write your own control flow. The compiler optimizes prompting/finetuning for the declarative steps of your pipelineaccording to your metrics"
X Link 2023-08-31T03:00Z [----] followers, 28.7K engagements
"Really enjoyed visiting my undergraduate alma mater Carnegie Mellon Qatar. I presented "ColBERT: Effective and Efficient Search with Late Interaction" to the CS faculty and a room full of the particularly sharp and engaged CMUQ ugrads. Thanks for hosting me @CarnegieMellonQ"
X Link 2023-08-31T22:03Z 24.9K followers, [----] engagements
"@annargrs @yoavgo @Francis_YAO_ This is the only reason I submit these to non-ACL confs. This is a common challenge now with foundation models/frameworks. I dont need to advertise a paperjust the latest artifacts. But the key goal is to help researchers on active projects so waiting months is infeasible"
X Link 2023-09-05T14:46Z [----] followers, [---] engagements
"@annargrs @yoavgo @Francis_YAO_ Obviously this type of research poses a different type of challenge for anonymity in general. Id be fine with an artifact-first track that has different rules or expectations but this is an increasingly essential and impactful type of NLP research IMO"
X Link 2023-09-05T14:48Z [----] followers, [--] engagements
"@sherjilozair @yacineMTB @attharave This realistic understanding is important. But it doesnt change what you should do in terms of the work"
X Link 2023-09-06T16:51Z [----] followers, [---] engagements
"@khoomeik Its actually deeper than that smh. RAG is an excellent paper but it didnt invent conditioning an LM on some retrieved context This is a much older idea The papers novel explorations are about joint training and marginalization etc which arent even applicable to langchain"
X Link 2023-09-08T19:14Z [----] followers, [---] engagements
"@marktenenholtz A lot of people use DPR-style models:"
X Link 2023-09-13T17:50Z [----] followers, [----] engagements
"@marktenenholtz Which actually includes OpenAIs own embedding models in a sense"
X Link 2023-09-13T17:51Z [----] followers, [----] engagements
"@douwekiela This seems awesome We need to add this to the DSPy compiler so we explore a larger search space"
X Link 2023-09-15T21:47Z 16.3K followers, [---] engagements
"@WenhuChen @b_arbaretier Oh Wenhu let's chat soon. We're adding a general-purpose ProgramOfThought module like a nn layer in pytorch which can be optimized to fit any multi-stage pipeline and any task/metric. Would be cool if you check it out and see if it matches how you'd generalize PoT yourself"
X Link 2023-09-16T17:50Z [----] followers, [---] engagements
"@generatorman_ai @datagenproc Yeah for instance colbertv2 can find the passage for this. CTRL-F for Tom Cruise in there"
X Link 2023-09-23T16:18Z [----] followers, [--] engagements
"@omarsar0 Prompt engineering is needed the same way hyperparameters are important. The architecture design matters a lot more than either. And both can be automatically optimized: https://github.com/stanfordnlp/dspy https://github.com/stanfordnlp/dspy"
X Link 2023-09-23T17:31Z 16K followers, [----] engagements
"@gazorp5 @jkronand Thanks for engaging. If youd like to read more start in this section:"
X Link 2023-09-24T10:01Z [----] followers, [----] engagements
"@MystiqCaleid @jkronand It works extremely well with agentsway better than existing stuff. Will put out some threads on that"
X Link 2023-09-24T12:23Z 16.3K followers, [----] engagements
"Right LLMs are fuzzy devices and everyone writing prompts is coding in assembly. You could be writing well-defined modules instead and asking a compiler to build and optimize the messy parts: With many 🧩 dropping recently a more complete picture is emerging of LLMs not as a chatbot but the kernel process of a new Operating System. E.g. today it orchestrates: - Input & Output across modalities (text audio vision) - Code interpreter ability to write & run https://t.co/2HsyslOG2F With many 🧩 dropping recently a more complete picture is emerging of LLMs not as a chatbot but the kernel process"
X Link 2023-09-28T17:23Z 16.3K followers, 93.1K engagements
"@karpathy A more mature analogy with all the pieces functional and powerful results here:"
X Link 2023-09-28T17:25Z [----] followers, 16.4K engagements
"DSPy and ColBERT are interesting academic experiments imo. Each is a multi-paper repo that has one coherent artifact combining our latest research together. We typically release the features as open sourcehence get users/feedbackwell before writing a paper on the new ideas"
X Link 2023-10-06T03:28Z [----] followers, 26.5K engagements
"Join us in [--] minutes if you want to learn more about DSPy and how it works. Chatting about DSPy LlamaIndex LLMs etc. DSPy (@lateinteraction @TomJoshi2) as an LLM framework has some really cool novel ideas: ✅ Emphasis on programming over prompting: compose declarative modules instead of hand-crafting prompts ✅ Let the compiler optimize for you: given these modules the compiler can auto-tune DSPy (@lateinteraction @TomJoshi2) as an LLM framework has some really cool novel ideas: ✅ Emphasis on programming over prompting: compose declarative modules instead of hand-crafting prompts ✅ Let the"
X Link 2023-10-06T21:51Z 16.3K followers, 12.6K engagements
"A cool thread yesterday used GPT4 ($50) a 500-word ReAct prompt and [---] lines of code to finetune Llama2-7B to get 26% HotPotQA EM. Let's use [--] lines of DSPywithout any hand-written prompts or any calls to OpenAI ($0)to teach a 9x smaller T5 (770M) model to get 39% EM 🧵"
X Link 2023-10-11T15:58Z [----] followers, 148.6K engagements
"And ahem use DSPy: It's easy and powerful but also controllable and transparent. You can control every step. Many (other) papers can be implemented in [---] lines of DSPy without any hand-writen prompts with much smaller LMs and with higher quality"
X Link 2023-10-11T15:58Z [----] followers, [----] engagements
"What should we do next"
X Link 2023-10-13T00:36Z [----] followers, [----] engagements
"@deliprao @ChrisGPotts I think I need more Twitter (X) threads to graduate @deliprao"
X Link 2023-10-13T00:49Z [----] followers, [---] engagements
"DSPy is all about compiling LM programs. Why "compile"❓ Traditional compilers translate programs to machine code for specific hardware A compiler for LM programs** translates code with LM calls into prompt or finetune chains for specific LMs ** i.e. only DSPy so far"
X Link 2023-10-25T00:00Z [----] followers, 20.2K engagements
"This is a great recent example of that where we compile a short program into a high-quality prompt chain for Llama2-13b and then compile that into a finetune chain for T5-770M. With really high quality starting from minimal data:"
X Link 2023-10-25T00:02Z [----] followers, [----] engagements
"@eggie5 There are PRs and public branches all over the place like this one:"
X Link 2023-10-31T21:49Z [----] followers, [--] engagements
"@atroyn Using representations (or tools in general) to find context and learning to use (and re-encode) contextfor tasksis a key idea thats here to stay. Its a good idea (direct tool use & context manipulation) embedded in a necessary hack (deep learning) not the other way around"
X Link 2023-11-07T17:12Z [----] followers, [---] engagements
"Q: What does compilation mean for DSPy programs Usually we have CPUs & GPUs. You compile your high-level code so it uses them efficiently. We now have LPUslanguage processing units DSPy compiles your code into control flow with instructions (prompts) that maximize quality"
X Link 2023-11-22T18:07Z [----] followers, [----] engagements
"Aargh I promise I do"
X Link 2023-11-29T03:07Z [----] followers, 14.7K engagements
"How come no one told me about this paper RobustQA extends our LoTTE IR benchmark to a large labeled out-of-domain QA dataset ColBERTv2 outperforms DPR BM25 Contriever and expensive cross-encoders by [--] [--] [--] and [--] HIT@20 resp. Atlas is strong on downstream QA"
X Link 2023-11-29T18:40Z [----] followers, 17.8K engagements
"@haldaume3 Yes Learning weights (or soft prompts) is often better Any DSPy module can be compiled into prompts or finetunes Prompts are easier to bootstrap from scratch (e.g. zero shot) then compiled into cheaper high-quality finetunes. Only in DSPy Example:"
X Link 2023-12-03T02:26Z [----] followers, [---] engagements
"This year there were a few cool proposals that create and save "skills" during learning. I find it fascinating that Ashwin Paranjape @XiangLisaLi2 @siddkaramcheti and I called this concept "skills" back in July [----]. It's a good idea and "skill" is indeed a natural name"
X Link 2023-12-05T04:40Z [----] followers, 13.3K engagements
"@pascalefung Hey Pascale I agree with this. But Im making a distinction between systems and models. Without defining the context of a system (what fact corpus are generations derived from) and its structure (how does it use the LM) it doesnt make sense to speak of accuracy/hallucination"
X Link 2023-12-09T16:13Z [----] followers, [---] engagements
"The incredible @hrishioa builds what might be one of the richest LM programs. The principle is simple: dont rely on the LM as a user-facing system use it as a building block the way you would solve a task. (RAG multi-hop QA self-refine etc are just special cases of this)"
X Link 2023-12-13T16:06Z [----] followers, 36.2K engagements
"@PierreRunavot @hrishioa Can I interest you in. Nvm Im sure you know about it 😆"
X Link 2023-12-13T20:07Z [----] followers, [--] engagements
"🚨Announcing a powerful construct by @ShangyinT* @slimshetty_* @arnav_thebigman* Your LM isn't following complex instructions Stop prompting Add a one-liner assertion in your program: up to 35% gains w auto-backtracking & self-refinement🧵"
X Link 2023-12-14T15:51Z [----] followers, 74.5K engagements
"@Thewimo The VLLM module is here: Are you saying it's not imported in the right places though"
X Link 2023-12-17T18:35Z [----] followers, [---] engagements
"@Thewimo ah good catch yes VLLM is on main but not on pip yet pip install -e /path/to/local/version will do or if you need it I can upgrade pypi now. Should I"
X Link 2023-12-17T18:43Z [--] followers, [--] engagements
"@roydanroy @ThomasW423 @QuanquanGu Heres my definition. Hallucination is indeed not well-defined for a statistical model without additional assumptions. Its definited for system in which we specify or assume a number of things:"
X Link 2023-12-18T02:49Z [----] followers, [---] engagements
"@roydanroy @ThomasW423 @QuanquanGu A system is said to hallucinate when it generates a statement not derived correctly from supplied cited or assumed facts. This extrapolates summarization i.e. to open-ended generation conditioned on a large and hopefully explicit corpus"
X Link 2023-12-18T03:00Z [----] followers, [--] engagements
"@roydanroy @ThomasW423 @QuanquanGu It's also a causal claim: it's not saying hallucination is when the system is wrong it's saying hallucination is when the system extrapolates past what the corpus can support. In particular reasonable deduction leading to incorrect statements is not hallucination"
X Link 2023-12-18T03:02Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala Here's a thread with the full analogy. It's a bit broader than DNN frameworks though they're a key piece"
X Link 2023-12-18T03:23Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala Right now most of the emphasis is incorrectly placed on aligning the LM itself. That's silly Why do you want to align an (uninitialized) layer in your architecture You should align the full userfacing system instead not its parts A DNN abstraction places the focus right"
X Link 2023-12-18T03:49Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala Also: What's the alternative DSPy solves problems no other framework is attempting to resolve: stacking prompting patterns and bootstrapping techniques for arbitrary pipelines. LangChain/LlamaIndex/AutoGen are higher-level (for app dev) but should adopt DSPy like HF torch"
X Link 2023-12-18T03:54Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala If I take this seriously DSPy's adoption is larger within its target audience certainly in my sphere at least. In the "LM development stack" DSPy is much lower-level than frameworks for app devs. Much the same way that app devs don't need to use JAX/torch/CUDA"
X Link 2023-12-18T04:54Z [----] followers, [--] engagements
"@shoggoth1313 @soumithchintala If you're trying to build and tune a new LM pipeline for a new task LangChain/LlamaIndex/AutoGen aren't useful perhaps even are counter-productive. I know several dozen in this space most using or learning to use DSPy. Their only other option is prompt hacking with strings"
X Link 2023-12-18T05:02Z [----] followers, [--] engagements
"@shoggoth1313 Definitely valuable But you perhaps betray a limited understanding of this space & its timeline LangChain is a great example of taking off for incidental reasons and then creating a bad reputation for LM frameworks. (LlamaIndex is different; it is useful) Hype begets hype"
X Link 2023-12-18T05:29Z [----] followers, [--] engagements
"@shoggoth1313 Both have built organic communities of new people excited to play with AI and that's their value. A gold rush into a one more tech stack. LangChain is a gateway to the space. LlamaIndex is very conveniant out of the box; pretty useful. DSPy solves new technical problems"
X Link 2023-12-18T05:33Z [----] followers, [---] engagements
"@shoggoth1313 Hey overall this was quite fun. Definitely lots to think about and you helped me articulate a few things better in short form. Hope it was useful for you as well and good luck with your stealth"
X Link 2023-12-18T05:34Z [----] followers, [--] engagements
"Progress on dense retrievers is saturating. The best retrievers in [----] will apply new forms of late interaction i.e. scalable attention-like scoring for multi-vector embeddings. A🧵on late interaction how it works efficiently and why/where it's been shown to improve quality"
X Link 2023-12-18T17:45Z 14K followers, 236.6K engagements
"Say you have 1M documents. With infinite GPUs what would your retriever look like Maybe a cross-encoder Finetune a large LM to take querydoc pairs. Run it 1M times to get a score for all docs. Expressive Given a query the LM can pay attention to every detail in the doc"
X Link 2023-12-18T17:45Z 14K followers, [----] engagements
"It was a last-ditch run on a Sunday night (3 Nov'19) after complex scoring failed. I spent weeks looking for a "bug". ColBERT w cheap scoring rivaled BERT-large cross-encoders with 10000x more FLOPs Called it ColBERT as a pun: late show / late interaction log scale latency:"
X Link 2023-12-18T17:45Z [----] followers, [----] engagements
"There's a little-known trick that was essential for ColBERT's results: Query Augmentation. ColBERT appends MASK tokens to the query encoder to allow BERT to create more query vectors that aren't there Is it the earliest form of a scratchpad / chain of thought From Nov 2019"
X Link 2023-12-18T17:45Z [--] followers, [----] engagements
"OK but how can ColBERT search 100M docs in [---] milliseconds Late interaction is pruning-capable: it only needs to "touch" 0.1% of the documents to find the top-K. This is by design: it's composed of monotonic functions (Max/Sum) which enable some neat algorithmic tricks"
X Link 2023-12-18T17:45Z 14K followers, [----] engagements
"We can decompose late interaction into dozens of tiny nearest-neighbor searches at the token level. We'll only fetch & score docs in which at least one token in close to (at least one token in) the query. Otherwise we can prove the score will be too small and we can skip it"
X Link 2023-12-18T17:45Z 14K followers, [----] engagements
"@jxmnop Are you saying this doesn't exist; is it different from 4-5 things like pause tokens This may be the earliest of the general form afaik:"
X Link 2023-12-20T15:56Z [----] followers, [----] engagements
"A🧵on beating the hardware lottery for retrieval: the internals of the late interaction stack. ColBERT introduced a quirky multi-vector retrieval architecture. It does wonders for quality. But how can it search 100M docs in [---] sec on CPU Or store [--] billion embeddings in 20GB"
X Link 2023-12-20T21:01Z [----] followers, 76.8K engagements
"OK I'll pause the🧵here. In the next thread I'll cover: 1) Why does late interaction work so well in dozens of eval/analysis papers We'll dig into evals with ColBERT-QA all the way to UDAPDR. 2) What's next for late interaction And for neural retrieval in general"
X Link 2023-12-20T21:01Z [----] followers, [---] engagements
"Next Thread:"
X Link 2023-12-20T21:04Z [----] followers, [---] engagements
"@omarsar0 @abacaj more innovation around how to prompt theze models"
X Link 2023-12-20T23:51Z [----] followers, [---] engagements
"Someone just reminded me not everyone knows where the find the ColBERT code free Google Colab notebook to play with it list of papers etc"
X Link 2023-12-21T01:13Z [----] followers, [---] engagements
"Someone just reminded me not everyone knows where the find the ColBERT code free Google Colab notebook to play with it list of papers etc"
X Link 2023-12-21T01:14Z [----] followers, [---] engagements
"There's an important missing perspective in the "GPT-4 is still unmatched" conversation: It's a process (of good engineering at scale) not some secret sauce. To understand let's go back to 2000s/2010s when the gap between "open" IR and closed Google Search grew very large. 🧵 The recent releases of many GPT-3.5 class AIs (Grok Mixtral Gemini Pro) are oddly unilluminating about the future of frontier AI Its been a year & no one has beat GPT-4. Will they Is there some magic there Does it indicate a limit to LLMs Will GPT-4.5 be another huge jump The recent releases of many GPT-3.5 class AIs"
X Link 2023-12-21T17:14Z [----] followers, 135.8K engagements
"Note: I have no inside OpenAI info & I'm uninterested in individual LMs. Expressive power lies in the program wielding the LM As commercial search matured the IR field went from a very active and hot area of research circa [----] to a much less active one 2004ish through 2018"
X Link 2023-12-21T17:14Z [----] followers, [----] engagements
"Setting this up well takes timeand takes a team willing to be objective/honest and optimize for the right things with a long-term vision. Everyone rushing to "catch up" won't invest in this. Most other efforts are like "what's the right data mix" as if this is a one-off thing"
X Link 2023-12-21T17:16Z [----] followers, 10.6K engagements
"Addendum 1: You could half-jokingly call this process collective human-policy RL. Addendum 2: OpenAI may apply this process to GPT-3.5 but it's small and cost-optimized. Size is a work multiplier. Others "match" GPT-3.5 differently: they focus on scale & forgo most cost cutting"
X Link 2023-12-21T18:21Z [----] followers, [----] engagements
"Addendum 3: Another way to put it is this. The community is looking for the silver bullet that will allow GPT-4 to "emerge". But GPT-4 doesn't emerge. It's built very carefully feature by feature or unit test by unit test. Some research is key (ICL) but otherwise engineering"
X Link 2023-12-21T18:55Z [----] followers, [----] engagements
"This can get around the key limitations of autoregressive LMs control grounding transparency planning and efficiency. And it can simultaneously leverage their main power: that well-scoped automated hypothesis testing over the space of prompts is extremely data-efficient"
X Link 2023-12-22T17:56Z [----] followers, [----] engagements
"👀 I'm being called out. But I agree. DSPy will be so huge pretty soon @mbusigin. I'm spending winter break revamping docs/examples and doing much-needed refactoring. The abstraction is incredibly elegant as many have noted. The DX (developer experience) will soon match. I think DSPy would be so huge if it had applied any thought to the DX at all I think DSPy would be so huge if it had applied any thought to the DX at all"
X Link 2023-12-22T19:08Z [----] followers, 54.3K engagements
"@McaleerStephen Looked into DSPy"
X Link 2023-12-22T21:22Z [----] followers, [---] engagements
"@hwchase17 @HanchungLee @mbusigin @LangChain 💯 yes what would be useful Han"
X Link 2023-12-23T00:03Z [----] followers, [---] engagements
"@panabee Indeed it could be very different. But it depends on the strategy from my thread: find all common failure modes create "unit tests" for them extensively and collectively work to pass them. The open community isnt doing anything like this yet. Thats the bottleneck"
X Link 2023-12-23T22:08Z [----] followers, [--] engagements
"@srchvrs Thanks for the insightful posts Leo I doubt we have a fundamentally distinct understanding here. Im speaking to people who think OpenAI has magical model ideas or that Google had magical IR modeling ideas. Both have excellent engineering with tens of small key pieces"
X Link 2023-12-27T05:01Z [--] followers, [---] engagements
"@olgias @jobergum @bclavie @huggingface @vespaengine has a few more things"
X Link 2023-12-27T13:29Z [----] followers, [--] engagements
"@Shroominic @akatzzzzz @jxnlco @Teknium1 @arnav_thebigman Let me know if you wanna explore this further @Shroominic. Im not 100% clear on the details of your tweet"
X Link 2023-12-27T15:11Z [----] followers, [--] engagements
"@mayhewsw @deliprao Honestly fwiw I thought about tweeting with ELMo originally credit where its due"
X Link 2023-12-28T01:00Z [--] followers, [--] engagements
"@paul_cal So the original NYT prompt is to demonstrate that the LM saw some NYT text at training time (which is perhaps already known more or less) and that theres a chance it reproduces that text on demand (perhaps not for long though and not in latest versions). Underwhelming"
X Link 2023-12-29T23:15Z [----] followers, [--] engagements
"I'd been planning to post the third thread in the late interaction / ColBERT sequence. But posting now (or around Jan 01) may mean a lot of people miss it. What do people think Obviously it's a biased sample but curious"
X Link 2023-12-31T01:39Z [----] followers, [----] engagements
"@trading_indian Haha smart but Twitter really doesnt work that way. Old stuff (i.e. over [--] hours) is basically unboostable"
X Link 2023-12-31T01:47Z [--] followers, [--] engagements
"@arnaudai @ChetanyaRastogi @lancehasson Think long term How long ago was it when there were people in ML/NLP who thought you really ought to tune your weights by hand for your classifier because human intuition is unmatched Code in a high-level language. Let compiling/optimization handle low-level instructions"
X Link 2023-12-31T21:19Z [----] followers, [---] engagements
"@MimansaJ Markup modification/parsing sounds like the one thing good LLMs can do exceptionally well so far Or is that not true"
X Link 2024-01-02T20:23Z [--] followers, [--] engagements
"Is there a way to try ColBERT as a complete noob I literally started learning Python a few months ago. Yes Run this free Colab notebook in your browser: @lateinteraction Is there a complete noob - I literally started learning python a few months ago - to try this out @lateinteraction Is there a complete noob - I literally started learning python a few months ago - to try this out"
X Link 2024-01-03T03:40Z [----] followers, 10.3K engagements
"Fun question. I don't know much about DPO but I can share IR intuition. IR pairwise loss functions (a b) are known to work much better than pointwise loss (a=1 b=0). Pairwise encourages healthy margins without pushing the model to treat a as perfect or b as the worst. @CFGeek my mental model of DPO is "push model away from dispreferred and towards preferred" and SFT is "pull the model towards the gold answer". if this is just like DPO where dispreferred is the current policy i dont have a good intuition for why this would be better than SFT. wdyt @CFGeek my mental model of DPO is "push model"
X Link 2024-01-03T06:16Z [----] followers, [----] engagements
"@H1iReimer I agree 100%. But I think there's some nuance. If NVIDIA stops selling Titan V GPUs does that mean Systems paper that report efficiency results on Titan V are no longer reproducible Only if the paper is poorly done and is basically testing Titan V not fundamental patterns"
X Link 2024-01-04T15:05Z [--] followers, [---] engagements
"@marktenenholtz Just to clarify now ColBERT = ColBERTv2 = PLAID. We stick to the latest i.e. PLAID + some more recent improvements. For all of Common Crawl (wow) I'd do a distributed PLAID. That is build [---] ColBERT indexes on [---] machines and route by query. There are a few ways to split"
X Link 2024-01-05T00:55Z [----] followers, [----] engagements
"@andrew_n_carr @Teknium1 Um who believes that"
X Link 2024-01-05T16:43Z [----] followers, [---] engagements
"@vivekkalyansk @TheXeophon @EugeneVinitsky Im missing any/all context but surprised not to see the right() explanation. LMs barely see text in 2D. A good LM can 100% do tic-tac-toe if its linearized/serialized (very) well"
X Link 2024-01-06T03:45Z [--] followers, [--] engagements
"@420_gunna @bclavie I have IR course material on youtube [--] videos starting with this one from stanford CS224U Spring [----]. The material covers things from BM25 to ColBERT"
X Link 2024-01-06T12:30Z [----] followers, [----] engagements
"@n0riskn0r3ward "Our company merged with Tesla yesterday. Don't say embarrassing things about our Tesla.""
X Link 2024-01-08T01:08Z [--] followers, [---] engagements
"@Prestigious_AI @Mentor_Nolan Working on it this week One challenge is that we dont have an Anthropic key its not open to all yet() afaik"
X Link 2024-01-11T03:53Z [----] followers, [--] engagements
"@labloke11 @TheGrizztronic Hey Id love to help you grok this. I gotta run to a meeting in [--] minutes but before this let me clarify that DSPy is not a drop-in replacement for your current workflow. You have to redesign your workflow with DSPy lessons in mind. You have to decide on a metric to optimize"
X Link 2024-01-12T16:24Z [----] followers, [---] engagements
"Anecdotal example of ColBERT in the wild. Dense retrieval (OpenAI ada-002) doesn't find anything about whether "@paulg is vegan". Returns a random passage about him. ColBERT notices that "Paul liked a chicken McSpicy". That's the sort of thing that makes or breaks a RAG system. query: "was Paul vegan" text splitter: langchain's recursive character text splitter with chunk length of [---] and chunk overlap of [--]. outcome: ColBERT wins this rounds https://t.co/DlybPCkgT9 query: "was Paul vegan" text splitter: langchain's recursive character text splitter with chunk length of [---] and chunk overlap"
X Link 2024-01-14T17:46Z [----] followers, 38.7K engagements
"@deliprao @Samhanknr Thank you for tagging me on this Delip I have a lot of thoughts about this Im torn between sharing them and saving time to speed up ongoing work so I can show more and tell less. Overall Id love to share my current intuition on this but cant tell if its just too abstract"
X Link 2024-01-16T00:57Z [----] followers, [---] engagements
"@deliprao @Samhanknr I dont think gradient descent or a single pass through a DNN or passing embeddings around without concrete form / text / provenience can ever solve a few essential problems. But if RAG means retrieve then generate by direct prompting thats obviously not enough either"
X Link 2024-01-16T01:15Z [----] followers, [---] engagements
"Paraphrasing 09:00 New users always notice how DSPy forces you to contend with "what is it that I am trying to optimize for". Working with ppl this leads to much better systems. LMs are cool at demos but to build reliable systems you must define & iterate on your metrics Hey everyone I am BEYOND EXCITED to publish our interview with Omar Khattab (@lateinteraction) from @stanfordnlp 🔥 Omar delivers remarkably clear explanations of DSPy and ColBERT as well as views on the state of AI I hope you find this useful 🎙 https://t.co/6kmlcItT4v https://t.co/38jOP0pW8p Hey everyone I am BEYOND"
X Link 2024-01-16T01:42Z [----] followers, [----] engagements
"Obligatory ICLR tweet: DSPy accepted as Spotlight. [--]. This could be a notable start of a pattern at ML confsas our metareviewer says "refreshing to see this unconventional research style (developing programming models) at ICLR" [--]. Time for a crisp thread on DSPy & what's new"
X Link 2024-01-16T18:06Z [----] followers, 59.3K engagements
"We spent a long time thinking about the many LLM frameworks and how they differ. Article below with @heathercmiller's CMU Composable Systems Lab (@Lambda_freak Haoze He) is the best place to understand this emerging stack. Agents optimizers chains schemas prompts A new thing Ive been up to lately along with Peter Zhong Haoze He @lateinteraction @ChrisGPotts & @matei_zaharia A Guide to LLM Abstractions https://t.co/oJfCY1tfsU its one thing to call the OpenAI APIs from a webapp its entirely another to build crazy rich https://t.co/yuWhLbFSHR A new thing Ive been up to lately along with"
X Link 2024-01-16T23:06Z [----] followers, 27K engagements
"You should read our new guide to LLM abstractions a stack with [--] layers To randomly help this tweet reach ppl see how happy DSPy power users feel when they get state-of-the-art scores using DSPy optimizersso happy in fact they make fancy slack emojis. cc: @KarelDoostrlnck We spent a long time thinking about the many LLM frameworks and how they differ. Article below with @heathercmiller's CMU Composable Systems Lab (@Lambda_freak Haoze He) is the best place to understand this emerging stack. Agents optimizers chains schemas prompts https://t.co/KlwatjMH5E We spent a long time thinking"
X Link 2024-01-17T00:50Z [----] followers, [----] engagements
"@regevbs Good luck Let me know if you face issues. We'd love to merge something like this visualizer in the original ColBERT or maybe in RAGatouille by @bclavie if there's interest :D"
X Link 2024-01-17T18:07Z [----] followers, [--] engagements
"This will be simple but effective on tasks where we need detailed feedback from the metric. Consider this quick-n-dirty metric for generating "correct" "faithful" "engaging" tweets under [---] characters and without hashtags. Seeing these & their values can improve optimization"
X Link 2024-01-19T19:03Z [----] followers, [----] engagements
"@stalkermustang Haha. I think were talking about different things Igor. I meant: if anyones first PR is to create a (draft incomplete messy) list of first PRs for others then Id happily merge and tweak it. But that may be too far. Thats what I meant by recursive here"
X Link 2024-01-20T13:03Z [----] followers, [--] engagements
"ChatGPT is going bananas at large scale I'm not even kidding this is really what it responded with https://t.co/7J7yIcmpF7 I'm not even kidding this is really what it responded with https://t.co/7J7yIcmpF7"
X Link 2024-01-20T16:37Z [----] followers, [----] engagements
"Generic agents will eventually work but not soon. Raw LMs and Agents are both text in - text out which is way too opaque for iterative development. LMs self-distract. You need to program them (maintain control flow in code) & deal with exceptions by actually debugging. 1/2 Are LLM agents just a gimmick Is anyone using them at scale Are LLM agents just a gimmick Is anyone using them at scale"
X Link 2024-01-20T17:30Z [----] followers, 34.3K engagements
"@ukituki No it doesn't. Structured output just means your output will parse that's a very very low bar. In [----] in the Demonstrate-Search-Predict paper (which is different from the [----] DSPy paper) we wrote about self-distraction. This example is parsed correctly but content is wrong"
X Link 2024-01-20T17:47Z [----] followers, [---] engagements
"@ecardenas300 @weaviate_io Thank you @ecardenas300 @CShorten30 and @weaviate_io team"
X Link 2024-01-21T17:15Z [----] followers, [--] engagements
"I really enjoy telling people: hey we just released a new optimizer. Just replace that line and re-compile. Your DSPy program will likely score much higher. Much nicer than: there's a new string that you should insert in your 6-prompt pipeline & keep tweaking until it works. Quite a few happy academic users for new DSPy optimizers this weekend. Fewer painstaking prompts higher scores Turns out if you want to help grad students you need to reach them directly. Grad students: DSPy is a research library la PyTorch (not HF). Its for you as well. Quite a few happy academic users for new DSPy"
X Link 2024-01-22T15:43Z [----] followers, 12.3K engagements
"It's even nicer IMO that optimizers can update weights not just write better instructions and few-shot examples. It reminds me of my first exposure to Adam. I replaced SGD with Adam and suddenly my tiny DNN for a deep learning course was actually competitive"
X Link 2024-01-22T15:43Z [----] followers, [---] engagements
"Many DSPy users are not sure what the optimizers do given a pipeline that calls the LM a few times to solve a task. So I added this to the mini FAQ on the README. It's quick-n-dirty and pretty general so feedback welcome. Also new optimizers very much welcome"
X Link 2024-01-22T18:56Z [----] followers, 15K engagements
"Prompt optimization solves the cold start problem"
X Link 2024-01-24T04:07Z [----] followers, [----] engagements
"I tried five extremely basic questions it got each one worse than the one before. Before I discuss more am I using the right Bard 🔥Breaking News from Arena Google's Bard has just made a stunning leap surpassing GPT-4 to the SECOND SPOT on the leaderboard Big congrats to @Google for the remarkable achievement The race is heating up like never before Super excited to see what's next for Bard + Gemini https://t.co/QPtsqZdJhC 🔥Breaking News from Arena Google's Bard has just made a stunning leap surpassing GPT-4 to the SECOND SPOT on the leaderboard Big congrats to @Google for the remarkable"
X Link 2024-01-26T23:26Z [----] followers, 70.2K engagements
"@simonw @marktenenholtz Lots of amazing pointers by @marktenenholtz and @imaurer. The paper's main figure or just this (really solid IMO) thread can also be useful: Progress on dense retrievers is saturating. The best retrievers in [----] will apply new forms of late interaction i.e. scalable attention-like scoring for multi-vector embeddings. A🧵on late interaction how it works efficiently and why/where it's been shown to improve quality https://t.co/2XG33TtM9R Progress on dense retrievers is saturating. The best retrievers in [----] will apply new forms of late interaction i.e. scalable"
X Link 2024-01-28T01:56Z [----] followers, [---] engagements
"@akatzzzzz @mathemagic1an Why dont we help you add it Which models do you need it for"
X Link 2024-01-28T22:01Z [----] followers, [--] engagements
"@swyx @CShorten30 @jxnlco @gordic_aleksa Theres an inline notation for prompts and theres a verbose version where you can write instructions. You start with inline notation to get the right program (control flow) and then compile. You only do more where iterative development tells you its strictly needed"
X Link 2024-01-30T21:20Z [----] followers, [---] engagements
"I'm glad that a lot more people understand the key ideas behind ColBERT and DSPy now. My only remaining goal is to make sure people can also say them correctly; both are quite tricky😆 * Col-BAIR (it's "the late" interaction retriever get it) * Dee-Ess-Pie (like num-pie)"
X Link 2024-01-31T18:33Z [----] followers, 33.8K engagements
"@srush_nlp The issue with one vector isn't capacity it's learnability. When SGD tries to "fix" a vector in one way it breaks it in other ways. That's why tiny ColBERT vectors outperform massive single vector. And also why ColBERT's sum-max is way better than sum-sum with identical dims"
X Link 2024-01-31T21:23Z [----] followers, [----] engagements
"There's a recent surge of interest in quicker replication of ColBERTv2 training. To make this easier I uploaded the examples file (64-way) and initial checkpoint (colbert v1.9) to HF hub. Here's the full code to launch training after pip install colbert-ai. (links below)"
X Link 2024-02-02T14:41Z [----] followers, 18.6K engagements
"And more here should include open-ended evaluation. Its always inspiring to see the kinds of eval campaigns that TREC held in early 2000s. This is why having a solid background in IR (or information access broadly) is one of the biggest assets in working with modern LM systems"
X Link 2024-02-03T05:57Z [----] followers, [----] engagements
"@markopolojarvi This is a surprisingly common take and I dont get it. I suspect its just bad tooling (do you use conda) or you work with long tail unpopular packages a lot. In my world pip and conda install almost always just work. The only exception is when cuda is involved. Thats it"
X Link 2024-02-03T06:43Z [----] followers, 31.4K engagements
"@markopolojarvi I mean you and I both know thats not true. Whatever packages are like in Ruby Java or for that matter C and C++ isnt much better than Python at all. And unless PHP got magically better over the past [--] years (its been a while) its worse than all of these. Idk about JS"
X Link 2024-02-03T06:51Z [----] followers, 10.5K engagements
"@420_gunna Read IR papers and take IR courses I have some super simple material that are admittedly 2-3 years old but theyre still solid. Five videos starting with this one:"
X Link 2024-02-03T07:06Z [----] followers, [--] engagements
"There are so many things I want to focus on in DSPy that we badly need to outsource the "integrations layer". It's really a time sink to manage all these LM and vector store integrations. Who has a uniform layer to enough of these in a uniform interface that people like"
X Link 2024-02-03T18:46Z [----] followers, 40.9K engagements
"@HanchungLee @LiteLLM @withmartian @PortkeyAI LangChain is especially great at this but these constraints are the problem: Requirements: * I don't want to get a lot of things we didn't ask for. Just shallow clients for LM and retrieval models / vector stores. * For LMs it must support requesting n [--] completions. It's odd how this isn't in many clients. Requirements: * I don't want to get a lot of things we didn't ask for. Just shallow clients for LM and retrieval models / vector stores. * For LMs it must support requesting n [--] completions. It's odd how this isn't in many clients"
X Link 2024-02-03T20:20Z [----] followers, [---] engagements
"@AliArmani Hi Ali yea theres vespa @vespaengine. Or you can index your documents in colab and do the search locally or on a cheap CPU server. RAGatouille is the right library for this kind of usage for ColBERT"
X Link 2024-02-04T15:20Z [----] followers, [--] engagements
"@HamelHusain @wasimlorgat @thesephist This is a whole new paradigm but it needs a fundamental shift in how people view LMs. If you view them as systems or agents that you need to chat with or ask nicely rather than optimize it will be hard to do good ML / software engineering with them like LM programming"
X Link 2024-02-04T19:29Z [----] followers, [---] engagements
"Im seeing so many should I switch from framework X to DSPy Well weve written extensivelymainly CMU composable systems labs @Lambda_freak and @HaozeHe_Hectoron the emerging LM stack. Different frameworks tackle fundamentally different concerns. Im taking this from Jason Liu https://t.co/PwgaXL8Vef via @YouTube @jxnlco @ConnorShorten So please help me understand how much is DSPy doing what Instructor does and how much something different Im taking this from Jason Liu https://t.co/PwgaXL8Vef via @YouTube @jxnlco @ConnorShorten So please help me understand how much is DSPy doing what Instructor"
X Link 2024-02-04T22:59Z [----] followers, 33.6K engagements
"@SearchDataEng @Lambda_freak @HaozeHe_Hector Yeah we actually have no competition"
X Link 2024-02-04T23:52Z [----] followers, [---] engagements
"A nice strong result by @fearnworks from the DSPy discord server. Also: we just made this server (originally created by the community; thanks @dronathon) our official Discord for DSPy. Feel free to join. Link below"
X Link 2024-02-05T00:41Z [----] followers, [----] engagements
"We now have an official Discord server for DSPy. Feel free to join to ask questions or just discuss cool ideas with the DSPy community. There are so many powerful ideas right at the edges of DSPy and I might start tweeting more of them. This feels exactly the same as when in-context learning was discovered and yet very few people realized just how big of a paradigm shift that was poised to create. There are so many powerful ideas right at the edges of DSPy and I might start tweeting more of them. This feels exactly the same as when in-context learning was discovered and yet very few people"
X Link 2024-02-06T17:57Z [----] followers, 13.4K engagements
"@bclavie Congrats Ben 🙌 Extremely insightful blog post Ill be looking forward to future posts"
X Link 2024-02-06T18:06Z [----] followers, [---] engagements
"gpt-3.5 achieves [----] average Jaccard score with [--] shot and [----] after compilation Trying to combine DSPy Pydantic types and JSON Schemas https://t.co/l54jOQvvf2 Trying to combine DSPy Pydantic types and JSON Schemas https://t.co/l54jOQvvf2"
X Link 2024-02-06T23:57Z [----] followers, 20.9K engagements
"New major release in DSPy With a whole lot of new results on a bunch of tasks. Language Model Assertions work by @arnav_thebigman @ShangyinT @slimshetty_. New: DSPy will now teach your LM to follow complex constraints (assertions) and will handle retries on top of that Theres huge interest now in programming with LMs but its unclear how to actually enforce constraints like make sure the output is engaging & has no hallucination. Just ask the LM nicely We built DSPy LM Assertions so you have far better controlup to 164% gains https://t.co/23NAOsljGz Theres huge interest now in"
X Link 2024-02-07T21:09Z [----] followers, 58.6K engagements
"@deter3 @hwchase17 @KevinAFischer Honestly there's no need to say things like that @deter3. LangChain and DSPy are solving very different problems. Thanks to the way @hwchase17 does things LangChain has taught a vast number of developers about LMs/AI far more quickly than it seemed to be happening otherwise"
X Link 2024-02-07T21:49Z [----] followers, [---] engagements
"From the DSPy discord by @fearnworks. New major release in DSPy With a whole lot of new results on a bunch of tasks. Language Model Assertions work by @arnav_thebigman @ShangyinT @slimshetty_. New: DSPy will now teach your LM to follow complex constraints (assertions) and will handle retries on top of that New major release in DSPy With a whole lot of new results on a bunch of tasks. Language Model Assertions work by @arnav_thebigman @ShangyinT @slimshetty_. New: DSPy will now teach your LM to follow complex constraints (assertions) and will handle retries on top of that"
X Link 2024-02-08T01:02Z 13K followers, 12.9K engagements
"Well we finally have a backronym for DSPy. now stands for eclarative elf-improving Language rograms (in pthon). What's the impact of this Nothing. You still pip install dspy-ai and work. It's just nice to be able to answer that exceedingly common question"
X Link 2024-02-08T14:10Z [----] followers, 15.2K engagements
"This is an AMAZING tutorial video on using the latest DSPy optimizers and on building advanced DSPy metrics with AI feedback. @CShorten30 improved his RAG program over Weaviate FAQ by 28% from 2.71/4 to 3.46/4. I'll postpone another announcement to tomorrow; watch this Hello world DSPy I am SUPER excited to share a new video walking through the end-to-end of how to use DSPy to optimize the CIFAR-10 for LLM programs RAG with FAQs 🛠 This tutorial contains 4 major parts: (1) library installation settings and creating a dataset with https://t.co/5UxH0Ye8uJ Hello world DSPy I am SUPER"
X Link 2024-02-12T15:52Z [----] followers, 25.1K engagements
"Incomplete list of DSPy internals experts. Follow them all Optimization: @kristahopsalong @michaelryan207 @KarelDoostrlnck @JoshPurtell Modules & Assertions: @arnav_thebigman @ShangyinT @slimshetty_ Abstractions: @Lambda_freak @HaozeHe_Hector @krypticmouse @cyrusofeden"
X Link 2024-02-13T16:43Z [----] followers, 14.5K engagements
"Seeing discussions about using DSPy for synthetic data generation and for building agents. These are two big usecases but remember: the real power of DSPy for that is that you can optimize (1) the properties of the data that will be synthesized and (2) the agent behavior"
X Link 2024-02-15T07:38Z [----] followers, [----] engagements
"One of the most frequently asked questions this week: "I optimized a DSPy program. How can I re-use it in the future" compiled_program.save("path.json") loaded_program = YourModule() loaded_program.load("path.json") (it's in the tutorials but gotta make a cheatsheet)"
X Link 2024-02-15T08:08Z [----] followers, [----] engagements
"New release of [--] rich (and free to run) tutorials on DSPy Assertions for teaching complex properties to LMs Example outputs (h/t @arnav_thebigman). Top: Prompting GPT-3.5 to generate engaging tweets Bottom: Using dspy.Assert to suggest engaging & hallucination-free tweets Lots of interest in using DSPy Assertions since the release [--] weeks ago Here are [--] notebooks on how to use DSPy Assertions for complex tasks with multiple nuanced constraints like JSON formatting or context faithfulness and engagement levels - judged by LMs. https://t.co/YJcNZn8ILj Lots of interest in using DSPy Assertions"
X Link 2024-02-16T18:29Z [----] followers, 12.3K engagements
"@arnav_thebigman Read the thread by Arnav and check out the three Google Colab notebooks. They're are all pre-cached so you can run them for free in your browser or locally. Many folks on the DSPy discord have been exploring assertions hopefully these three cool tasks/examples make it easier"
X Link 2024-02-16T18:31Z [----] followers, [---] engagements
"This is a very cool release of a ColBERTv2-style model by @JinaAI_ The model has several advantages like natively working with long text and using a more modern base than BERT-base. I expect @JinaAI_ team has even more up their sleeves for data mixtures in future releases.👀 Introducing jina-colbert-v1-en. It takes late interactions & token-level embeddings of ColBERTv2 and has better zero-shot performance on many tasks (in and out-of-domain). Now on @huggingface under Apache [---] licence https://t.co/snVGgI753H Introducing jina-colbert-v1-en. It takes late interactions & token-level"
X Link 2024-02-16T18:55Z [----] followers, [----] engagements
"Now that DSPy is popular some nuance. DSPy is not a prompting framework. It proposes a new paradigm: a framework for expressing & optimizing Language Programs. Think neural networks but with LM "layers" i.e. abstract modules that learn prompts/finetunes given a metric"
X Link 2024-02-17T06:07Z [----] followers, 43.8K engagements
"I always thought these remarks are not very serious but since theyre common it could be instructive to comment. Tradeoffs. Context (concatenation) & Retrieval pose fundamentally different tradeoffs in recall precision and latency/cost. You cant choose between RAM and disk. With the context size of 10M tokens de we still need vector databases With the context size of 10M tokens de we still need vector databases"
X Link 2024-02-17T06:38Z [----] followers, 24.8K engagements
"@madmaxbr5 Very well said. Also: At some level late interaction is (very cheap) attention. Being able to scale up lower-fidelity attention is hence not new. Getting perfect recall over 10M tokens is not new Getting good general-purpose attn with 10M tokens with low cost would be new"
X Link 2024-02-17T07:53Z [----] followers, [---] engagements
"The problem isn't with what the function space can represent -- the problem is the learning process. It's SGD. From [----] to [----] I was actually quite convinced that RNNs could be trained to learn any program. After all they're Turing-complete (or at least some of them are) and they learn a highly compressed model of the input:output mapping they're trained on (rather than mere pointwise From [----] to [----] I was actually quite convinced that RNNs could be trained to learn any program. After all they're Turing-complete (or at least some of them are) and they learn a highly compressed model of"
X Link 2024-02-18T15:26Z [----] followers, 10.2K engagements
"@GregKamradt Probably nothing good though happy to be proven wrong. Most wont release their slack dumps. Those who would probably wont have managed to create labels around them. Laziness around good data is nearly ubiquitous. But can be a new project over a public discord/gh issues crawl"
X Link 2024-02-21T23:33Z [----] followers, [---] engagements
"Just saw this interview at The Register with @rickbattlephoto on his recent paper on "The Unreasonable Effectiveness of Eccentric Automatic Prompts". Highlights his & Teja Gollapudi's experience w DSPy at VMware. It has some curious observations on LM behavior. Star Treck👀"
X Link 2024-02-22T16:09Z [----] followers, 20.4K engagements
"Automatic DSPy optimizers by @kristahopsalong and @michaelryan207 were used by this VMware research team and are now in the news (The Register) before they're even released in a preprint. Follow Michael and Krista for lots of cool stuff soon Just saw this interview at The Register with @rickbattlephoto on his recent paper on "The Unreasonable Effectiveness of Eccentric Automatic Prompts". Highlights his & Teja Gollapudi's experience w DSPy at VMware. It has some curious observations on LM behavior. Star Treck👀 https://t.co/t1FBP9vRAn Just saw this interview at The Register with"
X Link 2024-02-22T16:31Z [----] followers, [----] engagements
"@prabhatweet @ArizePhoenix Hmm we did merge a big update to dspy today I dont see how it affects this part of the code but maybe it does. Maybe you should update your notebook run for now to pip install dspy-ai==2.2.0 and it will work while we look into this (also cc @thomasahle fyi)"
X Link 2024-02-26T06:37Z [----] followers, [--] engagements
"Looks like both @krypticmouse and @ndzfs have been independently developing synthetic dataset generators for DSPy optimizers. So today we got two PRs for this with very different approaches Community feedback welcome; maybe we can merge the two into one robust version"
X Link 2024-03-02T18:34Z [----] followers, 23.9K engagements
"ICYMI from Friday. Our best DSPy prompt optimizer yet for complex language programs (though more coming soon) With great animated descriptions of how it works step by step and a free (cached) google Colab example: Got a pipeline with multiple prompts like a DSPy program What's the right way to jointly optimize these prompts Introducing MIPRO a Multi-prompt Instruction Proposal Optimizer. We integrated MIPRO into DSPy. It can deliver +11% gains over existing DSPy optimizers 🧵👇 https://t.co/hFfqSsELyT Got a pipeline with multiple prompts like a DSPy program What's the right way to"
X Link 2024-03-11T16:32Z [----] followers, 10.1K engagements
"AGI is nonsense but I like working on API Artificial Programmable Intelligence"
X Link 2024-03-20T01:58Z [----] followers, 17.1K engagements
"Oh wow this paper from VMwares @rickbattlephoto comparing manual prompts against prompts produced by DSPy is now highlighted in The Register IEEE Spectrum and Business Insider. On The Unreasonable Effectiveness of Eccentric Automatic Prompts https://www.businessinsider.com/prompt-engineer-ai-careers-tech-fad-2024-3amp https://www.businessinsider.com/prompt-engineer-ai-careers-tech-fad-2024-3amp"
X Link 2024-03-24T04:53Z [----] followers, [----] engagements
""Who validates LM-based metrics" is a good question. Luckily it's not recursive. Your language program may generate say free-form text (hard to evaluate). But your metrics/assertions are just classifiers which makes them much easier to optimize given a few labels. Getting reliable outputs from LLMs is tricky. Use LLMs to validate LLM outputs they say but who validates the validators My collaborators & I are interested in feedback on our ideas to solve this. If you have a spare hour in the next few days please DM/email me 🙏 thx Getting reliable outputs from LLMs is tricky. Use LLMs to"
X Link 2024-03-25T16:09Z 16.9K followers, 13.3K engagements
"Seems like the very Erik Meijer himself left Meta to. build a new DSPy Well we welcome some competition when he's done Been strange being the only programming model that compiles high-level code to optimized LM parameters i.e. instructions examples and weight updates. Folks I left Meta yesterday. This time there were no layoffs or anything thrilling involved. In fact I am more bullish than ever about Meta with the companys increased focus on AI. But given the incredible competitive pressure in the field there is really no advantage to be Folks I left Meta yesterday. This time there were no"
X Link 2024-03-26T22:52Z [----] followers, 37.8K engagements
"Any baseball fans around An example in the HoVer dataset asks systems to verify this claim: The MVP of a game that Red Flaherty umpired was elected to the Baseball Hall of Fame. I know nothing about baseball but this seems to be oddly phrased especially "game". 1/2"
X Link 2024-03-31T01:17Z [----] followers, [----] engagements
"I now make a joke about API (artificial programmable intelligence) every time someone asks me after a talk if DSPy is my path to AGI. Its funny because I absolutely mean it. AGI is nonsense but I like working on API Artificial Programmable Intelligence. AGI is nonsense but I like working on API Artificial Programmable Intelligence"
X Link 2024-03-31T20:22Z 10K followers, [----] engagements
"👀 Replits new code repair LLM uses DSPy for a few-shot prompt pipeline to synthesize diffs Pretty cool https://t.co/dSC2wS6pZ1 Replits new code repair LLM uses DSPy for a few-shot prompt pipeline to synthesize diffs Pretty cool https://t.co/dSC2wS6pZ1"
X Link 2024-04-03T16:15Z 11K followers, 17.4K engagements
"@ChiefScientist @databricks Thank you Alexy"
X Link 2024-04-03T19:38Z 10.1K followers, [---] engagements
"Due to interesting circumstances a 52-hour Amtrak may be unavoidable. One can enjoy this especially in a little roomette right"
X Link 2024-04-07T18:31Z 10.3K followers, 12.9K engagements
"@Lambda_freak thank you Peter I have VRZ phew"
X Link 2024-04-07T19:50Z 10.3K followers, [---] engagements
"I worry about a bubble burst once people realize that no AGI is nearno reliably generalist LLMs or agents. Might seem less ambitious but it's far wiser to recognize: LLMs mainly create opportunities for making general progress for building AIs that solve specific tasks"
X Link 2024-04-12T17:55Z 11.2K followers, 73.8K engagements
"@BrandoHablando Yes we support inference with TGI VLLM and SGlang. I think the first and third are a bit better tested imo than VLLM with DSPy. One nice thing about VLLM is that some people managed to use it with dspy.OpenAI because VLLM supports that API. Tagged some people on GitHub"
X Link 2024-04-14T23:32Z 10.6K followers, [---] engagements
"@letsrebelagain @jeremyphoward @enginoid @HamelHusain I dont know where [--] minutes came from tbh. I remember OpenAI was having trouble with timeouts around the day Hamel posted this. A few hundred API calls with DSPys parallelism and gpt-3.5s latency should finish in a few minutesat least if you have a decent rate limit"
X Link 2024-04-15T03:41Z 10.6K followers, [---] engagements
"@HamelHusain @letsrebelagain @jeremyphoward @enginoid I agree. I think this file doesnt normally take that long tbh. We saw github issues in mid Feb about openAI timeouts. Your run was probably just stuck Anyway the sequence weve generally recommended is on the README. The first and all runnables are free and take 2-3 mins"
X Link 2024-04-15T03:58Z 10.6K followers, [---] engagements
"True but not because people cant afford full SGD. Its because better more modular learning paradigms are emerging. Balancing fine-tuning & coarse-tuning is becoming essential. Systems now must efficiently learn high-level behavioral changes and compositions of skills. My easy prediction: We are moving to a macro-learning level where SGD will be seen as some technology for GPU-rich heavy players and most of the model design will be model merging + LoRa-like + prompt. A bit like engineering assembles pieces made by aluminum foundries. My easy prediction: We are moving to a macro-learning"
X Link 2024-04-26T13:57Z 11.1K followers, [----] engagements
"There goes my plan to teach the first course entirely on Programming with LMs Joking aside this is an exciting course topic from @davidschlangen at the University of Potsdam. And apparently then @ChrisGPotts's 224U is no longer the only NLP course that heavily features DSPy. Another class I'm teaching this semester is "Programming w/ LLMs". This sidesteps the whole chatbot / assistant / "an AI" theme and looks at LLMs as function approximators -- where weirdly the function needs to be "found" first. (Yes DSPy will feature heavily.) https://t.co/yqObFXVg5l Another class I'm teaching this"
X Link 2024-04-28T18:14Z 11.1K followers, 25.4K engagements
"Sufficient practice can often approximate system [--] goal-oriented planning as system [--] reflexive responses like when one internalizes how to drive. But for any degree of internalized system [--] capabilities reliable composition of maximal skills needs explicit system [--] structure"
X Link 2024-04-30T03:46Z 13.2K followers, 10.8K engagements
"@n0riskn0r3ward @dottxtai Masking out some tokens will not affect the ordering of other tokens. Merging token probabilities within equivalence class (like case insensitive decoding for example) can affect the relative ordering indeed though a challenge is that equivalence classes are often not syntactic"
X Link 2024-05-01T15:18Z 16.6K followers, [--] engagements
"There's often many "earliest" papers that have discovered or proposed something. We should think of it as a Pareto frontier: how early it was vs. how precise the proposal was at matching the eventual consensus. You'll find that this works out. Either way T5 was super early. Undoubtedly it is Raffel et al (2020) the T5 paper which came earlier. You can see the beginnings of supervised instruction/prompting there (see red highlights) and it was the first ever paper to show you can use formulate any task as a text to text task and train multiple https://t.co/zvbOnwjM3T Undoubtedly it is Raffel"
X Link 2024-05-01T15:28Z 19.3K followers, 19.1K engagements
"@deliprao 100% it was probably too early in fact. That's always a problem"
X Link 2024-05-01T15:32Z 11.2K followers, [---] engagements
"@deliprao Both in the sense that it doesn't work well enough yet and people don't grok it well enough yet"
X Link 2024-05-01T15:32Z 11.2K followers, [---] engagements
"@bclavie @huggingface lol the way they plot that increase makes the graph look like it was a zero before but it had been consistently [------] for several months"
X Link 2024-05-03T13:20Z 11.2K followers, [---] engagements
"Think of DNNs but replace: * Learnable layers - learnable LM calls. * Dimensions - Natural language types e.g. queries answers reasoning chains * Passing tensors - passing these text objects * CNNs/RNNs/etc - CoT/ReAct/etc * Differentiable loss - arbitrary metric"
X Link 2024-05-08T16:14Z 11.4K followers, [----] engagements
"The Bitter Lesson is just an artifact of lacking good high-level ML abstractions. Scaling deep learning helps predictably. But after every paradigm shift the best systems include modular specializations that basically look the same every time and should have been reuseable"
X Link 2024-05-11T21:28Z 11.4K followers, 18.8K engagements
"Crudely ML has [--] components: learning (where SGD shines) and composition (algorithms expressing problem structure). The latter won't go anywhere for almost any problem at a sufficiently high level of complexity. It's programming. We need learning-within-algorithms abstractions"
X Link 2024-05-11T21:48Z 11.4K followers, [----] engagements
"@ludgerpaehler Let's say SLURM But also let's say you get to choose/inform this. Say it's a small enough lab that maybe you can have "schedule over slack""
X Link 2024-05-15T22:41Z 11.5K followers, [---] engagements
"API achieved internally"
X Link 2024-05-16T02:59Z 11.5K followers, 15.6K engagements
"Folks ask how this will affect DSPy & ColBERT. Wellextremely positively Both grew dramatically in [----] while I was busy w faculty interviews & a broken (but now almost recovered) left hand. Looking forward you should expect a lot more research and OSS in both directions I'm excited to share that I will be joining MIT EECS as an assistant professor in Fall [----] I'll be recruiting PhD students from the December [----] application pool. Indicate interest if you'd like to work with me on NLP IR or ML Systems Stay tuned for more about my new lab. I'm excited to share that I will be joining MIT"
X Link 2024-06-03T17:18Z 12.9K followers, 29K engagements
"Despite all the theorems a surprising fraction of grad students believe in free lunch"
X Link 2024-06-08T00:40Z 13K followers, 28.2K engagements
"What are people doing with ColBERTv2 Apparently it receives 6-7 million downloads per month which is top-25 among all [------] models on HF"
X Link 2024-06-10T16:48Z 13.2K followers, 32.7K engagements
"I'm joining @hanlintang's 12:30pm keynote tomorrow at the @Data_AI_Summit to talk about DSPy. See you all there Sign up for this technical keynote with @hanlintang our CTO of Neural Networks and @lateinteraction Creator of DSPy. Theyll discuss the latest innovations that help enterprises build & deploy GenAI applications. Tomorrow at 12:30pm save your seat https://t.co/DwWzAYsbHK https://t.co/YVOnXmr5oB Sign up for this technical keynote with @hanlintang our CTO of Neural Networks and @lateinteraction Creator of DSPy. Theyll discuss the latest innovations that help enterprises build & deploy"
X Link 2024-06-12T04:01Z 13.3K followers, 12.1K engagements
"Super cool blogpost from @SnowflakeDB's Alejandro Herrera on using DSPy for higher-quality cost-effective RAG. "We demonstrate that by algorithmically tuning a @MistralAI Mixtral 8x7B pipeline with DSPy and @SnowflakeDB Cortex we can outperform a larger Llama370B pipeline while delivering 5X cost savings." https://t.co/2gX2US232Y "We demonstrate that by algorithmically tuning a @MistralAI Mixtral 8x7B pipeline with DSPy and @SnowflakeDB Cortex we can outperform a larger Llama370B pipeline while delivering 5X cost savings." https://t.co/2gX2US232Y"
X Link 2024-06-14T16:45Z 13.2K followers, 22.9K engagements
"three of the largest optimizer updates completed and on their way into DSPy"
X Link 2024-06-16T19:09Z 13.3K followers, 34.7K engagements
"🚨Announcing the largest study focused on how to optimize the prompts within LM programs a key DSPy challenge. Should we use LMs to Craft instructions Self-generate examples Handle credit assignment Specify a Bayesian model By @kristahopsalong* @michaelryan207* &team🧵"
X Link 2024-06-18T15:54Z 13.7K followers, 152.4K engagements
"one day I will figure out why RL people (and everyone else) use RL for the problem the solution space the specific implementations the area and some underlying philosophy"
X Link 2024-06-20T00:30Z 13.4K followers, [----] engagements
"@tom_doerr Would merge"
X Link 2024-06-20T18:48Z 13.3K followers, [---] engagements
"@OwainEvans_UK Awesome Thank you"
X Link 2024-06-22T17:25Z 13.3K followers, [---] engagements
"@OfirPress Thats a key approach for sure My understanding is that GPT-3.5 (davinci-02 onwards) has always been finetuned at a large scale on such trajectories since at least [----] if the WebGPT work was actually applied at all inside OpenAI. https://arxiv.org/abs/2112.09332 https://arxiv.org/abs/2112.09332"
X Link 2024-07-08T14:01Z 13.7K followers, [----] engagements
"Nice stuff. Posting with permission"
X Link 2024-07-09T17:57Z 13.7K followers, 13K engagements
"@dosco We wish we had your contributions in DSPy but I know you're building other awesome impactful things :D"
X Link 2024-07-09T18:56Z 13.7K followers, [--] engagements
"Exactly. What's surprising even in hindsight is that the difference between working extremely well (81.3% for ColPali) and not working at all (58.8% for BiPali) is the "Col" part of ColPali i.e. ColBERT late interaction. VLMs & vector representations alone don't do the trick It's fascinating how a small 3B model like ColPALI can disrupt the PDF extraction industry overnight It's fascinating how a small 3B model like ColPALI can disrupt the PDF extraction industry overnight"
X Link 2024-07-16T09:17Z 13.9K followers, 65.9K engagements
Limited data mode. Full metrics available with subscription: lunarcrush.com/pricing