[GUEST ACCESS MODE: Data is scrambled or limited to provide examples. Make requests using your API key to unlock full data. Check https://lunarcrush.ai/auth for authentication information.]

lateinteraction Avatar Omar Khattab @lateinteraction on x 22.6K followers Created: 2025-07-27 04:32:01 UTC

Yeah, meta-learning of sorts. BTW my answer was more about instruction following. I agree the question is much less non-trivial for example-based ICL.

For that, my intuition is that the weights are very good at aggregating all knowledge without forgetting fast. But ICL is much more overriding and loud.

In the sense that, perhaps due to overparameterization, with SGD you can teach the LLM X gazillion tricks, incrementally one at a time. But ICL is a tool where you can orient the LLM towards ~1 task at a time.

Just some vague intuitions.

XXX engagements

Engagements Line Chart

Related Topics llm singapore dollar

Post Link