Jake Van Clief’s Post

Big Picture Science Radio Show: Fuhgeddaboudit (ENCORE) A thousand years ago, most people didn’t own a single book. The only way to access knowledge was to consult their memory. But technology – from paper to hard drives – has permitted us to free our brains from remembering countless facts. Alphabetization and the simple filing cabinet have helped to systematize and save information we might need someday. But now that we can Google just about any subject, have we lost the ability to memorize information? Does this make our brains better or worse? Listen here: https://buff.ly/3YNfnjX

Low Rank Adaptation is a fundamental tool in being able to fine tune LLMs and serve them at scale. This article describes the theory, and the practice. https://buff.ly/3wzlRrt Author: https://buff.ly/3JVGHEn

 One of the primary takeaways from recent prompt optimization papers is the fact that LLMs are good at writing prompts. Assuming we provide the right information as context, we can create surprisingly powerful prompt optimization algorithms by just iteratively prompting an LLM to critique and improve a prompt. Larger (and more capable) LLMs tend to be better at this task - cool post Cameron R. Wolfe, Ph.D. - https://lnkd.in/dUVKU5N9

I'm a particular fan of this InnerDrive & Bradley Busch article: Specific Retrieval Questions. I often find that retrieval is just lumped together as one big thing... 'recalling prior learning', but there's more to it. Are you using specific or open questions? If push came to shove, I think specific questions are both more important, and also easily to use more effectively. But this isn't to say that open retrieval questions are not also crucial. Anyway, check out the article because it does walk you through the pros and cons, and literature, around specific retrieval questions. https://lnkd.in/gPjsYSDU

What is information and can information be negative? Read my latest take on the nature of information and its relation to multivariate contexts. Through illustrative interactive examples, I discuss what information is, and how to interpret negative information.

If you'd like to improve the effectiveness, coverage, and adaptability of HyDE (hypothetical document embeddings) for advanced RAG applications, Ian Ho presents his novel approach: AutoHyDE.

Speculative Sampling is the principle of using two language models simultaneously to produce output faster at a reduced cost. https://buff.ly/3WAGKNq Author: https://buff.ly/3JVGHEn

Here's the second article with my son about the computational algorithm of Relative Resolution https://lnkd.in/gu9ENJFa

The recent focus on inference scaling has brought attention to its potential and limitations. A recent paper exploring the boundaries of one specific type of inference scaling offers insightful perspectives on the subject. Paper-https://lnkd.in/gMEpUn8F

Complex adaptive systems are all around us from financial markets to ecosystems to the human immune system and even civilization itself. They consist of many agents that are acting and reacting to each other's behaviour, and out of this often chaotic set of interactions emerges global patterns of organization in a dynamic world of constant change and evolution where nothing is fixed. It's a fainting subject that I spent many years learning about. If you are interested in this here's the guidebook at Si for you to learn more about the area of complex adaptive systems: https://lnkd.in/eP9_SGTR and video course: https://lnkd.in/ekVFSECY Graphic inspired by this image https://t.ly/85rGb