Liam Brannigan

We asked Dvir Ben Or and Andrey Lisovoy what's new and valuable has happened in Data science recently: #xLSTM: Extended Long Short-Term Memory. It aims to overcome some limitations of traditional LSTMs by introducing two main modifications: 👉Exponential gating: Using exponential activation functions for the input and forget gates instead of sigmoid functions. This allows the gates to revise storage decisions more easily. 👉New memory structures: #sLSTM: A scalar memory cell with a new memory mixing technique across cells/heads. #mLSTM: A matrix memory cell that is fully parallelizable with a covariance update rule. These modifications are integrated into residual blocks to create xLSTM blocks, which can then be stacked to form xLSTM architectures. The results demonstrate that xLSTM models outperform other large language models like RWKV-4, Llama, and Mamba on various benchmarks, including next token prediction, common sense reasoning tasks, and fine-grained domain benchmarks across different model sizes. Notably, xLSTM maintains low perplexities even for very long context lengths, excelling at sequence extrapolation. 🐾Our Comment For mobile game development history, this means that xLSTM architectures could potentially lead to more efficient and powerful language models for tasks like game dialogue generation, quest text generation, or any other text-based content creation. The linear computation complexity could make xLSTMs more suitable for on-device deployment in mobile games compared to computational expensive Transformers. #tech #innovation #mobile #game #development

2

"...embeddings enable machine learning models to find similar objects." Read more of Gregory Hodum's article on embeddings https://lnkd.in/eFs6hhAP #ai #softwaredevelopment #embeddings #machinelearning

9

【Script-Based GPU-Ready ELM Development for Continuous Code Integration】 Full article: https://lnkd.in/gFM7gFdQ (Authored by Peter Schwartz, et al., from Oak Ridge National Laboratory, Environmental Sciences Division (USA), etc.) The E3SM Land Model (#ELM) is an integral part of the Energy Exascale Earth System Model (E3SM) framework, simulating the interactions between terrestrial land surfaces and other Earth system components. ELM has been instrumental in understanding hydrologic cycles, biogeophysics, and dynamics of terrestrial ecosystems. To facilitate the automatic porting of individual ELM modules, the researchers have developed a #software tool named SPEL. This work presents a SPEL extension that leverages the version control system (e.g., Git) to autonomous code generation and demonstrate its application to #continuous_code_integration and development of the ELM software system.  

2

Great Tables is now BYODF (Bring Your Own DataFrame). The Great Tables team have supported both Polars & Pandas DataFrames for a while, and have finally removed our hard dependency on Pandas. This means that you can still use either DataFrame, but don't have to install the other in order to use Great Tables. Removing dependencies is a little thing that goes a long ways for helping users in restricted computing environments, or who want a lightweight install. In this post, Michael Chow goes into detail on: * 👯‍♀️ The things great_tables currently depend on. * 🏗️ The challenge of removing hard dependencies. * 🫠 The approach chosen for making Pandas optional. https://lnkd.in/gsuB7dhj

88

We've been busy creating a video series aimed at beginners to open source research tools. From tips on how to access tools without coding knowledge, to the basics of the command line, our tech team are sharing their knowledge. Find the full playlist here: https://lnkd.in/eqKV8vyZ #techseries #tech #starter #developers #opensource #coding #jupyter #notebooks #learntocode #techadvice #research #tools #resource #youtube

154

5 Comments

There are always trade-offs to key data architecture decisions Do you go for the "tried and true" or "cutting edge tech"? In our latest tech blog, Roi Talpaz sheds more light on our choice of database architecture, including: - Why we liked a GraphDB approach - The questions & issues we found on the way - Experimenting with a change - The switch to Relational Database Management System (RDBMS) Enjoy and share with friends building cloud data & analytics platforms! #Database #GraphDB #RDBMS #Analytics

3

We are delighted to announce a round of updates for Shiny for R! 🎉 The Shiny team has released updates for *9* different R packages, including {shiny} and {bslib}. • Non-blocking operations: The biggest new feature in this release is the ability to run truly non-blocking operations in Shiny via ExtendedTask. {promises} can be used to prevent operations in one user session from blocking operations in another user’s session, helping scale your app to multiple concurrent users supported by a single R process. • As a companion to ExtendedTask, we’ve created bslib::input_task_button(), a special button that displays visual feedback when a task is in progress. • Another exciting new feature is Shiny’s JavaScript error dialog, which makes critical client-side errors more discoverable. • bslib received many improvements and fixes in this release. Most of these improvements are focused on sidebar()s, cards()s, layout_columns()s, and the default bs_theme(). There's so much, we can't fit them into a single post. Read more on the Shiny blog: https://lnkd.in/gxwGdQie

322

4 Comments

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

23

Expand your explainable-AI toolkit by following along Conor O'Sullivan's latest tutorial, where you'll learn how to apply the H-stat with the artemis package and interpret the pairwise, overall, and unnormalized metrics.

25

The new Cyclic partitioning method not only optimizes value assignments but can potentially accelerate algorithms like QuickSort and median-finding by up to 1.5 times when dealing with complex data types. Read more in Tigran Hayrapetyan's latest article. #ComputerScience

6

How often does an opportunity come to work on a greenfield platform like Pydantic Logfire? Build a database that will scale (if we get it right) to petabyte scale and change how develoeprs understand their applications.

115

3 Comments

When *open* isn't *open source* 🙋🏼‍♂️👀 Reflecting on the open-source landscape for Large Language Models, I explore the essentials: permissive licensing, transparency in training data and model architecture & more #LLM #LLMs #AI #GenerativeAI #opensource https://lnkd.in/gSJuK_J4

1

An underrated component of RAG evals is retrieval evals - is potentially more important than LLM evals! Some requirements for performing retrieval evals is 1) identifying the right retrieval metrics, and 2) having a unified dataset representation. This article by Ross Ashman takes you through tour of retrieval/ranking metrics (e.g. precision@K, NDCG, and more), and also shows you how to convert a dataset into the BEIR format in order to run these evals. Once retrieval evals is setup, run it over your LlamaIndex retrievers. https://lnkd.in/gfha8kiX

173

3 Comments

Stratagem 28 - Use Joblib for (easy!) multiprocessing ➡️ Python has built-in multiprocessing module to run code across multiple cores but a lot of boilerplate code needs to be written ✅ With Joblib, it's "embarrassingly" easy to run code in parallel; plus joblib makes caching of function results easy. More about it in Stratagem 18 (https://lnkd.in/gHBP62hc) 📝 Check out the notebook here - https://lnkd.in/gw7gmyBs ✨ Stratagems Collection - https://lnkd.in/gYEgpQfg #StratagemGIS

4