Marie Al Ghossein’s Post

📣 This is really important research for anyone using RAG, and more generally retrieval for LLM in-context learning. Most people today look to the MTEB (Massive Text Embedding Benchmark) to decide which embeddings to use for their RAG retrieval engine. The problem is that MTEB evaluates embeddings based on pure semantic relevance (basically text similarity), instead of looking to see whether the retrieved documents actually improve the LLM's performance on the downstream task at hand. The ICLERB paper and 🏆 leaderboard released today by the Crossing Minds research team proposes a better way to evaluate embeddings and reranker models when they are used to retrieve documents for in-context learning, which gives a much better sense of how they will perform on real-world tasks, instead of a theoretical text similarity. And the 🍒 on top is we demonstrate a new algorithm called RLRAIF (Reinforcement Learning-to-Rank from AI Feedback) which *beats* the top scoring embedding model on the MTEB leaderboard, at a 20x smaller model size! Check out the 📖 here: https://lnkd.in/d_r7FXZy And the 🏆 leaderboard: https://lnkd.in/dNDNsTbw

🎉✨ Happy to share some fantastic news! Our paper, "CIRCOD: Co-saliency Inspired Referring Camouflaged Object Discovery," has been accepted in the Algorithms Track at 𝐖𝐀𝐂𝐕'25 (𝐂𝐎𝐑𝐄 𝐀)! 🎉🎈 It’s the first publication of my second PhD student, Avi G. (jointly with Prof. Tammam Tillo)—congratulations, Avi and Prof. Tammam! CIRCOD introduces a novel setting, Referring Camouflaged Object Discovery (RCOD), designed to enhance visual object search in camouflaged scenarios. This work draws inspiration from a common experience we all share: searching for something specific but unable to find it because it’s camouflaged. Very proud of Avi for taking up this challenging problem and creating something impactful! #Research #ComputerVision #Algorithms #WACV #ProudAdvisor #Innovation #CamouflagedObjectDetection

My new paper is now available in arxiv! Letters of Reference and Job Market Outcomes using LLMs in this paper I developed a new method to extract sentiment from recommendation letters using prompt-based learning and a large language model. https://lnkd.in/dwb3BpdD

I would like to share the paper accepted at ICML 2024 entitled 'SignSGD with Federated Defense: Harnessing Adversarial Attacks through Gradient Sign Decoding.' Our distributed learning algorithm is 1) communication-efficient, 2) robust to adversarial attacks, and 3) guarantees convergence rates. Traditionally, there is a belief that the convergence rate in distributed learning decreases proportionally with the number of adversarial workers. However, our results show that the convergence rate can remain unchanged under some adversarial attacks, provided that we can sagaciously harness the gradient information received from adversarial workers. This counter-intuitive result is achievable by our method called SignSGD-FD. The idea of our algorithm is inspired by a coding-theoretical interpretation of SignSGD with a majority vote. I believe that this work demonstrates how communication theory can be essential in redesigning algorithms for distributed learning. See the details through Arxiv version at https://lnkd.in/gvE7DAsu and the attached presentation slides presented at ERC workshop.

Interesting to know what would be other arxiv papers in other disciplines.

Runtime Monitoring of Human-centric Requirements in Machine Learning Components: A Model-driven Engineering Approach 🔍 Presenting insights from Hira Naveed's #PhD research on enhancing trust in #machinelearning (ML) systems. As ML integration continues, the significance of ethical considerations like fairness, privacy, and transparency grows. Her research focuses on runtime monitoring, proposing a novel approach to ensure ML systems meet diverse human-centric requirements. Leveraging model-driven engineering, we aim to comprehensively monitor ML components, enhancing trust and adaptability in dynamic environments. Accepted at 2023 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C). IEEE, 2023 Read the full paper: https://lnkd.in/gXXj9ibR Monash Information Technology

We are pleased to share that Thomas Kolb was accepted to the Doctoral Symposium at ACM RecSys 2024. Thomas presented his ongoing research on cross-domain recommender systems, particularly focusing on large language models (LLMs) and fairness aspects in recommendations. Congratulations, Thomas, on this important achievement and for contributing valuable insights to the field! #RecSys2024 #DoctoralSymposium #RecommenderSystems #LLMs #FairnessInAI #CrossDomainRecommendations #ACM #Research

In our work on the HTSR Theory and the new SETOL approach [a SemiEmpirical Theory of (Deep) Learning], we have discovered that the generalization performance of DNNs are governed by a kind of Universality and a related Conservation Principle. Being fundamental principles, they are independent of the training data and the model architecture. They apply to simple Multi-Layer Perceptions, to more complex Convolutional Nets, and, even better to the very large Transformer models & LLMs. I don't think I'll make it to any more conferences for a while, so I'll just drop this link from our workshop at NeurIPS2023 where I presented some of these ideas for the first time. https://lnkd.in/gPZZX8ck It's an exciting time to do science.

I am pleased to announce the publication of our latest blog post on arXiv, titled "Benchmarking Advanced Text Anonymisation Methods: A Comparative Study on Novel and Traditional Approaches" (arXiv:2404.14465v1). In this comprehensive study, we compare the performance of transformer-based models and Large Language Models (LLM) against traditional architectures for text anonymisation using the CoNLL-2003 dataset. Our findings highlight the strengths and weaknesses of each approach, providing valuable insights for researchers in selecting the most suitable model for their anonymisation needs. Read the full article here: https://bit.ly/3Wbfx3G.

📣 New journal paper accepted! 📚 Our work "Learning Sub-Second Routing Optimization in Computer Networks requires Packet-Level Dynamics" has just been accepted for publication in Transactions on Machine Learning Research (TMLR). Adapting routing in computer networks to new situations quickly is an essential but challenging task. Deep Reinforcement Learning (RL) can bring us closer to general-purpose dynamic routing optimization algorithms that shine in any situation. To this end, our work contributes the following: - Experiments with existing RL algorithms show that the choice of training environment matters: Our new packet-level simulation framework "PackeRL" provides arbitrary yet realistic network conditions including UDP and TCP traffic simulations that, in contrast to more abstract simulation dynamics, bring out the best in your RL-based routing optimization algorithm. - We introduce two powerful new RL algorithms that provide competitive routing performance much more quickly than existing approaches. Within just a few milliseconds, our algorithms adapt to changing traffic conditions while outperforming static routing approaches in busy network conditions. For more information, check out the project webpage: https://lnkd.in/eJPqzKbG Joint work with Niklas Freymuth, Patrick Jahnke, Dr.-Ing., Holger Karl and Gerhard Neumann. #TMLR #ReinforcementLearning #RoutingOptimization #ns3 #GraphNeuralNetwork