Ramy Shahin

Aligning Large Language Models with Diverse User Preferences Using Multifaceted System Messages: The JANUS Approach Quick read: https://lnkd.in/gQAsT95M Paper: https://lnkd.in/gkBAuWiC

19

Discover DocETL—an open-source, low-code AI system developed by UC Berkeley researchers! This cutting-edge tool utilizes LLMs to tackle unstructured data processing with ease and efficiency. 🚀📊 #AI #dataprocessing #lowcode

8

Attention Mechanisms for Efficient and Effective Transformer Language Models Traditional attention mechanisms can be inefficient for long sequences. This post covers 4 attention techniques that improve efficiency while preserving performance. 1. Multi-Query Attention (Falcon,PaLM,StarCoder): - Addresses slow inference speed of transformers by sharing keys and values across attention heads, reducing memory usage and speeding up inference. - Ideal for applications requiring fast text generation like chatbots or real-time summarization, especially for large language models processing long sequences. - Trades off a slight reduction in generated text quality compared to multi-head attention for faster inference speed. 🔗https://lnkd.in/dUzRknj9 2. Grouped-Query Attention: - Compromise between MQA and standard multi-head attention, balancing speed and accuracy. - Better performance than multi-head attention while maintaining higher accuracy than MQA. - Suitable for applications requiring fast text generation with good quality. - If speed is the top priority and slight quality dip is acceptable, MQA might be preferable. If highest accuracy is essential, multi-head attention is still the best option. 🔗https://lnkd.in/dKhEzD_b 3. Sliding-Window Attention(Longformer, Reformer, Swin Transformer) - Focuses on a local window around the current token instead of the entire sequence, reducing computational cost while capturing relevant context. - Ideal for tasks involving long sequences like document summarization or question answering over long passages, especially for large language models with memory and computational limitations. - If the task requires considering the entire sequence for accurate results, Sliding-Window Attention might not be suitable. - Optimal window size needs to be carefully chosen depending on the specific task and dataset. 🔗https://lnkd.in/dAZn_gjn 4. Page Attention(Big Bird, Performer) - Partitions the sequence into "pages" and allows the model to focus on relevant pages and attend to tokens within those pages, reducing computational complexity while maintaining the ability to consider long-range dependencies. - Beneficial for processing extremely long sequences, where even sliding windows might not be sufficient, such as code generation or historical data analysis. - Choosing the optimal number of pages and the granularity of page division requires careful consideration. 🔗https://lnkd.in/dgRkCmE3 #Transformer #LLM##FastTextGeneration#EfficientInference #TransformerAcceleration #MemoryEfficientNLP#Computational Linguistics

4

Visualizing Progress in Broadening Participation in Computing: The Value of Context - Excellent article by Barr et al discussing new approaches like Cohort-based analysis, intersectionality, and entropy-based measures are needed to provide better insights into challenges and successes of BPC activities. https://bit.ly/3RQbxCY

4

In this paper, the authors focus on synchronous transmissions and propose a #multicast #triggered #synchronous #transmission #protocol, abbreviated as #MURIST, which enables contention based and coordinated data transmissions among distributed devices in order to reduce transmissions latency and energy consumption. Furthermore, they develop a novel analytical model based on an absorbing #Markov #chain to evaluate the performance of MURIST in a network cluster. Unlike existing models that are merely targeted at the behavior of a single device, the novelty of their model resides in a generic framework to assess the behavior of a cluster of devices for synchronous data transmissions. Based on the analytical model, they obtain closed-form expressions for the distributions of successful and discarded transmissions, number of collisions, and delay, as well as for energy consumption. ---- Debasish Ghose, Luis Tello-Oquendo, Vicent Pla, Frank Y. Li More details can be found at this link: https://lnkd.in/ef9R3hBt

3

Featuring one of our May 2024 papers published in the IEEE Transactions on Terahertz Science and Technology by Mingxiang (Stephen) Li, Mariam Abdullah, Jiayuan Estrid He, Ke Wang, Christophe Fumeaux, and Withawat Withayachumnankul. This paper proposes a novel frequency diverse antenna and a companion machine learning-assisted analysis of received signal spectra to determine the direction of arrival for terahertz communications.  Read more here: https://lnkd.in/egNMwMnY

73

🔬 Interesting insights from Microsoft Azure! We're thrilled to share Dr. Krysta M. Svore's recent post on defining logical qubits and criteria for resilient quantum computation. Understanding the path towards scaled quantum computing is crucial for the industry's progress and the development of a robust strategy to build a quantum-ready community. In June 2023, Microsoft Azure outlined the three implementation levels necessary for achieving utility-scale quantum computing: Level 1 Foundational, Level 2 Resilient, and Level 3 Scale. Currently, all quantum computing technologies are at Level 1, and while NISQ machines have been developed, they do not offer practical quantum advantage. 🚀 The next critical step towards Level 3 Scale is demonstrating resilient quantum computation on a logical qubit. Resilience, in this context, means showing that quantum error correction enhances non-trivial quantum computation. However, demonstrating resilience with just one logical qubit is not enough. It's essential to showcase the interaction between logical qubits and the entanglement they generate. Therefore, demonstrating two logical qubits performing an error-corrected computation that outperforms the same computation on physical qubits will mark a significant milestone in the field's history. 🌌 This advancement highlights the ongoing efforts to push the boundaries of quantum computing, paving the way for practical quantum advantage. Join us in the journey towards achieving scaled quantum computing! #Microsoft #MiddleMarket #Simulator #Supercomputer #MicrosoftAzure #QuantumComputing #QuantaStandard

3

🌟 The Quantum Algorithm Design Automation Workshop is Back! 🌟 Quantum Algorithm Design Automation Workshop is returning, featuring an outstanding lineup of invited talks and presentations of accepted papers. This event is a must-attend for anyone involved in the cutting-edge field of quantum computing. 🗓 Date: Sunday, September 15th, 2024 📍 Location: IEEE Quantum Week, International Conference on Quantum Computing and Engineering, Montréal, Canada ⏰ Time: 10:00 - 16:30 EDT This workshop is part of the prestigious IEEE Quantum Week, offering a unique platform to explore the latest advancements in quantum algorithm design. Whether you're a seasoned quantum researcher or new to the field, this event provides invaluable insights and networking opportunities. 🚀 Don’t Miss Out! Check it out here: https://lnkd.in/eCVinPxB Mark your calendars and get ready for a day filled with innovative ideas and groundbreaking research. Check out the full agenda and details here: Quantum Algorithm Design Automation Workshop. 🔔 Stay Connected: Follow Quantum University for more updates on the latest in quantum computing. And if you find this post helpful, please like, share, and repost to spread the word! Let's grow the quantum community together! 💡 #QuantumComputing #IEEEQuantumWeek #QuantumEngineering #QuantumUniversity #QuantumAlgorithms

1

📣Highly-Cited Papers #Review #recommendation Recent Advances of Smart Systems and Internet of Things (IoT) for Aquaponics Automation: A Comprehensive Overview by Mohamed Farag Taha, Gamal ElMasry, Mostafa Gouda, Lei Zhou, Ning Liang, Alwaseela Abdalla, David Rousseau, and Zhengjun Qiu. 👉https://lnkd.in/g6_BYiXN MDPI, Zhejiang University, Arish University, Suez Canal University, Nanjing Forestry University, Université d'Angers #aquaponics; #IoT #smart #deep #learning; #convolutionalneuralnetworks #intelligent #sensors #chemical #sensing #chemosensors This article belongs to the Special Issue Practical Applications of Spectral Sensing in Food and Agriculture: https://lnkd.in/gQkuXabw

5

This paper considers a remote inference system, where a neural network is used to infer a #time-#varying #target (e.g., robot movement), based on features (e.g., video clips) that are progressively received from a #sensing #node (e.g., a camera). Each feature is a temporal sequence of sensory data. The inference error is determined by (i) the timeliness and (ii) the sequence length of the feature, where the authors use the #Age #of #Information (#AoI) as a metric for timeliness. While a longer feature can typically provide better inference performance, it often requires more channel resources for sending the feature. To minimize the time-averaged inference error, they study a learning and communication #co-#design problem that jointly optimizes feature-length selection and transmission scheduling. ----Md Kamran Chowdhury Shisher, Bo Ji, I-Hong Hou, Yin Sun More details can be found at this link: https://lnkd.in/gauENaPM

3

🤖 Learn how to build a state-of-the-art chatbot application optimized for mobile processors, using tools like the OpenVINO™ Toolkit and PyTorch with Paula Ramos, PhD during her session at #AllThingsOpen happening today at 1:45pm.

42

Applied Affective Computing https://bit.ly/3naKZQw provides State-of-the-art and emerging themes in affective computing, including a comprehensive review of the existing approaches to affective computing systems and social signal processing. Authors: Leimin Tian, Sharon Oviatt, Michal Muszynski, Brent Chamberlain, & Jennifer Healey. #AI #AffectiveComputing #MachineLearning #SocialRobots #BuiltEnvironments #Multimodal #DataCollection #ReinforcementLearning #Synthesizing #Natural and #Believable #EmotionalExpressions ACM, Association for Computing Machinery

1

Kinetix: An Open-Ended Universe of Physics-based Tasks for Reinforcement Learning Self-supervised learning on offline datasets has permitted large models to reach remarkable capabilities both in text and image domains. Still, analogous generalizations for agents acting sequentially in decision-making problems are difficult to attain. The environments of classical Reinforcement Learning (RL) are mostly narrow and homogeneous and, consequently, hard to generalize. Read the full article: https://lnkd.in/e49pwBzf Paper: https://lnkd.in/ex4FgnSf

6

This paper propose a decomposed #cross-#correlation based #single-#input-#single-#output (#CC-#SISO) behavioral modeling architecture. The proposed solution utilizes a low-cost, novel cross-correlation based method to estimate and cancel the simultaneous nonlinear and reverse crosstalk from multiple power amplifier (PA) branches. Once the crosstalk is mitigated, the MIMO #digital #predistortion (#DPD) can be implemented with single-input DPD blocks which significantly reduces the complexity. Furthermore, CC-SISO DPD eliminates the requirement for signal feedback paths before and after the PA, and thus reduces overall hardware implementation complexity. ---- Thakshanth Uthayakumar, Abubaker (B.K.) Abdelhafiz, Xianbin Wang, Ming Jian More details can be found at this link: https://lnkd.in/eiD3wwHi

2