Tim Erdmann’s Post

🚀 Day 2 of my 10-day prompt engineering challenge: Today's revelation - the dynamic pathways to interact with large language models (LLMs)! 🤝 Interacting with LLMs unfolds in two distinct ways: through user interfaces like ChatGPT and Gemini, or via APIs for seamless integration into applications. 🌟 APIs serve as the backbone, enabling applications to harness the power of AI seamlessly. 🛠️ Key to successful integration? Playgrounds! These versatile platforms offer customizable settings such as model selection, frequency penalties, and temperature adjustments. 🔍 Thrilled to explore further, leveraging prompt elements like instruction, context, input data, and output data in this fascinating journey! #PromptEngineering #AIIntegration #LLMs

In my previous post, I shared an overview of our graduation project—an AI-powered PDF application called ChatPDF 📃. Now, let’s zoom in on the first key component that’s set to transform how we interact with PDFs: Chat with your PDF: Imagine a world where you can converse with your PDF files 📁—where natural language seamlessly interacts with the static text on the page. With ChatPDF, this vision becomes a reality. Our feature enables dynamic, human-like interactions with PDF content, bridging the gap between data and dialogue. How It Works: At the core of the Chat with PDF feature, we combine LLM and RAG (Retrieval-Augmented Generation). In this post, I will focus exclusively on the fine-tuning process. For this step, Belal Al-Tabbaa and I fine-tuned two LLMs: LLama2-7B and Mistral-7B, on the SQuAD2.0 dataset to enhance their ability to answer questions within a contextual framework. Fine-Tuning Process: Leveraging the PEFT/LORA approach, we fine-tuned LLama2-7B and Mistral-7B. • PEFT (Parameter-efficient Fine-tuning) streamlines the fine-tuning of language models by selectively updating only a subset of parameters, balancing efficiency and accuracy. • LoRA (Low-rank Adaptation) further reduces model complexity by creating low-rank transformations of existing weights. Together, they enhance adaptability while managing computational demands. Additionally, we optimised memory usage through 4-bit quantisation, allowing efficient fine-tuning on a single GPU. After fine-tuning, we seamlessly merged the adapter with the base model. These models are available on HuggingFace for exploration. https://lnkd.in/dehC7y-y In an upcoming post, I’ll dive into the Retrieval-Augmented Generation (RAG) technique. Stay tuned for more insights! 🌟 If you’d like to learn more about my project, feel free to send me a message. I’m always open to connecting and discussing exciting ideas! 🚀 #LLM #FineTuning #HugginFace #LLama #Mistral #PEFT #LoRA #RAG

How can you boost productivity with ChatGPT and other LLMs? Abhishek Saxena, Nino Weingart and Claudio Giorgio 🤖 Giancaterino are running the workshop “Unlock the Power of LLMs: from Prompt Engineering to Agents” at the Swiss Conference on Data Science | SDS Find more information here: https://lnkd.in/gSJJymPi #LLMs #GenerativeAI #Agents #PromptEngineering #doneai

My major challenge today was how to create a sigmoid function and apply it on input_tensor. (correct code below) input_tensor = torch.tensor([[0.8]]) nn.Sigmoid() sigmoid = nn.Sigmoid() probability = sigmoid(input_tensor) print(probability) I got it wrong in my first attempt and then I discovered that DataCamp has now integrated ChatGPT into their platform which explains error messages a lot easier. I love the innovation. NB: I'm learning courtesy of Laureates Science Innovation and Education Initiative - Laureates SIE.

🚀 用 SciSpace x ChatGPT 革新您的研究！在 AI 智能輔助下進行高效且深入的文獻探討，適合追求深度和創新的研究者。#學術研究 #SciSpace #ChatGPT 🌐📚 🚀 Revolutionize your research with SciSpace x ChatGPT! Dive into efficient and insightful literature reviews with AI-powered guidance. Perfect for researchers seeking depth and innovation. #AcademicResearch #SciSpace #ChatGPT 🌐📚

🎉 I'm thrilled to announce that my paper, "Effectiveness of ChatGPT for Static Analysis: How Far Are We?" has been accepted for presentation at the AIware Conference '24 Conference, co-located with the 32nd ACM International Conference on the Foundations of Software Engineering (FSE '24), a top-tier (A*) ranked conference in Software Engineering. As the first author, I find this achievement incredibly special, and I'm immensely proud of our team's hard work. I want to express my heartfelt thanks to my amazing co-authors: Reem Aleithan, Nima Shiri Harzevili, Moshi Wei, Alvine Boaye Belle, Hung Viet Pham, and Song Wang. 🌟 I also want to express my deepest gratitude and a big shoutout to my supervisor and co-supervisor, Prof. Song Wang, and Prof. Alvine Boaye Belle. Their guidance, expertise, and unwavering support have been invaluable in overcoming challenges and advancing our research. 🔍 About our paper: Our study conducted a novel exploration of ChatGPT's capabilities in static analysis tasks, showcasing remarkable performance in static bug detection and false positive warning removal. This paper not only pushes the boundaries of AI in software engineering but also offers practical evaluations that can significantly improve the productivity and efficiency of static analyzers using large language models. We look forward to sharing more insights at the upcoming AIware 2024 conference in Porto de Galinhas, Brazil. #AI #softwareengineering #research #staticanalysis #AIware24 #FSE24

Chat-GPT is now Thinking Slow ⏳ Yesterday, I opened the premium version of Chat Gpt and noticed the new O1 model. It's a more advanced version, capable of solving very complex tasks, but it also takes longer to process your prompts (minutes, even!) In the new post of IE Research Datalab, Dae-Jin Lee explains the O1 model and why this "slow thinking" approach can lead to better results 🚀 https://lnkd.in/dZ2PU7rt IE University IE School of Science and Technology #ImpactXcelerator #IEResearchDatalab #ChatGPT #O1 #LLM

Happy to share that I've successfully completed the 'Geospatial Data Analysis with ChatGPT and Google Earth Engine' course from the United Nations University Institute for Water, Environment and Health (UNU-INWEH)🌐💻 Throughout this transformative journey, I delved into the world of geospatial data analysis, mastering the integration of cutting-edge technologies like ChatGPT and Google Earth Engine. 🛰️💬 This course has been a fantastic learning experience, equipping me with the skills needed to make informed decisions and drive positive change through geospatial data. 🌟📊 #GeospatialDataAnalysis #EarthEngine #ChatGPT #UNUINWEH #EnvironmentalScience #DataAnalysis #SustainableDevelopment 🌱🌎

Please, join me in congratulating Mohammad Mahdi Mohajer who has just got a full paper accepted by the AIware Conference 🎉 🏆 🥇 . The latter is co-located with FSE (32nd ACM International Conference on the Foundations of Software Engineering), a top-tier conference in Software Engineering. The title of that paper is: "Effectiveness of ChatGPT for Static Analysis: How Far Are We? ". That is truly impressive, right 🎇? Warm thanks to all the co-authors of that paper for their precious collaboration. Mohammad just completed his MSc degree in computer science at York University. Dr. Song Wang was his primary supervisor, and I was his co-supervisor. This is the abstract of his newly published paper: "This paper conducted a novel study to explore the capabilities of ChatGPT, a state-of-the-art LLM, in static analysis tasks such as static bug detection and false positive warning removal. In our evaluation, we focused on two types of typical and critical bugs targeted by static bug detection, i.e., Null Dereference and Resource Leak, as our subjects. We employ Infer, a well-established static analyzer, to aid the gathering of these two bug types from 10 open-source projects. Consequently, our experiment dataset contains 222 instances of Null Dereference bugs and 46 instances of Resource Leak bugs. Our study demonstrates that ChatGPT can achieve remarkable performance in the mentioned static analysis tasks, including bug detection and false-positive warning removal. In static bug detection, ChatGPT achieves accuracy and precision values of up to 68.37% and 63.76% for detecting Null Dereference bugs and 76.95% and 82.73% for detecting Resource Leak bugs, improving the precision of the current leading bug detector, Infer by 12.86% and 43.13% respectively. For removing false-positive warnings, ChatGPT can reach a precision of up to 93.88% for Null Dereference bugs and 63.33% for Resource Leak bugs, surpassing existing state-of-the-art false-positive warning removal tools."

I would like to share our latest work titled: “State of Practice: LLMs in Software Engineering and Software Architecture” In collaboration with Jasmin Jahić from University of Cambridge, we've explored how Large Language Models like ChatGPT can tackle complex Software Engineering tasks, focusing on Software Architecture. Our paper sheds light on the current use and potential benefits of LLMs in the field, based on empirical surveys from 15 software engineering companies. Plus, we dive into how LLMs, especially ChatGPT, fare in architectural design through 50 experiments across 5 software projects. Excited to discuss our findings at 3rd International Workshop on Software Architecture and Machine Learning. Edinburgh Napier University's SCEBE Edinburgh Napier University #LLM #AIinTech #SoftwareEngineering #SoftwareArchitecture #GenAI