Josep Curto

Meta has just released their long-awaited #Llama #3 models. Everyone's talking about it, let me give it a try 😅! Some observations: ☑ #Llama #8B and #70B are available, and the #400B is currently in training ⁉ Yes... well, who cares, right? We see new models every morning? Well, not really, these models are a bit different. Not in terms of innovation – practically the same as #Llama #2 – but rather in the meticulous work done on the quality and quantity of data. ✅ For the first time, we have a model of 8B (and 70B as well) trained with more than 15 trillion tokens! For comparison, #Llama #1 used 1T, and #Llama #2 used 2T. That's 8 times more data. To give you an idea, 15T tokens =~ 69 terabytes. Training an #8B model with 15T far exceeds the norms; Chinchilla Scaling suggests 20 tokens per parameter, which would require 160B tokens for optimal training of a model this size. Here we are talking about 15T, which is 95 larger, that's just astronomical. Is it beneficial? it seems to be the case, at most, it won't hurt. ☑ 5% of the texts are not in English, which is not ideal, but still acceptable because LLMs generalize beyond the training language. ☑ More code in pretraining, as it is proven to improve the reasoning capabilities of LLMs. ☑ What about compute? Training the 8B model required 1.3 million GPU hours (I'm not sure if this includes failures and restarts, or if it's purely 1.3M of training), and the 70B was trained on 6.4 million hours. In total, 7.7 million hours, so at a rate of $2 per hour in a H100 GPU, that amounts to $15 million, not counting the additional costs (engineers, infrastructure, etc.). ☑ Why are these models cool? Because #Llama 1 was the main driver of the proliferation of open-source LLMs (open-weight to be precise). After #Llama 1 (7-65B) and 2 (7-70B), we've seen an explosion of open-source models and research that wouldn't have been possible without these models. ☑ What's next? The next step is #Llama 400B, which is already closing the gap with #GPT4 😲. We will then have all the quantizations, fine-tunings, etc., and we will achieve performances almost close to #GPT4 with smaller models. I thought we'd see this in a year! This will surely motivate OpenAI to release #GPT5 :D. ☑ We will definitely see #Llama with larger context windows than 8k tokens, because 8k is not satisfying. And you? Have you tested these models? Eager to hear your comments on their performance. #LLMs #Llama3 #Llama #Meta

13

2 Komen

Malaysia published its national AI guidelines. How do they stack up against those of other nations, such as the United Arab Emirates' "AI Ethics Principles and Guidelines"? Although the Malaysian and UAE AI guidelines share common principles, they differ in their scope, regulatory approaches, stakeholder engagement, sector focus, and implementation strategies. Key Differences 1. Regulatory Focus: The UAE guidelines are more regulatory in nature, incorporating tools like an AI Seal and AI Procurement Guidelines. In contrast, Malaysia offers a comprehensive framework for responsible AI practices without specific regulatory measures. 2. Stakeholder Involvement: The UAE emphasizes a collaborative, multistakeholder approach, while Malaysia focuses on government agencies playing a leading role in driving AI ethics. 3. Sector-Specific Focus: The UAE guidelines address sector-specific needs, while Malaysia provides a more general framework applicable across various sectors. 4. Implementation: The UAE includes clear provisions for monitoring and implementation, such as establishing an AI Office. Malaysia’s guidelines lean toward voluntary adoption with a focus on continuous updates to align with evolving technology and ethics. Key Similarities 1. Ethical Principles: Both countries emphasize fairness, accountability, transparency, and privacy in AI systems. 2. Human-Centered AI: Both guidelines advocate for AI that enhances human capabilities and respects societal values. 3. Sustainability: Sustainability is a priority in both, with a focus on environmentally friendly AI practices. 4. Inclusiveness: Both stress the importance of ensuring AI systems benefit all segments of society. 5. Privacy and Security: Data protection and ensuring privacy are key concerns in both sets of guidelines. In short, while both Malaysia and the UAE promote responsible AI development, the UAE guidelines are more regulatory and sector-specific, while Malaysia emphasizes a broader, government-led framework.

148

20 Komen

This week on the ShareTIGR lab blog (https://lnkd.in/edm5GTad): Why #metadata is important for FAIR data sharing and reuse? Metadata is an important topic in data management. In this post, Nina Profazi reflects on the relevance of descriptive, technical and administrative metadata for interactional linguistics. Later, we will discuss the specific choices to be made at this regard for our conversational data on the LaRS repository. #openscience #openresearchdata #opendata #spokenlanguage #corpuslinguistics #interactionallinguistics

7

I'm thrilled to share a bonus episode of the Data Today #podcast on #AIAppreciationDay. This one is particularly special to me: Data and Artificial Neural Networks with Damian Borth.    Neural networks are the backbone of many #AI and machine learning systems. But how can they be applied to real-world scenarios?    I had the incredible opportunity to sit down with Damian Borth, Professor of Artificial Intelligence and Machine Learning at the University of St.Gallen and director of its Computer Science Institute. Damian's ground-breaking work on neural networks spans several crucial areas, including climate change, financial fraud detection, and text-to-speech. It's no wonder he’s a recipient of the prestigious Google Research Scholar Award!    In this episode, we dive into:  What #neuralnetworks are and what they do How their application can help tackle global challenges Insights from Damian's vast experience and groundbreaking research   I found our conversation incredibly enlightening and inspiring, and I can't wait for you all to hear it. Tune in to discover how #ArtificialIntelligence and neural networks could help save the world! #TechForGood https://lnkd.in/ePTYn3AC #MachineLearning #DataScience #Innovation #Research #Data #Technology

44

🚀 Just shared a game-changing tutorial on advanced data analysis with AI using Google Gemini & Colab, all without coding! Check it out and be amazed! 📊🤖 Watch now: https://lnkd.in/dU3fh8yb #DataAnalysis #AI #NoCoding #GoogleGemini #GoogleColab #Python #DataScience

4

Join us: Data Literacy 101: Unlocking the Power of Data University Libraries Present: Today’s data-driven society necessitates data literacy skills for everyone. This workshop led by SBU Libraries’ Data Literacies Lead, Ahmad R. Pratama, will help you learn the foundational concepts of data literacy, the types of data, and the importance of reliable data sources. 9/26 3:30-4:30pm Zoom. Register via link. https://lnkd.in/eBdiGsQC

2

How is Generative AI significant for Product Development ? Generative AI in product development is a game-changer, but it comes with its own set of challenges. Let's explore! Key use Cases : 1️⃣ Generate Entire Products - Input a good detailed prompt, Gen AI does its magic, and generates a fully designed and coded feature product . https://lnkd.in/gguZmmpx 2️⃣ Co-Pilot Development - While development cannot be done in entirety without human intervention yet (Devin presented a stunning case but we know how the claims were exaggerated) , assisted development is on the rise. GitHub’s Co-Pilot is reportedly being actively used by 1.5ml developers across 37,000 orgs, with high levels of satisfaction and retention. Amazing feat in such a short span. 3️⃣ Power Your Product - Tech company area adding Gen Ai capabilities to their products and services, mostly using off the shelf models like GPT after finetuning. Digital Marketing , presales and Customer Service operations are already witnessing good adoption . https://lnkd.in/gK2dY6kA The Risks Involved: 1️⃣ Missing Context: Beware of generic solutions- one key reason for this is improperly formulated prompts. Prompt is the new code. 2️⃣ Hallucinations: AI's tendency to produce nonsense - There are several techniques that allow a user to remove or reduce hallucinations. This is an evolving topic and proper education is necessary 3️⃣ Costs and Risks: Trial and error can be costly - Compute and storage requirements require investment, Third Party Gen AI APIs are expensive. In such scenarios, iteration can be very expensive. Imagine having an AI co-pilot for your product development journey, guiding you through data analysis, idea generation, and even coding. But beware, like any powerful tool, Generative AI has its pitfalls. It might lead to a culture of copy-pasting solutions without critical thinking, ultimately churning out junk features at lightning speed. What are your thoughts on Generative AI's role in product development? Share your insights in the comments below and let's navigate this technological frontier together! 💬🔍 Remember, with great AI comes great responsibility! #AI #Simplilearn #ProductDevelopment #Innovation #TechTrends #FutureOfWork

17

As simple as they are, confusion matrices are an essential tool for building and training a classification model with imbalanced datasets. #datascience #AI #artificialintelligence #opensource https://hubs.li/Q02WQFY70 If you enjoyed this post, follow me for more insights on AI and data science!

8

🔊Featured Article, ComTech Journal 🔊 Comparison of the Symmetric and Asymmetric Generalized Autoregressive Conditional Heteroscedasticity (GARCH) Models in Forecasting the 2018-2023 Jakarta Composite Index Abstract The Autoregressive Integrated Moving Average with Exogenous Variables (ARIMAX) method assumes a homogeneous residual variance, but data with high volatility can cause violations of this assumption. Hence, it is interesting to compare the forecasting accuracy of symmetric and asymmetric Autoregressive Conditional Heteroskedasticity (ARCH) models in various data conditions. The research aimed to compare the accuracy of the symmetric ARCH/ Generalized Autoregressive Conditional Heteroscedasticity (GARCH) and asymmetric TGARCH models in forecasting weekly Jakarta Composite Index (JCI) data on January 1st, 2018, to April 24th, 2023, by involving the influence of COVID-19 as a covariate variable and applying several validation scenario models to training and testing data. Based on the best-selected model, forecasting was carried out from May 1st, 2023, to July 3rd, 2023. The data used were weekly JCI opening data from January 1st, 2018, to April 24th, 2023, with the COVID-19 period as a covariate variable. The analysis results show that symmetric and asymmetric methods can handle violations of the heteroscedasticity assumption in the ARIMAX model. The best model produced based on four data validation scenarios is the asymmetric ARIMAX(3,1,3)-TGARCH(1,2) model with an average MAPE value of 3.158%. In this model, the COVID-19 variable significantly influences the JCI movement. Forecasting is done with forecasting results that are stable with confidence intervals that widen in each period. Read full article: https://lnkd.in/g4W5YHPR

1

*AI and ontology for automating IM Compliance as-a-Service* I am absolutely stoked to see the Archives NZ AI Auto-classification for Information Management Proof of Concept, which demostrated the power of AI tools to automate Public Records Act, and Privacy Act compliance, and to identify and surface information of interest to Māori. The full report is now available. This was a concept I had been thinking about since 2017, but the tooling just wasn't easily available back then. So, we ran a project to assess how AI could automate Information Classification in SharePoint/M365, and Shared Drives hosted in the Cloud. I pitched the concept to Amazon Web Services (AWS) and Microsoft in 2020, and much to my surprise , immediately saw the potential for AI in the public sector years before #ChatGPT brought this technology to broad attention. We successfully navigated the highly competitve Dragon's Den pitch through the Westpac NZ Government Innovation Fund in 2021, to put the seed funding in place to run the project, which kicked off in early 2022. it was amazingly successful (but possibly a year, or two ahead of it's time). Anyway, I am super proud of this work, and in particular the awesome team at Archives New Zealand Te Rua Mahara o te Kāwanatanga who made the dream a reality. We also had the benefit of the All-of-Governmnet Ontology, which was a critical intellectual underpinning for the approach (https://lnkd.in/g6bp_wfp) I addition to our technology partners, it wouldn't have been possible without our Public Sector partners Ministry of Justice - New Zealand and Ministry for Primary Industries (MPI) The report also includes the AWS and Microsoft technical reports, on the tools we used and how the AI performed: AWS report on Document Understanding Proof of Concept — Archives New Zealand (PDF 1.2MB) Outcome Document — Proof of Concept with Microsoft Azure Datalake, Purview and Text Analytics for records disposal (PDF 1.6MB) https://lnkd.in/gPMGgUJx Special Thanks to: The governance group: Angela Beaumont (Director of digital delivery, (Archives), Joanna Condon (Archives), Joanne Koreman (Archives), Anahera Morehu (Archives); Antonia Aloe, Brian Nevin, Tony Eyles (DIA); Andre Post (MPI); and Tony Breen (MoJ). The project team: Shakil Akhter, Elisa Capecci, Joanne Koreman, Madeleine Pierpoint, Spencer Tasker (DIA); Tanya Hurley, Sara Knight (MPI); Vanessa Cuthbert-King (MoJ); Marzieh Cameron, Jan Hutar, Carly Lenz, and Joshua Ng (Archives). Technology Partners: Mark Gordon, Craig Hind, Matthew McCulloch (AWS); Nick Cater (Microsoft New Zealand); Bruce Anderson, Richard Malloch and JP Van Heerden (Insight). Information and technical specialists: Namita Parshotam (DIA); Annabel Snow (Deloitte); Team IM.  #Privacy #AI #ML #Cloud #AWS #Microsoft #Archives #InformationManagement #Autoclassification #AIforIM #Compliance #M365 #SharePoint #Government #ontology

41

4 Komen

Join us: Data Literacy 101: Unlocking the Power of Data University Libraries Present: Today’s data-driven society necessitates data literacy skills for everyone. This workshop led by SBU Libraries’ Data Literacies Lead, Ahmad R. Pratama, will help you learn the foundational concepts of data literacy, the types of data, and the importance of reliable data sources. 9/26 3:30-4:30pm Zoom. Register via link. https://lnkd.in/eKCAyrtg

2

🚀Responsible Retrieval Augmented Generation (ResRAG) 🌟 Happy to continue the series on Responsible Retrieval Augmented Generation (ResRAG) with #PwC AT AI Center of Excellence! 📘 In this part, I delve into the essential ResRAG Requirements: - Context Awareness - Domain Awareness - Document Structure Awareness - Facts Awareness - Time Awareness - Fine-Grained Access Rights - Robustness Against Adversarial Attacks - Privacy Preserving Feel free to share your thoughts or provide feedback on this content! Gerald Dipplinger Andreas Hladky Mark D. Orlic Marcel Tkacik Kristian Landegren #AI #ResponsibleRetrieval #PwC #Innovation #Tech #AIResearch

28

Thrilled to announce the launch of IBM watsonx.ai Text Extract ! Unlock content from your enterprise complex documents and make it amenable for RAG and other GenAI solutions 🚀 📚 Supports popular document formats .pdf (programmatic, scanned and hybrid), and images: .bmp, .gif, .jpg, .png, .tiff (single-page and multi-page) and exports to JSON and Markdown 📝 🌎 OCR for scanned documents and images with machine print in many languages and handwriting 🖋️ in English 📑 Advanced PDF document understanding, including reading order, page layout & table structures 🔍 Preserves positional information of extracted content, for downstream visual representation of the document with text highlighting 🔗 Available as REST API and Python SDK Try it out now 🚀 - Documentation: https://lnkd.in/gQmCMcZZ - Examples: https://lnkd.in/gf84v6ky #IBM #AI #GenAI

346

9 Komen

Exciting Announcement! We have released the latest version of the Open Data Initiatives by Local Governments on the Digital Agency’s website. Explore the updated materials to discover how local governments are harnessing open data to foster innovation and transparency. For more details, visit our website: Digital Agency Open Data Resources #OpenData #DigitalTransformation #LocalGovernments #Innovation #Transparency https://lnkd.in/gWVQFim4

4

OUR KNOWLEDGE, ON DEMAND. Our scientists frequently publish white papers and datasheets that dive deeper into our areas of expertise. This #datasheet on TNBS-induced #IBD is available to download at no cost >> https://lnkd.in/g-HSYuEr #PreclinicalModels #IrritialBowelSyndrome #PreclinicalResearch

10

Big fan of Andrew NG, one of those people who's opinion i have a level of confidence in. Do you he's right? https://lnkd.in/gqbPmP-u #AI #Artificialintelligence #technology #generativeai

4