Eshan Bhatt

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

It’s been a while since I have shared my experiences of working on AI projects using LLM’s that I have worked on in last 2 years. Hope these experiences will help folks. 😊 so here is the first one   Language Classification with GPT – We were working on a problem scenario where 2 languages to be classified in separate classes were very much similar in terms of words available in them and up to a good extent semantically also, but the languages were actually different. As the languages were domain specific, it was expected that we have to empower GPT. We used different models starting with GPT 3.5, GPT4 Turbo 0125, and then GPT 4 Omni and at last GPT-4 Turbo 2024-04-09. Of all the models we got the best accuracy with the last one. The last 2 are the latest ones available and as per Microsoft recommendation the go-to GPT models. However with GPT-4 Turbo 2024-04-09 also we didn’t got the desired result by the same pipeline we were using (words filter + vector search + RAG). While searching for the method to resolve the issue, I came across a research paper around ‘Feedback loop’ and how it can improve the results. (Sorry didn’t remember the paper name now , will try to find and post that).  We applied the same with some experimentations and it worked. So the focus here is – if you have a use cases where GPT/LLM’s is getting confused in language classification after application of latest techniques like vector search and RAG, feedback loop can be introduced after the final output as a last step. It can enhance the accuracy in very confusing but important cases. Hope this will help community. 😊 #NLP #LLM #GPT #RAG #DataScience #AI #OpenAI #Chatgpt OpenAI

40

2 Comments

One of the most common question asked in Data Engineering Interview: Difference between ETL, ELT, and Reverse ETL. 1️⃣ ETL: The Traditional Powerhouse Extract, Transform, Load (ETL) has been the go-to process for decades. It involves: 1. Extracting data from various sources 2. Transforming it to fit operational needs 3. Loading it into the target system (usually a data warehouse) 2️⃣ ELT: Flipping the Script Extract, Load, Transform (ELT) emerged with the rise of cloud computing and big data. The key difference: - Data is loaded into the target system before transformation - Leverages the power of modern data warehouses for transformation - Offers more flexibility and scalability 3️⃣ Reverse ETL: Closing the Loop A newer player in the field, Reverse ETL: - Moves processed data from warehouses back into operational systems - Enables data activation, turning insights into action - Bridges the gap between analytics and operations For career guidance in Data Analytics, job search strategies, resume review, interview preparation, etc., you can book a session with me here: https://topmate.io/itshb

53

We had this coming: MoA LLM -> Mixture of Experts (MoE) Agents -> #Mixture of Agents (MoA) Let's understand them in simple terms. #MoE: Within one large model, sub-models are created on specific parts of data to develop an expert on that part of data. E.g. One sub-model specializes in predicting health conditions for Native Hawaiians while another sub-model is specialized for Asians. MoE reduces training time, computational costs while maintaining (or improving) accuracies of #LLMs. Twitter / X's #Grok-1, Mistral AI's #Mixtral 8x7B (8 experts of 7 billion parameters each), Databricks's #DBRX are based on this architecture. Ensemble models (EM): Why am I suddenly jumping to EM? Because, in your discussions, this may come up as a concept similar to MoE at #LLM-level, just like EM is at traditional #ML-level (e.g. Gradient Boosting Random Forest). But there is one distinction - In EM, each model is trained on all the data while in MoE, each sub-model (expert) is trained on a subset of data. #MoA: While MoE combines multiple neutral networks (#experts), MoA combines outputs from multiple Agents (potentially accessing multiple LLMs internally). Advantage: More accurate response with improved quality. Disadvantage: Longer inference time; to the time order of 5x. Solution: Groq. It reduces inference time to as low as 1x! #ArtificialIntelligence #AI

20

Dear companies, take care of your AI talent, especially the researchers. Researchers? Really? Aren't researchers the least influential people in the company? - They are the cost-centers. They do not directly sell to customers & so, do not generate revenue. And company has to fund them from profits - They live far away from reality. They do not know the "ground-truth". - They are not enterprising or charismatic like the #marketing & #sales' dashing & street-smart folks. - They cannot converse in normal language but are deep inside their narrow, un-understandable world. - Even if they do not exist in the company, the current product portfolio can be enough to sustain sales for years, or even decades. Look at ROLEX. Their watches are as mechanical as they were decades ago! Clearly their R&D budgets for adopting consumer-side emerging technologies are near-zero. - They help achieve company's Reduction in Force (RiF) targets. This is probably the only reason a company & its other employees may want them - for own survival. As they get hit first. So, why should companies care about their AI researchers, if at all they have them? The world has changed. They are one of the most influential people in the tech world and its only likely to increase with time. Why? We do not need every researcher to come up with a newer neural architecture or a different LLM, VLM or LMM. Right? That's true. But their role has just expanded and we still do not know. They are some of the best equipped people to manage the speed of current AI changes, naturally. Look, the AI world has been so rapidly advancing in the past 2 years that leaders are finding it difficult to keep track of them. Even if they succeed to keep track of them, their AI developments & implementations are becoming obsolete within 6-9 months! Ask a leader, and you'll invariably hear this. So? How will AI researchers help? They know the next in-thing (or likely to be one) at least a year before the world. And so, they can guide the next #AI #development or implementation within an organization that can sustain itself for a little over an year and a half, at least. Even if they are not implemented, at least the leaders & organizations can plan their 3-year strategies (Short Range Planning) based on their inputs. Are there any examples? AI Agents. They have been in existence for over an year. But organizations are still implementing LLMs. In spite of #Devin going viral! So tell me, what's the next in-thing that companies should think of? While there are nuanced answers to this question, some of the prominent ones, in general, would be - Natural Conversations, Reinforcement Learning & #Humanoids. But not every future in-thing actuates in future. No? True. And hence, we even need a Discriminator & not just a Generator AI researcher ^⁠_⁠^ #GAN #Future #HLM

21

3 Comments

𝑻𝒊𝒑𝒔 𝒇𝒐𝒓 𝒏𝒆𝒘 𝑫𝒂𝒕𝒂 𝑺𝒄𝒊𝒆𝒏𝒄𝒆 𝑴𝒂𝒏𝒂𝒈𝒆𝒓𝒔 I became a manager in data science early in my career. It has been a journey fraught with failures and some successes. Here are some of my learnings along the way - 📈 𝑹𝒐𝒍𝒆, 𝒓𝒆𝒔𝒑𝒐𝒏𝒔𝒊𝒃𝒊𝒍𝒊𝒕𝒚 𝒂𝒏𝒅 𝒑𝒓𝒊𝒐𝒓𝒊𝒕𝒊𝒆𝒔 Early on, my vision was small which narrowed the value I was able to create. I had two focuses - 1) ship out high quality work, 2) coach & promote my team. After >10 years in leadership roles, here is how I NOW think of the responsibilities of a data science manager in the hierarchy of importance - 🔹 Build authentic relationships with stakeholders and team members 🔹 Identify and align on highest impact work the team can enable and unlock 🔹 Set up systems, processes, and people to execute on these projects 🔹 Manage projects and priorities 🔹 Coach your direct reports on execution Depending on the seniority of the role, the distribution across these may vary but the core buckets remain the same. 🏆 𝑩𝒆 𝒊𝒏 𝒕𝒉𝒆 𝑨𝒓𝒆𝒏𝒂 Don’t be carried away by the business of meetings that make you feel important but don’t create real value. Spend at least 1/3rd of your time towards hands-on work. Don’t get rusty on the technical chops. Be an SME in several areas in the workflow - tech stack/tools, data models, methodologies, algorithms and specific tables. This will build trust with your reports while reducing dependency, increasing team efficiency and productivity. 🔄 𝑻𝒂𝒊𝒍𝒐𝒓 𝑷𝒓𝒐𝒄𝒆𝒔𝒔𝒆𝒔 & 𝑷𝒓𝒐𝒅𝒖𝒄𝒕𝒔 Data science lacks standard processes, unlike software engineering, due to its evolving role, inherent variability in methods depending on project goals. Creating processes and systems that will help with the ground reality of the ecosystem and culture at your company, is important. 🎲 𝑼𝒏𝒅𝒆𝒓-𝑷𝒓𝒐𝒎𝒊𝒔𝒆 𝒂𝒏𝒅 𝑶𝒗𝒆𝒓-𝑫𝒆𝒍𝒊𝒗𝒆𝒓 In the unpredictable world of predictive modeling 😉, where factors such as data anomalies, algorithm biases, and unexpected market shifts can disrupt the accuracy of forecasts, it pays to temper expectations and try to exceed them. #datascience #analytics #businessanalytics #leadership #techleadership #careers #techmanagers

28

1 Comment

This is probably the best in-depth "musing" on building a Generative AI product I've ever read so far! Some notable learnings that I have picked up in this article: - There is trade-off of speed of development vs consistency in UX when parallelizing tasks across multiple teams to build different independent AI agents - Building the RAG pipeline (80%) was a breeze: they built dev sets and fitted them with prompt engineering and in-house models, but the generation bit (20%) was tough - In-house defensive YAML parser; re-prompting LLM to correct its YAML output mistakes adds non-trivial latency + GPU calls - "Poor man's fine-tuning" with Embedding-Based Retrieval (EBR) with in-memory DB to inject response examples into the prompts. Link in the comment

3

1 Comment

Many of ML practitioners might have heard the term "Extending context length" for LLMs. In case if you want to dive deeper into this topic along with its code implementation, give this blog a read. I have described about Position Interpolation, NTK aware and Dynamic NTK method, which are widely used for extending context length.

68

When it comes to "Retrieval" phase of RAG, choosing the right strategy makes all the difference. Sparse retrieval (like BM25/TF-IDF) is great for finding exact keyword matches, while Dense retrieval (like BERT) understands the meaning behind words for more accurate results. Hybrid retrieval combines both, offering the best of both worlds. Each method has its strengths. Understanding when to use them is key to better search results. 

10

4 Comments

Excited to share our interactive BlogApp on Generalized Linear Models (GLMs)! In collaboration with Venkatachalam Subramanian Periya Subbu, Satya Lasya N, and Pooja Baralu Umesh, we've developed an intuitive tool that makes understanding GLMs both engaging and accessible. If you are an aspiring Data Scientist, Actuary, Statistician, Economist, Epidemiologists or an Engineer, you definitely need to check it out at: https://lnkd.in/gMqf3M5Y View the source code: https://lnkd.in/g9HUm_7r Special thanks to Professor Cody Carroll for the guidance throughout this project. Let's discuss GLMs! What's your experience with these powerful statistical tools? Share your thoughts below. #DataScience #Statistics #MachineLearning #GLM #Analytics #Programming #USF #MSDS

34

2 Comments

I received two badges a few months back on LinkedIn for contributions to different articles; one was for Artificial Intelligence, and the other was for Data Engineering. Due to the policy and template of the LinkedIn profile, I could not display both on my profile. While the initial excitement of receiving these badges was gratifying, my motivation quickly waned, and my contribution rate steeply declined until I no longer cared. However, I still enjoyed reading the articles, as they offered perspectives from many people, often with 90% saying the same thing with slight variations. Nevertheless, reading was still far more enjoyable than trying to make a differentiated contribution that added any value. As I reflected on this experience, I realised the complexity of my motivations and the transient nature of such accolades. Here, I share some introspective questions that arose during this journey ❓𝗪𝗵𝘆 𝘄𝗮𝘀 𝗜 𝗰𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝘁𝗹𝘆 𝗺𝗮𝗸𝗶𝗻𝗴 𝗮𝗻 𝗲𝗳𝗳𝗼𝗿𝘁 𝘁𝗼 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗲 𝗯𝗲𝗳𝗼𝗿𝗲? - To make the best use of my time; writing and reading simultaneously and making a difference?  - Was I merely seeking the recognition of a 'top voice' badge on my profile? - To test myself to see if I could get one? ❓ 𝗪𝗵𝘆 𝗱𝗶𝗱 𝗺𝘆 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻 𝗱𝗲𝗰𝗹𝗶𝗻𝗲 𝗮𝗳𝘁𝗲𝗿 𝗿𝗲𝗰𝗲𝗶𝘃𝗶𝗻𝗴 𝘁𝗵𝗲 𝗯𝗮𝗱𝗴𝗲? - Did I face a reality check about the sustainability of staying in the top 5% of contributors? - It didn’t seem worth it when the same article already had thousands of people doing a seemingly good job? - I found it hard to find time to give the same advice on different articles repeatedly. ❓𝗪𝗼𝘂𝗹𝗱 𝗜 𝗱𝗼 𝗶𝘁 𝗮𝗴𝗮𝗶𝗻? - I'm not sure, as it hasn’t materially changed anything for me except that my profile had a 'thinking bulb' for a while. - However, this might change if there is a way for this badge to have a material impact. ❓ 𝗔𝗺 𝗜 𝘀𝘂𝗴𝗴𝗲𝘀𝘁𝗶𝗻𝗴 𝗻𝗼𝘁 𝘁𝗼 𝗱𝗼 𝗶𝘁? - Certainly not. I encourage those who are committed to learning and contributing consistently. I’ll continue to support and read their efforts. - In my personal opinion, I think I did it for mixed reasons but with good intentions. As long as you are gaining something from it, keep doing it. If you are still reading and curious about how to earn such badges, my advice is simple: remain consistent and authentic in your contributions. This approach not only enhances your chances of recognition but also enriches your professional journey. P.S. I have lost those badges due to my inconsistency, just underscoring what I said above about active engagement. #ProfessionalDevelopment #LinkedInTips #CareerAdvice #ContinuousLearning #Authenticity

18

Drama in AI history For the first time in the AI history, there is more drama from the humans who develop AI than the AI that these humans develop.   It started with #Geoffrey Hinton, the "Father of AI", when he announced his departure from Google because of the harm the latest #AI was likely to cause to #humanity. Satya Nadella was pushing boundaries of AI solutions & their availability, without the seeming regards for human #safety. And Sundar Pichai was following suit, which Geoff apparently did not like.   Ilya Sutskever, OpenAI cofounder & brain behind some of the high-impact breakthroughs in AI (not just OpenAI) of the past decade, also did not like the direction in which OpenAI was headed - not in terms of tech (because he was building it) but the business strategy in using this emerging tech. So, he ousted #Sam Altman from the company after taking its then Board into confidence. But Satya Nadella weighed in heavily, reinstated Sam with a new board & came out as one of the most influential #CEO of our times.   6 months later, Ilvy & #Jan Leike left the company. Ilvy left silently, but Jan shared his concerns of OpenAI not working on Safety & Security aspects of the future AI "horrors". Even after Sam / OpenAI publicly declared last July about dedicating 20% of the computing power to Superalignment team over next four years, they had not yet lived upto it. Meanwhile, #Elon Musk continued to question contradiction of OpenAI's existence vis-a-vis its founding principles. Starts #xAI to contest OpenAI and recently got the most funding anyone has ever received within A, B or C rounds - $6bn at the valuation of $24bn - all within 15 months of starting it!   There are two camps in the world right now: AI-doomers & AI-non-doomers. AI-doomers believe AGI (Artificial General Intelligence - the ultimate AI state, supposedly) is around the horizon and can doom humanity. AI-non-doomers believe AGI is far-far away and cannot doom humanity. AI-D has Geoff Hinton, Elon Musk (even though he himself is building the same with xAI), Steve Wozniak & 1000+ scientists who signed an open letter in March 2023 to pause AI-work for at least six months. AI-ND includes Yann LeCun, Andrew Ng and likes.   While we track these developments, only time will tell if Satya Nadella ends up being one of the most influential CEOs OR most #harmful CEO of our times (for his apparent disregard to malicious use of democratized & easily accessible AI). Either ways, he'll go down in the AI history for making history. Do you belong to either of the camps? If yes, then share your why? #ResponsibleAI #EthicalAI

34

3 Comments

🌟 New Episode Alert: Ever wondered what it takes to transition into the field of data engineering? 🤔 I'm thrilled to announce the release of the 7th and latest episode on "The Starlight Podium," where we dive deep into the world of data engineering with Kushagra Jaiswal, a seasoned Data Engineer at Deloitte, USA. 🚀 𝗪𝗵𝗮𝘁 𝗬𝗼𝘂'𝗹𝗹 𝗟𝗲𝗮𝗿𝗻: Kushagra takes us through his unique journey from infrastructure roles to mastering data engineering. He shares invaluable insights on the essential skills and tools that are shaping today's data solutions. Whether you're a fresher, a professional looking to pivot, or simply curious about the field, this episode is packed with practical advice and strategies to help you succeed. 𝗪𝗵𝗮𝘁 𝗬𝗼𝘂'𝗹𝗹 𝗟𝗲𝗮𝗿𝗻: 🔹 Foundational skills required for a career in data engineering 🔹 How to transition from IT infrastructure to data engineering 🔹 The critical role of cloud platforms in modern data engineering 🔹 Building practical data engineering projects for hands-on learning 🔹 Insights from career transitions within data roles If you're looking to break into data engineering or elevate your existing skills, this episode is a must-listen! 🎧 Link To The Episode on Spotify: https://buff.ly/3QQE62v Huge thanks to Kushagra for taking out time from his busy schedule for sharing such crucial insights from his journey in this field. Don't miss out on this opportunity to gain actionable insights and learn from industry professionals. #DataEngineering #CloudComputing #DataScience #DevOps #CareerTransition #TheStarlightPodium #PodcastRelease #DataAnalytics #LearningJourney #TechCareers #GoogleCloud #Deloitte

6

1 Comment

As a data scientist who is constantly working in pyspark for my position at CVS. I often encounter scenarios where I have to deal with complex data manipulations. One powerful technique that I have found for handling such situations is the `applyInPandas` function. However, I have long since noticed the tradeoff between when to use this method and when alternative approaches might be more efficient. I've condensed my standard exploration of the tradeoffs into the following cost-benefit analysis of utilizing `applyInPandas` in PySpark. When to Use applyInPandas: Complex Data Manipulations are particularly useful for: - Time series analysis - Statistical computations - Custom aggregations - Machine learning feature engineering ---Small to Medium-Sized Data Groups--- This method is most efficient when working with small to medium-sized data groups. The data from each group is collected to the driver and processed as a Pandas DataFrame, making it suitable for operations on manageable chunks of data. ---Efficiency Considerations--- There are trade-offs: - **Data Movement:** Data is shuffled to the driver, which can be costly for large datasets. - **Serialization Overhead:** Converting between Spark and Pandas DataFrames incurs serialization costs. - **Single-Node Processing:** Each group is processed on a single node, potentially limiting parallelism. --- Alternatives to Consider --- Before opting for `applyInPandas`, consider these alternatives: 1. **Native PySpark Functions:** Utilize built-in PySpark functions and SQL expressions for better performance on large-scale data. 2. **User-Defined Functions (UDFs):** For simpler operations, PySpark UDFs might offer better performance. 3. **Spark SQL:** Complex transformations can often be expressed efficiently using Spark SQL. --- Conclusion --- `applyInPandas` is a powerful tool for complex data manipulations in PySpark, especially when dealing with small to medium-sized data groups or when leveraging Pandas-specific functionality. However, it's essential to weigh the benefits against the potential performance costs, particularly for large-scale data processing. For optimal efficiency, use `applyInPandas` judiciously, considering the nature of your data and the specific requirements of your data manipulation tasks. Are you considering PySpark for your data science project but unsure where to begin? Let’s connect! I’d love to discuss your project needs and explore how I can help you harness the power of PySpark to manage and analyze your massive datasets effectively. Don’t hesitate to reach out!

1

1 Comment

Data aficionados, I've stumbled upon some incredible tools that are real game-changers in our industry. Here’s a quick rundown: 1. Vectorize is designed for Retrieval-Augmented Generation (RAG). It effortlessly converts unstructured data into high-performing vector search indexes. Imagine cutting through data chaos like never before. Do any of you folks already use this? 2. Spreadsheet is all you need—a mind-blowing nanoGPT pipeline neatly packed in a spreadsheet format. Perfect for anyone looking to get a handle on GPT's intricacies without diving into hardcore programming. 3. Replicate makes it a breeze to run and fine-tune open-sourced AI models using a straightforward API. Talk about simplifying complex operations! What AI models are you all experimenting with right now? 4. transformers.js brings the power of transformers right to your browser. Think about the possibilities here—AI execution without ever needing to leave your browser. Have you guys tried this yet? 5. Build your own X is a treasure trove of step-by-step guides for creating your own tech marvels from scratch. Whether it's AR, bots, or torrent tech, this resource has got you covered. Curious, which tech are you excited to recreate? 🚀 It's an exhilarating time to be in the data industry with tools like these at our fingertips. Let's elevate our game! #DataScience #AI #MachineLearning #VectorSearch #GPT #TechInnovation #DIYTech #Transformers Jump in and share your experiences or thoughts about these tools! 🌟

5

1 Comment

One common challenge that many junior data scientists face is the rush to start building ML or NLP models right after a client call. 📞 While enthusiasm is great, this often leads to the realization later that the core business problem wasn't fully understood. This scenario is not only frustrating but also counterproductive. 𝑯𝒆𝒓𝒆 𝒂𝒓𝒆 𝒂 𝒇𝒆𝒘 𝒕𝒊𝒑𝒔 𝒕𝒐 𝒏𝒂𝒗𝒊𝒈𝒂𝒕𝒆 𝒕𝒉𝒊𝒔 𝒄𝒓𝒊𝒕𝒊𝒄𝒂𝒍 𝒑𝒉𝒂𝒔𝒆 𝒆𝒇𝒇𝒆𝒄𝒕𝒊𝒗𝒆𝒍𝒚: ✅𝐀𝐬𝐤 𝐐𝐮𝐞𝐬𝐭𝐢𝐨𝐧𝐬 𝐁𝐨𝐥𝐝𝐥𝐲❓: It's essential to overcome the fear of appearing inexperienced. Asking questions is a sign of engagement and a desire to fully understand the problem. Remember, clarity leads to better solutions. ✅𝐑𝐞𝐪𝐮𝐞𝐬𝐭 𝐅𝐨𝐥𝐥𝐨𝐰-𝐮𝐩 𝐒𝐞𝐬𝐬𝐢𝐨𝐧𝐬 📅: Don’t hesitate to ask for additional meetings if needed. It’s better to invest time upfront to grasp the business problem than to redo work later. Clients appreciate thoroughness and commitment to solving their problems accurately. ✅𝐀𝐜𝐭𝐢𝐯𝐞 𝐋𝐢𝐬𝐭𝐞𝐧𝐢𝐧𝐠 👂: Focus on listening more during the initial call. Take detailed notes and summarize your understanding to the client to ensure you’re on the right track. This helps in aligning your perspective with the client's needs. ✅𝐁𝐮𝐢𝐥𝐝 𝐚 𝐂𝐥𝐞𝐚𝐫 𝐏𝐫𝐨𝐛𝐥𝐞𝐦 𝐒𝐭𝐚𝐭𝐞𝐦𝐞𝐧𝐭 📝: Before diving into model building, create a detailed problem statement. Outline the objectives, constraints, and expected outcomes. Share this with the client for validation. ✅𝐋𝐞𝐯𝐞𝐫𝐚𝐠𝐞 𝐌𝐞𝐧𝐭𝐨𝐫𝐬 𝐚𝐧𝐝 𝐏𝐞𝐞𝐫𝐬 👥: Don’t shy away from seeking guidance from more experienced colleagues. Their insights can help bridge gaps in understanding and provide new perspectives. ✅𝐂𝐨𝐧𝐭𝐢𝐧𝐮𝐨𝐮𝐬 𝐋𝐞𝐚𝐫𝐧𝐢𝐧𝐠 📚: Regularly update your knowledge about the industry and domain you’re working in. The more you understand the business context, the better you can tailor your solutions. Remember, the goal is not just to build a model but to solve a business problem effectively. Taking the time to understand the problem deeply will save time and resources in the long run and lead to more impactful solutions. Do you want more guidance, please connect with me https://lnkd.in/gnEdfBX8 #DataScience #MachineLearning #Business #ArtificialIntelleigence

7

One more session alert by Vidvatta for people interested in prompt engineering. Topic - Efficient Prompt Engineering and Tuning for Language Models! 🗓️ Date: 26th May 2024 🕒 Time: 11 AM IST ⏳ Duration: 1 hour 30 mins 🔗 Register Here https://lnkd.in/gurGZmeV ✨ Join us in this webinar where you will have a unique experience of doing hands on together with mentor in this session and get live feedbacks when you are stuck. Are you interested in Large Language Models (LLMs) and their potential? Curious about how to leverage advanced prompting techniques for various tasks? Our upcoming webinar is perfect for you! ✨ What to Expect: A brief overview of LLMs and the importance of prompting. Prompting and Task Complexity- Classification of tasks: Easy, Medium, Hard. Advanced Prompting Techniques: * Zero-Shot and Few-Shot Learning: Definitions and applications. * Task Decomposition: Simplifying complex tasks. * Chain and Tree of Thought: Enhancing reasoning capabilities. Interactive Q&A Session: Open floor for attendee questions and sharing experiences. Future trends in LLMs and prompt engineering. Don't miss this opportunity to delve into the intricacies of LLMs and enhance your skills. Register now and be part of the conversation shaping the future of prompt engineering! In case of any queries contact [email protected] #Webinar #LLM #PromptEngineering #MachineLearning #ArtificialIntelligence #FutureTech #LearningCommunity

13

1 Comment

Why Time Series Data Should Be Immutable? Recently, I came across a data model where mutation was enabled for time series data in a data model. It was an intended design choice but it sparked some red flags as I started using this dataset for my prediction models and reporting. While it may seem harmless, enabling mutability in time series datasets risks altering historical insights and impacts decision-making. IMO, the principle of immutability should be non-negotiable when working with time series data. "Once a data point is recorded, it should remain unchanged. " Here are top 3 potential issues I can think of: · Compromise of Data Integrity:  Altering historical data can lead to inconsistencies and inaccuracies, distorting the true nature of past events. · Bias in AI Prediction Models: Modified data can skew analytical results, leading to flawed insights and misguided decisions. · Audit Challenges: Now tracking changes to historical data becomes difficult, making it harder to identify the root cause of issues or fraudulent activities. While there may be rare edge cases where a seemingly benign modification might be tempting, it's crucial to weigh the risks vs benefits. It should be justified by compelling reasons. I would be interested to know, is there ever a valid case for allowing mutations in time series data. My curiosity is to understand where do we draw the line? #data #dataengineering #dataarchitects #datamodeling #aiml

5

7 Comments

Life’s Update: Professional: I am extremely happy to share that I have been promoted to Lead Data Scientist. My learnings: ✍️ Be responsible for deliverables and QA. Communicate well. ✍️ Take ownership of your work. This builds trust with your manager. ✍️ Expand your horizons. This way, seniors will become familiar with you and your work. Personal: I have completed my Umrah this EID 🌙. Stayed 11 days in Mecca and 6 days in Madina. Life’s most pure days. My learnings: ✍️ If you intend to perform Umrah/Hajj, do it at a younger age. ✍️ Avoid purchasing an internet plan unless necessary. Most hotels provide complimentary WiFi, which is usually sufficient. ✍️ Connect yourself totally to Allah and His mercy. Completely submit to His might. Special thanks to my current organization for their support ❤️. Comment your experience and advice.

75

10 Comments