Inderpreet Khanduja

Nitin Seth's book "Mastering the Data Paradox: Key to Winning in the AI Age" delves into the crucial role of data in the era of artificial intelligence and provides actionable insights for organizations to effectively leverage data and gain a competitive edge. Here are seven key lessons from the book: 1. Data as a Driving Force: The book highlights the emergence of a data-first world, where data has become a central driving force shaping industries and fueling innovation. It emphasizes the importance of recognizing the value and potential of data in driving business success 2. Navigating Data Paradoxes: Nitin Seth explores the complexities and paradoxes associated with data. Despite the abundance of data, actionable insights can be scarce. The book provides strategies and frameworks to navigate these paradoxes and effectively harness the power of data 3. Data as a Strategic Asset: Seth emphasizes the need to view data as a strategic asset. He explores how organizations can turn data into a valuable resource and leverage it to drive innovation, make informed decisions, and achieve success 4. The Role of Artificial Intelligence: The book discusses the dawn of the AI age and the role of artificial intelligence in leveraging data effectively. It explores the potential of AI technologies such as machine learning, predictive analytics, and IoT in driving innovation and staying ahead of the curve 5. Future Trends and Opportunities: Nitin Seth explores emerging trends and technologies that are shaping the future of data and AI. From advancements in machine learning to the rise of edge computing, the book identifies key trends and opportunities for organizations to capitalize on in the AI age 6. Data Literacy and Culture: The book emphasizes the importance of fostering a data-centric culture within organizations. It highlights the need for data literacy and the development of a data-driven mindset at all levels of an organization to effectively leverage data 7. The Unified Solution Framework: Nitin Seth proposes a thirteen-component solution framework called the Unified Solution Framework (USF) to help organizations succeed in the data-first world and the AI age. The framework provides a comprehensive approach to harnessing the power of data and driving business success These lessons from "Mastering the Data Paradox: Key to Winning in the AI Age" provide valuable insights into the pivotal role of data in the AI age and offer actionable strategies for organizations to effectively leverage data and gain a competitive edge. By understanding the complexities of data, embracing AI technologies, and fostering a data-driven culture, organizations can navigate the data paradox and harness the power of data for innovation and success.

1

Logarithm is a magical transformation. Before Calculators, Complex Multiplication, Division, were done using pre-computed Log-AntiLog Tables. Remember carefully checking the mantisa and the characteristic from last pages of standard- 6th Maths-textbook ? It helped converting complex Multiplication and Division, to much simpler operations - Addition and Subtraction. And, Slide-rule was a popular mechanical-device with logarithmic-scale for computation before our era of electronic calculators. Its believed, the device was used extensively in the first Moon-landing mission. See IBM's advertisement (1951) pitching how IBM-calculator can do work of 150 engineers with Slide-rules! PS: If you are a Modeler you might have been maximizing 'Joint-LOG-Likelihood' and not Just 'Joint-Likelihood' function, Simplifying multiplicative objective function into additive one and efficiently solving for parameter-coefficients in GLM. Moreover, Log is like a log stick, which turns a notorious mathematical function to a well behaved one ! There is a reason sab-log love log 🤗 #computation #statistics #datascience #predictivemodel

14

Addresses fundamental challenge of integrating contextual domain knowledge with RAG. Essential for real life applications in AI.

15

Question : Why does an AI product manager need to monitor #AI model❓ Once AI model is developed and deployed, the job doesnt end there. Model encodes pattern of the data it has seen during training phase. There may be several patterns or edge cases which are not part of training data. If model encounters any such patterns in testing or production phase, it will produce results which wont make sense without giving any warning about it. 👉 Data drift is a common challenge faced by AI-powered products. Over time, the distribution of input data can shift, leading to a mismatch between the model's training data and the real-world data it encounters. This can result in degraded model performance, unexpected outputs, and even complete model failure. 👉 The consequences of data drift can be severe, ranging from suboptimal user experiences to regulatory compliance issues. Product managers must be vigilant in detecting and addressing these challenges to ensure the continued success of their AI-powered offerings. 👉 Sudden changes in metrics can indicate input data drift. Additionally, monitoring the distribution of model outputs can reveal output drift, which may signal a need to retrain or fine-tune the model. Detecting data drift early is crucial. Product managers can leverage techniques such as A/B testing, anomaly detection, and continuous monitoring to identify and address these issues. Once detected, the product team can take appropriate actions, such as retraining the model, adjusting the data pipeline, or even considering a model architecture change. 💡 Quick Tip: Integrate model monitoring into your product roadmap and allocate dedicated resources to ensure the long-term success of your AI-powered offerings. Establish a comprehensive model monitoring dashboard to gain real-time visibility into the health and performance of your AI models. #artificialintelligence #productmanagement #datascience #productmanager

5

Another great talk from the PyData London 2024 - Backtesting and error metrics for modern time series forecasting by Kishan Manani Backtesting is the time series forecasting equivalent to the machine learning cross-validation approach for training and testing models. https://lnkd.in/g-DWgqmB #timeseries #forecasting #machinelearning #python

46

5 Comments

Follow up to the previous article about Aswhin - https://lnkd.in/gmaeXBSC #cricket #cricketdata #cricketanalytics #dataanalytics #sportsanalytics

11

1 Comment

📢 Best Advice from a Senior Data Scientist Packed in One Podcast: Insights from Abhishek Das!...🌟 🔷 Speaker: Abhishek Das Senior Data Scientist at Aptus Data Labs Ex-HSBC 9+ Years of Experience in Advanced Analytics and Machine Learning 🔷 Hosted by: Avinash Dara Here are some highlights from his talk: 🔷 His Personal Life: • He came from an Electronics and Communication Engineering background with an interest in research. • He was always interested in the business field to scale. • In his college placements, he joined e2open where the environment was new to him. He took some time to understand and scale in that industry. After that, his data science career started. In addition to this, he also started writing a book. 🔷 He worked across diverse industries such as: • Transportation, pharmaceuticals, manufacturing, supply chain, e-commerce, and banking. • Each sector has shown him the immense value data science brings to businesses, driving growth through informed decision-making. 🔷 He explained what Data Science is: • Data science is leveraging data to add value to a business. • He gave a practical example like Uber and Ola, which calculate the ride price based on other factors. 🔷 Career Paths in Data Science: 1. Data Engineer o Roles: Building and maintaining data infrastructure o Skills to acquire: Data engineering tools and technologies 2. Data Analyst o Roles: Extracting data, proficient in SQL, visualization, and providing insights o Skills to acquire: Strong SQL skills, data extraction abilities, and expertise in visualization tools 3. Data Scientist o Roles: Focusing on machine learning, algorithms, optimization, and prediction o Skills to acquire: Proficiency in machine learning, algorithms, and predictive analytics 4. ML Engineer o Roles: Researching, creating, and improving algorithms, and deploying models o Skills to acquire: Deep understanding of algorithm development and deployment 🔷 His advice for interview tips: 1. Be truthful about your skills. 2. Clearly indicate your proficiency levels. 3. Showcase 2-3 relevant projects, including ML projects. 4. Align your projects with industry-relevant problems, such as financial predictions or sales forecasting. 🔷 His approach for finding opportunities: • Apply to startups and MNCs. • Optimize your resume on LinkedIn. • Network with hiring managers and senior employees for referrals. • Target analytics-driven companies. 🔷 He also explained about LinkedIn: • LinkedIn is an incredible platform for networking and professional development. • Maintain a problem-solving mindset and continually showcase your skills and projects. #datascience #dataengineering #mlengineer #machinelearning #ccbp #advise #podcaste #datascientist #learning

35

One for the data architects / data engineers & professionals who work with Snowflake - which approach do you think is better for “moving data” from a 3rd party who uses Snowflake to your Snowflake instance: 1) via API using an integration tool like ADF, Informatica, Matillion etc 2) or using Snowflake direct share There are considerations for costs, risks, security and efficiency. My personal opinion is Direct share as it is so much more simpler and more secure - point to point, plus zero costs for moving data and mostly zero development effort on the subscriber side. But there would be advantages of api as well that I am not across. Would appreciate your comments and thoughts.

16

10 Comments

Why Multimodal RAG is Important? 🔹It provides more accurate and comprehensive information. Multimodal RAG models combine the power of information retrieval and generative language models to produce answers that are not just accurate but also contextually enriched. This is because they can access and process a wider range of data sources, including text, images, and videos. 🔹It is more adaptable to different domains and tasks. Multimodal RAG models can be easily customized to provide information on a wide range of topics by leveraging different knowledge sources. This makes them ideal for use in a variety of applications, such as customer service, education, and research. 🔹It improves the overall user experience. Multimodal RAG models can provide more informative, reliable, and relevant responses, which can lead to a better user experience. This is because they can take into account the user's context and preferences, and they can provide explanations and citations for their answers. Check out this super interesting new course from Deeplearning AI in partnership with Weaviate. https://lnkd.in/d3H6tFsf

30

1 Comment

#HIRING-FEEDBACK While hiring for data scientists or machine learning engineers, I always look out for one major thing in candidates take home assignments - their curiosity and adherence to the data science lifecycle - also known as CRISP-DM. The purpose why this framework exists is to streamline and standardize data science workflows. Yet I see innumerable cases while evaluating the scripts where numerous parts of this workflow are skipped. For e.g. when you are proposing model A is better than model B, cite your reasons, ground them in quantitative metrics like RMSE, or MAPE. Argue your case! Do not recommend a model without evaluating it on the test set. Talk about limitations of data and the approach, show you have taken time to understand the problem thoroughly Most importantly, perform EDA. There is a high chance you have not even understood the data if no EDA is performed, showing a casual attitude to the problem at hand. Data scientists are by nature curious. They are intrigued by the data mining process, making correlations and unearthing hidden patterns. Hopefully this feedback will help you crack more take home assignments to demonstrate your data skills. #Hiringfeedback #datascience

174

11 Comments

🚀 Are You Ready to Unlock the Full Potential of Small Language Models (SLMs)?🚀 Fine-tuning SLMs can significantly enhance their performance for specific tasks. Here’s an expanded approach, including tools and cloud services: 1. Dataset Preparation: Use high-quality, domain-specific datasets. Tools like Hugging Face's `datasets` library can help load and preprocess data efficiently. 2. Model Selection: Choose a pre-trained model such as Phi-3 or TinyLlama, accessible through libraries like Hugging Face Transformers. 3. Fine-Tuning Techniques: - LoRA: Fine-tunes smaller matrices to reduce parameters. - QLoRA: Combines quantization with LoRA for efficient memory usage. - DoRA/QDoRA: Further optimizations for resource-constrained environments. 4. Hyperparameter Tuning: Adjust learning rates, batch sizes, and other settings using tools like Optuna for automated optimization. 5. Cloud Services for Fine-Tuning: - AWS SageMaker: Offers powerful compute resources and simplifies the training process with built-in features. - Google Cloud Vertex AI: Provides a range of AI tools and customizable options for model training. - Microsoft Azure: Features NV-series VMs with NVIDIA GPUs and comprehensive ML lifecycle support via Azure Machine Learning. - Lambda Labs: Known for high-performance NVIDIA H100 Tensor Core GPUs and competitive pricing. - Oracle Cloud Infrastructure (OCI): Offers a Generative AI playground for fine-tuning models efficiently. 6. Evaluation and Iteration: After fine-tuning, evaluate model performance using relevant metrics and refine your approach based on results. By following these steps and utilizing these tools and cloud services, you can effectively fine-tune small LLMs to meet your specific needs! 💡 If you need further guidance, feel free to connect with Rohan Chikorde #ai #machinelearning #llm #finetuning #huggingface #cloudcomputing #generativeai #artificialintelligence #mlops #deeplearning #nlp #dataanalytics #bigdata #techtrends #innovation

2

Something I've written on how a T20 batting innings can be assessed. This framework can be extended to assessment of individual players as well, which I will do in the susequent posts. For now, please feel free to comment and if you like the content, please subscribe. #cricket #cricketanalytics #data #dashboards #datavisualization #sportsanalytics https://lnkd.in/gb7T-UzJ

28

6 Comments

🎇 Light Up Your Data Insights this Diwali with BigQuery & Vertex AI Agent Builder! 🎇 Tired of fumbling in the dark to find insights in your BigQuery data warehouse 🕶? What if you could search your massive BigQuery datasets as easily as Googling 🤔? This Diwali, illuminate your data with the power of Vertex AI Search 🚀! Dive into my latest blog post to learn how to build a powerful, AI-driven search application for the 'tourism' use case. Link: https://lnkd.in/grt36_fU #Vertex_AI hashtag#Search hashtag#BigQuery Google Cloud

1

A good start to my week! My article got featured in one of the best starting platforms like #LLaMAIndex and LangChain. I write a lot about LLMs AI and Healthcare. Follow me on https://www.ithinkbot.com

12

Unlock the Power of Data & AI for Supply Chain Analytics with Databricks on Azure/AWS 🚚📊 Supply chain logistics has never been more complex. From managing inventory levels and demand forecasting to optimizing delivery routes and handling disruptions, businesses are facing critical challenges that impact both their bottom line and customer satisfaction. 🌍 🔴 Key Pain Points in Supply Chain Logistics: Demand forecasting inaccuracies leading to overstock or stockouts. Inefficient routing causing delays and escalating transportation costs. Supply chain disruptions due to external factors like weather, political instability, or raw material shortages. As the pressure to streamline operations grows, Data & AI are becoming essential enablers for tackling these challenges. With Databricks on Azure or AWS, supply chain analytics are taken to the next level, driving better predictions, smarter decisions, and more resilient operations. Why Choose Diggibyte Technologies for Your Supply Chain Analytics Needs? 🔹 End-to-End Data Integration: We leverage Databricks to unify all your data sources—structured, unstructured, and real-time data—into a single platform. Say goodbye to data silos and hello to actionable insights. 🔹 AI-Driven Demand Forecasting: Using AI and machine learning, we improve demand forecasting accuracy, helping you balance supply and demand, reduce wastage, and maximize profits. 🔹 Smart Route Optimization: With real-time data analytics, we optimize your logistics network, identifying the best routes, reducing fuel consumption, and ensuring on-time deliveries. At Diggibyte Technologies, we help you harness the full potential of your supply chain data—from warehouse to delivery, making your supply chain smarter, more efficient, and future-proof. 🌟 Let’s drive smarter supply chain decisions together! 🚀 #SupplyChainAnalytics #AI #DataDriven #Databricks #Azure #AWS #Logistics #MachineLearning #DataIntegration #BusinessTransformation #CXO #DigitalTransformation #SmartLogistics #FutureOfSupplyChain

21

𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻: 𝗨𝗻𝘃𝗲𝗶𝗹𝗶𝗻𝗴 𝗗𝗮𝘁𝗮'𝘀 𝗛𝗶𝗱𝗱𝗲𝗻 𝗦𝘁𝗼𝗿𝗶𝗲𝘀 Regression models are the unsung heroes in the ever-evolving world of data science. They transform raw data into actionable insights. They don't just crunch numbers—they tell the powerful stories behind the data. 𝗪𝗵𝘆 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻𝘀 𝗠𝗮𝘁𝘁𝗲𝗿: 𝗣𝗿𝗲𝗱𝗶𝗰𝘁𝗶𝗼𝗻 Foresee future trends, sales, and customer behavior with precision. 𝗨𝗻𝗱𝗲𝗿𝘀𝘁𝗮𝗻𝗱𝗶𝗻𝗴 Decode complex relationships between variables to uncover hidden influences. 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Make more intelligent, data-driven decisions to drive better outcomes in any industry. 𝗥𝗲𝗮𝗹-𝗪𝗼𝗿𝗹𝗱 𝗔𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻𝘀  • Predict house prices based on square footage, location, and other key factors. • Forecast sales volume by analyzing marketing spend, seasonality, and economic trends. • Estimate customer churn probability using patterns in usage and demographics.  • Model the impact of various medical treatments on patient outcomes. 𝗞𝗲𝘆 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 𝗧𝘆𝗽𝗲𝘀 𝗬𝗼𝘂 𝗦𝗵𝗼𝘂𝗹𝗱 𝗞𝗻𝗼𝘄 𝗟𝗶𝗻𝗲𝗮𝗿 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 Perfect for straightforward, linear relationships between two variables. 𝗠𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝗟𝗶𝗻𝗲𝗮𝗿 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 Expand your analysis to include multiple independent variables for deeper insights. 𝗟𝗼𝗴𝗶𝘀𝘁𝗶𝗰 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 Ideal for tackling binary classification challenges—think yes/no, true/false scenarios. 𝗣𝗼𝗹𝘆𝗻𝗼𝗺𝗶𝗮𝗹 𝗥𝗲𝗴𝗿𝗲𝘀𝘀𝗶𝗼𝗻 Capture more complex, non-linear relationships with a twist. Whether you're a seasoned data scientist or just stepping into the field, mastering regressions is your ticket to uncovering the secrets hidden within your data. Let your data speak, and watch as the stories unfold. #DataScience #Regression #MachineLearning #Statistics #Analytics #PredictiveModeling #DataDriven

7

Q. Why Snowflake & Databricks moves ahead and did acquisitions in upstream & downstream, "Data Ingestion" & "AI Modeling/AI Ops" ? 💭 Embracing the evolution of the warehouse/lakehouse where storage, compute, and knowledge are independent yet scalable, adapting dynamically to the volume or changes in data. Q. Can the prime focus on scalability, elasticity, and performance lead to deep vendor lock-in, limiting options in their data maturity journey? Even with tight vendor lock-in, organizations must still push for rapid innovation and accelerate AI product development. But could this focus become a cost barrier that hampers the ability to see real ROI? #warehouse #lakehouse #modernwarehouse

21