Sharik Purkar

Week 3 at CUNY Tech Prep: I explored Tableau vs. Jupyter Notebooks. Tableau's data management was insightful, but I prefer Jupyter, possibly due to my Python background. I am looking forward to working more with Tableau, but for right now, I will stick to Jupyter Notebooks. #DataAnalysis #TechSkills #CUNYTechPrep

11

Join us next Thursday, 11/21, to learn more about the dashboard I worked nearly a year to create with the Data Collaborative for Justice to report on the current and anticipated future state of NYC's jail population. For those of you who do not know, NYC has been mandated to close Rikers Island by August 2027 and transition to a borough-based jail structure that will accommodate nearly 4,000 people. However, over 6,000 people are incarcerated in NYC jails today, and that number continues to rise. Based on statistical and data science methods, we created this tool to offer the public trustworthy information on the City's jail population and where it is headed in the next year. I aimed to create a resource that has something for everyone, whether you are unfamiliar with the movement to close Rikers Island or a seasoned data scientist. The ultimate goal of this dashboard, in my opinion, is to provide the public, policymakers, and advocates with accurate and compelling information to inspire impactful, data-driven policy changes that can improve the lives of New Yorkers. Register at the link below to listen in!

10

I'm guessing many of you in my data and AI networks can relate to this -- I regularly end up helping teams and clients sort out knowing when it's better to use LLMs or transformers or traditional ML algorithms, etc. The more I chew on this topic, the less trivial the decision seems. So inspired by the need those conversations exposed, I've started a series of articles on how to identify a good AI use case. Enjoy! https://lnkd.in/eHDPFkWT Part I takeaways: 🧠 What IS a good AI use-case 🐵 How a 13th century monk is relevant to AI 🤖 A classification for grouping #AI and #MachineLearning tasks into families 👩‍💼 Practical tips for aligning AI projects with business challenges Check out Part I and let me know what you think! What other articles would you like to see in a "good AI use-case" series? #AI #MachineLearning #DataScience #GenerativeAI #BusinessStrategy #Innovation #OccamsRazor

24

4 Comments

Looking forward to heading to Boston next week for the joint Data Engineering Summit & Open Data Science Conference (ODSC)! If you will be there (or in the area) let me know! Would love to sync up and say hi! I'll be presenting at 1pm on Tues, April 23rd on "Why the Hype Around dbt is Justified". I've built a lot of data warehouses with a lot of different solutions in my career and you'll learn why me (and many others) think this is one of the best data engineering tools out there. #ODSC #DataEngineeringSummit #BostonBound

21

Let’s break down the big jargon of AI, ML, and data science into simple, relatable terms! Interested to hear thoughts from others in this field like Nimra Ayaz Shashank Singh 🇮🇳 and Oun Muhammadon how they break down data concepts for a broader audience. #DataForEveryone #Data #DataIinSimpleTerm Abhay Mandal

2

My favorite part of the election is all of the interesting data visualization.

36

Learning a new skill or tool in data is super fun, because usually you realize you have to go back and learn five other foundational things first. Want to learn dbt? Okay first make sure you know cloud computing, CLI, OOP, CompSci 101, astrology, and galactic sociology first. #data #analytics #dataengineering #sql #dbt

14

🚀 New Project Alert: K-Means Clustering on High School Friends Chat Data🚀 ⚠️ Heads up! These word clouds might contain some colorful language, courtesy of my high school friends. Viewer discretion is advised! 😅 I'm thrilled to share a recent data science project I worked on, where I applied K-Means clustering to analyze chat data from a high school friends' group on WhatsApp. 📊🤖 Project Highlights: 🔍 Objective: To uncover hidden patterns and group similar messages into clusters. 🛠 Tools & Techniques: Data Preprocessing: Extracted date, time, sender, and message content using regex. TF-IDF Vectorization: Transformed text data into numerical format while removing common English and Hindi stop words. K-Means Clustering: Applied K-Means to identify 4 distinct clusters. Visualization: Created word clouds to represent the most frequent terms in each cluster. Key Findings: Positive & Affirmation Messages: This cluster included supportive and encouraging messages, reflecting the camaraderie within the group. Negative or Negation Messages: These messages often expressed disagreement or negative sentiments. Discussions & Debates: A cluster characterized by thoughtful discussions and debates on various topics. Numbers & Vague Phrases: Included messages with numerical data and ambiguous phrases, often used in planning or casual conversation. This project was not only a fun trip down memory lane but also a valuable exercise in text mining and clustering techniques. I'm excited about the potential applications of these methods in more complex and diverse datasets. Feel free to reach out if you have any questions or want to discuss the project further! 🤝 #DataScience #MachineLearning #KMeans #NLP #TextMining #DataAnalysis

7

2 Comments

What’s the difference between a Data Scientist and a Great Data Scientist? A great data scientist is a true storyteller! They can effectively communicate insights and dynamics to various stakeholders across the organization. 🗣️ Message me today so I can discuss the Data Science and Machine Learning roles I have available. #DataScience #Storytelling #Communication #StakeholderEngagement #DataInsights

12

📚 Excited to share my first Machine Learning (ML) project on predicting diabetes status using 📊 Machine Learning algorithms! 🌟 Whether you're in healthcare, tech, or simply interested in ML, this research dives into how advanced ML models can enhance early detection and treatment. Your thoughts and feedback are most welcome! #MachineLearning #DiabetesPrediction https://lnkd.in/e2GcwrGa

15

2 Comments

We had this coming: MoA LLM -> Mixture of Experts (MoE) Agents -> #Mixture of Agents (MoA) Let's understand them in simple terms. #MoE: Within one large model, sub-models are created on specific parts of data to develop an expert on that part of data. E.g. One sub-model specializes in predicting health conditions for Native Hawaiians while another sub-model is specialized for Asians. MoE reduces training time, computational costs while maintaining (or improving) accuracies of #LLMs. Twitter / X's #Grok-1, Mistral AI's #Mixtral 8x7B (8 experts of 7 billion parameters each), Databricks's #DBRX are based on this architecture. Ensemble models (EM): Why am I suddenly jumping to EM? Because, in your discussions, this may come up as a concept similar to MoE at #LLM-level, just like EM is at traditional #ML-level (e.g. Gradient Boosting Random Forest). But there is one distinction - In EM, each model is trained on all the data while in MoE, each sub-model (expert) is trained on a subset of data. #MoA: While MoE combines multiple neutral networks (#experts), MoA combines outputs from multiple Agents (potentially accessing multiple LLMs internally). Advantage: More accurate response with improved quality. Disadvantage: Longer inference time; to the time order of 5x. Solution: Groq. It reduces inference time to as low as 1x! #ArtificialIntelligence #AI

20

I have been working on a open source project that simplifies LLM Evaluation. Initially the project was supposed to be a standalone evaluator but last week took an alternate approach to support metrics that are available today Below is some early preview of results with and without using the Open Source package on a RAG based application. Planning to possibly release the package by Nov End As is comparison score {   "BLEU": 0.004631443593628837,   "GLEU": 0.13924050632911392,   "ROUGE-1": 0.2937062937062937,   "ROUGE-2": 0.28368794326241137,   "ROUGE-L": 0.2937062937062937,   "METEOR": 0.16377121738055922 } Score with Open Source package {   "BLEU": 0.7636669529586486,   "GLEU": 0.7674418604651163,   "ROUGE-1": 0.9047619047619047,   "ROUGE-2": 0.9,   "ROUGE-L": 0.9047619047619047,   "METEOR": 0.7963918933682945 } It will work on most LLM applications like Summarization, Q&A, Machine Translation etc Best part it does not need Alternate LLM #generativeai #artificialintelligence

163

17 Comments

Generative AI Effective or Not Effective?

12

Exciting News, LinkedIn Family! I just published another article to my snowflake series: "An Interactive Guide to Snowflake Stages and Storage Integration". In this article, I dive deep into: 1. The nuances of Snowflake Stages and their role in streamlining data movement 2. How to leverage COPY commands for bulk loading and the power of Snowpipe for continuous data ingestion 3. Practical examples of integrating storage solutions with Snowflake for optimal performance But wait, it’s more than just theory! I’ve included interactive examples and code snippets to help you apply these concepts in real time! Curious to enhance your Snowflake skills? Check it out and let me know what you think! #SnowflakeData #DataEngineering #CloudComputing #ETL #DataLoading #DataOptimization #RealTimeData #Snowpipe #CloudData #BigData

9

DataCamp, one more time...Here we go! I am thrilled to announce that I have completed the 48 hour Data Analyst using PowerBI! Throughout the track, I have learnt about preparing, transforming, modeling, DAX Functions, deploying, and report creating using one of the world’s most popular business intelligence tools. Finally, I applied my skills in 2 case studies: Telecom Databel Customer Churn Analysis & Atlas Labs HR Analytics. The better is yet to come! #datacamp #dataanalysis #powerbi

19

3 Comments

This is the second of three notes "on transitions" Let me excerpt the farewell note I sent to the #dataScience teams (previously part of #AI, before that #EDABI) at #Target that I was leading at the time of my recent departure, they were working on #forecasting and #generativeAI. I sent this on my penultimate day in role. "Dear Team, As many of you know, tomorrow, [], will be my last day in the office. This will indeed be a bittersweet transition for me as I have quite enjoyed my tenure at Target and the amazing warmth and friendliness of the Target culture. Reflecting on the past five years, it has been an honor and a privilege to have led this incredible team. From our humble beginnings with a handful of scientists, engineers and product owners building an uncertain product, the Demand forecasting engine (DFE) has transformed into a cornerstone of Target's enterprise operations model. This remarkable journey would not have been possible without each one of you. Thank you for your dedication, hard work, and unwavering commitment to making this vision a reality. More recently, I have been involved in building and scaling our generative AI efforts. I am confident that this initiative is on the right path and will achieve great success with your continued efforts. As I bid farewell, I want to leave you with some advice: take calculated risks, step out of your comfort zone, and set high standards for yourselves. Push beyond the boundaries set by others. Find your internal metrics that guide you to accomplish all that you are capable of. The future is bright, and I believe in each of your abilities to drive innovation and excellence. Good luck to all of you, individually and collectively. It has been a pleasure working together, and I look forward to seeing the amazing things you will achieve. I do hope our paths cross again in the future, and that this is au revoir rather than adieu. PS: Please do stay in touch and feel free to reach out at any time, even if it is just to say hello! " To be concluded.

195

13 Comments

One of the nice things about working with Manning Publications is I get to see a lot of the books produced by other authors there. One very impressive book I've been going through is The Quick Python Book by Naomi Cedar. This is one of the clearest introductions to python I've ever seen. Manning Publications Co. Naomi Ceder #Python #PythonFundamentals #ManningBooks #LearnwithManning

25

1 Comment

Check out this great podcast series focusing on Digital Public Goods, how digital technologies are applied to tackle wicked problems

3