Zishan Hussain

COVID-19 brought abrupt changes in consumer behavior and impacted the accuracy of forecasting models. Similarly, UPI (Unified Payment Interface) abrupted consumer's spending methods, bringing many of them online. Concept drift happens when the statistical properties of the target variable, which the model is trying to predict, itself change over time. This causes problems because the predictions become less accurate and become unreliable. Here's a short video for Data Science enthusiasts I have created on the topic of Data drift and model monitoring followed by a practical exercise at the end. Like and subscribe for more such interesting concepts. Also, like and share over here for maximum reach. : ) Video Link: youtu.be/tQjRQWfYQ10 YT channel Link:  youtube.com/@datatrek #datatrek #datascience #machinelearning #statistics #deeplearning #ai

5

COVID-19 brought abrupt changes in consumer behavior and impacted the accuracy of forecasting models. Similarly, UPI (Unified Payment Interface) abrupted consumer's spending methods, bringing many of them online. Concept drift happens when the statistical properties of the target variable, which the model is trying to predict, itself change over time. This causes problems because the predictions become less accurate and become unreliable. Here's a short video for Data Science enthusiasts I have created on the topic of Data drift and model monitoring followed by a practical exercise at the end. Like and subscribe for more such interesting concepts. Also, like and share over here for maximum reach. : ) Video Link: youtu.be/tQjRQWfYQ10 YT channel Link:  youtube.com/@datatrek #datatrek #datascience #machinelearning #statistics #deeplearning #ai

9

Question : Sampling Techniques : What are they❓ As we saw recently in exit polls on Indian elections, choosing samples from a population can have a significant impact on predictions. #AI enthusiasts need to have a solid understanding of statistical sampling techniques. Statistical sampling is essential for AI product managers because it helps them: 1️⃣ Understand the Target Population: Accurate sampling ensures that the data used to train and evaluate AI models is representative of the target population, reducing the risk of biased or skewed results. 2️⃣ Optimize Data Collection: Sampling techniques can help AI product managers determine the appropriate sample size and sampling method, ensuring efficient and cost-effective data collection. 3️⃣ Validate Model Performance: Sampling is crucial for testing and validating AI models, as it allows for the creation of representative test sets that accurately reflect the real-world scenarios the model will encounter. AI enthusiasts should be aware of the different types of sampling methods, including: 🔺 Probabilistic Sampling Techniques: methods where randomisation is used instead of deliberate choice. 👉 Simple Random Sampling: Each member of the population has an equal chance of being selected. 👉Stratified Sampling: The population is divided into subgroups (strata), and samples are drawn from each stratum. 👉Cluster Sampling: The population is divided into clusters, and a random sample of clusters is selected. 👉Systematic Sampling: Every nth member of the population is selected, starting from a random point. 🔺 Non-Probabilistic Sampling Techniques: methods where some selection criteria exists 👉Convenience Sampling: Selecting participants based on their availability and accessibility. 👉Quota Sampling: Selecting participants to match predetermined proportions in the population. 👉Purposive Sampling: Selecting participants based on specific characteristics or criteria. 👉Snowball Sampling: Selecting participants through referrals from initial participants. The choice of sampling method depends on the specific goals of the AI project, the available resources, and the characteristics of the target population. Choosing the wrong sampling technique can lead to biased results, skewed data, and ultimately, the development of AI models that fail to perform well in real-world scenarios. 💡 Quick Tip: When selecting a sampling technique, consider the diversity and representativeness of the target population, the level of precision required, and the resources available for data collection.  #AI #ArtificialIntelligence #productmanagement #productmanager #DataScience 

7

A data analyst and a social media nut case both tell stories, but what’s the difference? Data analyst digs up the data and tells a story based on relevant and concrete facts. A social media nut case tells a story first and then tries to find edge cases data points to support it. Don’t you agree ? #data #analysis #friday #thoughts

4

Another way of handling missing values in your modelling projects!! In data science, handling missing values is a common and crucial task. Common methods include imputation, deletion, or using algorithms that can handle missing values natively. However, another effective way is using missing value indicators. Scenario: Imagine we are modeling loan approval of loan applicants based on marital status, as one of the features. In our sample, 30% of individuals are unmarried, thus their marital status is missing. This missingness isn't random; it indicates that these individuals are actually unmarried. Instead of just imputing these missing values with the mode, we can use an indicator to signify missingness (1 for unmarried and 0 for married). This provides the model with additional information. More info and practical example here: https://lnkd.in/drMhfbHz #machinelearning #datascience #missingvalues #missingvalueimputation #handlingmissingvalues #statistics

17

“𝐉𝐮𝐩𝐲𝐭𝐞𝐫 𝐧𝐨𝐭𝐞𝐛𝐨𝐨𝐤𝐬 𝐬𝐩𝐚𝐫𝐤 𝐭𝐡𝐞 𝐜𝐫𝐞𝐚𝐭𝐢𝐯𝐢𝐭𝐲 𝐨𝐟 𝐃𝐚𝐭𝐚 𝐒𝐜𝐢𝐞𝐧𝐜𝐞, 𝐛𝐮𝐭 𝐌𝐋𝐎𝐩𝐬 𝐟𝐫𝐚𝐦𝐞𝐰𝐨𝐫𝐤𝐬 𝐢𝐠𝐧𝐢𝐭𝐞 𝐭𝐡𝐞 𝐟𝐥𝐚𝐦𝐞𝐬 𝐨𝐟 𝐩𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧-𝐠𝐫𝐚𝐝𝐞 𝐬𝐮𝐜𝐜𝐞𝐬𝐬.” Hello everyone, In this blog, I explained: 1. How jupyter notebooks aid data exploration, but MLOps frameworks are crucial for scaling Data Science projects and ML models, ensuring reliability, scalability, and reproducibility. 2. How to move your existing jupyter-notebook to a production-grade Data Science project by modularizing your codebase and using several MLOps tools. Please feel free to drop your comments/suggestions. Love. Peace. Aashish #datascience #mlops #ai

23

Possibly One of the Most Underrated Features of Tableau Cloud: ↳Tableau Pulse It won't: - Provide the full interactive experience of dashboards - Replace custom reporting for complex analysis - Offer in-depth visual exploration for every dataset But it does: - Speed insights with rapid access to critical metrics - Empower users by focusing on key, actionable information - Boost decision-making with easily consumable data at a glance Tableau Pulse is perfect for those moments when fast, impactful insights are the priority.

4

#Amdari21DaysDataChallenge Machine Learning for Jewelry Price Optimization In my deliverable today, I performed #Spearman #correlation #analysis using heatmap to visualise it. # Feature correlation correlations = real_categories.drop(labels = ['Order_datetime','User_ID', 'Order_ID', 'product_ID'], axis = 1).corr(method = 'spearman') # Correlation heatmap plt.figure(figsize = (8, 8)) sns.heatmap(correlations, annot = True, center = .3) plt.show(); plt.close() # set correlation limit upper = 1 lower = .2 # Explicitly obtain correlated features correlated_columns = {   column: correlations[(correlations[column]> lower) & (correlations[column]< upper)].index.tolist()   for column in correlations.columns } correlated_columns #Output {'SKU_Quantity': [], 'Category_ID': [], 'Category': ['Price_USD'], 'Brand_ID': ['Target_Gender'], 'Price_USD': ['Category'], 'Target_Gender': ['Brand_ID'], 'Main_Color': [], 'Main_metal': [], 'Main_gem': []} #CasmirAAmdari21DDC #Amdari21DaysDataChallenge

4

🌟 New Blog Post ! 🌟 I'm excited to share my latest article, "A Guide to Understanding Interaction Terms," published on Analytics Vidhya 🚀 In this article, I dive into the concept of interaction terms in statistical modeling— a crucial aspect that can reveal hidden relationships between variables in your data. Check out the full article here: https://lnkd.in/dEkmfjwF

30

2 Comments

Want to land more callbacks for Data Science roles in E-Commerce Sector? One strategy is to focus on building business relevant projects. One common business domain is - 𝗠𝗲𝗿𝗰𝗵𝗮𝗻𝗱𝗶𝘀𝗶𝗻𝗴 The goal of merchandising in e-commerce is to optimize the assortment, pricing, and display of products to maximize revenue, profit, and customer satisfaction. Some 𝗸𝗲𝘆 𝗯𝘂𝘀𝗶𝗻𝗲𝘀𝘀 𝗺𝗲𝘁𝗿𝗶𝗰𝘀 𝘁𝗼 𝗳𝗼𝗰𝘂𝘀 on are - - Revenue per Visitor (RPV): Measures the revenue generated per customer visit. - Average Order Value (AOV): The average dollar amount spent each time a customer places an order. - Conversion Rate: The percentage of visitors who make a purchase. - Stock Turnover Ratio: How quickly inventory is sold and replaced over a specific period. - Gross Margin Return on Investment (GMROI): Measures the profitability of inventory. To build an 𝗮𝘁𝘁𝗿𝗮𝗰𝘁𝗶𝘃𝗲 𝗽𝗼𝗿𝘁𝗳𝗼𝗹𝗶𝗼 in this space, check out the following ideas 𝗣𝗿𝗼𝗱𝘂𝗰𝘁 𝗕𝘂𝗻𝗱𝗹𝗶𝗻𝗴 𝗦𝘁𝗿𝗮𝘁𝗲𝗴𝘆 Identify product bundles that maximize customer purchase probability and increase AOV. Dataset: Retail Data Analytics from UCI https://lnkd.in/gjNcNcwf ML Formulation: Association Rule Learning (e.g., Apriori Algorithm) 𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗣𝗿𝗶𝗰𝗶𝗻𝗴 𝗢𝗽𝘁𝗶𝗺𝗶𝘇𝗮𝘁𝗶𝗼𝗻 Develop a model to adjust product prices in real-time based on demand, competition, and inventory levels. Dataset: Dynamic Pricing Dataset https://lnkd.in/g4-_jfhe ML Formulation: Regression, Time Series Analysis 𝗜𝗻𝘃𝗲𝗻𝘁𝗼𝗿𝘆 𝗗𝗲𝗺𝗮𝗻𝗱 𝗙𝗼𝗿𝗲𝗰𝗮𝘀𝘁𝗶𝗻𝗴 Predict the demand for products to optimize stock levels and reduce overstock/understock situations. Dataset: Demand Forecasting from Kaggle https://lnkd.in/gtTbWncT ML Formulation: Time Series Forecasting, Regression 𝗥𝗲𝗽𝗼𝘀𝘁 with comments to grow your own network! 𝗟𝗶𝗸𝗲 for more such content. Follow Karun Thankachan for everything DS/ML

331

51 Comments

One of the most common question asked in Data Engineering Interview: Difference between ETL, ELT, and Reverse ETL. 1️⃣ ETL: The Traditional Powerhouse Extract, Transform, Load (ETL) has been the go-to process for decades. It involves: 1. Extracting data from various sources 2. Transforming it to fit operational needs 3. Loading it into the target system (usually a data warehouse) 2️⃣ ELT: Flipping the Script Extract, Load, Transform (ELT) emerged with the rise of cloud computing and big data. The key difference: - Data is loaded into the target system before transformation - Leverages the power of modern data warehouses for transformation - Offers more flexibility and scalability 3️⃣ Reverse ETL: Closing the Loop A newer player in the field, Reverse ETL: - Moves processed data from warehouses back into operational systems - Enables data activation, turning insights into action - Bridges the gap between analytics and operations For career guidance in Data Analytics, job search strategies, resume review, interview preparation, etc., you can book a session with me here: https://topmate.io/itshb

53

Week 10 of the "Ultimate Big Data Masters Program" provided an insightful journey into Apache Spark Optimizations and Performance Tuning - 1, provided by Sumit Mittal Sir. Here's a summary of Week 10 key learnings: ·       Pyspark Optimizations – Internals of groupBy ·       Normal Join Vs Broadcast Join ·       Different types of Joins ·       Partition Skew ·       Adaptive Query Execution (AQE) ·       Join Strategies ·       Optimizing Join of 2 large tables – Bucketing Thank you Sumit Mittal Sir and TrendyTech for all the support and guidance whenever needed. #Bigdata #Spark #Python #DAG #Join #dataengineer #AWS #Azure #ADLS #HDFS #Linux #SQL #athena #redshift

22

The Data Analyst's Guide to Reproducibility (Part 1): https://wix.to/3cWX9tq #newblogpost #dataanalysis #reproducibility #newdataanalysts #careergrowth #datadocumentation #dataanalysistips

6

COVID-19 brought abrupt changes in consumer behavior and impacted the accuracy of forecasting models. Similarly, UPI (Unified Payment Interface) abrupted consumer's spending methods, bringing many of them online. Concept drift happens when the statistical properties of the target variable, which the model is trying to predict, itself change over time. This causes problems because the predictions become less accurate and become unreliable. Here's a short video for Data Science enthusiasts I have created on the topic of Data drift and model monitoring followed by a practical exercise at the end. Like and subscribe for more such interesting concepts. Also, like and share over here for maximum reach. : ) Video Link: youtu.be/tQjRQWfYQ10 YT channel Link:  youtube.com/@datatrek #datatrek #datascience #machinelearning #statistics #deeplearning #ai

9