Farooq Ahmed Siddique

🌟𝐖𝐞'𝐫𝐞 𝐇𝐢𝐫𝐢𝐧𝐠: 𝐏𝐲𝐭𝐡𝐨𝐧 𝐀𝐏𝐈 + 𝐄𝐥𝐚𝐬𝐭𝐢𝐜𝐬𝐞𝐚𝐫𝐜𝐡 𝐃𝐞𝐯𝐞𝐥𝐨𝐩𝐞𝐫𝐬! TekPillar®🌟 🔧𝐏𝐨𝐬𝐢𝐭𝐢𝐨𝐧 : Python API+ Elastic Search 📍𝐋𝐨𝐜𝐚𝐭𝐢𝐨𝐧 : Bangalore  📅𝐄𝐱𝐩𝐞𝐫𝐢𝐞𝐧𝐜𝐞 : 5+ Years      #KeyResponsibilities: 👉Develop custom search and analytics applications using FastAPI and Elasticsearch. 👉Index data from various sources, including big data systems like Databricks and Snowflake. 👉Query Elasticsearch data and create robust REST APIs. Implement AI/ML use cases and optimize Elasticsearch performance. 👉Create reusable libraries for index templates, custom analyzers, and scripts. 👉Collaborate with data engineers on pipelines and ETL jobs. 👉Guide developers on search, filtering, and aggregation requirements. 👉Research and prototype new Elasticsearch features. 🔗 𝐀𝐩𝐩𝐥𝐲 𝐇𝐞𝐫𝐞:  📞8799447189 | ✉️ [email protected] #Python #Elasticsearch #FastAPI #BigData #Azure #TechJobs #Bangalore #Hiring #tekpillarjobs

5

Very common rookie mistake candidates do while trying to get a job in the data science industry. Having the following 3 projects from #Kaggle in your resume: House Price Prediction Telco Churn Prediction Titanic Problem No, you will not get your resume selected with these projects. I get it. YouTube tutorials suggested doing these projects so you are wondering, “Why aren’t my resume getting picked after I’ve completed them?” Well think about it. As a hiring manager, how do they know you didn’t just copy an existing notebook? You might say, “But they can at least give me a chance to prove my skills!” Again think from their perspectives - What if they do give a chance but you don’t perform up to the mark, that’s valuable time wasted. Now imagine giving that time to all candidates. To really get noticed, work on projects that will show that you’re not just another candidate. And there are ways. Many ways. We will discuss some during https://lnkd.in/gMNckgRi on 14th October, 9pm. #DataScience #DataAnalytics #HiringTips #CareerAdvice #DataJobs #Kaggle #ResumeTips #StandOut #DataProjects #JobSearch #Job

9

I have seen 2 interesting bifurcation within industry profiles of data scientists : 1. Dude Data scientists - These profiles are tech heavy profiles. - These data scientists breathe in & breathe out docker, cloud , kubernetes , SQL , Deployment , production , ML , Analysis, APIs etc - These are not ML engineers either. They fight for this identity but when they are about to get it someone assigns them to an automation project spoiling everything. - These people have panic attacks if model/data sees drift in production. - They are often diagnosed with insomnia. 2. Data science Consultants who call themselves as data scientist just to look cool ! - These profile are blend of data scientist + product consultants - These data scientist spend 50% of their time working on ML , stats and 150% of time explaining to business why these framework will work. They always create 1 .ipynb file & 5 PPT decks. - They have immense confidence while explaining anything to anyone since they have been explaining meaning of p-value to non-tech stake holders often - They are often diagnosed with imposter syndrome - They dont know who they are, some see them as core data scientists , others see them as a consultants. - They face rejections in data science interviews since they are heavily working on consulting & face rejections in consulting roles since in tech rounds they dont know how the hell XGBoost works - They suffocate if chatgpt stops working some day. What kind of data scientist are you ? #datascientist #sarcasm #fun #humour #datascience #python #machinelearning #ai #learning #growth

194

22 Comments

Follow up to the previous article about Aswhin - https://lnkd.in/gmaeXBSC #cricket #cricketdata #cricketanalytics #dataanalytics #sportsanalytics

11

1 Comment

Week 10 of the "Ultimate Big Data Masters Program" provided an insightful journey into Apache Spark Optimizations and Performance Tuning - 1, provided by Sumit Mittal Sir. Here's a summary of Week 10 key learnings: ·       Pyspark Optimizations – Internals of groupBy ·       Normal Join Vs Broadcast Join ·       Different types of Joins ·       Partition Skew ·       Adaptive Query Execution (AQE) ·       Join Strategies ·       Optimizing Join of 2 large tables – Bucketing Thank you Sumit Mittal Sir and TrendyTech for all the support and guidance whenever needed. #Bigdata #Spark #Python #DAG #Join #dataengineer #AWS #Azure #ADLS #HDFS #Linux #SQL #athena #redshift

22

Another great talk from the PyData London 2024 - Backtesting and error metrics for modern time series forecasting by Kishan Manani Backtesting is the time series forecasting equivalent to the machine learning cross-validation approach for training and testing models. https://lnkd.in/g-DWgqmB #timeseries #forecasting #machinelearning #python

46

5 Comments

🏆 Winners of Data Odyssey 🏆 Breaking news: Team Chop The Data (aka the data warriors featuring Areej Aasna, Sukanta Bala, and yours truly) has officially chopped its way to the Championship title 🥇 at Data Odyssey, an Analytics Case Study Competition by T A Pai Management Institute, Manipal! 🎉 Out of 320+ teams, we stood tall—armed with machine learning models, predictive analytics wizardry, and enough coffee to power a small city. ☕🔮 Our task? Tame a wild dataset, unleash actionable insights, and present strategies sharper than Sukanta's Excel shortcuts! The competition challenged our creative problem-solving and teamwork skills, blending technical excellence with strategic business acumen (with a sprinkle of last-minute panic 😅). Special thanks to Prof. Amlan Chakrabarti for his valuable encouragement, and Dola Bhattacharjee Gupta for her unwavering guidance throughout our journey. Your support truly made a difference and inspired us to push our limits! 🙏 Highlights? Machine learning predictions so good they could almost predict our lunch options. Business strategies that made data come alive (seriously, the visuals practically danced!). A balance of analytics, storytelling, and power naps! 💤 Huge thanks to TAPMI Analytics Forum for being stellar hosts and giving us this platform to flex our data muscles. And yes, the buffet deserves its own award 🥗🏅. This journey was proof that when you mix a dash of humor, a splash of tech, and a heap of teamwork, magic happens! Looking forward to more adventures in data, learning, and competitions (and maybe upgrading from coffee to something stronger for the stress 🥲). #DataScience #MachineLearning #PredictiveAnalytics #Hackathons #Teamwork #HumorMe #DataOdyssey

36

5 Comments

Day 12 - 30-Day Data Engineering Interview Preparation Series PySpark (#Day12): 20 PySpark interview questions that can be relevant for a data engineering role 👇 📍 What is PySpark, and how does it differ from Apache Spark? 📍 Explain the architecture of Spark. 📍 How do you initialize a SparkSession in PySpark? 📍 What are RDDs in PySpark, and how do they work? 📍 What are the transformations and actions available in RDDs? 📍 How do you create RDDs in PySpark? 📍 What are DataFrames in PySpark, and how do they differ from RDDs? 📍 How do you perform operations on DataFrames? 📍 Explain the concept of a Dataset in PySpark. 📍 How do you work with SparkSQL in PySpark? 📍 Explain how to perform SQL operations on DataFrames. 📍 What are the advantages of using SparkSQL? 📍 How do you read and write data in various formats (CSV, JSON, Parquet) using PySpark? 📍 Explain the process of handling schema evolution in PySpark. 📍 What is Catalyst Optimizer in PySpark? 📍 How do you optimize a PySpark application? 📍 Explain the concept of lazy evaluation in PySpark. 📍 What is the significance of caching and persistence in PySpark? 📍 How do you handle skewed data in PySpark? 📍 What are broadcast variables and accumulators in PySpark? 🚨 We have launched our most affordable and industry-oriented "Complete Data Engineering 3.0 With Azure" bootcamp and ADMISSIONS ARE OPEN! 🔥 Only 7 days left and 80 seats remaining! 🚀 Enrollment link is in the first comment 👇 Cheers - Grow Data Skills Shashank Mishra 🇮🇳 SHAILJA MISHRA🟢 Sahil Choudhary Shubhankit Sirvaiya Aman Kumar 🙂 #DataEngineering #SQL #Hive #Hadoop #Kafka #Pyspark #AdvancedSQL #DataScience #BigData #TechInterview #InterviewPreparation #CareerGrowth #TechSkills #SQLQueries #azure

98

6 Comments

As a data engineer, I’m constantly amazed by the advancements in AI and NLP. Today, I want to take a moment to recognize and celebrate one of the key contributors from India to the revolutionary Transformer architecture—Ashish Vaswani, one of the brilliant minds behind the "Attention Is All You Need" paper, which transformed the field of natural language processing (NLP). - In short, “self-attention” helps the model "read" like a human, by looking at all parts of the text simultaneously and focusing on what matters most. The Transformer model is the foundation for groundbreaking AI systems, including the powerful tools we use today, like GPT. It’s incredible to see how the work of talented individuals like Ashish, hailing from India, is shaping the future of technology. Feeling proud and inspired by his global contributions driving innovation! 🇮🇳 🫡 Also, a big thank you to OpenAI for their continued efforts in advancing this technology—making it more powerful, scalable (trained on hundreds of petabytes of data, using thousands of GPUs/TPUs), and user-friendly while ensuring safety and accessibility. Their work empowers developers and engineers like me to push the boundaries of what’s possible! 🚀💻 #AI #NLP #Transformers #ProudIndian #TechInnovation #DataEngineering #AshishVaswani #India #MachineLearning #GoogleBrain #AdeptAILabs

37

2 Comments

🚀 Excited to share my latest project on Market Basket Analysis using association rules! 📊 In this project, I analyzed retail transaction data to uncover hidden patterns and relationships between products. By implementing the Apriori algorithm, I identified key association rules that reveal how often items are purchased together. I explored various metrics, including lift, leverage, conviction, and Zhang's metrics, to gain deeper insights into customer buying behavior. This analysis not only enhances marketing strategies but also informs inventory management and product placement decisions. 🔗 https://lnkd.in/gQvYCjAu Feel free to download the project and explore the code! I’d love to hear your thoughts and any feedback you have. Let’s see how data-driven insights can transform retail strategies! 🛒✨ #DataScience #MarketBasketAnalysis #Python #DataMining #AssociationRules

3

2 Comments

**𝐋𝐞𝐯𝐞𝐥 𝐔𝐩 𝐘𝐨𝐮𝐫 𝐃𝐚𝐭𝐚 𝐒𝐤𝐢𝐥𝐥𝐬 𝐰𝐢𝐭𝐡 𝐒𝐧𝐨𝐰𝐟𝐥𝐚𝐤𝐞 𝐓𝐫𝐚𝐢𝐧𝐢𝐧𝐠!** A huge shoutout to my friends, Vishal Kaushal , a Snowflake Squad member, and Sachin Mittal, a 4x Snowflake Data Superhero, who are starting Snowflake Training: Zero to Hero course starting May 18th! . **Why Snowflake? ** Snowflake is more than just a data warehouse. It's a powerful, cloud-native data platform that empowers data engineers to build next-generation data solutions. Here's why learning Snowflake can be a game-changer for your career. By mastering Snowflake, you'll equip yourself with the skills to design and implement robust, scalable data solutions that are in high demand across industries. This positions you as a valuable asset in the ever-evolving world of data engineering. This comprehensive training, will equip you with the knowledge and skills to become a Snowflake master. Here's what you'll gain: Solid Foundation: Grasp the core concepts and functionalities of Snowflake. Hands-on Learning: Get practical experience through real-world exercises and projects. Unlimited doubt support Real time Projects : 2 End to End Real time projects , 1 using Matillion as well. Ready to take your data engineering skills to the next level? Don't miss out on this amazing opportunity! Sign up for the Snowflake Training: Zero to Hero course starting May 18th. 👉🏻 Enroll Here - https://lnkd.in/gcfxzNx4 📅 Live classes starts from 18th May 2024 Discount Coupon : LIVE20 PS : This is not a paid post. #Snowflake #DataEngineering #CloudDataWarehouse #Training #Upskilling #DataSuperhero

121

4 Comments

My latest Medium post comparing SetFit, FastFit, and Semantic Router for Few-shot NLP Intent Classification has been published by Towards AI! This post covers: 1. Details of NLP Intent classification and why it is extremely important in the age of LLM chatbots 2. Algorithm details of SetFit, FastFit and Semantic Router 3. Detailed code experiments on a chatbot intent detection dataset 4. Conclusion on the best method as of today. As with most of my posts, it has lots of code you can directly use in your work! This post was jointly written by Shilpa and me, both of us humans 😅 🔗 Medium Friend Link (without paywall): https://lnkd.in/dVfAvjzr Follow me for more tips on building successful ML and LLM products! Medium: https://lnkd.in/g2jAJn5 X: https://lnkd.in/g_JbKEkM #generativeai #llm #nlp #artificialintelligence #mlops #llmops

78

9 Comments

Summary of the recently concluded test match. https://lnkd.in/g6hZNruW #cricketanalytics #cricketdata #cricket #sportsanalytics

13

State of Data Engineering in 2024 - Jobs | Salary | Skills | Projects 📝 As data-driven decision-making continues to drive business success, the role of data engineers has become pivotal. Here's a snapshot of the key trends and findings from the latest report by Analytics India Magazine: 📈 Market Growth: The data engineering market in India is projected to grow at a CAGR of 33.8%, from USD 29.1 billion in 2023 to USD 124.7 billion by 2028. 🧑🏻‍💻 Job Trends: Demand for Senior Talent: Data engineers with 6+ years of experience are increasingly in demand, rising from 27% in 2023 to 38% in 2024. 🤑 Salary Growth: Significant increase in data engineers earning ₹6-10 lakhs, from 24% in 2023 to 32.4% in 2024. 🚶🏻‍♂️ Attrition Rates: Overall attrition rates have improved, dropping from 18% in 2023 to 14% in 2024. 🟢 Sector Highlights: ☎️ Telecom Boom: The Telecom sector saw a 12.5% increase in data engineer employment due to advancements in 5G and data analytics needs. 🏦 BFSI Sector: Despite a slight decline, BFSI remains a major employer of data engineers, reflecting ongoing digital transformation efforts. 🧑‍🎓 Skills in Demand: Proficiency in Python, SQL, ETL, and PySpark is crucial, with cloud platform skills (AWS, Azure, GCP) gaining prominence among mid-level engineers. 💸 Compensation Trends: While there has been an increase in mid-range salaries, experienced professionals are seeing adjustments due to economic conditions. 🌆 City-Specific Insights: Significant salary reductions in tech hubs like Bengaluru and Mumbai highlight changing compensation dynamics influenced by remote work trends. For detailed insights, visit AIM latest report. 📊🔍 #dataengineering #career #jobs

172

12 Comments

Thanks for the invite Mahima Bhat Sejuti Das. Looking forward to speaking at the event about changing data infrastructure needs in the wake of developments in the generative AI space. Also, looking forward to other presentations at the event. #dataengineering #generativeai

38

1 Comment

Glad to hear 🤟 My Spark Streaming playlist on YouTube just helped someone to crack his JOB INTERVIEW 🤩 Congratulations Ravi Prakash 🏆 If you are reading this post, please tag yourself in the comments 😄 If you want to learn PySpark or Spark Streaming from basics to advanced Performance Optimization, check out the links in comments 👇 #spark #streaming #dataengineering

34

1 Comment

In case you've missed my previous posts on EDA Here's the full blog on Exploratory Data Analysis (EDA): Read Here: https://lnkd.in/dYRybKzz #datascience

2

1 Comment

📢 Exciting Workshop Alert: Machine Learning Introduction 📢 🗓 Date: May 25, 2024 🕚 Time: 11:00 AM IST ⏰ Duration: 1.5 Hours You will be hearing lot of jargons online like AI, ML, LLMs, Transformers, NLP, Prompting, Neural Nets etc etc. BUT many of those using these jargons don't even know how these technologies work. So join me in these session hosted by Vidvatta where I will take you through the basics. Not just one sided PPT presentation but I will also make you code a small ML model together with me. ✨ Workshop Highlights: Learn ML Fundamentals: Grasp the core concepts and principles of Machine Learning. Project Lifecycle: Understand the stages involved in a typical ML project, from data collection to model deployment. Job Market Insights: Gain insights into the current job market and opportunities in the field of Machine Learning. Hands-On Experience: Work on a practical ML project with real-time feedback from experienced mentors. 🔗 Register and Enroll Now! https://lnkd.in/dxXtcMsj Don't miss this opportunity to enhance your ML skills and advance your career! 🚀 For any support please contact - [email protected]

21

2 Comments