Nikolas Mancebo

How To Specialize In Data Science / Machine Learning https://ift.tt/RnSuZx4 Is it better to be a generalist or specialist? Photo by Saulo Mohana on Unsplash Eventually, in your data science career, you’ll be asked, “What do you want to specialise in?” It’s quite a daunting question, and knowing what’s best for you is hard to say the least. So, in this article, I will explain why you should specialise, which one is right for you, and how to start. Why should you specialise? In my opinion, you should specialise, but there is no need to rush into this decision. Spend your initial years, like 2–3, learning all the data science and machine learning fundamentals. The things you should cover are: Basic statistics Linear algebra Calculus Linear and logistic regression Random forests Gradient boosted trees Neural networks Support vector machines Modelling best practices: cross-validation, hyperparameter tuning, etc. Writing high-quality Python and SQL This list is non-exhaustive, but it’s a good place to start. It will take a couple of years to fully understand everything. Some may think that’s a long time, but careers are long. If you start as a data scientist in your 20s, you could potentially have four decades working as a data scientist. A couple of years is nothing on that horizon. So, start with the basics and essentially become a generalist. You will then be able to solve most data science and machine learning problems, but you are not necessarily a deep expert in one particular field, which is totally fine at this point. After you have nailed all your fundamentals and have a sound footing, it’s time to decide where you want your expertise to lie in. The specialist vs. generalist debate is years old now, and there are pros and cons to both. Generalist Pros Better for the management track. More job opportunities. Wide variety of work and learning. Stick closer to the business side. Cons Less compensation as not an expert in any one area. Hard to become an IC if that’s something you want to achieve. Harder to get into a research based position. Specialist Pros Deep expertise could be seen as a higher value. Can go down both IC and manager paths. Work in an area you really enjoy. Cons Your expertise may become irrelevant in the future. May have limited opportunities compared to a generalist. I think it’s better to be a specialist because it’s easier to go from a specialist to a generalist than vice versa, and you are likely to be compensated more. However, I don’t think you should be an expert in one niche area. I recommend having T-shaped skills where you know the fundamentals very well and know roughly three areas to a pretty good depth. T-shaped skills diagram by author in Canva. This is kind of the best of both worlds and allows you to be more flexible in the job market. A lot of this assumes you are working in industry. If you want to do research, then it’s probably better to be a deep subject...

1

#data_science_in_Brazil (hoping you are interested in it!😄). Recently, I had the pleasure of connecting with an awesome team of data scientists and exchanging thoughts and knowledge. These are the challenges we discussed: 1. Ethics in Data Science, especially in the era of Facial Recognition capabilities. 2. How to increase trust in AI for the business side and stakeholders—seems like this is a pain point for tech people worldwide. 😅 3. Career Development in Tech. 𝚆𝚑𝚊𝚝 𝙸 𝚛𝚎𝚊𝚕𝚕𝚢 𝚠𝚊𝚗𝚝 𝚝𝚘 𝚍𝚛𝚊𝚠 𝚢𝚘𝚞𝚛 𝚊𝚝𝚝𝚎𝚗𝚝𝚒𝚘𝚗 𝚝𝚘 is that, even though we come from different continents, industries, etc., the questions, challenges, and pain points are pretty much the same. 𝚃𝚑𝚎 𝙿𝚘𝚠𝚎𝚛 𝚘𝚏 𝙳𝚒𝚟𝚎𝚛𝚜𝚒𝚝𝚢 (which isn’t just about gender or geography) is not just a fancy phrase—it is a very powerful tool. When we stay in our little local bubbles, working with the same kind of people, we tend to get stuck in that mindset and miss out on a lot of opportunities. ✨✨ — Jasmin Weimüller and I will be discussing one of these topics in the next episode of our Diaries of a Data Scientist podcast - don’t miss it out!😉 — If you want to know more about the Power of Diversity → I invite you to book your seat by following this link. (http://e-fellows.net/) Patricia Martins de Novais, Dr. Ricardo Klein Sercundes, Luiz Matheus Barbosa, Hismael Costa, Fernanda Aiko Hamatsu, Pedro Ribeiro Monasterio

59

1 comentário

Let's talk about data science in the context of a work of art: ### Describe a Piece of Art You Like #### What the Work of Art Is The work of art I want to describe is not a traditional painting or sculpture but a data visualization piece called "Wind Map" by Fernanda Viégas and Martin Wattenberg. "Wind Map" is a real-time, artistic visualization of wind patterns across the United States, turning complex meteorological data into a dynamic and beautiful visual experience. #### When You First Saw It I first encountered "Wind Map" a few years ago while browsing online for interesting data visualizations. It immediately caught my attention with its elegant representation of something as intangible as wind, depicted through flowing lines that change in real-time based on live weather data. #### What You Know About It "Wind Map" leverages data science to transform raw meteorological data into a visual art form. The creators use real-time wind data from the National Digital Forecast Database, which is processed and displayed on a web interface. The visualization shows wind speed and direction as continuous, flowing lines that vary in density and motion, creating an almost hypnotic effect. Fernanda Viégas and Martin Wattenberg are pioneers in the field of data visualization. Their work bridges the gap between data science and art, demonstrating how complex data can be made accessible and visually compelling. "Wind Map" updates in real-time, providing an ever-changing display that reflects current wind conditions across the country. #### Explain Why You Like It I like "Wind Map" because it exemplifies the beauty that can emerge from data science. This piece transforms raw, often overlooked data into a living work of art that is both informative and aesthetically pleasing. It shows how data science can be used not only for analysis and decision-making but also for creative expression. The visualization's real-time nature adds a dynamic element, making it more than just a static piece of art; it’s a constantly evolving canvas. This interplay between art and science, where data becomes a medium for artistic expression, is fascinating and inspiring. "Wind Map" challenges the traditional notions of what art can be and how we can experience data, making it a standout example of the innovative potential of data science.

1

1 comentário

🎓 Learnings from the class with Professor Mateus Gerolamo, on Special Topics - Leadership in Data Science at the MBA USP/Esalq: ✨ Leadership and Personality: I discovered that the Trait Theory highlights enduring characteristics that can influence leaders' behavior. It makes me wonder: which personality traits are most relevant for leading teams in Data Science projects? 🤔 ✨ Importance of Leadership: Leadership was emphasized as one of the fundamental factors for achieving results in any human endeavor. 💡 How can I develop my leadership skills to effectively lead Data analysis projects? ✨ Models of Personality Assessment: I learned about the Big Five Personality Model, which is widely used and validated. How can understanding personality traits influence the formation of effective teams in Data Science projects? 🤔 ✨ Organizational Mission: I reflected on the overall mission of an organization, which involves meeting the needs of human beings. How can Data Science projects contribute to meeting the needs of society? 🌍 These learnings inspire me to think more broadly about how leadership, personality, and organizational mission relate to my Data Science and Analytics course at USP. How can I apply these concepts to lead data analysis teams more effectively and in alignment with my organization's values? ✨📊

54

1 comentário

If you could train a LLM to self reflect and correct its mistakes. Reflection LLM built on top of Llama 70B is just released. Will the new models take on this approach and allow self reflection to correct its own hallucinations?

3

Very interesting keynote talk that I listened to, in person, at the ML in PL conference 2023, about looking at LLMs, not through the lense of sci-fi (Matrix/Terminator), but rather through literature and philosophy https://lnkd.in/dd3CMp6A

6

How far would you be willing to go for your career? Data scientists and software engineers are some of the most dedicated people I know. After all, name one other profession where, after a long day in the office, people go home and do more of what they're paid for - for fun - as is the case with Kaggle competitions. Yet, most of them still wouldn't be willing to rap to spread "the gospel of ML". One data scientist who is willing to "go the extra mile" and then some, is Dr Eric Siegel (aka Dr Data) who wrote and performed the (surprisingly catchy) "Predict This!", which you can (and should) check out below: https://lnkd.in/gm-Yp9XA Dr Eric Siegel is also the author of the recently released "The AI Playbook", which we discussed in the latest episode of "Value Driven Data Science". You can listen to our entire conversation here: Episode 48: Overcoming the Machine Learning Deployment Challenge. You can listen to our entire conversation at the link in the comments. #datascience #ML #AI #podcast

4

2 comentários

How To Specialize In Data Science / Machine Learning Is it better to be a generalist or specialist? Photo by Saulo Mohana on Unsplash Eventually, in your data science career, you’ll be asked, “What do you want to specialise in?” It’s quite a daunting question, and knowing what’s best for you is hard to say the least. So, in this article, I will explain why you should specialise, which one is right for you, and how to start. Why should you specialise? In my opinion, you should specialise, but there is no need to rush into this decision. Spend your initial years, like 2–3, learning all the data science and machine learning fundamentals. The things you should cover are: Basic statistics Linear algebra Calculus Linear and logistic regression Random forests Gradient boosted trees Neural networks Support vector machines Modelling best practices: cross-validation, hyperparameter tuning, etc. Writing high-quality Python and SQL This list is non-exhaustive, but it’s a good place to start. It will take a couple of years to fully understand everything. Some may think that’s a long time, but careers are long. If you start as a data scientist in your 20s, you could potentially have four decades working as a data scientist. A couple of years is nothing on that horizon. So, start with the basics and essentially become a generalist. You will then be able to solve most data science and machine learning problems, but you are not necessarily a deep expert in one particular field, which is totally fine at this point. After you have nailed all your fundamentals and have a sound footing, it’s time to decide where you want your expertise to lie in. The specialist vs. generalist debate is years old now, and there are pros and cons to both. Generalist Pros Better for the management track. More job opportunities. Wide variety of work and learning. Stick closer to the business side. Cons Less compensation as not an expert in any one area. Hard to become an IC if that’s something you want to achieve. Harder to get into a research based position. Specialist Pros Deep expertise could be seen as a higher value. Can go down both IC and manager paths. Work in an area you really enjoy. Cons Your expertise may become irrelevant in the future. May have limited opportunities compared to a generalist. I think it’s better to be a specialist because it’s easier to go from a specialist to a generalist than vice versa, and you are likely to be compensated more. However, I don’t think you should be an expert in one niche area. I recommend having T-shaped skills where you know the fundamentals very well and know roughly three areas to a pretty good depth. T-shaped skills diagram by author in Canva. This is kind of the best of both worlds and allows you to be more flexible in the job market. A lot of this assumes you are working in industry. If you want to do research, then it’s probably better to be a deep subject matter expert in a...

2

Awesome list here! If you read Breiman's Two Cultures, be sure to read the version with comments, and you will find an awesome surprise. There is also a full special issue revisiting the paper https://lnkd.in/dVtpsR9f. The following were also some papers that I would add to the list : Roberts, David R., et al. "Cross‐validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure." Ecography 40.8 (2017): 913-929. https://lnkd.in/dWbAFCmy Rudin, C., Wagstaff, K.L. Machine learning for science and society. Mach Learn 95, 1–9 (2014). https://lnkd.in/dxdXeqAe Cawley, Gavin C., and Nicola LC Talbot. "On over-fitting in model selection and subsequent selection bias in performance evaluation." The Journal of Machine Learning Research 11 (2010): 2079-2107. https://lnkd.in/dtdzmPub Rudin, Cynthia, and David Carlson. "The secrets of machine learning: Ten things you wish you had known earlier to be more effective at data analysis." Operations research & management science in the age of analytics. Informs, 2019. 44-72. https://lnkd.in/dqbtBPyJ Guyon, Isabelle, and André Elisseeff. "An introduction to variable and feature selection." Journal of machine learning research 3.Mar (2003): 1157-1182. https://lnkd.in/dAHxN5dA

16

I’ve just completed the Introduction to Python Course on DataCamp! Learning a new skill was quick, interactive, and fun! Plus, everything you need is entirely in-browser. Discover which of the 500+ data and AI courses can help you build a stronger career.

2

1 comentário

Hey #datafam Hey everyone! I'm excited to share a project I've been working on: the Data Masking Tool! 🚀 🚀 🚀 Most of my tools originate from real needs within our community. Recently, Ivett Kovács brought to my attention a common challenge we all face: the need to protect sensitive information when working with data. We often require a reliable way to anonymize data for testing and development purposes, as well as creating projects for our Tableau Public based on a real use case we've built for a client, without relying on external services like GPT, which isn't recommended for handling sensitive data. To address this, I developed the Data Masking Tool—a Python application that helps you anonymize sensitive data in CSV and Excel files securely and efficiently. Key Features: 🔒 Secure Data Masking: Replace sensitive information with realistic fake data, all processed locally on your machine. ⚙️ Customizable Options: Adjust prefixes, suffixes, numeric ranges, date ranges, and create custom lists to tailor the anonymization. 🎛️ User-Friendly Interface: An intuitive GUI built with Tkinter makes it easy to select columns and configure settings (The current UI is not great - sorry, I'm not used to creating UIs with Python. Looking forward to collaborating on it 😊). 🔄 Consistent Mappings: Option to keep mappings consistent so that the same original value is always replaced with the same fake value. 📝 Introduce Blanks: Specify a percentage of blank values to simulate real-world incomplete datasets. We often need a reliable way to handle sensitive data without risking exposure. This tool processes everything locally, giving you peace of mind when anonymizing datasets. I'm inviting anyone interested to contribute! 🤝 Open-source projects thrive with community involvement. Whether you want to add new features, fix bugs, or improve the documentation, your help is welcome. 👉 GitHub Repository: https://lnkd.in/dMhvN_mU Let's collaborate to make data handling safer and more efficient. Huge thanks to the Swift Insights Tableau Team for providing feedback. Your feedback and suggestions are greatly appreciated. If you have any questions or ideas, don't hesitate to reach out! Thanks for your support! EDIT: I just added a Jupyter Notebook example in case you prefer it. #dataprivacy #datasecurity #opensource #python #dataanonymization #datamaskingtool #tableau #tableauambassador #hastech

59

Vitor Cerqueira presented our work at Discovery Science 2024 in Pisa, Italy! 🌍 Our paper, "Forecasting with Deep Learning: Beyond Average of Average of Average Performance," introduces a new Aspect-based Evaluation Framework that moves beyond the traditional focus on single summary metrics. This work is also co-authored by Carlos Soares. Our approach evaluates forecasting models by analyzing performance across several key dimensions: sampling frequency, forecasting horizon, problem difficulty, and anomalies (extreme cases outside the 99% prediction interval). To provide a clearer understanding of model strengths and weaknesses, we use different summarization techniques, including overall performance metrics, expected shortfall (focusing on how models perform in the worst 5% of cases), and win/loss ratios for comparing models without assuming a specific distribution. We also incorporate the region of practical equivalence (ROPE) to focus on differences that are practically significant. We showed how controlling for these dimensions gives a clearer, more reliable view of model performance. This approach leads to more informed model selection and a better understanding of trade-offs, which is crucial for responsible forecasting. Link to our paper: https://lnkd.in/d3EyUBHb #DiscoveryScience2024 #Forecasting #DeepLearning #AI #Research #TimeSeries #ModelEvaluation #ResponsibleAI #DataScience

128

2 comentários

Key questions to consider for a Successful Genai Implementation from Bruno Aziza. #genai #thoughtprocess #data

6

🎬 A day in the life of a media data scientist exec. 6 AM: Wake up screaming from nightmares about corrupt datasets. 7 AM: Explain to non-technical wife why her Netflix recommendations suck. Over coffee, obviously. 9 AM: Arrive at office, immediately regret career choices. 10 AM: Meeting with stakeholders who think AI is magic and budgets are suggestions. 11 AM: Crush dreams by explaining why we can't predict the next viral TikTok dance. 1 PM: Lunch break? You mean debugging someone's spaghetti code while questioning existence. 3 PM: Pitch revolutionary AI idea. Get told to focus on increasing ad clicks. Again. 5 PM: Realize day's work was based on mislabeled data. Contemplate early retirement. 7 PM: Leave office, vowing to quit... until remembering that NYC mega-mortgage. 9 PM: Drown sorrows in TikTok dances and a mocktail. This is "market research." 11 PM: Dream of a world without p-values. Ready to join the glamorous world of data science leadership? What doesn't kill you just makes you bitter enough to excel in this field. #DataScience #MediaAnalytics #ExecutiveNightmares

15

8 comentários

Visualizing the Relationship Between Location and Price: Exploring the complex interplay between location and price is crucial in many fields. To visualize this relationship effectively for a housing project in Brazil, I've built a multiple linear regression model that predicts price based on longitude and latitude. By leveraging scatter plots, we can now visualize these three dimensions simultaneously, providing a richer understanding of how location influences pricing. This powerful visualization technique allows for more intuitive insights and data-driven decision-making. #datavisualization #linearregression #locationanalysis #priceprediction #dataanalytics #python #visualization #machinelearning

8

Useful slide. #python #programming

3

📚 🧠 𝐀 𝐠𝐞𝐧𝐭𝐥𝐞 𝐢𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐭𝐨 𝐦𝐚𝐭𝐫𝐢𝐱 𝐜𝐚𝐥𝐜𝐮𝐥𝐮𝐬 🟪 ⚡ - a great, relatively easy and quite extensive intro to #matrix calculus by Jan R. Margus

20

Explore José Antonio Siqueira de Cerqueira's efforts in enhancing language models for greater safety and reliability. Dive into this article to discover his significant contributions to the field - https://lnkd.in/dA-EWMu2! 📚👨‍💻 #research #languagemodels

4