Joan Pau Campins García

Gradient boosting is currently one of the most popular techniques for efficient modeling of tabular datasets of all sizes. I learned how to use the powerful library XGboost alongside pandas and scikit-learn to build and tune supervised learning models. I worked with real-world datasets to solve classification and regression problems.

5

In the world of data, analysis is part of everyday life. With R, you can easily load datasets, clean them, run simple stats, and create clear visualisations. Check out my example here. Let's make data work for us.

9

2 comentarios

When comparing R and SAS codes for handling ZERO observation datasets/dataframes, several key points emerge. R programming shines with its minimal programming steps, achieving the desired output efficiently thanks to its flexibility and extensive package ecosystem. This flexibility allows for detailed manipulation at both the record and column levels, making it perfect for programmers who need to dive deep into data analysis. While both tools have their own strengths, R's adaptability truly sets it apart.   In the following R/SAS code, I'm counting the number of observations in a dataset and executing conditions based on the count. If the dataset has zero observations, a temporary variable is created with the message "Dataset has zero records!" Within the dataset. If there are more than zero observations, no action is taken, and the original dataset remains unchanged in the output. In R programming, certain tasks are quite straightforward, like using the “nrow( )” function to directly check the number of observations. In contrast, achieving the same result in SAS programming requires writing multiple lines of code, either through proc SQL or proc Freq, among other methods. Things seem more exciting in R programming, don't they?   Do share your favorite R function or keyword in the comment box if you find R programming more captivating than SAS. Also, it will be interesting to hear from you how you tackle routine R programming challenges! NOTE: SAS and R are distinct platforms, each offering unique and potentially simpler approaches to various tasks. Both tools have their own strengths and weaknesses.

128

12 comentarios

Excellent article from Iván Palomares Carrascosa on Statology! For professionals without a profound statistics background, choosing the right statistical test to perform on their data might sometimes not be a straightforward task. This article provides a decision tree-based guide aimed at helping them navigate the problem of choosing the right test depending on the data and problem they are facing, and the hypothesis to be tested.

2

A benefit of Bayesian statistics is the posterior distribution. Why? To name three reasons: - Shows the uncertainty for the estimate - We can calculate probabilities for different scenarios - It can be more intuitive to interpret Let's use an example. Imagine we have a binary treatment and a continuous outcome. We start with the prior information. We assume the treatment effect has a distribution with a mean of 0 and SD of 1. We also use our data. We can use the data through the likelihood. Then, we combine the two! Hurray! Now we have a posterior distribution. What do we do with this? Well, we can show the uncertainty (see image in post). We can also calculate what the probability is. For example, what's the probability of the effect being between -0.05 and 0.05? We can calculate this! It's 0.285. We can also calculate a range of where we think the "true" effect is: There is 95% probability of the value being between -0.29 and 0.15. #CausallyCurious #RealWorldEvidence #CausalInference #RealWorldData

74

11 comentarios

we could alternatively avoid the early evaluation of the objects in #r using the different functions and !! (Quasiquotation), # dplyr::enquo my_filter <- function(data, column, value) {   column <- dplyr::enquo(column)  data |> dplyr::filter(!!column > value)  } my_filter(mtcars, hp, 300) # rlang::ensym my_filter <- function(data, column, value) {   column <- rlang::ensym(column)  data |> dplyr::filter(!!column > value)  } my_filter(mtcars, hp, 300) # rlang::enexpr my_filter <- function(data, column, value) {   column <- rlang::enexpr(column)  data |> dplyr::filter(!!column > value)  } my_filter(mtcars, hp, 300) In all the above examples we use the object as a variable, what if we have to use the object as a string then we could follow the below approach # .data[[column]] my_filter <- function(data, column, value) {   data |> dplyr::filter( .data[[column]] > value) } my_filter(mtcars, 'hp', 300) #r #rstudio #posit #rlang

18

1 comentario

The Webinar on Data Science - R & Python yesterday. Thank you all for discussions, comments, support. In case you missed the webinar you may watch it using the link provided in the comments section. #biostatistics #datascience #r #rstats #python #ml #ai #programming

7

1 comentario

Please check my first articles on Polars out on Medium. #pydata #datscience #pydata #python #rstats

11

This new tool makes it easier for database users to perform complicated statistical analyses of tabular data without the need to know what is going on behind the scenes. GenSQL, a generative AI system for databases, could help users make predictions, detect anomalies, guess missing values, fix errors, or generate synthetic data with just a few keystrokes. #database #GenAI #data #GenSQL #SQL #datascience #GAI #technews #tech #IoT #AI

4

The #Romulus model series has been released on Hugging Face, continually pre-trained on #153K French #laws and intended to serve as a foundation for fine-tuning on labeled data 🤗 The training code, dataset and model weights are open and available free on #HF and the training was based on H100 provided by Microsoft for Startups using Unsloth AI by Daniel Han and Michael Han (Unsloth) 🦥 Please note that these #models have not been aligned for the #production of usable texts as they stand, and will certainly need to be refined for the desired tasks in order to produce satisfactory results. The training corpus consists of around 34,864,949 tokens (calculated with the tokenizer Meta-llama/Meta-Llama-3.1-8B-Instruct). Models are available on my personal HF page as a comment on this post 🎙️ #LLM #AI #GenAI #NLP #opensource #Legal #LegalTech

48

12 comentarios

The first thing I do is always plot my data. Why? If we know there is a variable with a mean of 50 and SD of 10. Can we draw it? Maybe. Maybe not. There are other reasons as well. To name just a few: - Sample size can affect this. N of 10, 100 and 1000 can look very different - Plots let you see the entire distribution (including kurtosis and skewness) - Easier to see outliers - Certain distributions are ideals (i.e., normal, binomial, etc). Plotting the data can help us understand "is this close enough to [insert distribution]?" I find plots much easier to interpret. Imagine you were buying a painting with the description "blue cat". Would you buy it? I'd want more information. Data is no different. #CausallyCurious #RealWorldEvidence #CausalInference #RealWorldData

56

12 comentarios

I’ve just completed the Introduction to SQL Course on DataCamp! Learning a new skill was quick, interactive, and fun! Plus, everything you need is entirely in-browser. Discover which of the 500+ data and AI courses can help you build a stronger career.

1

Nothing is better than being taught by an expert and that is what you will get with Kirk Paul Lafler with his in-person tutorial Southeast SAS Users Group in Bethesda on September 22. Kirk literally wrote the book on PROC SQL. With Kirk's dynamic personality and love for helping others you will definitely leave this tutorial feeling like you mastered this powerful tool. For more details and to get registered, visit https://lnkd.in/ghbY82qg #sesug #sesug2024 #sas #sasprogramming #sasprogrammer #sql

25

10 comentarios

Stratified sampling is a valuable technique for ensuring that different subgroups in a population are represented in a sample. This method is especially useful when you want to make sure that each subgroup is adequately represented, which can be crucial for accurate analysis. In R, you can perform stratified sampling using both base R and the `dplyr` package. For base R, you create a function that samples a fixed number of items from each subgroup. This approach helps you achieve a balanced representation of each group. With the `dplyr` package, stratified sampling becomes even more straightforward. You can use `group_by()` to group your data by a specific characteristic, and then use `sample_n()` to select a fixed number of samples from each group. If you prefer to sample a proportion of each group, `sample_frac()` lets you do just that. For example, you might sample 10 individuals from each gender group, or if you want a proportionate sample, you could select 20% from each subgroup. Both methods ensure that your sample reflects the diversity within your data. Give these techniques a try on your own data to see how they work in practice. Experiment with different sample sizes and strata to get a better understanding of how stratified sampling can enhance your analyses. Happy sampling and coding! #R #RStats #RProgramming #Sampling #dplyr #baser

35

3 comentarios

Time-varying covariates can be tricky for survival data. Why? They could violate the PH assumption! (if you're using a Cox PH model) One solution is to extend the Cox PH model to include time-varying covariates. An article I recently came across discussed just this! Zhang et al. (2018). The authors give tangible examples and R code! Building off this code, I've been able to explore a few scenarios myself. There are several options when the PH assumption is violated. It's been interesting comparing them. Recommend reading this one if you're curious about time-varying covariates and Cox regression! #CausallyCurious #RealWorldEvidence #CausalInference #RealWorldData [Link: https://lnkd.in/eu2W-S5r]

88

10 comentarios

i had really a good practice with kaggle SQL queries

5

As data professionals, we constantly work with messy, unstructured data that often requires cleaning and validation. This is where Regular Expressions (REs) come to the rescue! Regular Expressions offer incredible flexibility for matching and manipulating patterns in your data. Whether you’re validating emails, standardizing phone numbers, or ensuring the correct format of ID codes, REs can save you hours of manual work by automating these tasks directly in your SQL queries. Some common use cases where REs can make a huge difference: Ensuring fields like zip codes or IDs contain only valid digits. Validating the format of email addresses and phone numbers. Detecting or correcting patterns in inconsistent text data. Why are REs so important? Mastering Regular Expressions allows you to enforce data integrity, improve data quality, and streamline complex data cleaning tasks. In today’s data-driven world, REs are an essential tool for anyone working with SQL databases. Want to dive deeper and learn how to apply Regular Expressions in SQL? Check out my latest blog post to explore detailed examples and best practices for using REs effectively in SQL.

3

Are you getting tired of filling in Missing values constantly in your datasets? This weekend I decided to use my creativity and ChatGPT assistance with code optimisation to create a streamlit deployment that can fill missing values in any dataset without writing Python code but just input and export. I made this at first for my trainees with Ai DataYard to reduce their time in applying Machine learning algorithms for their datasets as a lot of time often went away trying to fill null values that model training and testing were overlooked. Now anyone can use it for their work, the link is here: https://lnkd.in/eGzVDHzB P.S I don't collect anyone's data, it's deleted from the deployment once you exit or refresh. This deployment has some limitations which I may solve overtime based on its need as for now it can only use 200MB of data due to Streamlit community cloud restrictions. You may check my GitHub code for validation if you require: https://lnkd.in/egZs_mWv #datascience #python #analytics #datacleaning

49

20 comentarios