Tomas Palazzo

Excited to share that we will be presenting our paper, "Can probing classifiers reveal the learning by contact-center large language models?: No, it doesn’t!" at NAACL 2024 in Mexico City! Our work investigates the effectiveness of probing classifiers in understanding domain-specific fine-tuning of large language models (LLMs) for contact centers. In this work, we fine-tune Flan-T5 and Llama models with in-house contact-center data and note significant improvement in response acceptability on in-domain tasks, as compared to their respective out-of-the-box counterparts. While this reinforces the importance of in-domain fine-tuning, we observe that the performance on probing tasks show no substantial difference between fine-tuned and out-of-the-box models. This questions the efficacy of traditional probing mechanisms in revealing the fundamental characteristics learned by LLMs and thus opens up avenues for further research into the learning dynamics of these models. A big thank you to my co-authors Varun Nathan and Ayush Kumar from Observe.AI! 🎉 Do check-out our blog in the comment below to get into the details of our work and feel free to interact with us if you are onsite!

136

10 comentarios

This week I have done a course about Bayesian optimization that is a part of a Gaussian Process and Bayesian optimization course. If you want to discover how to use these beautiful class of methods, we have uploaded several jupyter notebooks to Github, using Skopt and BOTorch. https://lnkd.in/dSUwBBWp

116

1 comentario

The other day I was interviewed about the capabilities of generating new models of deep learning as Vgg's, Gan's, Llamas and ChatGpt (Computer Vision, NLP, etc) in Mexico. I was saddened to say that we have several problems here in Mexico (People are going to be unhappy and quite upset with me, but they say as Cassandra... before Troy is burned to cinders): 1. Mathematics education and understanding, a lot of it, after all ask yourself why Andrew Ng has a class of embeddings models or the culmination of 170 years of studying vectorial spaces. Actually, the people at control use embedding also under a series of nifty assumptions for fault detection under complex and real fields. 2. Money for hardware and computational time, a lot of it, after all if we look at history we have several examples of that. For example, The Metropolis-Hastings has an interesting story... you needed to have the a lot of computational time that with Edward Teller and Nicholas Metropolis, the researchers were able to use the MANIAC, at los Alamos, during the night for the development of the algorithm. 3. People a lot of people where you can say automatic differentiation or Bert recall and they know you are referring to the way the Jacobian is generated for the gradient based trained algorithms or the way we see how much information is invented/hallucinated by a LLM. We do that at our research center, but I barely have four people at the class of Deep Learning... at Guadalajara... it is quite dispiriting... when you hear of 100 hundred at MIT or Stanford. You need numbers, when AI was still coming out of a winter, at the CISE, University of Florida, circa 2002, we had 20 people, PhD level, studying under other names (Intelligent Systems, Pattern recognition, whatever) the subject, 22 years ago. Eight years later they were 50 people already doing so Master and Phd Level.... and after that the explosion in Canada and USA. 4. The fact that given the hype, there is a lot of people drinking the hype kool aid at Mexico. Basically, posting themselves as the experts... instead of acknowledging, the hard fact that you need decades to prepare the critical mass of capable people to develop such skills and models. The stuff I have seen in Mexico is at the level of user mostly, nothing new, and if you think of the amount of people contributing at sklearn 2896 people... you can imagine the other projects given that takes time to generate new ideas, write the papers, read the papers and implement the algorithms. Yes I know... you are going to be quite upset, but I am the kind of people that have hopes and work for that in Mexico... I do not need problems of hype, but hopes of something else. Excuses, I don't know how to lie not with stuff I love and respect.

15

14 comentarios

🌟Hello Everyone!🌟 I'm happy to share my new Portfolio Website😀. Contains some EDA 🔍 projects, Kaggle competition 💥 notebooks, and some AI 🤖 projects. Feel free to explore my repositories and Kaggle contributions. 🔗 GitHub: https://lnkd.in/eqKtnncQ 🔗 Kaggle: https://lnkd.in/exZmyU_Q #datascienceportfolio #PortfolioLaunch #DataScience #GitHub #Kaggle #MachineLearning #DeepLearning #ArtificialIntelligence #Math #Portfolio #ARCgames #ClimSim #NaturalLanguageProcessing

10

2 comentarios