Sasan Hashemi

Happy to share that our guideline for AI called "Guide to the Practical, Intelligent and Responsible Use of Artificial Intelligence in Tehran University of Medical Sciences " has been published today. This guideline is the result of several months of effort by me and Professor Torabi. Whether you are a professional in education, research, or clinic, this guideline will help you interact fluently with Generative AIs like Gemini, ChatGPT, and so forth. Hope you enjoy🌹: https://lnkd.in/deE7Ui3D #TUMS #AI #guideline

18

This video is about my last interview with a professor for a #Directly #PhDPosition in England. In this video (which is in Persian), I spoke about a challenging question that the professor asked me! The English version of this video will be placed on my #YouTube, so if you don't understand Persian, I recommend that you watch that video. You can find my YouTube link on my LinkedIn profile. ❤️😍 #interview #England #experience #machinelearning #deeplearning #هوش_مصنوعی #مصاحبه #مهاجرت #مهاجرت_تحصیلی #انگلستان

129

22 Comments

I'm thrilled to announce the release of two new fine-tuned large language models (LLMs) and a groundbreaking dataset for Persian NLP. 🚀 Shotor (Llama 3 8B): Fine-tuned on the Persian Alpaca Deep Clean dataset. Paya (Aya 23 8B): Also fine-tuned on the same comprehensive dataset. These models are crafted for high-quality performance in various Persian language applications, leveraging the Alpaca Deep Clean dataset which I meticulously processed. The fine-tuning was achieved using Dora for efficiency. Explore the models: Shotor: https://lnkd.in/ddge8CAh Paya: https://lnkd.in/dT55qCH4 Additionally, I’ve released the first Persian DPO dataset for LLM alignment, aiming to set a new standard in Persian language processing. Check out the datasets: Persian Alpaca Deep Clean: https://lnkd.in/d8ejpYRf Persian DPO: https://lnkd.in/dMF2sihF Your feedback and insights are invaluable as we strive to advance Persian NLP. I invite the community to test these models and datasets, and share your thoughts. Feel free to engage and share this post to spread the word! Looking forward to your feedback and collaboration. #NLP #PersianNLP #MachineLearning #AI #LanguageModels #DataScience #HuggingFace

34

6 Comments

Meeting one of the researchers I respect in the language testing field Vahid Aryadoust, at AILA 2024

37

4 Comments

I am thrilled to share that our paper, "Mutation Testing Reinvented: How Artificial Intelligence Complements Classic Methods," co-authored by Kaan Kurtel and me, has been published in the proceedings of 2024 9th International Conference on Computer Science and Engineering (UBMK) 🙂 In this study, we explored mutation testing as a method for assessing the effectiveness of software test suites and investigated the integration of artificial intelligence (AI) to overcome its challenges, such as high computational costs and limited fault detection. By automating tasks and optimizing mutation operator selection, AI enhances the efficiency and accuracy of mutation testing. Our work highlights advancements in this area and explores effective models that improve both fault detection capabilities and computational performance.

67

2 Comments

We are thrilled to announce an upcoming seminar by Dr. Hedieh Sajedi, Associate Professor at the University of Tehran. Title: “Spectrum of Deep Learning Algorithms in Drug Discovery” Abstract: The journey of drug discovery is complex and costly, often spanning over a decade. However, advancements in AI, especially deep learning, are revolutionizing this field. This seminar will delve into how deep learning algorithms are transforming drug discovery, offering innovative solutions to accelerate the development of new therapeutics. Dr. Sajedi will explore the spectrum of these algorithms, addressing the challenges and opportunities they present, including data quality, computational demands, and model interpretability. Date & Time: September 20, 16:00 Location: YSU, AI lab

6

Conditional Computation in LLMs (MoEs): Understanding the Concept of MoEs The Mixture of Experts (MoE) works on the idea of conditional computation, allowing the model to only run through some parts (an Expert) for a given input compared to a dense model where all parameters are used. It consists of a gating mechanism that selects the experts for the given input, enabling the model to scale without increasing the required computation. Popular LLM architectures like Mixtral and Grok use the same concept. What is MoEs? The Mixture of Experts (MoE) makes a simple modification to the usual architecture by replacing all or some of the Feed Forward layers with an MoE layer. The MoE layer consists of: 1. Router: A parameter gating mechanism that selects the experts used to process the given input by producing a probability distribution over experts. 2. Experts: Standalone NN modules with independent sets of parameters trained jointly with routers. It can use more complex architecture to create hierarchical MoE modules by implementing each expert as another MoE. - Implementation in Switch Transformers : Training was found to be 7x faster than training the base T5, keeping computation and training time constant. - Gshard and MeshTransformer : These frameworks propose efficient methods to train the MoE architectures using the parallelisation of models across GPUs/TPUs. Challenges with MoEs and Solutions 1. Inconsistent Batch Size for Experts: Can be overcome with the use of "soft" constraints in training loss. 2. Load Balancing Across Experts: The issue of repeatedly using the same few experts while training can be overcome by including better balancing mechanisms. 3. Model Instability Issues: Use of router-z Loss as an auxiliary loss to encourage smaller values within the router mechanism. 4. Model Overfitting: Requires different hyperparameter settings for effective fine-tuning. It was found that models benefit from high learning rates and smaller batch sizes. Recommended Reading/References 1. Hugging Face Blog on MoE - https://lnkd.in/g36gQJTx 2. Cameron Wolfe's Substack on Conditional Computation - https://lnkd.in/gtpjMHnJ

60

1 Comment

6 Open Positions under National Elite Foundation grant, AI and Quantum Computing We are excited to announce that our project, “Creating Building Blocks for Hybrid Quantum-Classical Neural Networks,” has been awarded the annual grant from the National Elite Foundation of Iran, one of the highest prestigous and competetive national grants. This project integrates deep learning components (such as residual block, dense block, attention layers, etc.) with quantum computing, targeting complex high-variance problems. Successful candidates will receive support from the National Elite Foundation. Qualification Requirements and Conditions: Two Sub-Groups of Positions: * Sub-Group 1 - Three Positions: Candidates should be distinguished winners of prestigious events, including: - Medalists (gold, silver, bronze) from international or national student Olympiads. - Top 1000 rank holders in Regions 1, 2, or 3 of the National Competitive Examination; i.e. Iran Conquer. - Top 100 rank holders in the the National Competitive Examination of art and foreign languages. - Highest rank achievers in the student, scholar, and technologist sections of the Khawarzmi Youth Festival. - 1st to 3rd rank holders in the Kharazmi International Festival. - 1st to 3rd rank holders in the National Student Olympiads endorsed by the Ministry of Science. - 1st to 3rd rank holders in the Farabi Youth Festival. - 1st to 3rd place winners in the "Business Designers" section of the Sheikh Bahai Festival. - Top 10 rank holders in the National Nanotechnology Competition (Student). - 1st to 3rd place winners in problem-oriented competitive events (Shahid Babaei Project), as per the National Elite Foundation regulations. * Sub-Group 2 - Three Position: Candidates must solve the following task and submit the code via email: “Update a simple residual block using a quantum algorithm.” The choice of Python framework is open. Preferred Selection Criteria: - High motivation and goal orientation with a passion for AI and quantum algorithms. - Proficiency in Pytorch or Tensorflow frameworks. - Familiarity with QNN libraries, such as Qiskit. - Strong analytical skills for tackling complex problems. - Excellent problem-solving abilities and grasp of new theories. - Ability to work both independently and collaboratively. - Competence in conducting and reporting research professionally. Benefits of Joining Our Team: - Certificate opportunity from one of the most prestigious academies in the country. - Exciting job opportunities in one cutting-edge interdisciplinary field. - Professional development opportunities. - High scoring potential from the National Elite Foundation. How to Apply: Interested candidates should send their resumes to: - Head of the Group: Ali A. Kiaei, email: [email protected] - Research Assistant: Fatemeh Aqaei, email: [email protected] For more information, please contact Fatemeh Aqaei. We look forward to welcoming talented individuals to our innovative team

27

8 Comments

Quality in, Garbage out. Briefly, this recent work on Nature ( among others ) claims and gives evidence that LLMs seriously degrade their behavior if trained from LLM generated data. To the point of wreaking havoc if the process is repeated 4-5 times. This says a lot about how lossy and noisy is LLM generated content wrt the human produced training data. Not a surprise. Nice read. "We demonstrate that it must be taken seriously if we are to sustain the benefits of training from large-scale data scraped from the web. Indeed, the value of data collected about genuine human interactions with systems will be increasingly valuable in the presence of LLM-generated content in data crawled from the Internet."

65

9 Comments

I’m so happy of publishing this book 📚 for all respected Persian readers around the world who are interested in the amazing interdisciplinary field of Accelerated High Performance Computing for the Neural Networks and Artificial Intelligence applications.🪴❤️ Now many giant tech leaders like NVIDIA🥇are doing tremendous valuable works in this field and indeed they are considered as the key game player. The next step would be an original English book. Let’s go ahead and enjoy the game…🪻😊🌤️🙋🏻‍♂️ #ai #hardware #nvidia #hpc #acceleratedcomputing #highperformancecomputing #hardware #artificialintelligence #deeplearning #computerarchitecture

231

14 Comments

Our latest paper introduces the pre-evaluation of sentence encoders, similar to pertaining language models!! Pre-evaluation performs evaluation without a specific downstream task, which is why we call it task-free evaluation. #EMNLP_2024 #NLP #Generative_AI #ML

26

Check out this highly recommended course for those interested in optimizing their models: - Introduction to Quantization - Symmetric and Asymmetric Quantization Modes - Hands-on Implementation from Scratch using PyTorch - covering Per Tensor, Per Channel (Axis), and Per Group techniques - Weights Packing: Exploring 2 or 4 Bits Quantization Methods https://lnkd.in/gMAuWXHm

43

Kolmogorov–Arnold Networks seem like a paradigm shift in ML architectures. Curious to see how they evolve. "While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. " https://lnkd.in/gdPk_qCQ

29

Personally, I think that the concept of AI is really cool and it can be helpful for everyone, especially in jobs like programming. At this moment, I'm writing this post during a university seminar session discussing AI and its usage and potential in Iran. Before I came here, I thought this mandatory three-hour session could be fun and useful because I'm really into AI and use it constantly. So, there shouldn't be any problem.👍 But now I'm realizing the unique ability of university professors to turn such an interesting subject into something boring to even think about.🤦‍♂️ I think that, as a person who is not very experienced in AI tech, I could have given a better and more engaging seminar about using AI. (Just kidding!🤌😅) I hope that we have better seminars with better and more knowledgeable professors.🤘

6

I’m thrilled that former students and postdocs of mine won both of this year’s NeurIPS Test of Time Paper Awards. This award recognizes papers published 10 years ago that have significantly shaped the research field. The recipients included Ian Goodfellow (who, as an undergraduate, built my first GPU server for deep learning in his dorm room) and his collaborators for their work on generative adversarial networks, and my former postdoc Ilya Sutskever and PhD student Quoc Le (with Oriol Vinyals) for their work on sequence-to-sequence learning. Congratulations to all these winners! By nature, I tend to focus on the future rather than the past. Steve Jobs famously declined to build a corporate museum, instead donating Apple's archives to Stanford University, because he wanted to keep the company forward-looking. Jeff Bezos encourages teams to approach every day as if it were “Day 1,” a mindset that emphasizes staying in the early, innovative stage of a company or industry. These philosophies resonate with me. But taking a brief look at the past can help us reflect on lessons for the future. One takeaway from looking at what worked 10 to 15 years ago is that many of the teams I led bet heavily on scaling to drive AI progress — a bet that laid a foundation to build larger and larger AI systems. At the time, the idea of scaling up neural networks was controversial, and I was on the fringe. I recall distinctly that, around 2008, Yoshua Bengio advised me not to bet on scaling and to focus on inventing algorithms instead! A lesson I carry from that time is to not worry about what others think, but follow your convictions, especially if you have data to support your beliefs. Small-scale experiments performed by my Stanford group convinced me that scaling up neural networks would drive significant progress, and that’s why I was willing to ignore the skeptics. The diagram below, generated by Adam Coates and Honglak Lee, is the one that most firmed up my beliefs at that time. It shows that, for a range of models, the larger we scaled them, the better they perform. I remember presenting it at CIFAR 2010, and if I had to pick a single reason why I pushed through to start Google Brain and set as the team’s #1 goal to scale up deep learning algorithms, it is this diagram! I also remember presenting at NeurIPS in 2008 our work on using GPUs to scale up training neural networks. (By the way, one measure of success in academia is when your work becomes sufficiently widely accepted that no one cites it anymore. I’m quite pleased the idea that GPUs should be used for AI — which was controversial back then — is now such a widely accepted “fact” that no one bothers to cite early papers that pushed for it.😃) [...] See full text (with links) here: https://lnkd.in/gkHikVU9

3,782

109 Comments