Greetings to Fellow GenAI enthusiasts,
After testing out NVIDIA's new LLM Nemotron-70B's preview at https://lnkd.in/gvvhvwui, I was excited to find that they have enabled customizing key LLM parameters which enable me to enhance the quality of responses based on purpose.
Today, I'm excited to share my insights on how to tailor your LLM outputs for creativity, predictability, and diversity through parameter configurations.
1) Max number of tokens generated-> This may seem insignificant, but setting a low limit of tokens for simple questions can drastically save natural resources, as AI models reportedly consume 3 water bottles to generate every 100 words.
2) Temperature-> This determines the randomness of the model output. Lower temperatures lead to more predicable answers and are useful for responses that based on factual information such as the birth date of a famous person, a physics law, true/false question etc.
Higher temperature are useful for tasks involving creativity, such as writing, designing a logo or motto for a company, treating AI as a person to converse with or asking for views on new policies from government.
Low (0-0.25) Factual info (e.g., birth dates)
Medium (0.25-0.75) Balanced creativity & diversity (e.g., essays)
High (0.75-1) Highly imaginative & unpredictable outputs (e.g., artistic writing)
3) Top P-> Using this, you can control the diversity of the output. A higher top P value like 0.8 allows more choice of words to be considered while sampling, resulting in more diverse responses. Conversely, a lower top P value like 0.3, limits the choices and generates more focused responses.
High (>0.5) Richer language (e.g., creative writing, improving a speech)
Low (<0.5) Simpler responses geared towards easy comprehension (e.g., explaining complex topics, summarizing research paper)
4) Presence and Frequency Penalty-> Parameters to discourage the model from repeating words already in the generated response. By assigning a higher presence penalty value, such as 2.0, you can reduce the likelihood of repeating words. This is useful when you want to avoid certain content or bias in the generated text by over-fixating on a few words. Frequency penalty counts how many times the words has repeated too, penalizing more for additional counts of repetition, while Presence Frequency merely searches for presence, not frequency.
5) Seed-> Providing a new seed value generates a different response with same prompt, even if the choice of topics or words is similar to the previous response. This might seem not useful, but by changing the seed when analyzing a model's errors, you can consistently reproduce faulty outputs with different seeds, making it easier to track and address issues like bias or incorrect responses.
I hope this information about LLM parameters helps you get more out of Generative AI. Feel free to share these useful tips with fellow connections. Happy Prompting!
#GenerativeAI #LLMs #TechTrends #AI #NLP #NVIDIA
