@adamkeys_@emmycredible@imzftbi If you have a lead to protect then Mosquera is the perfect sub for JT if you need some goals then White offers more going forward. IMO
LLM Generation Parameters
These are the primary controls used to influence the output of a Large Language Model.
1./ Temperature
◾ Controls the randomness of the output. It is applied to the probability distribution of the next possible tokens.
◾ Low Temperature (e.g., 0.2): Makes the output more deterministic and focused. The model will almost always select the most probable next token. (ideal for factual tasks like summarization, code generation, and direct Q&A)
◾ High Temperature (e.g., 1.0): Makes the output more random and creative. The model is more likely to select less probable tokens, leading to more diverse and novel text.
(useful for creative writing, brainstorming, and open-ended conversation)
2./ Top-p
◾ Controls randomness by selecting from a dynamic subset of the most probable tokens. The model considers the smallest set of tokens whose cumulative probability is greater than or equal to the p value.
◾ Ex: If top_p is 0.9, the model considers only the tokens that make up the top 90% of the probability mass for the next choice, discarding the remaining 10%.
◾ It provides a good balance between randomness and preventing the model from choosing bizarre or nonsensical tokens. It is often recommended as an alternative to temperature.
3./ Top-k
◾ Controls randomness by restricting the model's choices to the k most likely next tokens.
◾ Ex: If top_k is 50, the model will only consider the 50 most probable tokens for its next selection, regardless of their combined probability.
◾ It prevents very low-probability tokens from being selected, which can make the output more coherent and less erratic than high-temperature sampling alone.
4./ Max Length / Max New Token
◾ This parameter sets a hard limit on the number of tokens (words or word pieces) the model can generate in a single response.
◾ It is essential for controlling response length, managing computational costs, and preventing runaway or endlessly rambling outputs.
5./ Frequency Penalty
◾ A value (typically between -2.0 and 2.0) that penalizes tokens based on how often they have already appeared in the generated text so far.
◾ + Value (e.g., 0.5): Decreases the likelihood of the model repeating the exact same words, encouraging more linguistic variety.
◾ Zero Value (0.0): No penalty is applied.
6./ Presence Penalty
◾ A value (typically between -2.0 and 2.0) that penalizes tokens simply for having appeared in the text at all, regardless of their frequency.
◾ + Value (e.g., 0.5): Encourages the model to introduce new concepts and topics, as it is penalized for reusing any token, even once.
◾ This is particularly useful for preventing the model from getting stuck on a single idea or topic.
7. / Stop Sequences
◾ A user-defined string or list of strings that will immediately stop the generation process if the model produces it.
◾ It is crucial for controlling the structure of the output, creating formatted text, or simulating conversational turn-taking.
Follow @techNmak