Set Up Parameters
Parameter allows you to adjust the behavior of your model during testing. Below is a breakdown of each parameter and its purpose:
| Name | Description | Type | Supported value |
|---|---|---|---|
| Log samples | Model’s outputs, and the text fed into the model, will be saved | bool | True / False |
| Max tokens | Maximum number of tokens to generate | int | (0, +∞) |
| No. of few-shot | Sets the number of few-shot examples to place in context. Must be an integer. | int | [0, +∞) |
| Temperature | The temperature for sampling | float | [0, +∞) |
| Repetition penalty | Float that penalizes new tokens based on whether they appear in the prompt and the generated text so far. |
- Values > 1 encourage new tokens.
- Values < 1 encourage repetition. | float | (0, 5) |
| Seed | Random seed for reproducibility | int | [0, +∞) |
| Top-K | Integer that controls the cumulative probability of the top tokens to consider. Set to -1 to consider all tokens. | int | -1 or (0, +∞) |
| Top-P | Float that controls the cumulative probability of the top tokens to consider. Set to 1 to consider all tokens. | float | (0, 1] |