pull/309/head
hako-mikan 2023-12-02 00:29:44 +09:00 committed by GitHub
parent 4cc20d073c
commit e4e5f1744c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 0 deletions

View File

@ -189,6 +189,7 @@ In this setup, we directly use *the two differential models*: **LoRA B** and **L
- **alpha (*α*)**: Controls the focus of feature extraction between **Model (LoRA)B** (***α* = 0**) and **Model (LoRA)C** (***α* = 1**).
- **beta (*β*)**: Controls the nature of feature extraction, with ***β* = 0** for **similar features** and ***β* = 1** for **dissimilar features**.
- **gamma (*γ*)**: Adjusts the selectivity in identifying feature (dis)similarity. **High *γ* (e.g., *γ* = 10)** emphasizes recognizing *more similar* features as similar. Conversely, **low *γ* (e.g., *γ* = 0.1)** emphasizes recognizing *more dissimilar* features as dissimilar.
**gamma** can be set in the option items of model merging as option(gamma), and in the case of LoRA, it can be set as gamma(smooth).
### Usage Scenarios
- ***α* = 0, *β* = 0**: Extracts features in **Model B** that are similar to those in **Model C**.