9 Extremely Experimental Libs

ArrowM edited this page 2023-01-23 11:09:41 -06:00

Table of Contents

• ⚠️Warning⚠️: Not Officially Supported! Do not open issues if you are using a modified library configuration.
• Issues
• Torch
• Xformers
• Potentially related issues
• BitsAndBytes
• Potentially related issues
• Diffusers

⚠️Warning⚠️: Not Officially Supported! Do not open issues if you are using a modified library configuration.

✅ Working
⚠️ Usable, but with issues
❌ Broken

Issues

1. Does not run.
2. May run, but training results are deformed.
3. VRAM does not clear properly; Must manually restart project to clear.

Torch

✅ torch-1.12.1+cu116 (default)
✅ torch-1.13.1+cu116
⚠️(2) torch-1.13.1+cu117
⚠️(3) torch-2.0.0.dev20230106

Xformers

✅ 0.0.14.dev0 (default)
❌(2) 0.0.14
❌(2) 0.0.15.dev0
❌(2) 0.0.15
❌(2) 0.0.16

Potentially related issues

• NaNs in backward pass when training with the huggingface diffusers-style trainer and unet. #631

BitsAndBytes

✅ 0.35.0 (default)
❌(2) 0.36.0.post2

Potentially related issues

• Regression in 0.36.0, training broken when fine-tuning RoBERTa on 3090 #121

Diffusers

✅ 0.10.2 (default)
❌(1) 0.11.1

Wiki

• Home

Getting Started

• Installing the extension
• Basics
• ELI5 Training
• Anime Training

Advanced Stuffs

• Class explained
• All settings explained
• API
• Batch Size
• Gradient Accumulation
• Learning Rate Scheduler
• Warmup

Troubleshooting

• Out of memory
• Overtrain
• Debugging