6d11cc7354
feat: Add basic text generation support with native models, initially supporting Gemma3 ( #12392 )
2026-02-18 20:49:43 -05:00
636d4bfb89
Fix hard crash when the spiece tokenizer path is bad.
2025-04-19 15:55:43 -04:00
e5ea112a90
Support Lumina 2 model.
2025-02-04 04:16:30 -05:00
0fd4e6c778
Lint unused import ( #5973 )
...
* Lint unused import
* nit
* Remove unused imports
* revert fix_torch import
* nit
2024-12-09 15:24:39 -05:00
a5f4292f9f
Basic hunyuan dit implementation. ( #4102 )
...
* Let tokenizers return weights to be stored in the saved checkpoint.
* Basic hunyuan dit implementation.
* Fix some resolutions not working.
* Support hydit checkpoint save.
* Init with right dtype.
* Switch to optimized attention in pooler.
* Fix black images on hunyuan dit.
2024-07-25 18:21:08 -04:00
10c919f4c7
Make it possible to load tokenizer data from checkpoints.
2024-07-24 16:43:53 -04:00
88ed893034
Allow SPieceTokenizer to load model from a byte string.
2024-07-23 14:17:42 -04:00
14764aa2e2
Rename LLAMATokenizer to SPieceTokenizer.
2024-07-22 12:21:45 -04:00
Jukka Seppänen
comfyanonymous
Chenlei Hu
comfyanonymous