1580158
/
automatic
mirror of https://github.com/vladmandic/automatic
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
12,597 Commits
3 Branches
29 Tags
109 MiB
Commit Graph

233 Commits (0e0b607cfaabf59167e3ab2a44d491b4e4e1b08e)

Author SHA1 Message Date
Disty0 2cf9938d97 SDNQ fix sdxl unet quant config not getting saved 2025-10-17 00:08:17 +03:00
Disty0 63aad89676 remove the unused state_dict arg 2025-10-16 16:29:23 +03:00
Vladimir Mandic 070edb20b0 update transformers and fix quant params 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-16 09:21:20 -04:00
Disty0 845869079d Fix sdnq unset config 2025-10-14 17:58:09 +03:00
Disty0 4aee524ddf SDNQ add NaDiT keys 2025-10-14 17:18:58 +03:00
Disty0 b601f0d402 SDNQ expose svd_steps and update module skip keys 2025-10-14 00:15:09 +03:00
Disty0 d4d24214b3 SDNQ use a better way of loading pre quants and cleanup 2025-10-13 14:06:13 +03:00
Vladimir Mandic 2e4e741d47 seedvt2 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-12 15:35:08 -04:00
Disty0 a376f89fd6 Add type checking to SDNQConfig 2025-10-12 01:02:47 +03:00
Disty0 9206d9443e SDNQ add dequantize model 2025-10-12 00:00:53 +03:00
Disty0 9a8ba0fc90 SDNQ unset device specific configs on save 2025-10-11 19:24:09 +03:00
Disty0 f7286c90d5 SDNQ add native pre-quant loader support to from_pretrained 2025-10-11 16:19:11 +03:00
Disty0 6bc83bc296 Prevent accelerate from splitting Linear and Conv layers and causing device mismatch errors 2025-10-11 03:19:30 +03:00
Disty0 0f785880ee SDNQ fix a singular bias not getting offloaded 2025-10-11 02:38:49 +03:00
Disty0 c7aba8589b SDNQ fix Qwen loading 2025-10-11 00:05:09 +03:00
Disty0 2a3deaa064 Check T5 keys before override 2025-10-09 22:46:27 +03:00
Disty0 6995d8c3c6 SDNQ fix T5 loading 2025-10-09 22:42:20 +03:00
Disty0 612df3abbb cleanup 2025-10-09 20:09:34 +03:00
Disty0 a9de8ef152 cleanup 2025-10-09 19:58:57 +03:00
Disty0 e19fb2d833 SDNQ keep the quant configs inside the module subfolder, add dtype cast and don't send to GPU 2025-10-09 19:34:48 +03:00
Vladimir Mandic 70defe6d06 handle load shards 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-09 11:29:36 -04:00
Vladimir Mandic 6907fcd320 speedup prequant model load 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-08 13:47:36 -04:00
Disty0 35277a79d3 cleanup x3 2025-10-08 01:21:11 +03:00
Disty0 9c16e2234a cleanup 2025-10-08 01:18:12 +03:00
Disty0 25303bb182 cleanup 2025-10-08 01:16:25 +03:00
Disty0 bdcd07f713 Add add_module_skip_keys to pre-load quant too 2025-10-08 01:11:40 +03:00
Disty0 7fdf400e8b cleanup 2025-10-08 00:41:04 +03:00
Disty0 df03ea9ba8 SDNQ add sdnq_post_load_quant and update Qwen keys 2025-10-08 00:29:36 +03:00
Vladimir Mandic 962cb7115d infra for full-model load/save with quant 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-07 14:30:45 -04:00
Vladimir Mandic 7fdc880a73 sdnq patches 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-07 09:43:34 -04:00
Disty0 1cd7b6d63a fix upcast scale check 2025-10-07 01:27:54 +03:00
Disty0 aa0c10440f SDNQ make the loader don't touch the model options by default 2025-10-07 00:15:23 +03:00
Disty0 5306376b2a improve contiguous mm performance 2025-10-06 19:05:46 +03:00
Disty0 be91bbff75 SDNQ add SVD support for Convs 2025-10-06 18:26:42 +03:00
Disty0 c931bf9efa SDNQ add dtype casting to loader 2025-10-06 17:44:52 +03:00
Disty0 5c042c5fb8 cleanup 2025-10-06 11:30:26 +03:00
Vladimir Mandic a315a004e9 linting 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-10-05 20:25:33 -04:00
Disty0 23f2deaa58 fix enable_quantized_mamtul 2025-10-06 02:04:28 +03:00
Disty0 1f81a37e8e Set the default svd rank to 32 2025-10-06 01:27:29 +03:00
Disty0 ebb26ac123 SDNQ make load file name configurable 2025-10-06 01:04:00 +03:00
Disty0 0acb571472 SDNQ ass load and save model funcs 2025-10-06 00:57:23 +03:00
Disty0 9e52d0c1fb SDNQ add SVDQuant quantization method 2025-10-05 22:50:30 +03:00
Disty0 428600613a SDNQ fix new transformers again 2025-10-05 15:30:15 +03:00
Disty0 a164f3e0c2 SDNQ Improve UINT3 and below quant speed 2025-10-05 03:12:05 +03:00
Disty0 f2e12a682f SDNQ remove use_contiguous_mm path in re_quant 2025-10-04 19:17:05 +03:00
Disty0 df142afe81 don't use triton mm for nvidia 2025-10-04 18:48:03 +03:00
Disty0 5c5d7d5a86 cleanup 2025-10-04 18:38:18 +03:00
Disty0 99113947bf SDNQ add RDNA2 INT8 support via Triton 2025-10-04 18:31:25 +03:00
Disty0 95a7da7e75 SDNQ use non-contiguous re-quantize 2025-10-03 18:54:58 +03:00
Disty0 54acf1760b Make SDNQ scales compatible with balanced offload 2025-10-03 18:13:55 +03:00
Disty0 c5cab96223 SDNQ simplify check_mats 2025-10-03 02:58:17 +03:00
Disty0 34c2a624aa SDNQ autodetect fp8 tw fallback and disable dynamic compile 2025-10-02 19:40:07 +03:00
Disty0 03382bdd4c SDNQ simplify check_mats 2025-10-01 01:35:51 +03:00
Disty0 0c1d34721c SDNQ use contiguous for intel 2025-09-30 02:37:58 +03:00
Disty0 6b67a9d0c4 SDNQ add check_mats to matmul 2025-09-30 01:58:13 +03:00
Disty0 1b45c145e9 SDNQ re-enable dynamic compile 2025-09-28 20:40:24 +03:00
Disty0 503a178794 Don't load rocm hsa on non wsl envs and remove unused lib hijacks 2025-09-27 11:39:24 +03:00
Disty0 71fde8a897 ROCm and Zluda don't fallback to CPU and clenup strings 2025-09-27 11:32:46 +03:00
Vladimir Mandic ae25cb8880 linting 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-09-25 14:33:21 -04:00
Disty0 a776c7c615 SDNQ fix new transformers 2025-09-24 21:32:30 +03:00
Disty0 41e297ba2c SDNQ fix new transformers 2025-09-23 16:17:09 +03:00
Vladimir Mandic 5b43c66a92 nunchaku sdxl and sdxl-turbo support 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-09-20 21:01:21 -04:00
Disty0 e6715ba8d3 Cleanup SDNQ compile 2025-09-19 19:29:36 +03:00
Disty0 39027ccfea SDNQ fix conv matmul 2025-09-16 17:12:04 +03:00
Disty0 a12edc1e90 SDNQ use nan_to_num_ with fp8 quantization in case of zeros 2025-09-15 20:22:39 +03:00
Vladimir Mandic 8cd5fbc926 lint 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-09-12 16:28:53 -04:00
Disty0 a8b850adf4 move hf quantizer hijacks to sdnq 2025-09-12 20:54:44 +03:00
Disty0 4f2b829450 cleanup 2025-09-11 15:42:36 +03:00
Disty0 6a954ffcde SDNQ fuse repeating bitwise ops 2025-09-06 13:18:17 +03:00
Vladimir Mandic 9743c8e4bf keep previous processed state 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-08-31 15:20:15 -04:00
Disty0 9035f4299c SDNQ fix new transformers 2025-08-30 22:44:57 +03:00
Disty0 bbb345cf44 Fix bias dtype mismatch 2025-08-30 02:31:41 +03:00
Disty0 6c36433a14 SDNQ fix row-wise FP8 matmul with fp32 and fp16 inputs 2025-08-30 02:27:15 +03:00
Disty0 4ec8603f63 SDNQ re-add bitpacking for uint1 2025-08-29 23:06:11 +03:00
Disty0 a20ab65359 SDNQ override matmul group size with int8 as well 2025-08-29 22:54:38 +03:00
Disty0 d49e954918 SDNQ listen to dequantize_fp32 option with re_quantize 2025-08-29 22:48:28 +03:00
Disty0 a8de3f7282 SDNQ add quantized matmul support for all quantization types and group sizes 2025-08-29 22:26:47 +03:00
Disty0 36bf998302 cleanup 2025-08-29 18:15:28 +03:00
Disty0 6e68dff381 Fix AuraFlow quant 2025-08-29 18:13:53 +03:00
Disty0 f324b7c0e5 SDNQ remove unnecessary .contiguous() 2025-08-21 02:21:05 +03:00
Disty0 e49814098e Add sdnq_modules_dtype_dict 2025-08-20 14:58:54 +03:00
Disty0 0946710662 Add sdnq_modules_to_not_convert to UI settings 2025-08-20 04:38:20 +03:00
Disty0 47ff01fd3b SDNQ add "*" support and upcast only the first and last layer's img_mod to 6 bit with Qwen Image 2025-08-20 03:24:19 +03:00
Disty0 47154db8b1 SDNQ Flux lora, use shape from sdnq_dequantizer 2025-08-18 19:53:57 +03:00
Vladimir Mandic fc547a3ccd sdnq with diffusers lora loader 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-08-18 10:29:01 -04:00
Disty0 85d494ee84 revert .t_().contiguous().t_() 2025-08-17 05:19:44 +03:00
Disty0 8460be662c SDNQ use inplace transpose and use view instead of reshape 2025-08-17 05:07:55 +03:00
Disty0 8ca74d0cd2 SDNQ rename unused param_name arg to op 2025-08-13 22:10:30 +03:00
Disty0 cb0c5414a3 SDNQ use uint with minimum_bits <= 4 2025-08-13 00:37:44 +03:00
Disty0 7085db9add Update changelog 2025-08-13 00:17:15 +03:00
Disty0 15cb8fe9f8 SDNQ add modules_dtype_dict and fix Qwen Image with quants less than 5 bits 2025-08-13 00:07:36 +03:00
Disty0 9992338187 sdnq fix convs 2025-08-11 23:24:13 +03:00
Disty0 26461f1d8d fix conv in8 matmul 2025-08-11 23:15:30 +03:00
Disty0 f45e3342e6 Cleanup 2025-08-11 15:11:29 +03:00
Disty0 afb3a5a06d SDNQ move non_blocking to quant config 2025-08-11 15:07:02 +03:00
Disty0 dc7b25d387 Cleanup SDNQ and add SDNQ_USE_TENSORWISE_FP8_MATMUL env var 2025-08-11 14:50:17 +03:00
Disty0 3f45c4e570 Cleanup SDNQ and skip transpose on packed int8 matmul 2025-08-10 19:31:34 +03:00
Disty0 69db77e365 SDNQ remove eps 2025-08-09 02:39:01 +03:00
Disty0 22d86acda3 Make SDNQ MatMul listen to the dequantize fp32 setting 2025-08-09 01:10:07 +03:00
Disty0 aa0652caa9 SDNQ fix new transformers 2025-08-07 00:18:24 +03:00
Disty0 ab8badfe0d SDNQ use non-blocking ops 2025-08-06 17:07:36 +03:00
Disty0 1d5dce1fb1 cleanup 2025-08-02 17:41:53 +03:00
Disty0 c3d007b02c SDNQ split forward.py into layers and cleanup 2025-08-02 17:36:55 +03:00
Vladimir Mandic fa44521ea3 offload-never and offload-always per-module and new highvram profile 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-07-31 11:40:24 -04:00
Vladimir Mandic 2656d3aa68 lint 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-07-24 15:42:29 -04:00
Disty0 444974a6ff cleanup 2025-07-23 19:02:44 +03:00
Disty0 7a08e1a7f2 SDNQ always use custom tensorwise fp8 matmul 2025-07-23 19:01:10 +03:00
Disty0 25a4731a97 SDNQ use static compile 2025-07-20 16:25:57 +03:00
Disty0 30af1f8fb0 Use inference_context with SDNQ 2025-07-14 13:07:44 +03:00
Vladimir Mandic e6d97f4d44 monkeypatch numpy for gradio 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-07-07 19:45:30 -04:00
Disty0 cf90e5621a Add _skip_layerwise_casting_patterns to SDNQ skip list 2025-07-04 00:04:01 +03:00
Vladimir Mandic c4d9338d2e major refactoring of modules 
Signed-off-by: Vladimir Mandic <mandic00@live.com>
 2025-07-03 09:18:38 -04:00
Disty0 3406083d14 Override lumina2 to use diffusers lora loading 2025-07-02 06:40:27 +03:00
Disty0 dc8fd006b2 Add modules_to_not_convert to pre-mode quants 2025-06-26 02:47:10 +03:00
Disty0 e43d1d2ba7 SDNQ use strings as target_dtype 2025-06-25 23:25:49 +03:00
Disty0 bbf986d3a5 Fix LTXVideo 2025-06-23 17:37:20 +03:00
Disty0 87e6d3f4fc SDNQ add modules_to_not_convert and don't quant _keep_in_fp32_modules layers in post mode 2025-06-20 20:41:11 +03:00
Disty0 7fc7797a1d SDNQ fix group size calc on odd shapes 2025-06-19 11:36:41 +03:00
Disty0 86cd272b96 SDNQ fix Dora 2025-06-18 16:24:42 +03:00
Disty0 e657cf790d SDNQ fix int8 matmul with qwen 2025-06-18 02:12:34 +03:00
Disty0 26800a1ef9 Cleanup sdnq 2025-06-17 02:05:13 +03:00
Disty0 d31df8c1eb SDNQ fuse bias into dequantizer with matmul 2025-06-14 22:10:10 +03:00
Disty0 25fc0094a9 SDNQ use quantize_device and return_device args and fix decompress_fp32 always being on 2025-06-14 21:29:08 +03:00
Disty0 c01802d9ff SDNQ fix transformers llm 2025-06-14 01:13:51 +03:00
Disty0 8f8e5ce1b0 Cleanup x2 2025-06-14 01:08:25 +03:00
Disty0 2ba64abcde Cleanup 2025-06-14 00:54:18 +03:00
Disty0 5e013fb154 SDNQ optimize input quantization and use the word quantize instead of compress 2025-06-12 12:06:57 +03:00
Disty0 2d05396b4e SDNQ simplify sym scale formula 2025-06-12 02:26:04 +03:00
Disty0 5cefa64a60 SDNQ update accepted dtypes 2025-06-11 20:58:54 +03:00
Disty0 78f99abec8 SDNQ use group_size / 2 for convs 2025-06-10 15:29:24 +03:00
Disty0 33fadf946b SDNQ add 7 bit support 2025-06-10 11:33:06 +03:00
Disty0 5bd7a08877 don't use inplace ops in quant layer 2025-06-10 03:29:07 +03:00
Disty0 5eed9135e3 Split SDNQ into multiple files and linting 2025-06-10 03:18:25 +03:00