docs: update per new version
MaggieGao02
parent
69a176f832
commit
1843dc404e
|
|
@ -8,8 +8,9 @@ Deploying this solution (ComfyUI portion) on Amazon Web Services primarily invol
|
|||
- Step 1: Deploy the middleware of the solution.
|
||||
- Step 2: Deploy ComfyUI frontend.
|
||||
|
||||
!!! tip
|
||||
You can refer to the ComfyUI section in the FAQ chapter if you encounter deployment issues.
|
||||
After the successfully deployment, please refer to [ComfyUI User Guide](../user-guide/ComfyUI/inference.md) for more details.
|
||||
|
||||
|
||||
|
||||
## Deployment Steps
|
||||
### Step 1: Deploy the middleware of the solution
|
||||
|
|
@ -17,104 +18,57 @@ This automated Amazon CloudFormation template deploys the solution in Amazon Web
|
|||
|
||||
1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/)，and use [Extension for Stable Diffusion on AWS](https://console.aws.amazon.com/cloudformation/home?#/stacks/create/template?stackName=stable-diffusion-aws&templateURL=https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/latest/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"} to create the stack.
|
||||
2. By default, this template will launch in the default region after you log in to the console. To launch this solution in a specified Amazon Web Services region, please select the desired region from the region drop-down list in the console's navigation bar.
|
||||
3. On the **Create Stack** page，confirm that the correct template URL has been entered in the **Amazon S3 URL** text box, then select **Next**.
|
||||
4. On the **Specify stack details** page, assign a unique name within your account that meets the naming requirements for your solution stack. Refer to the table below for deployment parameters. Click **Next**.
|
||||
3. In the **Create Stack** page，confirm that the correct template URL has been entered in the **Amazon S3 URL** text box, then select **Next**.
|
||||
4. In the **Specify stack details** page, assign a unique name within your account that meets the naming requirements for your solution stack. Refer to the table below for deployment parameters. Click **Next**.
|
||||
|
||||
|Parameter|Description|Recommendation|
|
||||
|:-------------|:--------------|:--------------|
|
||||
|APIEndpointType|For API calls, define the category of the API. The options are REGIONAL, PRIVATE, EDGE| Regional by default|
|
||||
|Bucket|Enter a valid new S3 bucket name (or the name of a previously deployed S3 bucket used for the ComfyUI section of this solution)||
|
||||
|email|Enter a valid email address for further notification receivement||
|
||||
|SdExtensionApiKey|请输入一个包含数字和字母组合的20个字符的字符串|默认为"09876543210987654321"|
|
||||
|LogLevel|择您心仪的Lambda Log日志打印级别|默认ERROR才打印|
|
||||
|SdExtensionApiKey|Please enter a 20-character string that includes a combination of numbers and letters |"09876543210987654321" by default|
|
||||
|LogLevel| Please select a desired Lambda Log leval| Only ERROR logs will be printed by default|
|
||||
|
||||
5. 在**配置堆栈选项**页面，选择**下一步**。
|
||||
6. 在**审核**页面，查看并确认设置。确保选中确认模板将创建Amazon Identity and Access Management（IAM）资源的复选框。并确保选中AWS CloudFormation需要的其它功能的复选框。选择**提交**以部署堆栈。
|
||||
7. 您可以在 AWS CloudFormation 控制台的 **状态** 列中查看堆栈的状态。您应该会在大约 15 分钟内收到**CREATE_COMPLETE**状态。
|
||||
5. In the **Configure stack options** page, select **Next**.
|
||||
6. In the **Review** page, review and confirm the settings. Ensure that the checkbox for confirming that the template will create Amazon Identity and Access Management (IAM) resources is selected. Also, make sure to select the checkboxes for any other AWS CloudFormation-required features. Choose **Submit** to deploy the stack.
|
||||
7. You can check the **status** of the stack in the Status column of the AWS CloudFormation console. You should receive a **CREATE_COMPLETE** status within approximately 15 minutes.
|
||||
|
||||
!!! tip "贴士"
|
||||
请及时检查您预留邮箱的收件箱，并在主题为“AWS Notification - Subscription Confirmation”的邮件中，点击“Confirm subscription”超链接，按提示完成订阅。
|
||||
!!! tip
|
||||
Please check your reserved email inbox promptly and click the "Confirm subscription" hyperlink in the email with the subject "AWS Notification - Subscription Confirmation" to complete the subscription as instructed.
|
||||
|
||||
|
||||
### 步骤2: 部署ComfyUI前端
|
||||
步骤2将会为客户安装ComfyUI的前端。该前端自动内置了汉化插件、工作流发布云上等按钮，为客户提供更友好的UI交互界面。此自动化Amazon CloudFormation模板在亚马逊云科技中部署。
|
||||
### Step 2: Deploy ComfyUI frontend
|
||||
This step will install the ComfyUI frontend for the customer. This frontend automatically includes Chinese language plugins and buttons for publishing workflows to the cloud, providing a more user-friendly UI interaction. This automated Amazon CloudFormation template is deployed within Amazon Web Services (AWS).
|
||||
|
||||
1. 登录到[AWS管理控制台](https://console.aws.amazon.com/)，点击控制台右上角**Create Stack**, **With new resource(standard)**，页面跳转至创建堆栈。
|
||||
2. 在**创建堆栈**页面上，选择**Choose an existing template**，在**特定模版**区域选择**Amazon S3 URLe**，填入该[部署模版链接](https://aws-gcr-solutions.s3.amazonaws.com/extension-for-stable-diffusion-on-aws/comfy.yaml)，然后选择**下一步**。
|
||||
3. 在**制定堆栈详细信息**页面，为您的解决方案堆栈分配一个账户内唯一且符合命名要求的名称。在**参数**部分，部署参数说明如下。点击**Next**。
|
||||
1. Sign in to the [AWS Management Console](https://console.aws.amazon.com/), click **Create Stack** in the upper right, **With new resource(standard)** to launch the AWS CloudFormation template.
|
||||
2. In the page of **Create Stack**, select **Choose an existing template**，**Specify template** field **Amazon S3 URLe**, enter [Amazon S3 URL](https://aws-gcr-solutions.s3.amazonaws.com/extension-for-stable-diffusion-on-aws/comfy.yaml)，and click **Next**。
|
||||
3. In the page of **Specify stack details**，assign a unique and name-compliant name for your solution stack within your account. In the Parameters section, the deployment parameter descriptions are as follows. Click **Next**。
|
||||
|
||||
!!! tip "贴士"
|
||||
此处的EC2 Key Pair主要用于本地远程连接EC2。如果没有现有的，可以参考[官方手册](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html){:target="_blank"}来创建。
|
||||
!!! tip
|
||||
The EC2 Key Pair here is primarily used for remote connections to EC2 instances from your local machine. If you don’t have an existing one, you can refer to [Create Key Pairs](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/create-key-pairs.html){:target="_blank"}.
|
||||
|
||||
|参数|说明|建议|
|
||||
|Parameter |Description|Recommendation|
|
||||
|:-------------|:--------------|:--------------|
|
||||
|InstanceType |部署的ec2的实例类型 | 如果是涉及推理动图、视频等，建议G6、G5机器 |
|
||||
|NumberOfInferencePorts|推理环境数量|建议不超过5个|
|
||||
|StackName|来自于部署步骤1中成功部署堆栈的名称||
|
||||
|keyPairName|选择现有的一个EC2 Key Pair||
|
||||
|InstanceType |Instance type of EC2 | For tasks involving inference animations, videos, etc., it is recommended to use G6 or G5 instances |
|
||||
|NumberOfInferencePorts|Number of inference interface|It is recommended not to exceed 5|
|
||||
|StackName| Stack Name from the successfully deployed stack in Step 1 of the deployment process||
|
||||
|keyPairName|Select a desired existing EC2 Key Pair||
|
||||
|
||||
4. 在**配置堆栈选项**页面，选择**下一步**。
|
||||
5. 在**审核**页面，查看并确认设置。确保选中确认模板将创建Amazon Identity and Access Management（IAM）资源的复选框。并确保选中AWS CloudFormation需要的其它功能的复选框。选择**提交**以部署堆栈。
|
||||
6. 您可以在 AWS CloudFormation 控制台的 **状态** 列中查看堆栈的状态。您应该会在大约 3 分钟内收到**CREATE_COMPLETE**状态。
|
||||
7. 选择部署成功的堆栈，打开**Outputs**，点击**Designer**对应的链接，即可打开解决方案部署的ComfyUI前端，Designer的访问可能需要关闭VPN或者去掉10000端口后访问。**NumberOfInferencePortsStart**代表推理环境地址起始路径端口，按照部署数量端口地址依次增加，例如：当NumberOfInferencePorts填写2时，地址范围时，可访问的推理环境地址依次为：http://EC2地址:10001，http://EC2地址:10002.
|
||||
4. In the page of **Configure stack options**, select **Next**。
|
||||
5. In the **Review** page, review and confirm the settings. Ensure that the checkbox for confirming that the template will create Amazon Identity and Access Management (IAM) resources is selected. Also, make sure to select the checkboxes for any other AWS CloudFormation-required features. Choose **Submit** to deploy the stack.
|
||||
6. You can check the **status** of the stack in the Status column of the AWS CloudFormation console. You should receive a **CREATE_COMPLETE** status within approximately 3 minutes.
|
||||
7. Select the successfully deployed stack, open **Outputs**, and click the link corresponding to **Designer** to open the ComfyUI front-end of the solution deployment. Access to Designer may require disabling the VPN or removing port 10000. **NumberOfInferencePortsStart** represents the starting port for the inference environment address, and the port addresses increase sequentially according to the deployment quantity. For example, when NumberOfInferencePorts is set to 2, the address range and the accessible inference environment addresses are sequentially as follows:"http://EC2地址:10001，http://EC2地址:10002.
|
||||
|
||||
|角色|功能|端口|
|
||||
|Role|Functions|Ports|
|
||||
|:-------------|:--------------|:--------------|
|
||||
|主美/工作流管理| 能够安装新的custom nodes，在EC2上调试工作流，发布工作流、环境至Amazon SageMaker。同时可以调用SageMaker资源、选中已发布的工作流进行推理验证 | http://EC2地址|
|
||||
|普通美术| 从该端口进入的界面，可以选择主美已发布的工作流，简单修改推理参数后，勾选“Prompt on AWS”后、调用Amazon SageMaker进行推理|当NumberOfInferencePorts填写3时，地址范围时，可访问的推理环境地址依次为：<ul><li>http://EC2地址:10001 </li><li>http://EC2地址:10002 </li><li>http://EC2地址:10003</li></ul>|
|
||||
|Senior Artist/ Staff of workflow management| Able to install new custom nodes, debug workflows on EC2, and deploy workflows and environments to Amazon SageMaker. You can also call SageMaker resources and select published workflows for inference validation | http://EC2 Address|
|
||||
|Junior Artists| From the interface accessed through this port, you can select the workflows published by the Art Director, simply modify the inference parameters, check 'Prompt on AWS', and then call Amazon SageMaker for inference. When NumberOfInferencePorts is set to 3, the address range and accessible inference environment addresses are sequentially as follows:"：<ul><li>http://EC2Address:10001 </li><li>http://EC2Address:10002 </li><li>http://EC2Address:10003</li></ul>|
|
||||
|
||||
!!! tip
|
||||
After the initial deployment, you need to wait for a while. If you open the link and see the message 'Comfy is Initializing or Starting,' it means that the backend is in the process of initializing ComfyUI. Please wait a bit and then refresh the page to confirm.
|
||||
|
||||
|
||||
|
||||
!!! tip "贴士"
|
||||
刚部署好贴士以后，需要稍作等待。如果打开链接后，看到提示“Comfy is Initializing or Starting”，表示后端在初始化ComfyUI过程中，请稍作等待，再次刷新页面确认。
|
||||
|
||||
### Step3: 在ComfyUI页面调试并创建一个可用的工作流。
|
||||
可以参考[这里](../user-guide/ComfyUI/inference.md)中的“工作流的调试”子章节部分
|
||||
|
||||
### Step 4: Deploy new Amazon SageMaker inference endpoint
|
||||
After successfully completing step 1, you need to deploy the required Amazon SageMaker inference nodes using API. Subsequent deployments of new ComfyUI workflow inferences will utilize the computational resources of these inference nodes.
|
||||
|
||||
The `ApiGatewayUrl` and `ApiGatewayUrlToken` required in the following API code can be found in the **Outputs** tab of the stack deployed successfully in step 1.
|
||||
|
||||
Please open any command-line interface capable of running code, such as Terminal on a local MacBook, and execute the following API code.
|
||||
|
||||
```
|
||||
curl --location ‘YourAPIURL/endpoints’ \
|
||||
--header ‘x-api-key: Your APIkey’ \
|
||||
--header ‘username: api’ \
|
||||
--header ‘Content-Type: application/json’ \
|
||||
--data-raw ‘{
|
||||
“workflow_name”:“Please fill the name of template you just released“,
|
||||
“endpoint_name”: “When you don't need to associate it with a workflow, you should fill in the name of the inference endpoint you want to create",
|
||||
“service_type”: “comfy”,
|
||||
“endpoint_type”: “Async”,
|
||||
“instance_type”: “instance type”,
|
||||
“initial_instance_count”: 1,
|
||||
“min_instance_number”: 1,
|
||||
“max_instance_number”: 2,
|
||||
“autoscaling_enabled”: true,
|
||||
“assign_to_roles”: “test”
|
||||
“assign_to_roles”: [ “test” ]
|
||||
}’
|
||||
```
|
||||
|
||||
!!! Important
|
||||
If your workflow is relatively complex, it's important to select asynchronous inference node types. Otherwise, you may encounter timeout issues due to the service's maximum wait time of 30 seconds for synchronous calls.
|
||||
|
||||
|
||||
|
||||
Delete corresponding Amazon SageMaker endpoint, can be executed as below:
|
||||
```
|
||||
curl --location --request DELETE 'https://please fill ApiGatewayUrl/endpoints' \
|
||||
--header 'username: api' \
|
||||
--header 'x-api-key: please type the ApiGatewayUrlToken' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{
|
||||
"endpoint_name_list": [
|
||||
"comfy-real-time-test-34"//type the name of the endpoint
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
!!! Important
|
||||
It's not recommended to directly delete endpoints from the SageMaker console as it can potentially lead to inconsistencies in data.
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 167 KiB After Width: | Height: | Size: 194 KiB |
|
|
@ -1,5 +1,6 @@
|
|||
| Date | Version |Changes |CloudFormation Template |
|
||||
|----------------------------|-------------------|-------------------|-------------------|
|
||||
| 2024/07/25 | V1.7.0| - **Decouple** workflow templates from environments, allowing multiple workflow templates to run on the same endpoint while ensuring both stability and ease of use. <br> - Support deploy Amazon SageMaker endpoint through **UI** <br> - Support **image** real-time synchronize <br> - Bugs fix | [V1.7.0](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.7.0/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
| 2024/06/28 | V1.6.1| - Support **automatic template rendering** in the frontend when a published template is selected. This facilitates subsequent parameter adjustments and inference based on the selected template. <br> - Bugs fix | [V1.6.1](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.6.1/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
| 2024/06/14 | V1.6.0 | - Support **ComfyUI** on SageMaker <br> - Support **ComfyUI template workflow** to release and inference <br> - Support Amazon SageMaker **G6 series** inference endpoint(s) <br> - Support for saving intermediate status of **Kohya_ss** trained model <br> - Support for visual evaluation after training **Kohya_ss** model <br> - Support image caption through **WD14** <br> - Bugs fix | [V1.6.0](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.6.0/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
| 2024/03/22 | V1.5.0 | - Support **webUI** V1.8.0 <br> - Support extension **ReActor for Stable Diffusion** <br> - Add **API Debugger** feature <br> - Support LoRa model training through **Kohya_ss** <br> - Improve customer experience on **Resource management** in webUI <br> - Optimize the method of customizing BYOC extensions, and support optimization startup <br> - Support **auto-scale** ability for real-time inference endpoints <br> - Support the **customizable range** for the numbers of instances during auto-scaling <br> - Optimize **installation time** by 67%, down to 4 minutes <br> - Upgrade Python version from 3.9 to 3.10 <br> - Deprecate support for **Dreambooth** <br> - Bugs fix | [V1.5.0](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.5.0/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
|
|
|
|||
|
|
@ -4,9 +4,9 @@ After successfully deploying the solution, you can open the native **ComfyUI** p
|
|||
|
||||
* Step 1: Connect to the EC2 that deploys ComfyUI frontend.
|
||||
* Step 2: Open the **Designer** link provided by the stack of solution, debug the new workflow locally (on the EC2 virtual machine), install the missing nodes, upload the required inference models, and ensure they can be successfully loaded and inferred locally (on the EC2 virtual machine).
|
||||
* Step 3: Release the workflow as a template.
|
||||
* Step 4: Create the endpoints needed for the workflow through API.
|
||||
* Step 5: On the ComfyUI inference page, select the released template, modify the inference parameters (for example: prompts) and models as needed, and perform inference on images/videos. This will utilize Amazon SageMaker resource.
|
||||
* Step 3: Repeat Step 2 as needed to **publish multiple workflow templates**.
|
||||
* Step 4: After debugging one (or more) workflow templates, package and **publish the current environment** with one click, and deploy a new Amazon SageMaker inference endpoint.
|
||||
* Step 5: On the ComfyUI inference page, select the published templates and, if needed, modify the inference prompts, inference models, and inference images/videos. This step will use Amazon SageMaker resources.
|
||||
|
||||
|
||||
## Step 1: Connect to the EC2 that deploys ComfyUI frontend
|
||||
|
|
@ -40,36 +40,32 @@ The steps for using the senior design version of ComfyUI are summarized as follo
|
|||
4. (Optional) If step 3 results in an error prompt, follow the instructions to resolve it. For example, if a missing model is indicated, download the model to the corresponding directory on the EC2 instance; if missing custom nodes are indicated, click **Manager** and then **Install Missing Custom Nodes** to install the missing nodes. After resolving the error, repeat step 3 to test again.
|
||||
|
||||
|
||||
5. When the workflow is completed and the generated results are displayed on the interface, it indicates that the workflow debugging has been successful.
|
||||
|
||||
## Step 3: Release workflow as new template
|
||||
Once the workflow can successfully infer images/videos locally (on the EC2 virtual machine), you can follow these steps to quickly release the debugged workflow as a template, making it convenient and stable for future inference calls through Amazon SageMaker.
|
||||
|
||||
1. Click **New Workflow** in the right navigation bar or the **plus sign** above the workflow list module on the right side.
|
||||
2. In the pop-up window, enter the name of the template to be published and click **OK**.
|
||||
|
||||
5. Once the workflow is complete and the generated results are displayed on the interface, it indicates that the workflow debugging has been successful.
|
||||
6. Click the plus sign above the **Template List** module in the right navigation bar. In the pop-up window, enter the name of the template to be published and the runtime environment to be bound, then click **OK**.
|
||||
|
||||
!!! tip
|
||||
The new template name must not exceed 20 characters in length, combining letters and numbers. It is case-sensitive. Additionally, the name must be unique within the same region. If it conflicts with an existing template name, an error message will be displayed.
|
||||
Please note that the new template name must not exceed 20 characters in length, should be a combination of letters and numbers, and is case-sensitive. Additionally, the name must be unique within the same region; if it conflicts with an existing template name, a creation error will be prompted. If the runtime environment is not yet published, you can return to modify the published templates after completing Step 4 to bind the correct runtime environment.
|
||||
|
||||
|
||||
3. During the workflow publishing process, no updates should be made on the ComfyUI frontend. Once the publishing is complete, a pop-up notification will confirm the successful publication.
|
||||
4. During publication, the workflow you are currently debugging will be temporarily saved. Each time you switch environments, the temporarily saved workflow corresponding to the current environment will be loaded first.
|
||||
7. During the workflow publishing process, no updates should be made on the ComfyUI frontend. This process generally takes about ten seconds. After the publication is complete, a pop-up message will appear on the frontend indicating that the publication is finished.
|
||||
8. Once the publishing is complete, refresh the page to see the newly published template in the workflow template list on the right navigation bar.
|
||||
|
||||
## Step 4: Deploy new inference endpoint for future inference of released workflow
|
||||
After completing the workflow releasing, you'll need to create a Amazon SageMaker inference endpoint to perform cloud-based inference based on the workflow:
|
||||
## Step 3: Repeat Step 2 as needed to publish multiple templates
|
||||
To ensure workflow stability and compatibility, and to maximize resource utilization, it is recommended that users continuously debug and publish multiple workflow templates within the ComfyUI default environment. Once these workflows are ready, the combined running environment should be published and bound to an Amazon SageMaker Inference Endpoint. This setup allows the published environment to stably run multiple templates while maximizing the resource utilization of the inference endpoint. Additionally, consider the environment size when publishing; a larger environment may result in longer cold start times during subsequent auto-scaling. Balancing the number of templates bound to a single environment is a trade-off decision users need to make.
|
||||
|
||||
1. You'll need to call the creation via API, referencing the "Deploying New Amazon SageMaker Inference Nodes" subsection in [this documentation](../../deployment/deployment_comfyui.md).
|
||||
|
||||
2. Once the inference endpoint is created and in the **InService** state, the published workflow will be ready for inference.
|
||||
## Step 4: Deploy new inference endpoint for future inference of released template
|
||||
After completing Step 3, users can package and publish the workflow runtime environment with one click and create an Amazon SageMaker Inference Endpoint for cloud-based inference based on the selected workflow template.
|
||||
|
||||
1. Click **New Environment** in the right navigation bar.
|
||||
2. In the pop-up dialog, enter the name for the environment to be published. In the *Endpoint Config* section, select the parameters for the Amazon SageMaker Inference Endpoint to be bound to this environment, including instance type, auto-scaling options, etc. Click **OK** when finished.
|
||||
3. The environment publishing process may take an uncertain amount of time, potentially up to 10+ minutes, depending on the size of the existing environment. During this time, the page will be locked to prevent any other operations that might disrupt the environment and cause the publishing process to fail.
|
||||
4. Once the creation is successful, refresh the page. In the **Template List** section, select the templates to be bound, click the pencil icon, choose the newly published environment in the pop-up dialog, and click **OK**.
|
||||
|
||||
## Step 5: Inference of released template
|
||||
In the view of **Designer** or **InferencePot**, you can easily perform inference based on a template using the following steps:
|
||||
In the view of **Designer** or **Junior Artists**, you can easily perform inference based on a template using the following steps:
|
||||
|
||||
1. Open the ComfyUI page and select a released template from the right-hand navigation bar. If in **Designer** view, also need to select the **Prompt on AWS** checkbox in the right-hand navigation bar.
|
||||
|
||||
2. The selected template will be automatically render in the ComfyUI page. Adjust the parameters as needed, and click **Queue Prompt** to submit the inference task.
|
||||
|
||||
3. Once the inference task is completed, the generated results will automatically be displayed on the page.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -79,8 +79,3 @@ You can open the **img2img tab** and use the original region along with the **Am
|
|||
4. Check inference result. Refresh the dropdown list of **Inference Job JDs**, check the topmost **Inference Job ID** that match the inference submission timestamp to review the inference result.
|
||||
|
||||
|
||||
### Continuous Inference Scenario
|
||||
|
||||
1. Following the **General Inference Scenario**, complete the parameter inputs and click **Generate on Cloud** to submit the initial inference task.
|
||||
2. Wait for the appearance of a new **Inference ID** in the right-side **Output** section.
|
||||
3. Once the new Inference ID appears, you can proceed to click **Generate on Cloud** again for the next inference task.
|
||||
|
|
|
|||
|
|
@ -2,19 +2,14 @@
|
|||
|
||||
部署时间：约 20 分钟
|
||||
|
||||
## 前提条件
|
||||
|
||||
|
||||
## 部署概述
|
||||
在亚马逊云科技上部署本解决方案（ComfyUI部分）主要包括以下过程：
|
||||
|
||||
- 步骤1：部署本解决方案中间件。
|
||||
- 步骤2：部署ComfyUI前端。
|
||||
- 步骤3: 在ComfyUI页面调试并创建一个可用的工作流。
|
||||
- 步骤4：根据创建的工作流部署新的Amazon SageMaker推理节点。
|
||||
|
||||
!!! tip "贴士"
|
||||
遇到部署问题时可以查看“常见问题解答”章节中ComfyUI相关部分。
|
||||
部署完成后，具体使用流程，请参考[ComfyUI用户手册](../user-guide/ComfyUI/inference.md)
|
||||
|
||||
|
||||
## 部署步骤
|
||||
### 步骤1: 部署解决方案中间件
|
||||
|
|
@ -27,6 +22,7 @@
|
|||
|
||||
|参数|说明|建议|
|
||||
|:-------------|:--------------|:--------------|
|
||||
|APIEndpointType|如需API调用，定义该API的类别，选项REGIONAL / PRIVATE / EDGE|默认Regional|
|
||||
|Bucket|填入一个有效的新的S3桶的名字（或之前部署的、用于本解决方案ComfyUI部分的S3桶名字）||
|
||||
|email|输入一个正确的电子邮件地址，以便接收将来的通知||
|
||||
|SdExtensionApiKey|请输入一个包含数字和字母组合的20个字符的字符串|默认为"09876543210987654321"|
|
||||
|
|
@ -70,54 +66,7 @@
|
|||
!!! tip "贴士"
|
||||
刚部署好贴士以后，需要稍作等待。如果打开链接后，看到提示“Comfy is Initializing or Starting”，表示后端在初始化ComfyUI过程中，请稍作等待，再次刷新页面确认。
|
||||
|
||||
### 步骤3: 在ComfyUI页面调试并创建一个可用的工作流。
|
||||
可以参考[这里](../user-guide/ComfyUI/inference.md)中的“工作流的调试”子章节部分
|
||||
|
||||
### 步骤4: 部署新的Amazon SageMaker推理节点
|
||||
在步骤1成功完成后，需要通过API方式部署所需的Amazon SageMaker推理节点。后续的新发布的ComfyUI工作流推理都将使用该推理节点的计算资源。
|
||||
|
||||
以下API代码中所需的ApiGatewayUrl及ApiGatewayUrlToken，可以在步骤1部署成功的堆栈**Outputs**标签页找到。
|
||||
|
||||
请打开任何可以运行代码的窗口，比如本地Macbook电脑的Terminal，运行如下API代码。
|
||||
|
||||
```
|
||||
curl --location 'api地址/endpoints' \
|
||||
--header 'x-api-key: 此处填写您的apikey' \
|
||||
--header 'username: api' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{
|
||||
“workflow_name”:“此处填写您创建的workflow名称“,
|
||||
“endpoint_name”: “当无需关联workflow时需要填写您要创建的推理端点名字",
|
||||
“service_type”: “comfy”,
|
||||
“endpoint_type”: “Async”,
|
||||
“instance_type”: “实例类型.ml.xxx”,
|
||||
“initial_instance_count”: 1,
|
||||
“min_instance_number”: 1,
|
||||
“max_instance_number”: 2,
|
||||
“autoscaling_enabled”: true,
|
||||
“assign_to_roles”: [ “test” ]
|
||||
}'
|
||||
```
|
||||
|
||||
!!! Important "注意"
|
||||
如果是相对复杂的workflow 注意选择异步推理节点类型，否则受限于service最长等待30s，会出现调用推理超时的情况。
|
||||
|
||||
|
||||
后续如需要删除Amazon SageMaker推理节点，可通过以下API完成。
|
||||
```
|
||||
curl --location --request DELETE 'https://此处填ApiGatewayUrl地址/endpoints' \
|
||||
--header 'username: api' \
|
||||
--header 'x-api-key: 此处填ApiGatewayUrlToken' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--data-raw '{
|
||||
"endpoint_name_list": [
|
||||
"comfy-real-time-test-34"//填要删除的endpoint名字
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
!!! Important "注意"
|
||||
不建议直接进去SageMaker console直接删除endpoint，容易造成数据不一致的隐患。
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 167 KiB After Width: | Height: | Size: 194 KiB |
|
|
@ -1,5 +1,6 @@
|
|||
| 日期 | 版本号 | 版本发布说明 | CloudFormation模版地址
|
||||
|----------------------------|--------|--------|--------|
|
||||
| 2024年07月25日 | V1.7.0| - 将工作流模版与运行环境**解耦**，以实现多个工作流模版可以在同一个Endpoint推理节点运行的效果，同时满足稳定性和易用性 <br> - 新增支持通过**UI**创建推理节点 <br> - 新增针对**上传图片**的实时云上同步 <br> - 修复若干已知问题 | [V1.7.0](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.7.0/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
| 2024年06月28日 | V1.6.1| - 新增支持已发布模版被选取时，前端**自动渲染模版**，以供后续基于此的简单参数调整推理 <br> - 修复若干已知问题 | [V1.6.1](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.6.1/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
| 2024年06月14日 | V1.6.0 | - 新增支持**ComfyUI** on SageMaker <br> - 新增支持云上**ComfyUI工作流模版**的发布和使用 <br> - 新增支持Amazon SageMaker **G6系列**推理节点 <br> - 新增**Kohya_ss**模型训练中间状态的保存 <br> - 新增支持**Kohya_ss**模型训练后的可视化评估 <br> - 新增支持**WD14**的图片自动打标 <br> - 修复若干已知问题 | [V1.6.0](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.6.0/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
| 2024年03月22日 | V1.5.0 | - 升级对**webUI**的版本支持至1.8.0 <br> - 新增支持换脸插件**ReActor for Stable Diffusion** <br> - 新增**API推理调试器功能** <br> - 新增支持**Kohya_ss**的LoRa模型训练 <br> - 优化**云上资源管理**的用户体验 <br> - 优化**BYOC自定义插件的方式**，并支持启动优化<br> - 支持实时推理节点的**弹性伸缩**功能 <br> - 支持所有实例**自定义实例数范围**，即便自动伸缩，也可以设置最小实例数，而不是 0 <br> - 优化中间件**部署时间**67%，降低至4分钟 <br> - 升级Python版本从3.9至3.10 <br> - webUI**资源管理**界面用户体验优化 <br> - 修复若干已知问题 | [V1.5.0](https://aws-gcr-solutions.s3.amazonaws.com/stable-diffusion-aws-extension-github-mainline/v1.5.0/custom-domain/Extension-for-Stable-Diffusion-on-AWS.template.json){:target="_blank"}
|
||||
|
|
|
|||
|
|
@ -1,13 +1,15 @@
|
|||
# 在comfyUI上推理
|
||||
|
||||
在成功部署解决方案后，您可以打开部署成功的堆栈所提供的**ComfyUI**原生页面，在工作流的调试、发布和推理，总结步骤如下：
|
||||
在成功部署解决方案后，您可以打开部署成功的堆栈所提供的**ComfyUI**的**Designer**端口页面，在工作流的调试、发布和推理，总结步骤如下：
|
||||
|
||||
* 步骤1: 连接部署了ComfyUI前端的EC2。
|
||||
* 步骤2: 在本方案提供的**Designer**链接页面，可进行新工作流的调试，安装缺失的节点并上传所需的推理模型，并能够成功在本地（虚拟机EC2）加载并推理成功。
|
||||
* 步骤3: 在发布该工作流为模版。
|
||||
* 步骤4: 创建工作流所需endpoint（通过API方式创建）。
|
||||
* 步骤2: 在本方案提供的**Designer**链接页面，可进行新工作流的调试，安装缺失的节点并上传所需的推理模型，并能够成功在本地（虚拟机EC2）加载并推理成功，发布此工作流为模版（template）。
|
||||
* 步骤3: 按需重复步骤2以发布多个工作流模版（template）。
|
||||
* 步骤4: 在调试完一个（或多个）工作流模版（template）后，一键打包发布当前环境，并配合部署新的Amazon SageMaker推理节点。
|
||||
* 步骤5: 在ComfyUI推理页面，选择已发布模版，按需修改推理词、推理模型并推理图片/视频。本步骤将会使用Amazon SageMaker资源。
|
||||
|
||||
!!! tip "贴士"
|
||||
用户可以创建一个或多个包含相近功能或插件组合的工作流模版（template）。调试完这些模版后，将当前的并集环境打包成一个运行环境（environment），并为其配置推理节点（Amazon SageMaker Endpoint）。这种方法确保了工作流的稳定性和兼容性。遇到部署问题时可以查看“常见问题解答”章节中ComfyUI相关部分。
|
||||
|
||||
|
||||
## 步骤1: 连接部署ComfyUI前端的虚拟机（EC2）
|
||||
|
|
@ -27,7 +29,7 @@ sudo journalctl -u comfy --no-pager -n 200 用于查看comfy运行日志最后20
|
|||
docker images -q | xargs docker rmi -f
|
||||
```
|
||||
|
||||
## 步骤2: 调试工作流
|
||||
## 步骤2: 调试工作流并发布为模版
|
||||
打开本方案部署后，提供的**Designer**链接，即为工作流调试员/主美视角的**ComfyUI**，您可以按单机版ComfyUI的使用方法来进行新的工作流的调试。有关模型的管理、定制节点的管理等可以通过连接部署ComfyUI的虚拟机（EC2）来进行。
|
||||
|
||||
|
||||
|
|
@ -41,33 +43,41 @@ docker images -q | xargs docker rmi -f
|
|||
|
||||
|
||||
5. 当工作流完成后，看到界面显示出生成结果，即表示该工作流的调试工作已经成功。
|
||||
|
||||
|
||||
## 步骤3: 发布工作流为新模版
|
||||
待工作流在本地（虚拟机EC2）已经可以成功推理图片/视频后，可以快速跟随以下步骤，将调试好的工作流发布成为模版，便于后续稳定、便利的调用推理。
|
||||
|
||||
1. 点击右侧导航栏中的**New Workflow** 或 右侧导航工作流列表模块上方加号。
|
||||
2. 在弹框中，填入待发布模版的名字，点击**确定**。
|
||||
6. 点击右侧导航栏中工作流模版列表模块上方加号，在弹框中填入待发布模版的名字，及需要绑定的运行环境，点击**OK**。
|
||||
|
||||
!!! tip "贴士"
|
||||
新模版命名长度不能超过20个字符，英文与数字结合，大小写敏感。同时该命名需要是同一个region内唯一，如果与现有模版名字冲突会提示创建错误。
|
||||
新模版命名长度不能超过20个字符，英文与数字结合，大小写敏感。同时该命名需要是同一个region内唯一，如果与现有模版名字冲突会提示创建错误。另外，此处的运行环境如还未发布，可以等到步骤4完成后，返回修改已发布的模版，以绑定正确的运行环境。
|
||||
|
||||
3. 工作流发布过程中，ComfyUI前端不可以有更新操作。在发布完成后会在前端有弹框提示发布完成。
|
||||
4. 发布时会将您页面当前调试时的workflow暂存，每次切换环境时会优先加载当前环境对应暂存的workflow。
|
||||
|
||||
|
||||
## 步骤4: 创建工作流在云上推理的推理端点
|
||||
待工作流发布完成以后，还需要创建推理端点来依据工作流进行云端推理。
|
||||
1. 需要通过api的方式调用创建，可以参考[这里](../../deployment/deployment_comfyui.md)中的“部署新的Amazon SageMaker推理节点”子章节部分。
|
||||
2. 待推理端点创建完成后，且为InService的状态后，已发布的工作流便处于可推理的状态。
|
||||
7. 工作流发布过程中，ComfyUI前端不可以有更新操作，该过程大致持续十几秒。在发布完成后会在前端有弹框提示发布完成。
|
||||
8. 发布完成，刷新页面即可在右侧导航栏的工作流模版列表看到新发布的模版。
|
||||
|
||||
## 步骤3: 按需重复步骤2以发布多个模版
|
||||
为确保工作流的稳定性和兼容性，以及最大化资源利用率，建议用户在ComfyUI的默认环境中持续调试并发布多个工作流模版（template）。在调试完成后，将当前的并集运行环境发布并绑定到一个Amazon SageMaker推理节点（Inference Endpoint）。这样发布的环境可以稳定运行多个绑定的模板，最大化推理节点的资源利用率。同时，需要注意发布环境的大小，如果环境过大，可能会在后续自动弹性伸缩时导致冷启动时间过长。用户需要在多个模板绑定一个环境的数量上做出权衡。
|
||||
|
||||
## 步骤4: 发布环境并创建推理节点
|
||||
待步骤3完成后，用户可以一键打包发布工作流运行环境，并创建推理端点来依据工作流进行云端推理。
|
||||
|
||||
1. 点击右侧导航栏中**New Environment**.
|
||||
2. 在弹框中填入待发布环境的名字。在*Endpoint Config*区域，选择为该环境绑定的Amazon SageMaker Inference Endpoint参数，包括机型、是否自动扩展等。完成后点击**OK**。
|
||||
|
||||
!!! Important "注意"
|
||||
如果是相对复杂的workflow 注意选择异步推理节点类型，否则受限于service最长等待30s，会出现调用推理超时的情况。另外，不建议直接进去SageMaker console直接删除endpoint，容易造成数据不一致的隐患。
|
||||
|
||||
3. 发布环境过程会基于现有环境大小等不定时间，可能长达10+分钟。等待期间，页面锁住，不可以进行其他操作，以防环境被破坏导致发布失败。
|
||||
4. 创建成功后，请刷新页面，在**Template List**区域选中需要绑定的模版，点击铅笔图标，在弹框中选择新发布环境，点击**OK**。
|
||||
|
||||
## 步骤5: 基于已发布工作流的推理
|
||||
在ComfyUI的推理页面，可以简单通过以下步骤完成基于模版的推理。
|
||||
|
||||
1. 打开ComfyUI的推理页面，在右侧导航栏选择一个已发布的模版。如果是调试环境，还需要选中右侧导航栏中“Prompt on AWS”复选框。
|
||||
1. 打开ComfyUI的推理页面，在右侧导航栏选择一个已发布的模版。如果是调试环境，还需要选中右侧导航栏中**Prompt on AWS**复选框。
|
||||
2. ComfyUI页面即会自动渲染该模版的工作流，按需调整参数，并点击**添加提示词队列（Queue Prompt）**，提交推理任务。
|
||||
3. 当推理任务完成后，生成结果会自动展示在页面。
|
||||
|
||||
|
||||
|
||||
|
||||
## 模型管理
|
||||
### 模型上传
|
||||
新模型需要上传到EC2，才能保证在模型调试阶段能够顺利调取。在EC2中上传模型，可以通过进入models目录下对应的模型类别子文件夹，通过直接拖拽或者wget + 模型下载地址的方式来实现。考虑到网速，推荐优先使用wget模型下载方式。
|
||||
|
|
|
|||
|
|
@ -1,91 +0,0 @@
|
|||
# 使用img2img进行云上推理
|
||||
|
||||
您可以打开**img2img**标签页，通过结合使用**img2img**原生区域及解决方案新增面板**Amazon SageMaker Inference**，实现调用云上资源的**img2img**推理工作。
|
||||
|
||||
|
||||
## img2img的使用方法
|
||||
### img2img不同标签功能的标准流程
|
||||
|
||||
1. 进入**img2img**标签页，**Amazon SageMaker Inference**面板。
|
||||
2. 输入推理所需参数。同于本地推理，您可以按需编辑**img2img**原生的推理参数，包括模型(stable diffusion checkpoint, VAE, extra networks:Lora,Hypernetworks, VAE等)，提示词，负提示词，取样参数，推理参数等。对于**img2img**，**sketch**，**inpaint**，**inpaint sketch**和**inpaint upload**，都可以按照原生方式进行图片上传和修饰。
|
||||
|
||||
!!! Important "提示"
|
||||
选择的模型文件需要通过云上资源管理章节介绍的方式进行推理模型上传到云上，才能使用该模型进行云上推理
|
||||
|
||||
3. 选择云上推理所需要的模型。点击**Stable Diffusion Checkpoint Used on Cloud**列表选择需要云上推理的模型, 则会触发右上角**Generate**按钮变为**Generate on Cloud**。
|
||||
|
||||
|
||||
!!! Important "提示"
|
||||
此项为必选项。
|
||||
|
||||
4. 参数设置完成后，点击**Generate on Cloud**。
|
||||
6. 查看推理结果。通过点击**Inference Job Histories: Time-Type-Status-UUID**右侧的刷新按钮进行下拉列表刷新，查看最上方的、符合推理提交时间戳的**Inference Job ID**。img2img标签页右上方的**Output**区域会显示推理的结果，包括图片，提示词以及推理的参数等。在此基础上，可以点击**Save**或者**Send to extras**等，进行后续工作流。
|
||||
> **补充：** 列表按照推理时间倒序排列，即最近的推理任务排在最上方。每条记录的命名格式为**推理时间->任务类型（txt2img/img2img/interrogate_clip/interrogate_deepbooru）->推理状态（succeed/in progress/fail） ->inference id**。
|
||||
|
||||
|
||||
### img2img标签操作示例
|
||||
|
||||
1. 上传原始图片到**img2img**标签并输入提示词，点击**Generate on Cloud**。
|
||||
2. 选择对应的**Inference Job ID**，推理结果会展示在 **Output** 区域。
|
||||
|
||||
### Sketch标签操作示例
|
||||
|
||||
1. 启动**Stable Diffusion WebUI**时在命令行带上‘--gradio-img2img-tool color-sketch’，上传白板底图到**Sketch标签**。
|
||||
2. 通过画笔，绘制下面的草图并输入提示词，点击**Generate on Cloud**。
|
||||
|
||||
|
||||
3. 选择对应的**Inference Job ID**，推理图片会展示在**Output**区域。
|
||||
|
||||
|
||||
|
||||
### Inpaint标签操作示例
|
||||
|
||||
1. 上传原始图片到**Inpaint**标签。
|
||||
2. 通过画笔建立掩膜并输入提示词，点击**Generate on Cloud**。
|
||||
|
||||
|
||||
3. 选择对应的**Inference Job ID**，推理图片会展示在**Output**区域。
|
||||
|
||||
|
||||
|
||||
### Inpaint Sketch标签操作示例
|
||||
|
||||
1. 启动**Stable Diffusion WebUI**的时候带上‘--gradio-img2img-tool color-sketch’，然后上传原始图片到**Inpaint Sketch**标签，并输入提示词。
|
||||
2. 通过画笔建立掩膜，并点击**Generate on Cloud**。
|
||||
|
||||
|
||||
3. 选择对应的**Inference Job ID**，推理图片会展示在**Output**区域。
|
||||
|
||||
|
||||
|
||||
### Inpaint Upload标签操作示例
|
||||
|
||||
1. 上传原始图片和mask图片到**Inpaint Upload**标签并输入提示词，点击**Generate on Cloud**。
|
||||
|
||||
|
||||
2. 选择对应的**Inference Job ID**，推理图片会展示在**Output**区域。
|
||||
|
||||
|
||||
|
||||
|
||||
### Interrogate clip/deepbooru功能的使用
|
||||
|
||||
1. 进入**img2img**标签页，展开**Amazon SageMaker Inference**面板。
|
||||
2. Interrogate只需要在**img2img**标签页把图片上传即可。
|
||||
|
||||
|
||||
3. 点击**Interrogate CLIP on cloud**或**Interrogate DeepBooru on cloud**。
|
||||
4. 查看推理结果。通过点击**Inference Job JDs**右侧的刷新按钮进行下拉列表刷新，查看最上方的、符合推理提交时间戳的**Inference Job ID**。
|
||||
img2img标签的提示词区域就能看到结果。
|
||||
|
||||
|
||||
### 连续使用场景
|
||||
|
||||
1. 按**通用场景**使用流程，完成参数录入，并点击**Generate on Cloud**提交第一次推理任务。
|
||||
2. 等待右侧**Output**部分出现了新的**inference id**。
|
||||
3. 在新的**Inference Job ID**出现后，便可再次点击**Generate on Cloud**进行下一次推理。
|
||||
|
||||
|
||||
|
||||
## Inference Job
|
||||
Inference Job下拉列表默认列出最近10条推理任务，命名格式遵循 Time-Type-Status-Uid。如果需要列出账户历史上所有推理任务，只需选中**Show All**。如果需要应用筛选器，缩小显示列表，可以勾选**Advanced Inference Job filter**，页面下方会自动增加本解决方案提供的筛选器维度，用户可以按需选择。
|
||||
|
|
@ -81,13 +81,7 @@
|
|||
img2img标签的提示词区域就能看到结果。
|
||||
|
||||
|
||||
### 连续使用场景
|
||||
|
||||
1. 按**通用场景**使用流程，完成参数录入，并点击**Generate on Cloud**提交第一次推理任务。
|
||||
2. 等待右侧**Output**部分出现了新的**inference id**。
|
||||
3. 在新的**Inference Job ID**出现后，便可再次点击**Generate on Cloud**进行下一次推理。
|
||||
|
||||
|
||||
|
||||
## Inference Job
|
||||
Inference Job下拉列表默认列出最近10条推理任务，命名格式遵循 Time-Type-Status-Uid。如果需要列出账户历史上所有推理任务，只需选中**Show All**。如果需要应用筛选器，缩小显示列表，可以勾选**Advanced Inference Job filter**，页面下方会自动增加本解决方案提供的筛选器维度，用户可以按需选择。
|
||||
|
|
|
|||
Loading…
Reference in New Issue