Modify FlowMatch Scale Noise #8678
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| @@ -852,7 +852,7 @@ def __call__( | |||
| # 4. Prepare timesteps | |||
| timesteps, num_inference_steps = retrieve_timesteps(self.scheduler, num_inference_steps, device, timesteps) | |||
| timesteps, num_inference_steps = self.get_timesteps(num_inference_steps, strength, device) | |||
| latent_timestep = timesteps[:1].repeat(batch_size * num_inference_steps) | |||
| latent_timestep = timesteps[:1].repeat(batch_size * num_images_per_prompt) | |||
There was a problem hiding this comment.
😱😱😱😱
thanks for the fix!
|
I copied the almost the same logic than the |
| @@ -129,7 +129,31 @@ def scale_noise( | |||
| if self.step_index is None: | |||
There was a problem hiding this comment.
I think we can also remove this here
Let's remove it and run the example for img2img to see if the results match from main and this branch
|
Slow test fails for me with this branch but also with main: This branch: Main: But I can tell that at least the slices are the same in both, probably something with my env? I also get the same results than the initial comment, the pear has a tiny difference but mostly the same (they're identical with the latest changes and from the initial commit for each branch though)
Should I look into what's making the difference between them? Or this is negligible? Edit: I forgot, probably the visual difference is because of the fix in the pipeline |
|
the test look good! since you got same results from main and branch |
I don't think the fix in the pipeline will affect the output, but I can't tell the difference, and the test slice is the same so I think we are fine |
|
ohh wait a second .... let's not merge just yet |
|
where is the difference in the image? |
|
you can open each one in a different tab and circle between them. The pear is a little bigger to the left and the stem is also a little higher |
pear.mp4 |
| if self.step_index is None: | ||
| self._init_step_index(timestep) | ||
| # Make sure sigmas and timesteps have the same device and dtype as original_samples | ||
| sigmas = self.sigmas.to(device=sample.device, dtype=sample.dtype) |
There was a problem hiding this comment.
I think the difference comes from casting simgas dtype
do you think the quality is worse when we cast the sigmas to fp16? (this branch)
There was a problem hiding this comment.
no, IMO is not worse, probably if I didn't use an example with a single fruit against a black background, I wouldn’t have even noticed the difference.
What does this PR do?
This PR modifies
FlowMatchEulerDiscreteSchedulerso it can work with differential diffusion, also fixesStableDiffusion3Img2ImgPipelineto work with the new change.I just made it analogous the other schedulers, but I found that there's a tiny change in the image to image generation, so I don't know if there's something wrong with what I did.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@yiyixuxu