NPU Adaption for FLUX

[leisuzz]

What does this PR do?

Implement Flash Attention for NPU machine
Add cleaning memory function for NPU machine

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline?
• Did you read our philosophy doc (important for complex PRs)?
• Was this discussed/approved via a GitHub issue or the forum? Please add a link to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[leisuzz]

@yiyixuxu @sayakpaul Please let me know if there is any improvement for this PR, and thanks for your help!

[yiyixuxu]

thanks!

[yiyixuxu]

can you run make style and make fix-copies to the CI would pass?

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[leisuzz]

@yiyixuxu I've fixed the code quality issue, please approve the workflow. Thanks

[leisuzz]

@yiyixuxu Sorry about my mistake, could you please approve the workflow? Thanks

[yiyixuxu]

hi @leisuzz
if you installed our dev environment, you can run make style and make fix-copeis from the terminal that will automatically adjust the style for you so the CI will pass

see here https://huggingface.co/docs/diffusers/en/conceptual/contribution#how-to-open-a-pr

[leisuzz]

@yiyixuxu Thanks for the information!

[leisuzz]

Hi @yiyixuxu ,
Sorry for the trouble, I think there is something wrong with the ruff in my env as there is no code quality in my end. But I think I've fix it manually. Please approve the workflow. Thanks for your help

[leisuzz]

@yiyixuxu
Could you please redo the approval of the workflow? Thanks for your help and sorry for all the troubles!

[leisuzz]

Hi @yiyixuxu ,
There was a copyright / consistency issue with "def fuse_qkv_projections" as it has to match models.unets.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections. Does that mean I have to change both of these at the same time? For now, please accept this workflows as fuse_qkv_projections can be added later in another PR.

[leisuzz]

@yiyixuxu @sayakpaul Could you please approve the workflow? Thanks for your help!

[sayakpaul]

Failing tests are unrelated.

[a-r-r-o-w]

@yiyixuxu I'm not completely sure why we merged this PR with the following changes:

I believe we should default to just the normal SDPA processors otherwise there is inconsistencies introduced. Users can already set custom attention processors, no?

[yiyixuxu]

@a-r-r-o-w
yeah we should revert this change - it was my fault, I realized we should not do that here https://github.com/huggingface/diffusers/pull/10465/files