(Re)run terragrunt run-all destroy even if some resources tore down successfully while others failed

[karlschriek]

Describe the bug

I am currently evaluating terragrunt's run-all capabilities and am getting unstuck on the terragrunt run-all destroy, in particular in the case of failure. I am not able to simply rerun this command if some modules destroyed correctly, but others failed. Let's say for example that terragrunt run-all apply generates the following dependency graph:

Group 1
- Module /root/myproject/base

Group 2
- Module /root/myproject/module1
- Module /root/myproject/module2

Group 3
- Module /root/myproject/module1_extension

Above both "module1" and "module2" depend on "base"; "module1_extension" depends on "module1".

In this case terragrunt run-all destroy will destroy all the "Group 3" modules, then the "Group 2" modules, then the "Group 1" modules. In the case where it successfully tears down both "Group 3" and "Group 2", but fails to fully remove "Group 1" I was hoping to be able to just again run terragrunt run-all destroy, but this does not work, erroring out with:

ERRO[0004] Module /root/myproject/module1_extension has finished with an error: /root/myproject/module1/terragrunt.hcl is a dependency of /root/myproject/module1_extension/terragrunt.hcl but detected no outputs. Either the target module has not been applied yet, or the module has no outputs. If this is expected, set the skip_outputs flag to true on the dependency block.  prefix=[/root/myproject/module1_extension]

Steps To Reproduce

Steps to reproduce the behavior, code snippets and examples which can be used to reproduce the issue.

Create a project structure as above and then destroy module1, module2 and module1_extension. If you now try to run "terragrunt run-all destroy" you will get the above error. (You can also just apply "module1" to begin with and then "run-all destroy")

Expected behavior

To be clear, in this specific example only the "base" project still needs to be torn down (everything else is gone). I could tear it down now with --terragrunt-include-dir *base, but since errors in destroy steps are quite common I would expect terragrunt to be able to deal with repeating the command to destroy all. Since destroy happens sequentially (in a reverse order to apply), terragrunt should know which states have been torn down and which haven't and should always be able to retry the ones that are still there (since, unless you specified --terragrunt-ignore-dependency-errors, it would not have gone on and torn down modules that anything that hasn't been torn down yet depends on).

EDIT: I am not entirely sure about the "unless you specified --terragrunt-ignore-dependency-errors" part. Would terragrunt go ahead and destroy "module1" even if "module1_extension" failed to destroy correctly with --terragrunt-ignore-dependency-errors set? Because if it still respects that dependency then just setting that flag is probably the solution to this post (although somewhat messy and full of irrelevant warnings).

Nice to haves

• Terminal output
• Screenshots

Versions

• Terragrunt version: 0.58.10
• OpenTofu/Terraform version: tf 1.8.3
• Environment details (Ubuntu 20.04, Windows 10, etc.):

Additional context

Add any other context about the problem here.

[denis256]

Hello,
Terragrunt is simply the runner of Terraform and doesn't have transactional capabilities to execute all or nothing, once dependency was destroyed, outputs reading will not work.

It can be also used --terragrunt-strict-include and specify path to specific depdencnies to use them in destroy
https://terragrunt.gruntwork.io/docs/reference/cli-options/#terragrunt-strict-include

[karlschriek]

But there are no dependencies that are needed for the destroy that doesn't work Let's say Module B depends on Module A. I cannot apply Module B before applying module A. I also cannot destory Module A before destroying Module B. That makes sense. What doesn't make sense is that I cannot (using the run-all directive) destroy Module A after I already destroyed Module B.

…

On Tue, 11 Jun 2024, 21:42 Denis O, ***@***.***> wrote: Hello, Terragrunt is simply the runner of Terraform and doesn't have transactional capabilities to execute all or nothing, once dependency was destroyed, outputs reading will not work. It can be also used --terragrunt-strict-include and specify path to specific depdencnies to use them in destroy https://terragrunt.gruntwork.io/docs/reference/cli-options/#terragrunt-strict-include — Reply to this email directly, view it on GitHub <#3183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGBE4OFP3EDAVAPKVOCZZKDZG5HK7AVCNFSM6AAAAABI2MJWQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRRGQ4DKMZVGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[denis256]

Hello,
in the current implementation, Terrgrunt "doesn't know" what was already destroyed and if the module should be skipped or not, so it is trying to do destroy each time.

[karlschriek]

Yes, that is clear. I am pointing out that that is very problematic in a non-interactive setting where you want to be able to robust tear down everything, with automatic retries etc. As it is right now if even just one module tears down correctly and the next one fails you will not be able to rerun the destroy

…

On Wed, 12 Jun 2024, 12:56 Denis O, ***@***.***> wrote: Hello, in the current implementation, Terrgrunt "doesn't know" what was already destroyed and if the module should be skipped or not, so it is trying to do destroy each time. — Reply to this email directly, view it on GitHub <#3183 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGBE4OAK5JWVCIWKFQ7LCXLZHASNLAVCNFSM6AAAAABI2MJWQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNRSG4YDSMRRHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>