Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TargetGroupBindings can now manipulate target groups from different aws accounts #3691

Open
wants to merge 23 commits into
base: main
Choose a base branch
from

Conversation

marcosdiez
Copy link
Contributor

@marcosdiez marcosdiez commented May 13, 2024

Issue #3634

Description

Now, every time a TargetGroupBinding has the alb.ingress.kubernetes.io/IamRoleArnToAssume (and optionally alb.ingress.kubernetes.io/AssumeRoleExternalId) annotations, we assume it before interacting to it (actually, the assume role operation is cached).

In real life, I changed:

  • targetgroupbinding_validator
  • targetgroupbinding_mutator
  • target manager (which has RegisterTargets / DeregisterTargets / ListTargets

This way we can easily manipulate target groups from target group bindings for every single AWS account in the world, as long as there is a role we can assume. In order to prevent the confused deputy problem, external id is supported.

The way this was implemented was actually way simpler than initially imagined:

the service.elbv2Client object now supports a new method called AssumeRole which returns service.elbv2Client on the specified role. The catch is that if AssumeRole is called with a blank role, the object returns itself and life goes on.

In other words, every time we call some method from elbv2Client from something related to target groups, we also call AssumeRole and that's basically it.

Caching the assumed role was also trivial, a simple map that is stored on cloud.go

I decided to have cloud.go manage the caching becuase in the future it could assume role for other objects if needed and all the assume role logic would be in the same place.

The only unfortunate change that needed to be made is that aws.Cloud became services.Cloud else golang would complain about circular dependency :(

This is what the logs look like:


{"level":"info","ts":"2024-05-13T03:24:31Z","msg":"awsCloud","method":"GetAssumedRoleELBV2","AssumeRoleArn":"arn:aws:iam::7550XXXXXXXX:role/alb-controller-policy-to-impersonate","externalId":"XXXXXX"}
{"level":"info","ts":"2024-05-13T03:24:33Z","msg":"registering endpoints using the targetGroup's vpcID vpc-00XXXXXXXXXXXXXXX which is different from the cluster's vpcID vpc-04XXXXXXXXXXXXXXX"}
{"level":"info","ts":"2024-05-13T03:24:33Z","msg":"registering targets","arn":"arn:aws:elasticloadbalancing:us-east-1:7550XXXXXXXX:targetgroup/helloworld-8080-a8054/5741da332459b560","targets":[{"AvailabilityZone":"all","Id":"10.244.0.22","Port":8080}]}
{"level":"info","ts":"2024-05-13T03:24:34Z","msg":"registered targets","arn":"arn:aws:elasticloadbalancing:us-east-1:7550XXXXXXXX:targetgroup/helloworld-8080-a8054/5741da332459b560"}

Documentation and tests were properly updated in this PR

Checklist

  • Added tests that cover your change (if possible)
  • Added/modified documentation as required (such as the README.md, or the docs directory)
  • Manually tested
  • Made sure the title of the PR is a good description that can go into the release notes

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

  • Backfilled missing tests for code in same general area 🎉
  • Refactored something and made the world a better place 🌟

image

A working amd64 container with this code is available here: marcosdiez/aws-load-balancer-controller:v2.7.2-m2

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels May 13, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @marcosdiez. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 13, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: marcosdiez
Once this PR has been reviewed and has the lgtm label, please assign johngmyers for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@marcosdiez marcosdiez changed the title [WIP] TargetGroupBindings can now manipulate target groups from different aws accounts TargetGroupBindings can now manipulate target groups from different aws accounts May 13, 2024
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 13, 2024
@marcosdiez
Copy link
Contributor Author

marcosdiez commented May 13, 2024

@shraddhabang I believe my code is ready. Feel free to review at will! Thank you.

Also, if you don't want the CRDs to be changed, here:
126c9c8

they are untouched.

@k8s-ci-robot k8s-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 13, 2024
@shraddhabang
Copy link
Collaborator

Thank you @marcosdiez for this PR. This looks excellent.
I will take a look and test it out. And would also ask our team to review it. Please note that we will have to go through our formal security process to review this. We will add this to our next minor release v2.9.0.
Please feel free to reach out to us.

@shraddhabang
Copy link
Collaborator

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels May 13, 2024
@marcosdiez marcosdiez force-pushed the CI-14222_manipulate_target_groups_from_different_aws_accounts_v3 branch from 70267e1 to bb3c617 Compare May 14, 2024 20:38
@oliviassss oliviassss added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label May 17, 2024
@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 21, 2024
@marcosdiez marcosdiez force-pushed the CI-14222_manipulate_target_groups_from_different_aws_accounts_v3 branch from 7b5b35f to 85bbf7f Compare May 21, 2024 17:03
@k8s-ci-robot k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants