403 Forbidden when using private AWS ECR as HelmRepository

[alexandermarston]

As discussed in Slack here, I am having some problems successfully using an OCI HelmRepository with an AWS Private ECR repository.

I have the following resources created:

apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: HelmRepository
metadata:
  namespace: default
  name: db-toolbox
spec:
  interval: 1h0m0s
  provider: aws
  type: oci
  url: oci://000000.dkr.ecr.eu-west-1.amazonaws.com/db-toolbox

apiVersion: helm.toolkit.fluxcd.io/v2beta1
kind: HelmRelease
metadata:
  name: db-toolbox
  namespace: default
spec:
  releaseName: db-toolbox
  chart:
    spec:
      chart: db-toolbox
      sourceRef:
        kind: HelmRepository
        name: db-toolbox
        namespace: default
      version: "1.3.5"

Unfortunately, the source-controller is unable to fetch this HelmChart and is complaining with a 403.

➜  ~ k get helmchart
NAME                 CHART        VERSION   SOURCE KIND      SOURCE NAME   AGE   READY   STATUS
default-db-toolbox   db-toolbox   1.3.5     HelmRepository   db-toolbox    12m   False   chart pull error: failed to download chart for remote reference: pulling from host 000000.dkr.ecr.eu-west-1.amazonaws.com failed with status code [manifests 1.3.5]: 403 Forbidden

I believe I've ruled out a permissions issue as if I assume the role attached to the source-controller SA locally I am able to fetch the chart and if I attach the source-controller SA to a test aws-cli pod I am able to do the same too.

Flux version as follows:

➜  ~ flux version
flux: v0.38.2
helm-controller: v0.28.1
kustomize-controller: v0.32.0
notification-controller: v0.30.2
source-controller: v0.33.0

Any advice would be appreciated.

[stefanprodan]

The issue is that your HelmRepository URL is wrong, it should contain the URL to the repository instead of the chart, like this: url: oci://000000.dkr.ecr.eu-west-1.amazonaws.com.

[alexandermarston]

The issue is that your HelmRepository URL is wrong, it should contain the URL to the repository instead of the chart, like this: url: oci://000000.dkr.ecr.eu-west-1.amazonaws.com.

That's solved the issue - thank you for your quick help!

[stefanprodan]

@darkowlzz @hiddeco could we add the OCI artifact full URL to the error message, sadly ECR only mentions the domain name so users can't tell that the chart URL contains the chart name twice.