This guide is for platform operators running Chainloop on-prem in AWS. If your users want to bring their own storage (OCI registry, their own S3 bucket, Azure Blob, etc.) instead, see the CAS backend concepts page.
How It Works
- Single shared bucket per Chainloop installation. There is no per-org bucket to provision or rotate.
- Storage is provisioned automatically for every organization while managed CAS is enabled — no per-org action from you or your users. Each org gets:
- one S3 Access Point on the shared bucket,
- one IAM managed policy, used as a session policy to scope credentials to that access point,
- one storage backend in Chainloop, named
chainloop-cloud-storageand marked as managed.
- Chainloop reaches the access point through short-lived, per-request
sts:AssumeRolesessions scoped to a single organization. - No long-lived AWS credentials are stored. Chainloop persists a small record per org in its secrets backend holding only the access point ARN, the region, the tenant role ARN and the per-org session policy ARN — no access keys.
- Every object is keyed under
<org-uuid>/sha256:<digest>inside the bucket, and the key prefix is derived from the authenticated org on each request rather than read from stored configuration. - Encryption at rest is delegated to a customer-managed KMS key on the bucket (
SSE-KMS).
Provisioning is currently one-way: Chainloop creates access points, session policies and objects, but never deletes them. Removing storage for a deleted or downgraded organization is a manual operation today.
Authentication Model
S3 Access Points have no credentials of their own. Authentication is ordinary AWS IAM — the access point is a resource-side scoping primitive, not an identity. Getting this model right is what most of the setup below is about. Two properties drive everything:- A request must be allowed by every layer it passes through. Effective permissions are the intersection of the tenant role’s identity policy, the per-org session policy, and the resource policies — not their union.
- A request through an access point is authorized twice, once against the access point ARN and once against the bucket ARN. AWS grants the access only if both contexts allow it. This is why the bucket policy must delegate to access points: without a bucket-context
Allow, requests fail even when every access point and session policy is perfect.
How Pods Obtain AWS Credentials
Chainloop supports the two standard EKS mechanisms for giving a pod an AWS identity, and you pick one for your installation:- EKS Pod Identity (recommended) — the workload role trusts the
pods.eks.amazonaws.comservice principal, and you bind service accounts to it with pod identity associations. - IRSA — the workload role trusts your cluster’s OIDC provider, and each pod exchanges a projected service-account token via
sts:AssumeRoleWithWebIdentity.
Which identity performs which call matters when reading logs:
- The backend worker provisions: it creates access points, writes their resource policies, and creates the per-org session policies.
- The cas and controlplane pods read and write objects. They only ever assume the tenant role.
- The tenant role creates nothing. It is the principal named in every access point policy, and the role whose temporary sessions are narrowed per org.
aws:userid to equal <tenant-role-id>:cas-<org-uuid> — a session name Chainloop derives from the authenticated request, not from stored configuration. A tampered record pointing org A at org B’s access point still assumes a session named for org A, and is denied.
Prerequisites
You will provision the following resources in your AWS account before enabling the feature in Chainloop. We recommend managing them with Terraform or OpenTofu.- A single S3 bucket for the installation
- A customer-managed KMS key for
SSE-KMSon the bucket - A tenant IAM role — the role Chainloop workloads assume per-request to reach the access points
- Workload identities (EKS Pod Identity or IRSA) attached to the Chainloop
backend,controlplane, andcaspods so they can call AWS APIs
Resource Naming
Chainloop derives access point and session policy names — they are not configurable. Both include a slug taken frombackend.environment, and the bucket and IAM policies you write below must match it:
<org-uuid-hex> is the organization UUID with hyphens stripped (32 characters); <12-hex> is its last 12 characters.
Throughout the policies below, substitute [env] with prod or canary to match the environment you will configure. Setting backend.environment is mandatory in production: if you leave it unset, Chainloop creates chainloop-dev-* access points that your bucket policy — scoped to chainloop-prod-* — will reject.
1. Create the S3 Bucket
Create one bucket in the region you want to operate from. Block all public access. Enable default encryption with the KMS key you create in the next step. Attach this bucket policy exactly as shown, substituting only[bucketName], [region], [accountId], and [env]. It is the AWS-documented delegation pattern for access points: the bucket defers every access decision to the access point policies, which Chainloop generates per organization with a per-tenant aws:userid guard. Any request that does not arrive through one of your installation’s access points is denied by default.
Deny statement alongside the delegation Allow — never instead of it. Narrow that Deny to object actions rather than s3:*: a blanket s3:* deny conditioned on the absence of an access point also denies bucket administration, including PutBucketPolicy, which leaves account-root access as the only way back in.
2. Create the KMS Key
Create a customer-managed KMS key in the same region as the bucket and set it as the bucket’s default encryption key. The key policy must admit the tenant role, either directly or by delegating to IAM for the account (a statement granting the account root, so identity-based policies can grant key use). An identity-based grant alone is not enough — a KMS key policy cannot be overridden by an IAM policy, so a key that does not admit the tenant role fails every upload and download with a KMS error. The tenant role is the only identity that touches objects; the backend workload identity only manages access points and needs no key access. Chainloop’s per-org session policies grantkms:GenerateDataKey and kms:Decrypt, matching SSE-KMS on PutObject and GetObject.
3. Create the Tenant IAM Role
This is the role the Chainloop workloads assume per-request via STS. Note its name — you will pass it to the chart asbaseRoleName.
Trust policy — allow the backend and controlplane workload identities to assume it. Grant sts:TagSession alongside sts:AssumeRole: workload sessions minted by EKS can carry session tags, and the assume call fails if propagating them is not permitted.
4. Configure Workload Identities
The Chainloop pods need AWS credentials to call AWS APIs. We recommend EKS Pod Identity; IRSA works equivalently. You need two workload identity roles: a controlplane workload role (shared by the controlplane and cas pods) and a backend workload role. Each one needs a trust policy that lets the pod’s Kubernetes service account assume it, plus the permission policy shown below.Trust policy
The trust policy is what bootstraps the pod’s credentials. If it doesn’t allow the service account, every AWS call fails withAccessDenied before any tenant sts:AssumeRole is attempted — so getting this right is a prerequisite for everything else. The shape depends on the mechanism:
- EKS Pod Identity
- IRSA
The role trusts the EKS service principal. Which service accounts may use it is expressed as an association (see Enable in Chainloop EE), not in the trust policy:
Permission policies
Controlplane workload role — used by the controlplane and cas pods on the data path. Needs to callsts:AssumeRole against the tenant role:
aws:userid condition in every access point policy), then creates each org’s access point, writes its resource policy, and creates its session policy.
s3:TagResource is required even though Chainloop never calls the TagResource API: it tags each access point as part of the CreateAccessPoint call, and AWS requires both actions to allow tagging on create. Without it, provisioning fails with AccessDenied on CreateAccessPoint.
s3:DeleteAccessPoint and iam:DeletePolicy are not exercised today — provisioning is one-way — but granting them now means teardown support will not require a policy change.
The IAM “1500 customer-managed policies per account” service quota applies because Chainloop creates one managed policy per organization. Request a quota increase if you expect to scale past it.
Enable in Chainloop EE
In yourchainloop-ee Helm chart values.yaml, set backend.environment and configure backend.managedCas:
chainloop-ee values.yaml
chainloop-ee values.yaml
aws_eks_pod_identity_association per (namespace, service account) pair instead — including one binding the cas service account to the controlplane workload role, matching the IRSA annotation above.
Apply the values and roll out the chart.
Restricting Access Points to a VPC
By default the per-org access points are internet-facing (access is still governed entirely by the policies above). To make them reachable only from inside a VPC, set:chainloop-ee values.yaml
- Existing access points are not migrated. The network origin is fixed at creation, and reconciliation treats an already-existing access point as done. Only orgs provisioned after you set
vpcIdget VPC-scoped access points. - An invalid VPC ID stalls provisioning. The value is not validated up front; a wrong ID turns into a per-org
CreateAccessPointfailure that blocks the reconciler from progressing to the remaining orgs.
Verify
- Create a new organization in the Chainloop UI. Open Storage Backends for that org — you should see a default backend named
chainloop-cloud-storageprovided by Chainloop.
- Confirm the access point exists in the AWS console under S3 → Access Points, named
chainloop-[env]-<org-uuid-hex>. If it isn’t there yet, give it a moment and check the backend logs — provisioning runs in both the backend and its worker, so select by label to avoid reading just one pod: - As a user of that org, run an attestation or upload an artifact:
- Inspect the bucket — the object should be under the org’s prefix, encrypted with your KMS key:
Troubleshooting
Read any chained STS error inside-out — the innermost operation is the one that actually failed. AHeadObject error wrapping an AssumeRole error wrapping AssumeRoleWithWebIdentity is a credential-bootstrap failure, not an S3 policy problem.
When you hit a bare 403 Forbidden with no error code or message, that is HeadObject: it is an HTTP HEAD request, so S3 cannot return an error body and cannot tell you which layer denied it. Re-run the same access as a request that can return a body. Both of these name the denying layer in one line — use the access point ARN as --bucket:
