I ran across an issue with Terraform where I couldn't destroy an Amazon S3 bucket created in a region other than the one provided at the prompt.
To get around this I had to configure an AWS provider for each region I wanted to add a bucket to, and use the alias property to reference the provider from the s3 resource.
Without the region based providers security errors were showing up when trying to manipulate the bucket in different regions.
For example, Create two buckets in different regions:
resource "aws_s3_bucket" "s3_bucket_1" {
bucket = "com-ginocoates-testbucket1"
region = "us-east-1"
}
resource "aws_s3_bucket" "s3_bucket_2" {
bucket = "com-ginocoates-testbucket2"
region = "us-east-2"
}
Plan and apply them...
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform plan
provider.aws.region
The region where AWS operations will take place. Examples
are us-east-1, us-west-2, etc.
Default: us-east-1
Enter a value:
+ aws_s3_bucket.s3_bucket_1
acceleration_status: "<computed>"
acl: "private"
arn: "<computed>"
bucket: "com-ginocoates-testbucket1"
force_destroy: "false"
hosted_zone_id: "<computed>"
region: "us-east-1"
request_payer: "<computed>"
versioning.#: "<computed>"
website_domain: "<computed>"
website_endpoint: "<computed>"
+ aws_s3_bucket.s3_bucket_2
acceleration_status: "<computed>"
acl: "private"
arn: "<computed>"
bucket: "com-ginocoates-testbucket2"
force_destroy: "false"
hosted_zone_id: "<computed>"
region: "us-east-2"
request_payer: "<computed>"
versioning.#: "<computed>"
website_domain: "<computed>"
website_endpoint: "<computed>"
Plan: 2 to add, 0 to change, 0 to destroy.
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform apply
provider.aws.region
The region where AWS operations will take place. Examples
are us-east-1, us-west-2, etc.
Default: us-east-1
Enter a value:
aws_s3_bucket.s3_bucket_2: Creating...
acceleration_status: "" => "<computed>"
acl: "" => "private"
arn: "" => "<computed>"
bucket: "" => "com-ginocoates-testbucket2"
force_destroy: "" => "false"
hosted_zone_id: "" => "<computed>"
region: "" => "us-east-2"
request_payer: "" => "<computed>"
versioning.#: "" => "<computed>"
website_domain: "" => "<computed>"
website_endpoint: "" => "<computed>"
aws_s3_bucket.s3_bucket_1: Creating...
acceleration_status: "" => "<computed>"
acl: "" => "private"
arn: "" => "<computed>"
bucket: "" => "com-ginocoates-testbucket1"
force_destroy: "" => "false"
hosted_zone_id: "" => "<computed>"
region: "" => "us-east-1"
request_payer: "" => "<computed>"
versioning.#: "" => "<computed>"
website_domain: "" => "<computed>"
website_endpoint: "" => "<computed>"
aws_s3_bucket.s3_bucket_1: Still creating... (10s elapsed)
aws_s3_bucket.s3_bucket_1: Creation complete
Error applying plan:
1 error(s) occurred:
* aws_s3_bucket.s3_bucket_2: Error putting S3 ACL: TemporaryRedirect: Please re-send this request to the specified temporary endpoint. Continue to use the original request endpoint for future requests.
status code: 307, request id: 9D4A2839B33A45F4
We get an error, however, the two buckets were created successfully. We can query s3 to verify...
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ aws s3 ls | grep bucket
2017-01-03 14:16:01 com-ginocoates-testbucket1
2017-01-03 14:15:59 com-ginocoates-testbucket2
Lets try to destroy the buckets...
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform destroy
Do you really want to destroy?
Terraform will delete all your managed infrastructure.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
provider.aws.region
The region where AWS operations will take place. Examples
are us-east-1, us-west-2, etc.
Default: us-east-1
Enter a value:
aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
Error refreshing state: 1 error(s) occurred:
* aws_s3_bucket.s3_bucket_2: error reading S3 bucket "com-ginocoates-testbucket2": Forbidden: Forbidden
status code: 403, request id: E081D8D9A009E943
Forbidden error, since the AWS provider is initialized with us-east-1, we can't delete the s3 bucket that was created in us-east-2. What I think this means is that the default provider is locked in to the region it was initialized with and can't call the s3 API in another region to refresh the state of bucket 2.
Lets try to fix it. Configure the AWS provider so we can create buckets in different regions. We can do this with alias properties.
provider "aws" {
alias = "us-east-1"
region = "us-east-1"
}
provider "aws" {
alias = "us-east-2"
region = "us-east-2"
}
resource "aws_s3_bucket" "s3_bucket_1" {
bucket = "com-ginocoates-testbucket1"
region = "us-east-1"
provider = "aws.us-east-1"
}
resource "aws_s3_bucket" "s3_bucket_2" {
bucket = "com-ginocoates-testbucket2"
region = "us-east-2"
provider = "aws.us-east-2"
}
Lets plan these changes...
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform plan
aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
No changes. Infrastructure is up-to-date. This means that Terraform
could not detect any differences between your configuration and
the real physical resources that exist. As a result, Terraform
doesn't need to do anything.'
So TF now sees the state as correct. Can we now destroy the s3 buckets successfully in each region?
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform destroy
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform destroy
Do you really want to destroy?
Terraform will delete all your managed infrastructure.
There is no undo. Only 'yes' will be accepted to confirm.
Enter a value: yes
aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
aws_s3_bucket.s3_bucket_1: Destroying...
aws_s3_bucket.s3_bucket_2: Destroying...
aws_s3_bucket.s3_bucket_2: Destruction complete
aws_s3_bucket.s3_bucket_1: Destruction complete
Destroy complete! Resources: 2 destroyed.
Yes! Now is it possible to move bucket2 to us-east-1? Lets first recreate them.
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform apply
aws_s3_bucket.s3_bucket_1: Creating...
acceleration_status: "" => "<computed>"
acl: "" => "private"
arn: "" => "<computed>"
bucket: "" => "com-ginocoates-testbucket1"
force_destroy: "" => "false"
hosted_zone_id: "" => "<computed>"
region: "" => "us-east-1"
request_payer: "" => "<computed>"
versioning.#: "" => "<computed>"
website_domain: "" => "<computed>"
website_endpoint: "" => "<computed>"
aws_s3_bucket.s3_bucket_2: Creating...
acceleration_status: "" => "<computed>"
acl: "" => "private"
arn: "" => "<computed>"
bucket: "" => "com-ginocoates-testbucket2"
force_destroy: "" => "false"
hosted_zone_id: "" => "<computed>"
region: "" => "us-east-2"
request_payer: "" => "<computed>"
versioning.#: "" => "<computed>"
website_domain: "" => "<computed>"
website_endpoint: "" => "<computed>"
aws_s3_bucket.s3_bucket_1: Still creating... (10s elapsed)
aws_s3_bucket.s3_bucket_2: Still creating... (10s elapsed)
aws_s3_bucket.s3_bucket_1: Creation complete
aws_s3_bucket.s3_bucket_2: Creation complete
Apply complete! Resources: 2 added, 0 changed, 0 destroyed.
Lets use change testbucket to to use the provider us-east-1. In theory this should destroy the bucket and create a new one in the desired region. Fingers crossed....
provider "aws" {
alias = "us-east-1"
region = "us-east-1"
}
provider "aws" {
alias = "us-east-2"
region = "us-east-2"
}
resource "aws_s3_bucket" "s3_bucket_1" {
bucket = "com-ginocoates-testbucket1"
region = "us-east-1"
provider = "aws.us-east-1"
}
resource "aws_s3_bucket" "s3_bucket_2" {
bucket = "com-ginocoates-testbucket2"
region = "us-east-1"
provider = "aws.us-east-1"
}
Plan the changes...
gino@gino-MacBookPro:~/sourcecode/s3-multiregion$ terraform plan
aws_s3_bucket.s3_bucket_1: Refreshing state... (ID: com-ginocoates-testbucket1)
aws_s3_bucket.s3_bucket_2: Refreshing state... (ID: com-ginocoates-testbucket2)
Error refreshing state: 1 error(s) occurred:
* aws_s3_bucket.s3_bucket_2: error reading S3 bucket "com-ginocoates-testbucket2": Forbidden: Forbidden
status code: 403, request id: 83855B6D6780B04D
Oops, error. The state has the bucket in us-east-2, but the tf files reference us-east-1. So the provider is initialized with the us-east-1 endpoint when we run terraform plan. But it can't refresh the state of the bucket in us-east-2 as its configured to use a different AWS endpoint. Kinda makes sense.
So, with TF we can't destroy a bucket in one region and recreate it in another with this setup. Something to watch out for!
To achieve the move we'd have to remove bucket2 from our tf files, apply to allow TF to destroy it, then add it back with the desired region, and apply these changes.
Learn Terraform Here....