localstack | longer actively maintained ( see README | Serverless library

I confirm I can reproduce the issue, and indeed this is due to vpc. Just to create your SG in a default VPC, you can remove the vpc_id = var.vpc_id. Also its good practice to add egress:

Probably because the following:

API Gateway v2 is only supported in pro version of LocalStack. So if you want to use it, please buy the pro version.

The S3Hook takes aws_conn_id as parameter. You simply need to define the connection once for your airflow installation and then you will be able to use that connection in your hook.

Default name of the connection is aws_default (see https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html#default-connection-ids). Simply create the connection first (or edit if it is already there) - either via Airflow UI or via environment variable or via Secret Backends

Here is the documentation describing all the options you can use:

https://airflow.apache.org/docs/apache-airflow/stable/howto/connection.html

As described in https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/connections/aws.html - login in the connection is used as AWS_ACCESS_KEY_ID and password is used as AWS_SECRET_ACCESS_KEY, but AWS connection in the UI of Airflow is customized and it shows hints and options via custom fields, so you can easily start with the UI.

Once you have the connection defined, S3 Hook will read the credentials stored in the connection it uses (so by default: aws_default). You can also define multiple AWS connections with different IDs and pass those connection ids as aws_conn_id parameter when you create hoook.

If you are trying to access an s3 bucket with localstack via an aws api; you would need to withPathStyleEnabled flag turned on.

for eg:

Summary: To use NoSQL Workbench with LocalStack, set the region to localhost in your code and Terraform config, and fix the resulting validation error (saying there isn't a localhost region) by setting skip_region_validation to true in the aws provider block in the Terraform config.

The problem is disclosed in the screenshot above:

NoSQL Workbench uses the localhost region.

When using DynamoDB Local, it appears the region is ignored, so this quirk is hidden (i.e. there is a mismatch between the region in the Terraform file and my code on the one hand and NoSQL Workbench on the other, but it doesn't matter with DyanmoDB Local).

But with LocalStack region is not ignored, so the problem popped up.

I wouldn't have written this up except for one more quirk that took a while to figure out. When I updated the Terraform configuration thus:

Try adding the following dependency:

I think your ports are incorrect. From docs:

A major (breaking) change has been merged in PR #2905 - starting with releases after v0.11.5, all services are now exposed via the edge service (port 4566) only! Please update your client configurations to use this new endpoint.

By default, when you list objects in a bucket, S3 collapses any objects with the same prefix before the separator as one entry, and marks this collection as PRE in the output. This lets you treat the contents of S3 buckets as a traditional filesystem with directories and files in those directories.

You can either use aws s3 ls --recursive to list all objects in a bucket, or query the object directly by doing something like aws s3 ls s3://bucket-name/path/to/object to view that single object, if it exists.