aws-sdk-go | AWS SDK for the Go programming language | AWS library

Scanning the table returns the map of items and you want to convert map into the struct so in order to convert the list of maps to the struct you want to use attributevalue.UnmarshalListOfMaps under aws-sdk-go-v2. In the previous version it was in the dynamodbattribute but they decided to change the package in the v2.

Not sure if the easiest way, however you can iterate over your objects list of interest - https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html, and aggregate the size locally.

I found the answer in GoDoc, I just had to change a config setting and use NewSessionWithOptions:

The code on the page you linked is not 100% correct. They actually link the "complete example" at the bottom of the page, which has slightly different code:

https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/gov2/cloudwatch/PutEvent/PutEventv2.go#L108-L117

Likely you are using an old version of Go. Overlapping method sets are allowed since Go 1.14, and the code compiles on the Go Playground.

Quoting from Go 1.14 release log:

Per the overlapping interfaces proposal, Go 1.14 now permits embedding of interfaces with overlapping method sets: methods from an embedded interface may have the same names and identical signatures as methods already present in the (embedding) interface. This solves problems that typically (but not exclusively) occur with diamond-shaped embedding graphs. Explicitly declared methods in an interface must remain unique, as before.

If you get a compile-time error for the code you posted, that suggests you're using a Go prior to 1.14. Urgently update! Please note that only the last 2 major versions are supported (currently 1.17 and 1.16). You using a version like 1.13 is a major risk!

no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send failed, Get \"http://169.254.169.254/latest/meta-data/iam/security-credentials/\": dial tcp 169.254.169.254:80: connect: host is down

Your snippet of code is attempting to use the EC2 instance meta-data service to read an IAM role to use. For that to work, you need to be able to communicate with it, and the role must be attached to the instance.

Is your Go service running on an EC2 instance? If not, that would explain your error. If it is, make sure the process or container has appropriate network access (eg: network namespace) to communicate with 169.254.169.254.

Based on the comments, my suggestion is to add $HOME/go/bin to the path and use the default GOPATH.

Go mod depends on the bin directory inside the GOPATH. It installs new packages there. The go binary itself can actually reside somewhere else. If you follow these install instruction https://go.dev/doc/install, go itself will actually be in /usr/local/go but the GOPATH is still $HOME/go.

I would also recommend, not involving apt in this at all. This looks like trouble in the form of conflicts with different installations.

It seems that sdk v2 offers a func wrapper to satisfy logging.logger:

The short answer is: You can run your object lambda function in a VPC as long as you allow it to route to s3-object-lambda..amazonaws.com through the internet, e.g. through a NAT gateway. You were on the right track and basically figured it out in your question already.

The S3 gateway interface endpoint is necessary to enable download of the input object.

When writing the result, the request goes to s3-object-lambda, which is technically a different service than S3 (at least on network level). AWS currently doesn't provide an interface endpoint for s3-object-lambda and the S3 gateway endpoint doesn't cover it either (which can be verified by comparing the IP address WriteGetObjectResponse request goes to and the routes created by the gateway endpoint).

So the only way is to route WriteGetObjectResponse requests via opened access to the internet. For future reference, one way to set this up is with a NAT gateway. Quoting AWS docs:

• The NAT gateway must be in a public subnet with a route table that routes internet traffic to an internet gateway.
• Your instance must be in a private subnet with a route table that routes internet traffic to the NAT gateway.
• Check that there are no other route table entries that route all or part of the internet traffic to another device instead of the NAT gateway.

In other words:

• Provision a public NAT Gateway in a public subnet and allocate it an elastic IP
• Make sure the public subnet has an internet gateway and the default route (0.0.0.0/0) points to it.
• Set up a default route (0.0.0.0/0) from the subnet hosting your lambda and point it to the NAT Gateway.

You're right that a NAT Gateway is priced by the hour, unfortunately, and you need one per subnet.

In theory, you could at least limit the egress with a security group to IP addresses of the s3-object-lambda service, but I'm not aware these IP ranges are published anywhere.