java-sdk | Baidu AI Open Platform Java SDK | SDK library

You need to use hadoop-aws version 3.2.0 for spark 3. In --packages specifying hadoop-aws library is enough to read files from S3.

The third party gradle plugin to use for jOOQ code generation isn't ready for jOOQ 3.16 yet. A fix is being discussed here: https://github.com/etiennestuder/gradle-jooq-plugin/pull/208

You need to add dependency for the jar required by your gradle version. Trying adding this dependency and refreshing your gradle.

implementation group: 'com.amazonaws', name: 'aws-java-sdk-core', version: '1.11.948'

or

I managed to get the error to go away. All I needed to do it all the dependency config for the related tasks (in build.gradle):

You have declared 'org.springframework.boot' as the parent module of both modules. So if some jars and artifacts like 'com.amazonaws' do not exist in 'org.springframework.boot', they won't be resolved in your project. These dependencies are not announced in 'Spring' module in your project and whatever you have declared in it, can be found in 'org.springframework.boot', then resolved. If you do not declare a 'version' tag in your pom, I guess the version of the parent (here 2.6.1) will be considered for your module version.

You should look at using the AWS SDK for Java V2. You are referencing V1, not the newest Amazon S3 Java API. If you are not familiar with V2, start here:

Get started with the AWS SDK for Java 2.x

To perform Async operations via the Amazon S3 Java API, you use S3AsyncClient.

Now to learn how to upload an object using this client, see this code example:

2021.3 IDE version has updated the version of the bundled Maven to 3.8.1. In this version, Maven blocks the access to http repositories by default. Before that, Maven itself has moved from using the http repositories.

So now one needs to explicitly configure Maven to allow http repositories if they are used in the project. E.g. in settings.xml add a mirror to your http repository that allows HTTP:

This might help: The message group ID is used for ordering of SQS messages in groups.

So, if you have multiple group IDs like 1 and 2, then within those groups, the SQS messages will always be in order. The ordering will be preserved relative to the group and not the queue as a whole.

This can be used in cases like, if you are sending out events for different user details updates, you would want to group the data for a particular user in order, and at the same time you don't care about the ordering of the queue as a whole. You can then set the group ID as the unique user ID in this case.

If there is no use-case of creating groups, and there is only one group and you want the message order to be preserved as a whole, just use the same groupID for all messages.

more info here: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagegroupid-property.html

The containers in your pods must use an AWS SDK version that supports assuming an IAM role via an OIDC web identity token file.

Check if you meet the minimum requirement for boto3 is 1.9.220, for botocore is 1.12.200.