s3cmd | Official s3cmd repo -- Command line tool | Cloud Storage library

by s3tools Python Version: v2.3.0 License: GPL-2.0

X-Ray Key Features Code Snippets(3)Community Discussions(10)Vulnerabilities Install Support

kandi X-RAY | s3cmd Summary

s3cmd is a Python library typically used in Storage, Cloud Storage, Amazon S3 applications. s3cmd has no bugs, it has no vulnerabilities, it has build file available, it has a Strong Copyleft License and it has high support. You can download it from GitHub.

S3cmd (s3cmd) is a free command line tool and client for uploading, retrieving and managing data in Amazon S3 and other cloud storage service providers that use the S3 protocol, such as Google Cloud Storage or DreamHost DreamObjects. It is best suited for power users who are familiar with command line programs. It is also ideal for batch scripts and automated backup to S3, triggered from cron, etc. S3cmd is written in Python. It's an open source project available under GNU Public License v2 (GPLv2) and is free for both commercial and private use. You will only have to pay Amazon for using their storage.

Support

Quality

Security

License

Reuse

Support

s3cmd has a highly active ecosystem.

It has 4180 star(s) with 888 fork(s). There are 111 watchers for this library.

It had no major release in the last 12 months.

There are 241 open issues and 617 have been closed. On average issues are closed in 963 days. There are 44 open pull requests and 0 closed requests.

It has a negative sentiment in the developer community.

The latest version of s3cmd is v2.3.0

Quality

s3cmd has 0 bugs and 0 code smells.

Security

s3cmd has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

s3cmd code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

s3cmd is licensed under the GPL-2.0 License. This license is Strong Copyleft.

Strong Copyleft licenses enforce sharing, and you can use them when creating open source projects.

Reuse

s3cmd releases are available to install and integrate.

Build file is available. You can build the component from source.

Installation instructions are not available. Examples and code snippets are available.

s3cmd saves you 2908 person hours of effort in developing the same functionality from scratch.

It has 6282 lines of code, 412 functions and 25 files.

It has high code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed s3cmd and discovered the below as its top functions. This is intended to give you an instant insight into s3cmd implemented functionality, and help decide if they suit your requirements.

Fetch local list of files
Get the md5 of a file
Get the md5 of a hard link
Creates a new instance of the instance
Fetch remote files
Convert date to Python datetime
Convert a date string to Python object
Get list of objects in a bucket
Parse config file
Modify a CloudFront distribution
Create a distribution from a list of buckets
Display information about the distribution
Return an HTTPSConnection instance
Return the expiration information for a bucket
Adds a permission to the account
Send a message to the server
Parse the XML tree
Set the accesslog for a bucket
Creates a bucket
Create a directory and all parents
List invalidations
Send a HTTP request
Store an object in S3
Read the AWS credential file
Compare two files
Invalidate an S3 bucket

Get all kandi verified functions for this library.

s3cmd Key Features

No Key Features are available at this moment for s3cmd.

s3cmd Examples and Code Snippets

GitHub Action to Sync S3 Bucket using s3cmd ,Usage,workflow.yml

Shell

Lines of Code : 25

License : Permissive (MIT)

Copy

name: Publish

on:
  push:
    branches:
    - master

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v2
    - uses: ThiagoAnunciacao/s3cmd-sync-action@v0.2.5
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_A

s3cmd Example

Python

Lines of Code : 6

License : No License

Copy

export AWS_ACCESS_KEY_ID="JSHSHEUESKSK65"
export AWS_SECRET_ACCESS_KEY="kskjskjsAEERERERlslkhdjhhrhkjdASKJSKJS666789"

pip install s3cmd

s3cmd --requester-pays ls s3://sdms-clif-2007/

s3cmd --requester-pays --recursive --human du s3://sdms-clif-200

Arch Linux Repository,Requirements

Shell

Lines of Code : 3

License : Permissive (MIT)

Copy

[default]
access_key = 
secret_key =

Community Discussions

Trending Discussions on s3cmd

How to move a large folder to s3?

Why below EBS deployement failing?

AWS: how to see the user1 created bucket by user2

Alpine 3.11 diff: unrecognized option: c BusyBox v1.31.1 () multi-call binary

How to preceed the output generated by "exec &" with a time/Date in linux bash scripts?

Is there a way to backup all a bucket with s3cmd

How can I list all empty folders with s3cmd

apt-get install can't find pinned versions within Dockerfile

How do I fix incorrect public url on s3cmd response?

Download zip file from s3 using s3cmd issue

QUESTION

How to move a large folder to s3?

Asked 2022-Apr-11 at 10:32

I'm looking for the most suitable tool to transfer 600 GB of media from a Linux server to s3, so far I found s3 sync and s3cmd , but they do not work in background mode, tell me the best option?

...

ANSWER

Answered 2022-Apr-11 at 10:32

You can run your command in tmux, or nohup. This way the AWS CLI command will persist after you logout. There are other ways, but I personally find tmux being my preferred choice.

Source https://stackoverflow.com/questions/71825111

QUESTION

Why below EBS deployement failing?

Asked 2022-Mar-22 at 11:11

I am running the this on corretto 8 running on amazon linux 2/2.3.13.. it gets the error as "An error occurred during the execution of command [app-deploy] - [CheckProcFileForJavaApllication]. Stop ruuning the command. Error: There is no profile and .jar file at the root level of your source bundle.

The commands while deployment are prestep

...

ANSWER

Answered 2022-Mar-22 at 11:11

linux2 is vastly different from linux1 and is incompatible with it. Yo may probably need to fully re-design your application to work with linux2. Please check AWS docs:

Migrating your Elastic Beanstalk Linux application to Amazon Linux 2

Source https://stackoverflow.com/questions/71568384

QUESTION

AWS: how to see the user1 created bucket by user2

Asked 2022-Mar-09 at 12:31

Step 1: User1 created the test-bucket & uploaded couple of files

Step 2: below policy is created and attached to the bucket

...

ANSWER

Answered 2022-Mar-09 at 07:57

If both IAM Users are in the same AWS Account

The s3cmd ls command will list all buckets in the AWS Account. It uses the s3:ListAllMyBuckets permission. Permissions to run this command are not granted by a Bucket Policy because it lists all buckets.

If you want to grant permission to use s3cmd ls, then add this permission to the IAM User:

Source https://stackoverflow.com/questions/71405573

QUESTION

Alpine 3.11 diff: unrecognized option: c BusyBox v1.31.1 () multi-call binary

Asked 2022-Feb-28 at 17:13

I am using an alpine 3.11 to build my image, everything goes well during the build the dockefile is here below :

...

ANSWER

Answered 2022-Feb-28 at 16:56

Well I just had to add the diffutils package to the list after installing it everything works well

Source https://stackoverflow.com/questions/71297838

QUESTION

How to preceed the output generated by "exec &" with a time/Date in linux bash scripts?

Asked 2021-Sep-02 at 13:21

I have the following script file that writes files to s3 from a local file system:

...

ANSWER

Answered 2021-Sep-02 at 13:21

Rather than redirect output on each line, you can wrap the body of the script in a single block and then handle the output of the entire block in one place. You can then process that output with the stream editor sed. For example:

Source https://stackoverflow.com/questions/69025336

QUESTION

Is there a way to backup all a bucket with s3cmd

Asked 2021-Jul-05 at 07:52

I can upload, get, delete or list files in a bucket with s3cmd:

...

ANSWER

Answered 2021-Jul-05 at 07:52

You can use --recursive with get command. Example:

Source https://stackoverflow.com/questions/68228189

QUESTION

How can I list all empty folders with s3cmd

Asked 2021-Jul-04 at 00:34

I have uploaded 5k+ folders, each of which should have one file and one subfolder (this subfolder then holds various files) to DigitalOcean Spaces (S3 storage).

It looks like some of the uploads failed (long story). Is there a way to list all empty folders using s3cmd?

...

ANSWER

Answered 2021-Jul-04 at 00:34

My comments here are about Amazon S3, but should apply equally to DigitalOcean Spaces.

'Folders' do not actually exist in S3. For example, you could upload a file to invoices/january/inv1.txt and S3 will magically create the invoices and january folders. Then, if you delete that object, those folders will magically disappear. Thus, folders automatically appear when objects are 'in' them.

It is possible to create an empty folder by creating a zero-length object with the same name as a path. For example, creating a zero-length object with a key of invoices/ will force a folder to appear even when it is empty (because it isn't actually empty!). This is how the S3 management console creates a folder when people click the "Create Folder" button.

So, when you ask how to "list all empty folders", it really depends on how those folders were originally created, or if they were actually created at all! It is quite likely that the folders were never created in the first place.

If your goal is to fix a failed upload, you could use the s3cmd sync function that can re-upload objects, but is smart enough to only copy files that are not present in the destination, or have changed.

Source https://stackoverflow.com/questions/68237161

QUESTION

apt-get install can't find pinned versions within Dockerfile

Asked 2021-Jan-27 at 10:56

During the building of a simple Dockerfile, the pinned version of apt-get install is never found and gives me the following output:

...

ANSWER

Answered 2021-Jan-27 at 10:56

The ruby1.9.1 and ruby1.9.1-dev are not available for ubuntu:18.04.
You can find here the list of supported packages by ubuntu version.

You can also read this askubuntu question

Source https://stackoverflow.com/questions/65915832

QUESTION

How do I fix incorrect public url on s3cmd response?

Asked 2020-Nov-20 at 10:03

I am uploading a file to my S3 instance using s3cmd. When I run s3cmd put test_py3.csv.gz s3://my.bucket/path/ --acl-public after the upload it gives the public url as http://my.bucket.my.bucket/path/test_py3.csv.gz instead of http://my.bucket.s3.amazonaws.com/path/test_py3.csv.gz

I have tested and confirmed that http://my.bucket.s3.amazonaws.com/path/test_py3.csv.gz works fine, the only issue seems to be that s3cmd is adding my bucket a second time instead of s3.amazonaws.com when it is displaying the public url string.

How can I make it display the correct public url?

...

ANSWER

Answered 2020-Nov-20 at 10:03

This is apparantly not possible with s3cmd. It needs to be done with awscli and the command aws s3 cp test_py3.csv.gz s3://my.bucket/path/ --acl public-read

Source https://stackoverflow.com/questions/64916854

QUESTION

Download zip file from s3 using s3cmd issue

Asked 2020-Nov-16 at 10:50

I have two issues I need help with on bash, linux and s3cmd.

First, I'm running into linux permission issue. I am trying to download zip files from a s3 bucket using s3cmd with following command in a bash script.sh:

...

ANSWER

Answered 2020-Nov-14 at 00:41

Fist of all ask 1 question at a time.

For the first one you can simply change the permission with chown like :

Source https://stackoverflow.com/questions/64828331

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install s3cmd

You can download it from GitHub.
You can use s3cmd like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: