s4cmd | Super S3 command line tool

 by   bloomreach Python Version: Current License: Apache-2.0

kandi X-RAY | s4cmd Summary

kandi X-RAY | s4cmd Summary


Super S3 command line tool

            Top functions reviewed by kandi - BETA

            kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
            Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of s4cmd
            Get all kandi verified functions for this library.

            s4cmd Key Features

            No Key Features are available at this moment for s4cmd.

            s4cmd Examples and Code Snippets

            Pythondot img1Lines of Code : 1dot img1License : Permissive (Apache-2.0)
            copy iconCopy
            pip install instaclone
            Instaclone,Running tests
            Pythondot img2Lines of Code : 1dot img2License : Permissive (Apache-2.0)
            copy iconCopy
            $ TEST_BUCKET=my-s3-bucket tests/run.sh

            Community Discussions


            Multipart upload to S3 with hash verification
            Asked 2019-Aug-20 at 09:13

            I am looking for a command line tool or a Python library which allows uploading big files to S3, with hash verification.

            There is an AWS article explaining how it can be done automatically by supplying a content-md5 header.

            Yet, it is not clear which command line tools do or do not do this:

            • rclone's documentation states that

              files uploaded with multipart upload don’t have an MD5SUM.

            • s3cmd doesn't say anything about this, but it supports md5 for the sync feature

            • s4cmd has a whole paragraph in the manual, but it's still not clear if an upload is actually verified

            • boto3 / s3transfer's upload_file() method doesn't really say anything

            Do you have information about any of these tools, or some other tool or Python library or boto3 snippet which handles big file uploads to s3 with the reliability of rsync?



            Answered 2017-May-06 at 06:41

            Why not use aws cli? It has support for md5 verification, even for multipart uploads. See this. Both upload-part and put-object commands of s3api support content verification using hash.

            And s4cmd does verify uploads using custom content header containing md5.

            As for Etag calculation mentioned in S4cmd doc, see this.

            Source https://stackoverflow.com/questions/43794838


            Does s4cmd support signature version 4 because i am unable to upload files to s3 bucket (LONDON)
            Asked 2017-Jul-06 at 08:51

            I am trying to upload files to s3 bucket(LONDON) i.e. eu-west-2. S4cmd is not working.

            s4cmd put /home/username/Documents/file-1.json s3://[BUCKETNAME]/file-1.json

            error when i run this command is : -



            Answered 2017-Jul-06 at 08:51

            The aws s3 cp command in the AWS Command-Line Interface (CLI) uses multi-part upload to fully utilize available bandwidth, so it should give you pretty much the best speed possible.

            Source https://stackoverflow.com/questions/44883168

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network


            No vulnerabilities reported

            Install s4cmd

            No Installation instructions are available at this moment for s4cmd.Refer to component home page for details.


            For feature suggestions, bugs create an issue on GitHub
            If you have any questions vist the community on GitHub, Stack Overflow.
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries