zip-bucket | zips files in a Google Cloud Storage bucket | Cloud Storage library
kandi X-RAY | zip-bucket Summary
kandi X-RAY | zip-bucket Summary
zips files in a Google Cloud Storage [tm] bucket
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of zip-bucket
zip-bucket Key Features
zip-bucket Examples and Code Snippets
Community Discussions
Trending Discussions on zip-bucket
QUESTION
Project currently maintains S3 bucket which holds a large zip size 1.5 GB containing .xpt and .sas7dbat files. Unzipped file size is 20 GB.
Trying to unzip file and push the same folder structure to S3
Following code works for a small zip files but fails for large Zip file (1.5GB) :
...ANSWER
Answered 2018-May-11 at 16:00As described in this AWS Lambda Limits link:
But there are limits that AWS Lambda imposes that include, for example, the size of your deployment package or the amount of memory your Lambda function is allocated per invocation.
Here, the issue you are having is because of "amount of memory Lambda function is allocated per invocation" needed. Unfortunately, Lambda is not an applicable solution for this case. You need to go with EC2 approach.
When your overall memory requirement is high, I don't think Lambda is great solution. I am not about how the specified file types work, but in general read/processing large files use chunked approach to avoid large memory requirements. Whether chunked approach works or not depends on your business requirement.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install zip-bucket
Pass the storage object when setting up zipBucket for usage.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page