aws-lambda-redshift-loader | Amazon Redshift Database Loader implemented in AWS Lambda | Cloud Functions library

by awslabs JavaScript Version: v2.7.8 License: Non-SPDX

X-Ray Key Features Code Snippets Community Discussions(2)Vulnerabilities Install Support

kandi X-RAY | aws-lambda-redshift-loader Summary

aws-lambda-redshift-loader is a JavaScript library typically used in Serverless, Cloud Functions, Amazon S3 applications. aws-lambda-redshift-loader has no bugs, it has no vulnerabilities and it has low support. However aws-lambda-redshift-loader has a Non-SPDX License. You can download it from GitHub.

With this AWS Lambda function, it’s never been easier to get file data into Amazon Redshift. You simply drop files into pre-configured locations on Amazon S3, and this function automatically loads into your Amazon Redshift clusters. For automated delivery of streaming data to S3 and into to Redshift, please consider using [Amazon Kinesis Firehose] instead of this function.

Support

Quality

Security

License

Reuse

Support

aws-lambda-redshift-loader has a low active ecosystem.

It has 588 star(s) with 172 fork(s). There are 50 watchers for this library.

It had no major release in the last 12 months.

There are 53 open issues and 122 have been closed. On average issues are closed in 100 days. There are 3 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of aws-lambda-redshift-loader is v2.7.8

Quality

aws-lambda-redshift-loader has 0 bugs and 0 code smells.

Security

aws-lambda-redshift-loader has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

aws-lambda-redshift-loader code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

aws-lambda-redshift-loader has a Non-SPDX License.

Non-SPDX licenses can be open source with a non SPDX compliant license, or non open source licenses, and you need to review them closely before use.

Reuse

aws-lambda-redshift-loader releases are available to install and integrate.

Installation instructions are not available. Examples and code snippets are available.

aws-lambda-redshift-loader saves you 8 person hours of effort in developing the same functionality from scratch.

It has 379 lines of code, 1 functions and 31 files.

It has low code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed aws-lambda-redshift-loader and discovered the below as its top functions. This is intended to give you an instant insight into aws-lambda-redshift-loader implemented functionality, and help decide if they suit your requirements.

Handler for upgrade .
Create the cluster command
Load the pending changes for the given batch .
Add a new pending transaction to the batch .
Copies the encrypted key encoding for the encrypted key .
Creates an event source for a given lambda function
Re - re - processing a batch request .
Creates the list of files for DynamoDB
Loads and creates new cluster with the given config .
Run a pg command

Get all kandi verified functions for this library.

aws-lambda-redshift-loader Key Features

No Key Features are available at this moment for aws-lambda-redshift-loader.

aws-lambda-redshift-loader Examples and Code Snippets

No Code Snippets are available at this moment for aws-lambda-redshift-loader.

Community Discussions

Trending Discussions on aws-lambda-redshift-loader

AWS Lambda times out connecting to RedShift

Lambda using python 3.6 & boto3 in VPC times out when connecting to Redshift

QUESTION

AWS Lambda times out connecting to RedShift

Asked 2019-Dec-17 at 20:35

My Redshift cluster is in a private VPC. I've written the following AWS Lamba in Node.js which should connect to Redshift (dressed down for this question):

...

ANSWER

Answered 2017-May-22 at 19:15

The timeout is probably because your lambda in VPC cannot access Internet in order to connect to your cluster(you seem to be using the public hostname to connect). Your connection options depend on your cluster configuration. Since both your lambda function and cluster are in the same VPC, you should use the private IP of your cluster to connect to it. In your case, I think simply using the private IP should solve your problem.

Depending on whether your cluster is publicly accessible, there are some points to keep in mind.

If your cluster is configured to NOT be publicly accessible, you can use the private IP to connect to the cluster from your lambda running in a VPC and it should work.
If you have a publicly accessible cluster in a VPC, and you want to connect to it by using the private IP address from within the VPC, make sure the following VPC parameters to true/yes:
- DNS resolution
- DNS hostnames

The steps to verify/change these settings are given here.

If you do not set these parameters to true, connections from within VPC will resolve to the EIP instead of the private IP and your lambda won't be able to connect without having Internet access(which will need a NAT gateway or a NAT instance).

Also, an important note from the documentation here.

If you have an existing publicly accessible cluster in a VPC, connections from within the VPC will continue to use the EIP to connect to the cluster even with those parameters set until you resize the cluster. Any new clusters will follow the new behavior of using the private IP address when connecting to the publicly accessible cluster from within the same VPC.

Source https://stackoverflow.com/questions/44112128

QUESTION

Lambda using python 3.6 & boto3 in VPC times out when connecting to Redshift

Asked 2019-Feb-07 at 04:34

I am trying to use boto3 in python3.6 to connect to my Redshift cluster using the get_cluster_credentials API. The following code times out 100% of the time when the Lambda function is added to the VPC. It runs without issue when Lambda is not added to the VPC.

I can't figure out if get_cluster_credentials uses the public or private IP to access Redshift. I also can't figure out if there is a way to force it to use one or the other.

...

ANSWER

Answered 2019-Feb-07 at 04:34

As per your other question, when an AWS Lambda function is added to a VPC, it does not receive a Public IP address. Therefore, if the function wishes to access the Internet (in this case to make the get_cluster_credentials() call), you should:

Add a NAT Gateway in a Public subnet
Attach the Lambda function to a Private subnet
Set routing on the private subnet to use the NAT Gateway for 0.0.0.0/0

It will not work if you have only one subnet, since the Lambda function will not be able to access the NAT Gateway.

I have also had success manually assigning an Elastic IP address to the Lambda function's ENI (instead of using a NAT Gateway), but this will not scale because Lambda might deploy additional containers and therefore additional ENIs. It might be sufficient if the function runs rarely and never concurrently.

Source https://stackoverflow.com/questions/54565960

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install aws-lambda-redshift-loader

You can download it from GitHub.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: