athena-express | simplify executing SQL queries in Amazon Athena | AWS library
kandi X-RAY | athena-express Summary
kandi X-RAY | athena-express Summary
Athena-Express can simplify executing SQL queries in Amazon Athena AND fetching cleaned-up JSON results in the same synchronous or asynchronous request - well suited for web applications.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Gets the results from a S3 query string
- Checks if the query has been completed
- Adds provided data type to the object .
- Start query execution
- Clean up the line of markdown .
- Formats an array of D3D results into a JSON object .
- Clean input string
- Return an array with raw results from a stream .
- Validates the constructor parameters
- Get the data types in the S3 .
athena-express Key Features
athena-express Examples and Code Snippets
Community Discussions
Trending Discussions on athena-express
QUESTION
I have written a Lambda function using athena-express that queries AWS Athena with S3 Parquet files as destination. Also using AWS Glue for ETL processes to S3. I am receiving: "Task timed out after 6.01 seconds". Increasing the timeout gives me the same message but with higher amount of timeout seconds. The strange thing is that it works locally, but not when it's deployed to AWS. Code should work. I think something is missing in AWS configurations.
I have followed the instructions and using lambda settings: https://www.npmjs.com/package/athena-express. I have added AmazonAthenaFullAccess and AmazonS3FullAccess policies to the Execution role for the Lambda function.
...ANSWER
Answered 2020-Apr-22 at 10:33When you say it works locally I assume that when you run it locally you have no timeout on the code. Athena is not a low latency database, and queries very rarely run in less than a few seconds, and often in minutes – it all depends on how much data they scan and other factors.
Your Lambda function must have a timeout that is longer than the longest run time of the queries you run with it. If those queries run for more than a minute, the timeout must be more than a minute, if they run for more than ten minutes, your timeout must be more than ten minutes, and so on.
In general, long running queries is not a great fit for Lambda and API Gateway that are made for more low latency situations. Since your queries seem to run for minutes I suggest that you look other ways of solving your problem. If you provide more context about what you are trying to do, we may be able to help you with that.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install athena-express
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page