pypyr-aws | create & manage AWS infrastructure | Cloud Functions library

 by   pypyr Python Version: v1.2.0 License: Apache-2.0

kandi X-RAY | pypyr-aws Summary

kandi X-RAY | pypyr-aws Summary

pypyr-aws is a Python library typically used in Serverless, Cloud Functions, Docker applications. pypyr-aws has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can install using 'pip install pypyr-aws' or download it from GitHub, PyPI.

Run anything on aws. No really, anything! If the aws api supports it, the pypyr aws plug-in supports it. It's a pretty easy way of invoking the aws api as a step in a series of steps without having to write code. pypyr is a cli & api to run pipelines defined in yaml. All documentation for the pypyr aws plugin is here: Why use this when you could just use the aws-cli instead? The aws cli is all kinds of awesome, but I find more often than not it's not just one or two aws ad hoc cli or aws api methods you have to execute, but especially when automating and scripting you actually need to run a sequence of commands, where the output of a previous command influences what you pass to the next command. Sure, you can bash it up, and I do that too, but running it as a pipeline via pypyr has actually made my life quite a bit easier because of not having to deal with conditionals, error traps and input validation.
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              pypyr-aws has a low active ecosystem.
              It has 6 star(s) with 1 fork(s). There are 3 watchers for this library.
              OutlinedDot
              It had no major release in the last 12 months.
              There are 1 open issues and 1 have been closed. On average issues are closed in 63 days. There are no pull requests.
              It has a neutral sentiment in the developer community.
              The latest version of pypyr-aws is v1.2.0

            kandi-Quality Quality

              pypyr-aws has 0 bugs and 0 code smells.

            kandi-Security Security

              pypyr-aws has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              pypyr-aws code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              pypyr-aws is licensed under the Apache-2.0 License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              pypyr-aws releases are available to install and integrate.
              Deployable package is available in PyPI.
              Build file is available. You can build the component from source.
              Installation instructions, examples and code snippets are available.
              It has 2517 lines of code, 110 functions and 27 files.
              It has high code complexity. Code complexity directly impacts maintainability of the code.

            Top functions reviewed by kandi - BETA

            kandi has reviewed pypyr-aws and discovered the below as its top functions. This is intended to give you an instant insight into pypyr-aws implemented functionality, and help decide if they suit your requirements.
            • Determine if the step needs to be run
            • Waits for a named service
            • Executes a boto3 client
            • Return the payload of an object
            • Get poll arguments from a dict
            • Validate and return args
            • Get s3Fetch input
            • Execute an AWS API call
            Get all kandi verified functions for this library.

            pypyr-aws Key Features

            No Key Features are available at this moment for pypyr-aws.

            pypyr-aws Examples and Code Snippets

            pypyr aws plug-in,custom waiters
            Pythondot img1Lines of Code : 14dot img1License : Permissive (Apache-2.0)
            copy iconCopy
            awsWaitFor:
              awsClientIn: # required. awsClientIn allows the same arguments as pypyraws.steps.client.
                serviceName: elasticbeanstalk
                methodName: describe_environments
                methodArgs:
                    ApplicationName: my wonderful beanstalk default app  
            pypyr aws plug-in,usage
            Pythondot img2Lines of Code : 13dot img2License : Permissive (Apache-2.0)
            copy iconCopy
            context_parser: pypyr.parser.keyvaluepairs
            steps:
              - name: pypyraws.steps.client
                description: upload a file to s3
                in:
                  awsClientIn:
                    serviceName: s3
                    methodName: upload_file
                    methodArgs:
                      Filename: ./testfi  
            pypyr aws plug-in,installation
            Pythondot img3Lines of Code : 1dot img3License : Permissive (Apache-2.0)
            copy iconCopy
            $ pip install --upgrade pypyraws
              

            Community Discussions

            QUESTION

            Automatic delete Cloud Logging logs
            Asked 2022-Mar-23 at 16:41

            I would like to know if there is a way to periodically delete logs from inside Cloud Logging.

            I have setup Firebase with Cloud Functions and i have an automatic Cloud Logging logs injection done for each function call.
            I don't want especially to stop sending logs to Cloud Logging, but i would like to be able to manage my costs by deleting older logs.

            ...

            ANSWER

            Answered 2022-Mar-23 at 16:41

            You can set a retention policy on your Cloud Logging bucket to match with your requirements, which can auto-delete logs after between 1 day and 10 years.

            Source https://stackoverflow.com/questions/71590582

            QUESTION

            Firebase function failing to deploy
            Asked 2022-Feb-17 at 15:31

            I'm trying to create a Firebase Function but I'm running into a deploy error, even when deploying the default helloworld function.

            The firebase-debug.log file mentions this: Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld.

            I have been trying to debug and so far have not been able to solve it...

            firebase-debug.log

            ...

            ANSWER

            Answered 2022-Feb-06 at 14:36

            Could not find image for function projects/picci-e030e/locations/us-central1/functions/helloWorld.

            The Firebase Function deployment failed because it cannot find the image built based on your function app. There might be a problem building in your app, it could be your dependencies or files.

            I replicated your issue, received the same error and solved it. There's a problem with the package.json file and package-lock.json. If you just add(without installing) your dependency in package.json you should delete or remove your package-lock.json that will be found in function directory before you deploy it again using the deployment command:

            Source https://stackoverflow.com/questions/70027316

            QUESTION

            Implementation of Distributed-Counter-Extension for local emulator
            Asked 2022-Jan-24 at 17:55

            The firebase extension for a distributed counter can be directly installed for the cloud and works just fine. To develop new features for an app I need to do this on the emulator to not interrupt the running server.

            As the firebase extensions simply are cloud Functions*, I thought about implementing the cloud function in my emulator by getting the source code from the extension itself. This worked fine for other extentions so far...

            Error and Disfunction when implementing

            When implementing the javaScript version that i get the following error:

            function ignored because the unknown emulator does not exist or is not running.

            This problem can be fixed by rewriting the export line of the index.jsfunctions, but is wont provide the expected functionality of the extension anyhow:

            ...

            ANSWER

            Answered 2022-Jan-24 at 17:55

            firebaser here

            Firebase Extensions normally declare their triggers in the extension.yaml file, instead of in the code itself. Therefore, in order to emulate an extension in this way, you'd need to move the triggers over to the code.

            For your specific example of the 'worker' function, the extension declares what document to listen to here, so we'll copy the document over to the code:

            Source https://stackoverflow.com/questions/70836970

            QUESTION

            Build a container image from inside a cloud function
            Asked 2021-Dec-22 at 00:59

            Context: I am training a very similar model per bigquery dataset in Google Vertex AI, but I want to have a custom training image for each existing dataset (in Google BigQuery). In that sense, I need to programatically build a custom Docker Image in the container registry on demand. My idea was to have a Google Cloud Function do it, being triggered by PubSub topic with information regarding which dataset I want to build the training container for. So naturally, the function will write the Dockerfile and pertinent scripts to a /tmp folder within Cloud Functions (the only writable place as per my knowledge). However, when I try to actually build the container within this script, apparently, it doesn't find the /tmp folder or its contents, even though they are there (checked with logging operations).

            The troubling code so far:

            ...

            ANSWER

            Answered 2021-Dec-21 at 11:07

            I've locally tested building a container image using Cloud Build Client Python library. It turns out to have the same error even the Dockerfile file is existing in current directory:

            error:

            Step #0: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /workspace/Dockerfile: no such file or directory

            build steps:

            Source https://stackoverflow.com/questions/70428362

            QUESTION

            Error when import firebase-functions-test when testing with mocha
            Asked 2021-Dec-02 at 09:53

            I am trying to setup a Firebase Cloud Functions repo to run mocha test. However, it throws the following error when I use import * as firebase from "firebase-functions-test"; or const firebase = require("firebase-functions-test")();. You can see in my code that I haven't even called the actual firebase functions yet so I think this a setup issue.

            Question: What change do I need to make mocha test running for Firebase Functions testing using import syntax?

            Working test code

            ...

            ANSWER

            Answered 2021-Dec-02 at 09:53

            This error should be resolved after specifying the latest version of the

            • firebase-functions, v3.16.0, and

            • firebase-functions-test, v0.3.3.

            Source https://stackoverflow.com/questions/70183270

            QUESTION

            Parsing error: Cannot read file '\tsconfig.json' eslint after following Firebase Cloud Functions initialization instructions
            Asked 2021-Nov-16 at 16:17
            Problem

            Right after my TypeScript project initialization in VSCode using firebase tools for composing Firebase Cloud Functions following the official documentation the very first line of the index.ts file displays an error:

            Parsing error: Cannot read file '\tsconfig.json' eslint [1,1]

            and the .eslintrc.js displays an error:

            File is a CommonJS module; it may be converted to an ES6 module.ts(80001)

            Since all files are auto-generated these errors are a complete surprise and I want to get rid of them.

            Versions

            For the record, here are the versions installed:

            ...

            ANSWER

            Answered 2021-Nov-16 at 16:17

            Ok, I have solved the problem with a great help of this github thread False positive Error - TS6133 error (declared but its value is never read) report.

            I have changed "noUnusedLocals" setting in the tsconfig.json file from the default true to false, so the file becomes:

            Source https://stackoverflow.com/questions/69897000

            QUESTION

            firebase function with realtime database error
            Asked 2021-Nov-08 at 12:28

            I am new to firebase function and trying to use firebase function with Realtime database (Emulator suite).But when i try to set the value in firebase using the firebase function,it gives an error and doesn't set the value in database.

            Error:

            ...

            ANSWER

            Answered 2021-Nov-05 at 13:59

            I'm unsure as to the cause of that log message, but I do see that you are returning a response from your function before it completes all of its work. In a deployed function, as soon as the function returns, all further actions should be treated as if they will never be executed as documented here. An "inactive" function might be terminated at any time, is severely throttled and any network calls you make (like setting data in the RTDB) may never be executed.

            I know you are new to this, but its a good habit to get into now: don't assume the person calling your function is you. Check for problems like missing query parameters and dodgy data before you blindly action something. The Admin SDK bypasses your database's security rules and if you are not careful a malicious user can cause some damage (e.g. a user that updates /users/$theirUid/roles/admin to true).

            Source https://stackoverflow.com/questions/69853038

            QUESTION

            Firebase Cloud Functions min-instances setting seems to be ignored
            Asked 2021-Oct-30 at 06:35

            Firebase has announced in September 2021 that it is possible now to configure its cloud function autoscaling in a way, so that a certain number of instances will always be running (https://firebase.google.com/docs/functions/manage-functions#min-max-instances).

            I have tried to set this up, but I can not get it to work: At first I have set the number of minimum instances in Google Cloud Console: Cloud Console Screenshot After doing this I expected that one instance for that cloud function would run at any time. The metrics of that function indicate that it instances were still scaled down to 0: Cloud functions "Active Instances Metric"

            So to me it looks a bit as if my setting is ignored here. Am I missing anything? Google Cloud Console shows me that the number of minimum instances has been set to 1 so it seems to know about it but to ignore it. Is this feature only available in certain regions?

            I have also tried to set the number of minimum instances using the Firebase SDK for Cloud Functions (https://www.npmjs.com/package/firebase-functions). This gave me the same result, my setting is still ignored.

            ...

            ANSWER

            Answered 2021-Oct-30 at 06:35

            According to the Documentation, the Active Instances metrics shows the number of instances that are currently handling the request.

            As stated in the Documentation :

            Cloud Functions scales by creating new instances of your function. Each of these instances can handle only one request at a time, so large spikes in request volume often causes longer wait times as new instances are created to handle the demand.

            Because functions are stateless, your function sometimes initializes the execution environment from scratch, which is called a cold start. Cold starts can take significant amounts of time to complete, so we recommend setting a minimum number of Cloud Functions instances if your application is latency-sensitive.

            You can also refer to the Stackoverflow thread where it has been mentioned that

            Setting up minInstances does not mean that there will always be that much number of Active Instances. Minimum instances are kept running idle (without CPU > allocated), so are not counted in Active Instances.

            Source https://stackoverflow.com/questions/69717844

            QUESTION

            Receiving inconsistent data in Go api
            Asked 2021-Oct-23 at 12:31

            I was learning the Go language and tested Google Cloud Functions with go + Google Firestore as the database.

            While I was testing the response I got inconsistent responses.

            I have used the json Marshaller to convert Firebase data to Json object to return from the API, this API is hosted in the Google Cloud Functions.

            ...

            ANSWER

            Answered 2021-Oct-23 at 12:31

            The solution i got after marshal and unmarshal, it works as expected.

            Source https://stackoverflow.com/questions/69072757

            QUESTION

            Get the client_id of the IAM proxy on GCP Cloud composer
            Asked 2021-Oct-15 at 15:02

            I'm trying to trigger Airflow DAG inside of a composer environment with cloud functions. In order to do that I need to get the client id as described here. I've tried with curl command but it doesn't return any value. With a python script I keep getting this error:

            ...

            ANSWER

            Answered 2021-Sep-28 at 13:00

            Posting this Community Wiki for better visibility.

            As mentioned in the comment section by @LEC this configuration is compatible with Cloud Composer V1 which can be found in GCP Documentation Triggering DAGs with Cloud Functions.

            At the moment there can be found two tabs Cloud Composer 1 Guides and Cloud Composer 2 Guides. Under Cloud Composer 1 is code used by the OP, but if you will check Cloud Composer 2 under Manage DAGs > Triggering DAGs with Cloud Functions you will get information that there is not proper documentation yet.

            This documentation page for Cloud Composer 2 is not yet available. Please use the page for Cloud Composer 1.

            As solution, please use Cloud Composer V1.

            Source https://stackoverflow.com/questions/69269929

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            No vulnerabilities reported

            Install pypyr-aws

            pypyraws depends on the pypyr core. The above pip will install it for you if you don't have it already.

            Support

            Well, you know. No one's perfect. Feel free to create an issue.
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries

            Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link

            Explore Related Topics

            Consider Popular Cloud Functions Libraries

            Try Top Libraries by pypyr

            pypyr

            by pypyrPython

            pypyr-example

            by pypyrPython

            docs

            by pypyrCSS

            pypyr-slack

            by pypyrPython

            pypyr.github.io

            by pypyrHTML