dg | A programming language for the CPython VM | Interpreter library
kandi X-RAY | dg Summary
kandi X-RAY | dg Summary
A (technically) simple language that compiles to CPython bytecode. DISCLAIMER: this project is just for fun, please don't use it for anything serious, thanks.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Load dgb bundle .
dg Key Features
dg Examples and Code Snippets
Lines of Code : 24License : Non-SPDX (Apache License 2.0)
Copydef _BatchNormWithGlobalNormalizationGrad(op, grad):
"""Return the gradients for the 5 inputs of BatchNormWithGlobalNormalization.
We do not backprop anything for the mean and var intentionally as they are
not being trained with backprop in th def get_support(cluster):
"""
Returns support
>>> get_support({5: {'11111': ['ab', 'ac', 'df', 'bd', 'bc']},
... 4: {'11101': ['ef', 'eg', 'de', 'fg'], '11011': ['cd']},
... 3: {'11001': ['ad'], def preprocess(edge_array):
"""
Preprocess the edge array
>>> preprocess([['ab-e1', 'ac-e3', 'ad-e5', 'bc-e4', 'bd-e2', 'be-e6', 'bh-e12',
... 'cd-e2', 'ce-e4', 'de-e1', 'df-e8', 'dg-e5', 'dh-e10', 'ef-e3',
. Community Discussions
Trending Discussions on dg
QUESTION
I have created an AWS Lambda with the new function URL feature enabled.
Since the URL isn't that easy to remember, I would like to create a Route 53 alias like lambda.mywebsite.com.
There is no Route 53 alias for Lambda function URLs in the drop-down menu for aliases in Route 53.
How can I call my AWS Lambda function URL via a custom domain?
Is a CNAME record the way to go?
...ANSWER
Answered 2022-Apr-10 at 10:38Is a CNAME record the way to go?
Yes.
If you want to call your AWS Lambda function URL via a custom domain, you will need a CNAME record.
There is no support currently for a Route 53 alias record.
They are meant to be the simplest & fastest way to invoke your Lambda functions via a public endpoint without using other AWS services like API gateway, so a lack of support for a custom domain name makes sense.
QUESTION
I have a graph g
ANSWER
Answered 2022-Apr-01 at 10:31You can use subgraph.edges:
QUESTION
I want to create a random ID variable considering an actual ID. That means that observations with the same id must have the same random ID. Let me put an example:
...ANSWER
Answered 2022-Mar-01 at 09:56To create a new id by group, use match with sample, or cur_group_id in dplyr. The ids will start from 1 until the number of total groups is reached.
QUESTION
To give more context about my problem:
I am using python to build an API connecting to the TWS of interactive brokers. I managed to build something functional and able to fetch live data from contracts using the methods given in the doc of IB. Now that I want to use all the data to build other parallel systems with it, I have encounter problems organising the data that arrives from the IB server.
My program loops a list of 30 symbols to get live data from and then I want to put the data (ex. 'HIGH', 'LOW', 'CLOSE', 'VWAP' etc) from each symbol all together in one dataframe to calculate indicators and from there come up with an alert system that is based on them indicators.
This objective I have already accomplished it using only one symbol for the whole program. Is easy to store the data in instances or variables, pass it to a DataFrame and then calculate to set up alerts.
Now when looping a list of 30 values and receiving the data of all of them I have struggled trying to store the data for each symbol together and then calculate and set up alerts. Specially when I have to use several methods to receive the data (ex. I use tickPrice for some data and tickString for some other data) this several methods execute themselves one after the other but they wont necessarily have all the data at the same time, some values take more time than others to show.
I will show an example of my code to give even more context of my objective:
This is my EWrapper class:
...ANSWER
Answered 2022-Feb-26 at 12:14It's easy to create a Pandas dataframe from a Python dictionary that contains lists. For example, the following code creates a dictionary containing ticker symbols, bid prices, and ask prices:
QUESTION
I am trying to schedule a data-quality monitoring job in AWS SageMaker by following steps mentioned in this AWS documentation page. I have enabled data-capture for my endpoint. Then, trained a baseline on my training csv file and statistics and constraints are available in S3 like this:
...ANSWER
Answered 2022-Feb-26 at 04:38This happens, during the ground-truth-merge job, when the spark can't find any data either in '/opt/ml/processing/groundtruth/' or '/opt/ml/processing/input_data/' directories. And that can happen when either you haven't sent any requests to the sagemaker endpoint or there are no ground truths.
I got this error because, the folder /opt/ml/processing/input_data/ of the docker volume mapped to the monitoring container had no data to process. And that happened because, the thing that facilitates entire process, including fetching data couldn't find any in S3. and that happened because, there was an extra slash(/) in the directory to which endpoint's captured-data will be saved. to elaborate, while creating the endpoint, I had mentioned the directory as s3:////, while it should have just been s3:///. so, while the thing that copies data from S3 to docker volume tried to fetch data of that hour, the directory it tried to extract the data from was s3://////////(notice the two slashes). So, when I created the endpoint-configuration again with the slash removed in S3 directory, this error wasn't present and ground-truth-merge operation was successful as part of model-quality-monitoring.
I am answering this question because, someone read the question and upvoted it. meaning, someone else has faced this problem too. so, I have mentioned what worked for me. And I wrote this, so that StackExchange doesn't think I am spamming the forum with questions.
QUESTION
I followed the aws documentation ( https://docs.aws.amazon.com/sagemaker/latest/dg/serverless-endpoints-create.html#serverless-endpoints-create-config) to create a model and to use that model, i coded for a serverless endpoint config (sample code below) ,I have all the required values but this throws an error below and i'm not sure why
parameter validation failed unknown parameter inProductVariants [ 0 ]: "ServerlessConfig", must be one of : VairantName, ModelName, InitialInstanceCount , Instancetype...
...ANSWER
Answered 2022-Feb-17 at 03:45You are probably using old boto3 version. ServerlessConfig is a very new configuration option. You need to upgrade to the latest version (1.21.1) if possible.
QUESTION
I am building a Telegram bot in C#, deployed with AWS Lambda. Telegram bot and Lambda are connected via a webhook and work fine. I need to schedule deleting a bot's message in a few minutes without blocking the bot. It must keep accepting and process new requests.
As for now I see the solution in using Task.Delay. However, the instance created by AWS to execute lambda doesn't scale and users have to wait until the delay is ended to handle the following request from the queue.
From the official documentation:
The first time you invoke your function, AWS Lambda creates an instance of the function and runs its handler method to process the event. When the function returns a response, it stays active and waits to process additional events. If you invoke the function again while the first event is being processed, Lambda initializes another instance, and the function processes the two events concurrently. As more events come in, Lambda routes them to available instances and creates new instances as needed. When the number of requests decreases, Lambda stops unused instances to free up scaling capacity for other functions.
The default regional concurrency quota starts at 1,000 instances.
As far as I understand the whole Lambda thing is about delegating concurrent execution to AWS. If a handler takes some time to fulfil a request, then AWS automatically creates the second instance to process the following request. Isn't it?
How can I implement concurrency/configure lambda/rewrite code to enable handling multiple bot events?
I've already watched through AWS Step Functions and EventBridges to solve the problem, but before diving deeper into them it would make sense to clarify that there is no a simple and straightforward solution that I missed.
P.S. Please keep in mind that this is my first experience in building a telegram bot and using AWS Lambda functions. The problem may lie completely outside AWS and Telegram Bot API.
...ANSWER
Answered 2022-Feb-14 at 13:45You need to realize that when you trigger that delay in a Lambda function, that instance of the function becomes suspended and will not handle another request. A Lambda function instance will not be sent another request until it returns a response. The Lambda function instance is effectively blocked, just watching its system clock waiting for the 2 minute delay to finish.
When you trigger another request while the first request is waiting for the delay, all you are doing is starting another instance, which is then also going to sit and wait for its own 2 minute delay to complete.
The way you've coded this Lambda function, each request is going to trigger a 2 minute delay and wait for that delay before it returns a response. And you are getting charged for each of those 2 minute delays, because you are still occupying AWS compute resources, although all they are doing is monitoring a system clock for 2 minutes.
I suggest having your Lambda function quickly push the message into an SQS delay queue and exit as soon as it has done that. Then have another Lambda function configured with the SQS queue as an event source, that takes the SQS message and does your delete.
QUESTION
I have a large number of JSON requests for a model split across multiple files in an S3 bucket. I would like to use Sagemaker's Batch Transform feature to process all of these requests (I have done a couple of test runs using small amounts of data and the transform job succeeds). My main issue is here (https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html#batch-transform-errors), specifically:
If a batch transform job fails to process an input file because of a problem with the dataset, SageMaker marks the job as failed. If an input file contains a bad record, the transform job doesn't create an output file for that input file because doing so prevents it from maintaining the same order in the transformed data as in the input file. When your dataset has multiple input files, a transform job continues to process input files even if it fails to process one. The processed files still generate useable results.
This is not preferable mainly because if 1 request fails (whether its a transient error, a malformmated request, or something wrong with the model container) in a file with a large number of requests, all of those requests will get discarded (even if all of them succeeded and the last one failed). I would ideally prefer Sagemaker to just write the output of the failed response to the file and keep going, rather than discarding the entire file.
My question is, are there any suggestions to mitigating this issue? I was thinking about storing 1 request per file in S3, but this seems somewhat ridiculous? Even if I did this, is there a good way of seeing which requests specifically failed after the transform job finishes?
...ANSWER
Answered 2022-Feb-10 at 21:26You've got the right idea: the fewer datapoints are in each file, the less likely a given file is to fail. The issue is that while you can pass a prefix with many files to CreateTransformJob, partitioning one datapoint per file at least requires an S3 read per datapoint, plus a model invocation per datapoint, which is probably not great. Be aware also that apparently there are hidden rate limits.
Here are a couple options:
Partition into small-ish files, and plan on failures being rare. Hopefully, not many of your datapoints would actually fail. If you partition your dataset into e.g. 100 files, then a single failure only requires reprocessing 1% of your data. Note that Sagemaker has built-in retries, too, so most of the time failures should be caused by your data/logic, not randomness on Sagemaker's side.
Deal with failures directly in your model. The same doc you quoted in your question also says:
If you are using your own algorithms, you can use placeholder text, such as ERROR, when the algorithm finds a bad record in an input file. For example, if the last record in a dataset is bad, the algorithm places the placeholder text for that record in the output file.
Note that the reason Batch Transform does this whole-file failure is to maintain a 1-1 mapping between rows in the input and the output. If you can substitute the output for failed datapoints with an error message from inside your model, without actually causing the model itself to fail processing, Batch Transform will be happy.
QUESTION
We have several lambda functions, and I've automated code deployment using the gradle-aws-plugin-reboot plugin.
It works great on all but one lambda functions. On that particular one, I'm getting this error:
...ANSWER
Answered 2021-Dec-09 at 10:42I figured it out. You better not hold anything in your mouth, because this is hilarious!
Basically being all out of options, I locked on to the last discernible difference between this deployment and the ones that worked: The filesize of the jar being deployed. The one that failed was by far the smallest. So I bloated it up by some 60% to make it comparable to everything else... and that fixed it!
This sounds preposterous. Here's my hypothesis on what's going on: If the upload takes too little time, the lambda somehow needs longer to change its state. I'm not sure why that would be, you'd expect the state to change when things are done, not to take longer if things are done faster, right? Maybe there's a minimum time for the state to remain? I wouldn't know. There's one thing to support this hypothesis, though: The deployment from my local computer always worked. That upload would naturally take longer than jenkins needs from inside the aws vpc. So this hypothesis, as ludicrous as it sounds, fits all the facts that I have on hand.
Maybe somebody with a better understanding of the lambda-internal mechanisms can add a comment to this explaining how this can happen...
QUESTION
I have Pandas DataFrame with 3 columns:
...ANSWER
Answered 2022-Jan-11 at 08:36Use GroupBy.agg with first and last in sorted DataFrame, so get values for minimal and maximal dates, last subtract values with DataFrame.pop for remove columns first, last:
If need last dates per groups use named aggregation also for date column:
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install dg
You can use dg like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
