hydra | agnostic ML Platform that will enable Data Scientists | Machine Learning library
kandi X-RAY | hydra Summary
kandi X-RAY | hydra Summary
A cloud-agnostic Machine Learning Platform that will enable Data Scientists to run multiple experiments, perform hyper parameter optimization, evaluate results and serve models (batch/realtime) while still maintaining a uniform development UX across cloud environments.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Runs the Hydra info
- Run the job
- Check git repo
- Product of a dictionary
- Uploads code to S3
- Convert a dict to a string
- Insert job attributes
- Get the git repository url
- Inflates a list of dictionaries
- Return True if the current branch is unsynced
- Run the model
- Get the SHA of the current git commit
- Return True if the repo is empty
- Checks if the index has been modified
- Return whether there are uncommitted uncommitted changes
- Return True if the repo is untracked
- Run the script
- Run a command
- Run script
hydra Key Features
hydra Examples and Code Snippets
Community Discussions
Trending Discussions on hydra
QUESTION
I have downloaded the corpus of articles Aminar DBLP Version 11. The corpus is a huge text file (12GB) which each line is a self-contained JSON string:
...ANSWER
Answered 2022-Mar-23 at 13:51Reading the file without providing the schema is taking longer time. I tried to split the huge file in smaller chunks to understand the schema and it failed with Found duplicate column(s) in the data schema:
I tried the below approach on the same dataset with provided schema and it worked.
QUESTION
The documentation in https://hydra.cc/docs/patterns/select_multiple_configs_from_config_group/ shows how to pick multiple configs from a Config Group and place them in a dictionary-like structure.
However, as mentioned in the very last paragraph there, "example uses an explicit nesting level inside each of the configs to prevent them stepping over one another". For my use-case, this would prove extremely cumbersome and I would like to avoid it at all costs if possible.
Is there a way to achieve a similar result without resorting to explicitly adding the level in the individual configs? Thanks in advance :)
...ANSWER
Answered 2022-Mar-06 at 14:58You can use a defaults-list @package
keyword to achieve a similar result.
Let's assume you have changed the yaml files server/site/fb.yaml
and server/site/google.yaml
to not contain the "explicit nesting", so e.g. server/site/fb.yaml
contains only the data domain: facebook.com
.
You can achieve the same output as from the docs webpage using the following defaults list in server/apache.yaml`:
QUESTION
After upgrading to Api platform 2.6, the non ApiResource
entities started being serialized (ld-json) along with a fake identifier and their type. For example:
ANSWER
Answered 2022-Feb-20 at 09:54No idea how to decorate the serializer, but I will handle your issue with an event subscriber.
The "POST_SERIALIZE" priority contains the serialized response body, and you can fetch it and modify it.
Like this:
QUESTION
Hydra provides a way to dynamically create a hierarchical configuration by composition and override it through config files and the command line, leveraging OmegaConf. I look for a recommended way to document the parameters but I could not find (a documented) one. What are best practices for that? Coming from argparse, I like the way of documenting the parameter inline, i.e. close to the code.
...ANSWER
Answered 2022-Feb-15 at 02:47Excellent question! The answer is: parameter-by-parameter documentation is not yet implemented (as of Hydra v1.1, OmegaConf v2.1).
The future plans are:
- in OmegaConf, expose an API allowing users to attach documentation (and other metadata) to each field of a structured config. See this open OmegaConf issue.
- Once the above is complete, implement a Hydra feature allowing a parameter-specific help messages to be printed based on the parameter's metadata. See this open Hydra issue.
For now, the best we can do is to customize the general application help message (i.e. the --help
command line flag).
QUESTION
Please, see this question for context and the answer by MrC aka Shaun Curtis
This question is about something I've tried to solve in the past without great success. Are you familiar with this sample I've once downloaded and run it. It did not work. I then realized that I must add the base url to the url in the browser's address bar in order to run the first project, for instance: https://localhost: 44302/FirstApp
That is, the Client project. And for the SecondClient it should be https://localhost: 44302/SecondApp
. This is exactly how the sample app by MrC aka Shaun Curtis works, though he added a Razor Pages app to provide a menu for redirection to the four projects.
What I tried to do without much success is to make the first WebAssemby front end project, which is hosted, to be the default; that is when I run the app, or type in the address bar https://localhost: 44302.
And if I type https://localhost: 44302/FirstApp
I see the first stand alone WebAssembly project I added to the solution. And a second project, and a third project, and so on, all are WebAssembly projects. I could not do that: When I ran the default project everything is fine... I can navigate within the bounds of the project, route to the Counter page, FetchData page, etc.
But when I add the segment /FirstApp to the url in the address bar and hit enter, the Router displays the message "Sorry, there's nothing at this address." instead of navigating to the project represented by the base url /FirstApp/
Does anyone here have any idea how to achieve the requested feature I am looking for ?
...ANSWER
Answered 2022-Feb-14 at 17:57This is a summary of the Repo that demonstrates how to do this.
The sub-folder Web Assembly projects have project files like this. The import bit is setting the
QUESTION
Recently I have started to use hydra to manage the configs in my application. I use Structured Configs to create schema for .yaml config files. Structured Configs in Hyda uses dataclasses for type checking. However, I also want to use some kind of validators for some of the parameter I specify in my Structured Configs (something like this).
Do you know if it is somehow possible to use Pydantic for this purpose? When I try to use Pydantic, OmegaConf complains about it:
...ANSWER
Answered 2022-Jan-10 at 05:58See pydantic.dataclasses.dataclass, which are a drop-in replacement for the standard-library dataclasses with some extra type-checking.
QUESTION
In Hydra I have the following configuration:
...ANSWER
Answered 2022-Feb-10 at 19:48Currently the recommended approach is:
- compose a mapping whose values are the desired callbacks, and then
- use the
oc.dict.values
OmegaConf resolver to get a list of values from that dictionary.
QUESTION
I have a project where I want to build a full-blown IDP (using Golang). So technically, a user wants to SSO into another system using my service. I am looking to build this service from scratch. Upon researching for open-source IDP solutions, I came across ory/Hydra and ory/Kratos. I went through their documentation and did a quick-start tutorial. I am still confused about which of the above 2 libraries are suitable for the development of this service.
From a high-level standpoint, this is what I am trying to do.
- There is an existing third-party web app X.
- A user has already login credentials for my service Y.
- A user wants to post some comments in app X.
- He/She is redirected to my service Y login screen if not logged in.
- After login, the Callback URL returns him back to app X comment section.
ANSWER
Answered 2022-Feb-09 at 19:16ory/Kratos is not what you are looking for because it is designed to answer your user management basic needs.
The right tool for you is ory/Hydra. I have copied this from its documentation:
If you want apps and websites you don't own to use your application as a potential sign in (e.g. be listed alongside "Sign in with Google", "Sign in with Apple"), ORY Hydra is the right tool for you.
source: https://www.ory.sh/hydra/docs/concepts/before-oauth2
QUESTION
Is there a better way to reload a hydra config from an experiment with enumerations? Right now I reload it like so:
...ANSWER
Answered 2022-Feb-05 at 13:52This is a good question -- reliably reloading configs from previous Hydra runs is an area that could be improved.
As you've discovered, loading the saved file config.yaml
directly results in an untyped DictConfig object.
The solution below involves a script called reload.py
that creates a config node with a defaults list that loads both the schema base_config_cs
and the saved file config.yaml
.
At the end of this post I also give a simple solution that involves loading .hydra/overrides.yaml
to re-run the config composition process.
Suppose you've run a Hydra job with the following setup:
QUESTION
I would like to use python datatypes - both built-in and imported from libraries such as numpy, tensorflow, etc - as arguments in my hydra configuration. Something like:
...ANSWER
Answered 2022-Jan-19 at 22:13Does the hydra.utils.get_class
function solve this problem for you?
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install hydra
Entrypoint for Hydra CLI. Submit a training job to the selected cloud platform. You need to run this from inside a git hosted repository that contains your model code and a conda yaml file environment.yml . The command takes a number of options to tailor your training job. These options can also be provided via a yaml file.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page