cmdb | cmdb saltstack | Configuration Management library

I'm using Ansible to build inventories dynamically according to several parameters.

I get raw info from our CMDB's API, then use set_fact to format a list of FQDNs.

Those FQDNs always follow the same formula, so it looks easy, but Ansible only seems to apply the templating to the first item.

The code:

I have a excel sheet that consist of over 6000 rows. There are two column , "IP Address CMDB" that contain IP addresses and another column called "IP Address LM". I am trying to look for IP address that belongs to "IP Address CMDB" in "IP Address LM" and if "IP Address LM" contain that IP address return ABCD. I could not attach excel sheet so I have attached screenshot of it.

I have a dynamodb table called environments and a column within that table called env_name When running the class in the command line:

I have the following code. In testing, I found that when I get several hundred concurrent child processes (somewhere around 400?), I get "OSError Too Many Open Files". Any idea why?

I can solve the problem with the time.sleep(.005) call, but I shouldn't have to.

This is a part of a larger program. A typical call will set a server string, token string, and a list of many thousands of devices. For the REST API call used, the server can only handle a single device at a time. In testing, this resulted in a 20 min execution time, but indications are that using a multiprocessing approach can reduce it to around 30 sec.

I have a repository with Ansible playbooks. In one playbook, gather-installed.yml, I use package_facts_module together with ansible-cmdb. In next following playbook, which I use to install software, I have a variable of type dict defined: packages:

I have an associate array format like $aa. Need to form an array list in the different format.

I have a class that scans a column from a dynamo db table, whilst using the aws sdk for java(main method taken out for simplicity):

I'm building a parameter optimizer that essentially generates configurations, benchmarks all of them, collects all results, sorts them, and then picks the best performing configuration relative to the benchmark result.

The benchmark by itself works fine, but takes between 50 ms an 2 sec per run depending on the configuration. The crux is, the optimizer generates a very large number of configuration, that means, between 100k on the lowest end and about 40 million on the higher end with about 1 - 5 million as a good normal range. Obviously, the single threaded version takes forever and CPU load is actually very low as the task is relatively light.

I already designed the benchmark in a way to make it play nicely with concurrency, that is, the runner is encapsulated in a separate struct (called agent) and the benchmark is essentially a pure function that takes all state as a parameter. Essentially, each run creates its own state and then runs independently of all others, but all functions using the same (referenced) shared dataset. The function is shown below.

However, I struggle with dealing with the return value per Benchmark. Back in the days, in Scale we used Async / Await for task parallelism and just let the results roll on. Go Routines, afaik only work well with functions that have no return value. In practice, channels are the most natural way to fetch a value from a goroutine. And that's the crux I'm mulling over:

Considering that I usually have > 1 million tasks, how do I catch the return values correctly and efficiently?

Related to that, is there actually a very fast parameter optimizer for Golang? For python, I remember optuna delivering excellent results.

Thank you

I want to execute cmdA, cmdB and cmdC in parallel before cmdD. This is what I have so far: