linkages | mechanical linkage simulator

In OpenMDAO V3.9.0 a feature was added that detects rows and columns of all 0 in the total derivative Jacobian. A row of all 0's means that an objective or constraint is not impacted by any of the design variables. A column of all 0's means that a design variable does not impact any constraint or objective values.

Both of these situations are potentially problematic. A 0 column means that there are less degrees of freedom than you might think, since that DV doesn't affect anything. This isn't fatal, but it is still something that is worth warning a user about.

A 0 row is much more problematic. If it the row is associated with a constraint, it means the optimizer has no ability to satisfy that constraint. You may get "lucky" and find that the constraint happens that the constraint happens to be satisfied at the initial condition anyway, and so you can technically solve the optimization problem (your specific case is likely one of these). However, mathematically the problem is singular, and unless the optimizer you using has specific code to handle this corner case it can make things difficult.

One of the primary reasons this feature was added was that the OpenMDAO dev team noticed that Dymos users were particularly prone to accidentally creating 0 rows when adding linkage and path constraints. Often these 0 rows seem to not cause harm, but we have definitely also seen cases where they give the optimizer fits. The warning helps you identify the problem so you can correct it.

In this case, it looks like you have two separate 0 rows.

1. traj.linkages.stage_1_grav_turn:alpha_final|coast_1:alpha_initial means that none of the design variables you've given to the optimizer affect that constraint. Likely this means that you have specified both alpha_final and alpha_initial as fixed_final and fixed_initial respectively. You're getting away with it because the initial conditions you provided must have both alphas equal by construction. You would still be better off either removing the constraint, or adding at least one end of the linkage or the other as a design variable.

2. traj.phases.stage_1_maneuver.path_constraints.path:q_alpha means that at least one of the entries in your path constraint is not affected by any DV. It is likely not the entire path constraint, but just one end of it that is fixed because its computed from fixed boundary conditions. In this case, you can simply add indices to the add_path_constraint call to exclude the first or last point from the constraint.

Since your optimization is running, the 0 rows aren't killing you. However, its good practice to clean this up. It's possible that not having them will improve performance now, or it may save you from a future situation where the optimization "mysteriously" stops working because you somehow trigger a situation where the optimizer can no longer handle the 0 rows.

Not the most elegant solution, but here I go
Basically first step is to filter the step for each GENENAME, after that I create a loop to gather all the CellType2 and keep appending them.

I think you can use PIVOT to achieve this. Using directly this functionality would not work because you are looking for symmetry of columns and rows.

The trick is to force that symmetry, appending values from Point-B into values of Point-A.

1. Create a secondary table and reorder the columns in the opposite way that the original table, so Tag, Point-B and Point-A.
2. On the secondary table, rename the columns to Tag, Point-A and Point-B in that order. Append usually take column names literally, so without renaming it would append the names of the same columns.
3. Pivot on column Point-B without aggregating data.
4. Reorder the columns using Point-A as a reference, so you have symmetry of columns and rows.

It's worth mentioning that's good practice to Buffer the source table because is used multiple times across the calculations.

There is a pattern in your dataframe. You can remove every fifth row to get your desired result:

Does this work?

demo:db<>fiddle

I tend to use the table that shows the norms from the driver and model perspectives.

In this case the driver values are all scaled on the order of 1 or so. If they're huge, then the scalers/refs may need to be adjusted. (Scaling to the order of around 1 isn't guaranteed to be a good strategy, but it's usually a pretty good going in position.

You may want to do this after limiting the driver iterations. For a converged case in dymos, the values of the defect constraints are going to be close to zero and this may not give you a good idea of their initial values.

For defect_refs, for instance, if I note that the value of the constraint were something like 5.5E4, then I might set that defect ref to 1.0E-4. Again, unit scaling isn't always correct, but it does frequently work.

Here is query that will give you all the relationships but as I posted in comments, there most likely are many relationships between tables that just dont have keys for the associations.

As pointed out by @r2evans, this problem could be solved by igraph library:

At the request of the OP, I am applying the method used in a previous answer Visualizing the result of dividing the network into communities to this problem.

The network in the question was not created with a specified random seed. Here, I specify the seed for reproducibility.