vignette | Guesses and gossip about counts

The vignette you offer says the logsum is calculated as:

To my reading that is similar to the calculation used to construct the "linear predictor". the lp is t(coef(clog)) %*% Xhat. If I'm correct on that interpretation, then that is stored in the clog-object:

TL;DR: set a new column in every list-element of data in the setup_data function.

It seems that setup_params and setup_data are meant for this exact use case.

That's right, but I get the impression from your question that some confusion exists about the order of operations of data ingestion. Facets and coordinates are part of the 'layout' of a plot. Before the layout is setup, layers setup their data (sometimes making a copy of the global data). Then, the layout can inspect the data and make adjustments (typically appending a PANEL column). If we inspect/print to console ggplot2:::Layout$setup, we see the following (comments by me):

The (currently very new) tsetsad package does this for the context of ETS ("smoothing") time series models, relying on facilities of package TMB. This is then used by package tsets which itself uses RcppArmadillo.

All this is fairly new and I have not had a chance to poke around much myself -- but it provides a working demonstration which is quite exciting.

I think it is best to understand how things work by example. I will comment on how DataFrames.jl is documented as this is a pretty standard approach:

1. Functions are documented using docstrings, here is an example of a docstring of function names; These docstrings are then discoverable interactively via help system (by pressing ?)
2. A standard way to generate a documentation for a package is to use Documenter.jl; by the way: the package has a great team of maintainers who are very helpful and responsive; here you have a link to the make.jl file that is executed to generate the documentation; note in particular the option doctest=true which makes sure that all code examples that are properly anoteted following Documenter.jl rules are producing an expected output
3. In order to set-up auto-generation of package documentation you need to set up CI integration on GitHub; again - there are many ways to do it; a standard one is to use GitHub Actions; here you have a link to the part of the ci.yml specification file that ensures that documentation is built as a part of CI; then in any PR, e.g. this one (I am giving a link to a currently open PR that is documentation related) you can see in the section reporting CI results that after running tests also documentation was generated. Here you can see how a documentation generated using this toolchain looks like (e.g. note that at the bottom you can switch the version of the package you want to read manual of dynamically which shows you that all here is really well integrated - not just a bunch of PDF or HTML files).

I hope this will help you to get started. I have pointed you to all essential pieces that are normally used by packages hosted on GitHub (i.e. this is not the only way to do it, but it is a standard way most commonly used).

To master all the details of the above you need to read the documentation in the Julia Manual and Documenter.jl carefully. Unfortunately writing a proper documentation is not easy (in any programming language). The good thing is that Julia has a really excellent toolchain that supports this process very well.

You can try with rowSums -

There are a number of issues here.

The proximal problem is that you have a (near) singular fit: glmmTMB is trying to make the variance zero (5.138e-08 is as close as it can get). Because it fits on a log-variance (actually log-standard-deviation) scale, this means that it's trying to go to -∞, which makes the covariance matrix of the parameters impossible to estimate.

The main reason this is happening is that you have a very small number of groups (3) in your random effect (experiment).

These are extremely common issues with mixed models: you can start by reading ?lme4::isSingular and the relevant section of the GLMM FAQ.

The simplest solution would be to treat experiment as a fixed effect, in which case you would no longer have a mixed model and you could back to plain glm().

Another slightly worrying aspect of your code is the response variable cbind(SARA_ph58, 1). If SARA_ph58 is a binary (0/1) variable you can use just SARA_ph58. If you pass a two-column matrix as you are doing, the first column is interpreted as the number of successes and the second column as the number of failures; it looks like you may have been trying to specify that the total number of trials for each observation is 1 (again, if this is the case you can just use SARA_ph58 as the response).

One final note is that lme4::glmer is a little more tolerant of singular fits than glmmTMB.

The chapter "Using %:% with %dopar%" from documentation you provided gives a useful hint:

all of the tasks are completely independent of each other, and so they can all be executed in parallel

The %:% operator turns multiple foreach loops into a single loop. That is why there is only one %do% operator in the example above. And when we parallelize that nested foreach loop by changing the %do% into a %dopar%, we are creating a single stream of tasks that can all be executed in parallel.

When you combine two %dopar% and measure execution time, you see that only the outer loop is executed in parallel, this is probably not what you're looking for :

You can use the sample() function to grab a value at random. If your dataframe is called df, I'd think something like this for your gghighlight() line: