grain | The Entire History of You

I have a table that looks something like

...

I'm working in overleaf, and the vertical lines I've inserted into my table (using tabular) are different lengths. I've attached code and a pic of the output.

Specifically, I'm wondering why the two lines on either side of the "year" column are longer than all the others and why they extend beyond the top and bottom horizontal lines. I'm also wondering why the vertical lines on either side of the other columns are not continuous (see the gaps under the horizontal line under each crop name and below the double horizontal line at the top of the table). If it's important, this table was generated using stargazer in RStudio. I manually added the vertical lines. Thanks!

...

I have a bunch of instances, each having a unique tempfile for its use (save data from memory to disk and retrieve them later).

I want to be sure that at the end of the day, all these files are removed. However, I want to leave a room for a fine-grained control of their deletion. That is, some files may be removed earlier, if needed (e.g. they are too big and not important any more).

What is the best / recommended way to achieve this?

May thoughts on that

• The try-finalize blocks or with statements are not an option, as we have many files, whose lifetime may overlap each other. Also, it hardly admits the option of finer control.

• From what I have read, __del__ is also not a feasible option, as it is not even guaranteed that it will eventually run (although, it is not entirely clear to me, what are the "risky" cases). Also (if it is still the case), the libraries may not be available when __del__ runs.

• tempfile library seems promising. However, the file is gone after just closing it, which is definitely a bummer, as I want them to be closed (when they perform no operation) to limit the number of open files.

  • The library promises that the file "will be destroyed as soon as it is closed (including an implicit close when the object is garbage collected)."

    How do they achieve the implicit close? E.g. in C# I would use a (reliable) finalizer, which __del__ is not.

• atexit library seems to be the best candidate, which can work as a reliable finalizer instead of __del__ to implement safe disposable pattern. The only problem, compared to object finalizers, is that it runs truly at-exit, which is rather inconvenient (what if the object eligible to be garbage-collected earlier?).

  • Here, the question still stands. How the library achieves that the methods always run? (Except in a really unexpected cases with which is hard to do anything)

In ideal case, it seems that a combination of __del__ and atexit library may perform best. That is, the clean-up is both at __del__ and the method registered in atexit, while repeated clean-up would be forbidden. If __del__ was called, the registered will be removed.

The only (yet crucial) problem is that __del__ won't run if a method is registered at atexit, because a reference to the object exists forever.

Thus, any suggestion, advice, useful link and so on is welcomed.

...

I have AWS Elastic service domain setup, I'm trying to push some data from an ec2 instance to the AWS elasticservice via Logstash.

I have setup the AWS ES domain with Open access policy - Allow all traffic. I have also enabled Fine Grained Control and setup a master user account to access the AWS ES service.

It all works fine with Kibana or regular cURL calls, but logstash fails with a request sending to https:///_license with a 401 response.

I'm not able to figure out why is this call happening. When I try to hit this in browser, I get {"Message":"Your request: '/_license' is not allowed."}

Here is the sample log that I get from logstash:

...

There is a couple of confusing points in the documentation that make me struggle to understand how exactly distribution across the cluster happens in Orleans. Hence, the questions.

Question #1

Orleans claims to have a built-in distribution capabilities to distribute across multiple servers. To me it sounds that Orleans can act as a load balancer itself and can scale out automatically. Thus, if I deploy Orleans app to several servers, then service discovery and load management should happen automatically, correct?

In this case, why some docs and articles suggest using other tools, like Ocelot or Consul, as a single entry point to Orleans cluster?

Question #2

I would like to use simple but distributed in-memory storage across several servers, like Redis or Apache Ignite, and I would like to know if it's possible to use a simple grain as this kind of a data storage?

Let's say, one grain will store a collection of restaurants and some other grain will keep track of the last 1000 visitors for selected restaurant. Can I activate these 2 grains only once as a singleton collection, add or remove records to each collection, and use these 2 grains as in-memory storage evenly available to all nodes in the cluster? Also, if answer is yes, do I need to add locks to these collections or each grain always exists in a single thread?

...

I am pretty good with simple Pandas but am struggling with data reshaping and multi indices. I have a multindex dataframe that looks like so (it doesnt have to be a multindex but it seems the right thing to do)

name index f1 f2 f3 calc1 calc2 calc3 fox 1 red white fur 0.21 1.67 -0.34 2 0.76 2.20 -1.02 3 0.01 1.12 -0.22 chicken 1 white yellow feathers 0.04 1.18 -2.01 2 0.18 0.73 -1.21 grain 1 yellow bag corn 0.89 1.65 -1.03 2 0.34 2.45 -0.45 3 0.87 1.11 -0.97

and all I want is:

name f1 f2 f3 calc1_1 calc2_1 calc3_1 calc1_2 calc2_2 calc3_2 calc1_3 calc2_3 calc3_3 fox red white fur 0.21 1.67 -0.34 0.76 2.20 -1.02 0.01 1.12 -0.22 chicken white yellow feathers 0.04 1.18 -2.01 0.18 0.73 -1.21 NaN NaN NaN grain yellow bag corn 0.89 1.65 -1.03 0.34 2.45 -0.45 0.87 1.11 -0.97

I figure this has got to be an easy one for the pandas gurus out there. Thanks all for your help!!

Drew

...

I want to aggregate raster data to each polygon in a custom shapefile.

In this case, I want to obtain the mean degree of urbanization within subnational regions in Subsaharan Africa.

My sf looks like this:

...

I am trying to iterate over a dictionary (inside another dictionary specifically, but I don't think that part matters), and I'm unable to get the for loop to iterate over the values in the order that they were placed in. I would like to be able to take the first value of each dictionary. I thought that after python 3.6, dictionaries kept their order (here), but these wont stay in order.

This is what my dictionary looks like:

...