random-scripts | Just some random scripts | Generator Utils library

Imagine we have an original API that returns a generator (it really is a mechanisms that brings pages/chunks of results from a server while the providing a simple generator to the user, and lets him iterate over these results one by one. For simplicity:

I am working on a large Pandas DataFrame which needs to be converted into dictionaries before being processed by another API.

The required dictionaries can be generated by calling the .to_dict(orient='records') method. As stated in the docs, the returned value depends on the orient option:

Returns: dict, list or collections.abc.Mapping

Return a collections.abc.Mapping object representing the DataFrame. The resulting transformation depends on the orient parameter.

For my case, passing orient='records', a list of dictionaries is returned. When dealing with lists, the complete memory required to store the list items, is reserved/allocated. As my dataframe can get rather large, this might lead to memory issues especially as the code might be executed on lower spec target systems.

I could certainly circumvent this issue by processing the dataframe chunk-wise and generate the list of dictionaries for each chunk which is then passed to the API. Furthermore, calling iter(df.to_dict(orient='records')) would return the desired generator, but would not reduce the required memory footprint as the list is created intermediately.

Is there a way to directly return a generator expression from df.to_dict(orient='records') instead of a list in order to reduce the memory footprint?

For the below code

Why does

I have been struggling for a long time to figure how to define a generator function of a ruler sequence in Python, that follows the rules that the first number of the sequence (starting with 1) shows up once, the next two numbers will show up twice, next three numbers will show up three times, etc.

So what I am trying to get is 1, 2, 2, 3, 3, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 7 etc.

I understand that the way to do this is to have two separate count generators (itertools.count(1)) and then for every number in one generator yield number from the other generator:

I'm wondering about code like this:

I'm applying a CNN to classify a given dataset.

My function:

Let's say I want to rotate class names for my button on click. Clicked once becomes button-green, twice - button-yellow, thrice - button-red. And then it repeats, so fourth click makes it button-green again.

I know other techniques how to do it, I'm not asking for implementation advice. I made up this example to understand something about generators in JavaScript.

Here's my code with generator:

I have a list of generators in a function alternate_all(*args) that alternates between each generator in the list to print their first item, second item, ..., etc. until all generators are exhausted.

My code works until a generator is exhausted and once the StopIteration occurs, it stops printing, when I want it to continue with the rest of the generators and ignore the exhausted one: