json_dump | JSONをフォーマットして見やすくするツール | JSON Processing library

Below is the given dataframe coming from sql for eg:

id days type delta A B 1 30 X 1 0.1 0.5 1 30 Y 2 0.2 0.6 1 60 X 1 0.3 0.7 1 60 Y 2 0.4 0.8

Transforming data frame, I want to achieve something like this:

id day_30 day_60 1 {A: {X1: 01, Y2: 0.2}, B: {X1: 0.5, Y2: 0.6} {A: {X1: 0.3, Y2: 0.4}, B: {X1: 0.7, Y2: 0.8}

Currently I am able to achieve this by group by Id, then group by Days and do iterrows. But it's very slow for millions of rows. How can I make it faster any apply or vectorization example to do the same would be very helpfull.

current code:

...

I have a tkinter app with about 1000 users which is collecting data from a game. Every 30 seconds, it saves the collected data to a json file. After having a couple of users experiencing corrupted save files after power outages, I tried implementing a safe json saving function that would always ensure an intact file.

However, after implementing this function a few other users (maybe 1 out of 100) are now getting permission errors during the autosave function. I am not able to replicate the bug on my own machine.

I use the function below for saving, which was inspired from this thread How to make file creation an atomic operation?

...

How do I do a random click somewhere on the page after n seconds to keep the session alive while using selenium? Scraping is going on, if we don't do something it will disconnect the session. Is there any other way to do some activity/keep the session alive?

...

I want to use my keras model on different computers with different python versions

I don't want to use Pickle and numpy.savez because it causes problems in different environments

I start with that and it works fine

...

I have a use case where I am required to append multiple json files and then convert them into 1 single Avro file. I have written the code below which appends the json files together and then convert them into AVRO file. But the issue I am having is that the JSON file gets appended but the entore JSON is enclosed in [] brackets and so I get error while converting it into AVRO file. I am trying to figure out how can I get rid of the [] from the first and the last line in JSON file? Any help is appreciated.

The error I am getting is (snippet of the error, error is too long to paste : avro.io.AvroTypeException: The datum [{'event_type': 'uplink'.....}] is not an example of the schema

My code: Laird.py

...

I am new to AVRO and Python. I have a use case where I want to convert the JSON file into an Avro file. I have stored my schema in .avsc format and the JSON data in .json format. Now I want to put the JSON file and .avsc file together and serialize the JSON file into Avro. Below is my code, I am getting an error that "avro.io.AvroTypeException: The datum file.json is not an example of the schema". I am not sure what I am doing wrong. In my writer.append statement I want the script to take data from JSON file and append it in .avro file but in a serialized format. I am not sure how to handle this, any help will be highly appreciated.

Here is my code:

...

UPDATE 7/23/2020

I have a Django project where I am trying to pass a list of values from a python function to my 'index.html' file. This list of values comes from a DB query in my views.py file

my views.py

...

I have a c program which sends the data on every even index over MQTT and once the data is sent, the framed json is cleared and start constructing the new one and the process repeats.

...

Below is the code which I have been using to send data through MQTT on every even index while iterating for loop,

...