virgil | The font that powers Excalidraw | User Interface library

You may use this single line mapfile + grep + printf solution:

Using Array.filter and Array.some seems to be the easiest way:

Why after clearing a device or using a new device does the code tell me that a user is already registered?

Short answer: There is no local private key within a cleared device or within a new device.

From the official documentation.

The EThree.register() function checks whether a user already has a private key saved in local storage, and a published public key on the Virgil Cloud. If the user doesn't have them, the function generates a new keypair for the user, saves the private key locally...

Why does hasPrivateKey return true but then when using authEncrypt there is an error saying there it no private key in the device?

LocalKeyStorage.retrieveKeyPair() can be used to define private key presence.

To support login across multiple-devices Key Backup mechanism can be used.

When you use item() on a Series you should actually have received:

Without trying it out I believe the issue is the int() in the return of MatchIDLookup() function. Pandas usually doesn't allow this. Instead, return the value without conversion to int and then add below:

df_pm['MatchID'] = df_pm['MatchID'].astype(int)

P.S. Also, I would generally advise against converting any type of IDs to integers but keeping it as strings - simple reason if an id starts with zero (0654 or 0012) by converting it to integer you will lose the 4 digit format.

EDIT:

you given URL in HttpWebRequest is looking wrong comparing with this URL https://bdm.insee.fr/series/sdmx/data/SERIES_BDM/000008630?startPeriod=2019

try this

In terms of performance you should always try to use the pyspark functions over python functions. Pyspark functions are optimized to utilize the ressource of your cluster and the data doesn't need to be converted to python objects.

The appropriate pyspark functions for your use case is rlike. Have a look at the example below:

I am not only looking at the code solution but more on the off-the-shelf code provided from PySpark.

Unfortunately, Spark is designed to operate in parallel on a row-by-row basis. Filtering out columns is not something for which there will be an "off-the-shelf code" solution.

Nevertheless, here is one approach you can take:

First collect the counts of the invalid elements in each column.