Popular New Releases in Dictionary
addict
freeze/unfreeze
Box
Version 6.0.2
pyglossary
PyGlossary 4.5.0
bidict
v0.22.0
yomichan
22.4.4.0 (testing)
Popular Libraries in Dictionary
by goldendict c
3881 NOASSERTION
A feature-rich dictionary lookup program, supporting multiple dictionary formats (StarDict/Babylon/Lingvo/Dictd) and online dictionaries, featuring perfect article rendering with the complete markup, illustrations and other content retained, and allowing you to type in words without any accents or correct case.
by skywind3000 python
2785 MIT
Free English to Chinese Dictionary Database
by mewwts python
2057 MIT
The Python Dict that's better than heroin.
by cdgriffith python
1950 MIT
Python dictionaries with advanced dot notation access
by jjgod c
1510
Dictionary conversion tool for Mac OS X 10.5 and above
by ilius python
1485 NOASSERTION
A tool for converting dictionary files aka glossaries. Mainly to help use our offline glossaries in any Open Source dictionary we like on any modern operating system / device.
by jab python
1047 MPL-2.0
The bidirectional mapping library for Python.
by ChestnutHeng python
771
有道词典的命令行版本,支持英汉互查和在线查询。
by Debdut shell
686 Apache-2.0
A Global Exhaustive First and Last Name Database
Trending New libraries in Dictionary
by Debdut shell
686 Apache-2.0
A Global Exhaustive First and Last Name Database
by jegnux swift
378 MIT
Powerful property wrapper to back codable properties.
by felixonmars python
365 Unlicense
Fcitx 5 Pinyin Dictionary from zh.wikipedia.org
by ijemmao javascript
145
An API exposing Igbo words, definitions, and more
by BoboTiG python
90 MIT
Finally decent dictionaries based on Wiktionary for your beloved eBook reader.
by ikey4u rust
88 MIT
Wikit - A universal dictionary
by lzxue javascript
83
新冠肺炎,疫情地图
by mcnaveen javascript
71 MIT
🦄 Get Random Words (with pronunciation) for Free using this API
by pry0cc python
60
A wordlist that is kept up to date with the latest headlines to provide relevant words to human society
Top Authors in Dictionary
1
6 Libraries
224
2
4 Libraries
777
3
4 Libraries
12
4
4 Libraries
10
5
4 Libraries
9
6
3 Libraries
64
7
3 Libraries
31
8
3 Libraries
21
9
3 Libraries
32
10
3 Libraries
15
1
6 Libraries
224
2
4 Libraries
777
3
4 Libraries
12
4
4 Libraries
10
5
4 Libraries
9
6
3 Libraries
64
7
3 Libraries
31
8
3 Libraries
21
9
3 Libraries
32
10
3 Libraries
15
Trending Kits in Dictionary
In Python, you can access the elements in a dictionary by following methods:
- To retrieve the value connected to a particular key, use the square bracket notation "[]".
- The "get()" function can also be used to retrieve dictionary items.
- get(): The dictionary's built-in Python get() function enables you to obtain the value associated with a particular key without throwing a KeyError if the key cannot be located in the dictionary.
- The "in keyword" may also be used to determine whether a key is included in the dictionary.
- in keyword: In Python, the in keyword is used to check if a specific item or value is present in a collection, such as a list, tuple, string, or dictionary.
- You can use the “keys()” method to access all the keys in a dictionary and the “values()” method to access all the values.
- keys(): In Python, the keys() method is a built-in method of dictionaries that returns a view object containing the keys of the dictionary.
- values(): In Python, the values() method is a built-in method of dictionaries that returns a view object containing the dictionary’s values.
Please review the code below to access elements in Python.
Fig : Preview of the output that you will get on running this code from your IDE.
Code
Instructions
Follow the steps carefully to get the output easily.
- Install python on your IDE(Any of your favorite IDE).
- Copy the snippet using the 'copy' and paste it in your IDE.
- Run the file to generate the output.
I hope you found this useful.
I found this code snippet by searching for 'how to access elements in Python dictionary' in kandi. You can try any such use case!
Environment Tested
I tested this solution in the following versions. Be mindful of changes when working with other versions.
- The solution is created in PyCharm 2021.3.
- The solution is tested on Python 3.9.7.
Using this solution, we are able to access elements in python dictionary with simple steps. This process also facilities an easy way to use, hassle-free method to create a hands-on working version of code which would help us to access elements in python dictionary.
Support
- For any support on kandi solution kits, please use the chat
- For further learning resources, visit the Open Weaver Community learning page.
A built-in data structure in Python called a dictionary holds things in a key-value manner. It resembles a list or an array, but instead of utilizing an index number to retrieve the objects, it uses a key.
A list is a pre-built data structure in Python that is used to hold an ordered collection of things. These items, such as numbers, strings, or other objects, can be of any type. You can add a value to a dictionary from a list in Python by using the following methods:”
- Using a for loop: Iterate through the list and add each item to the dictionary using the dictionary's “update()” method.
- update(): In Python, the update() method is a built-in method of the dictionary data structure that allows you to add, update, or remove items from a dictionary.
- Using the dict() constructor: Pass the list of items as a key-value pair to the “dict()” constructor.
- dict(): Python’s dict() constructor is used to create a new dictionary.
- Using a list comprehension: Create a new list of key-value pairs, then pass it to the dict() constructor.
Please review the code below to learn how to add values to a dictionary in Python from a list.
Fig : Preview of the output that you will get on running this code from your IDE.
Code
Instructions
Follow the steps carefully to get the output easily.
- Install python on your IDE(Any of your favorite IDE).
- Copy the snippet using the 'copy' and paste it in your IDE.
- Run the file to generate the output.
I hope you found this useful.
I found this code snippet by searching for 'how to add a value into dictionary from a list in Python' in kandi. You can try any such use case!
Environment Tested
I tested this solution in the following versions. Be mindful of changes when working with other versions.
- The solution is created in PyCharm 2021.3.
- The solution is tested on Python 3.9.7.
Using this solution, we are able to add a value to a dictionary from a list in python with simple steps. This process also facilities an easy way to use, hassle-free method to create a hands-on working version of code which would help us to add a value to a dictionary from a list in python.
Support
- For any support on kandi solution kits, please use the chat
- For further learning resources, visit the Open Weaver Community learning page.
By first getting the list using the key and then adding an element to the list using the "append()" function, you may append to a list that is a value in a dictionary in Python.
- append(): In Python, an element is added to the end of a list using the append() function. It does not produce a new list but rather alters the existing list. It just requires a single parameter, the element to be added, and updates the list in place.
You can also use the “setdefault()” method to append to a list value in a dictionary if the key does not exist.
- setdefault(): The setdefault() method is a built-in method of dictionaries in Python that is used to insert a key-value pair into a dictionary if the key is not already present.
You can also use “defaultdict” from the collections library.
- defaultdict: defaultdict is a class in the collections module in Python that is used to create a dictionary with a default value for keys that do not exist. It creates a dictionary-like object that returns a default value when a non-existent key is accessed.
Please review the code below to learn how to append values to a list in a dictionary in Python.
Fig : Preview of the output that you will get on running this code from your IDE.
Code
Instructions
Follow the steps carefully to get the output easily.
- Install python on your IDE(Any of your favorite IDE).
- Copy the snippet using the 'copy' and paste it in your IDE.
- Run the file to generate the output.
I hope you found this useful.
I found this code snippet by searching for 'Appending to a list in a dictionary in python' in kandi. You can try any such use case!
Environment Tested
I tested this solution in the following versions. Be mindful of changes when working with other versions.
- The solution is created in PyCharm 2021.3.
- The solution is tested on Python 3.9.7.
Using this solution, we are able to appending to a list in a dictionary in python with simple steps. This process also facilities an easy way to use, hassle-free method to create a hands-on working version of code which would help us to appending to a list in a dictionary in python.
Support
- For any support on kandi solution kits, please use the chat
- For further learning resources, visit the Open Weaver Community learning page.
In Python, you may loop through a dictionary's key-value pairs using the "items()" function. Then, you can use the "in" operator to check if the key is in the other dictionary. You may compare the values if it's there. You can include it in a list of differences if it is absent.
- The items() method in Python retrieves a view object. It contains a list of (key, value) pairs. The view object can be iterated over to access the key-value pairs of the dictionary. This is useful when you want to loop through a dictionary's key-value pairs. You can then operate on each of them. It is also useful when you want to create a copy of a dictionary and keep it updated with the original dictionary.
- The in-operator in Python checks if a value is in a container. This could be a list, tuple, set, or dictionary. It returns True if the value is found in the container and False otherwise. It is a very useful operator when you want to check if a value is present in a container data type. It saves time and features of code.
For more information about comparing two dictionaries with more values in one in Python, refer to the code below.
Fig : Preview of the output that you will get on running this code from your IDE.
Code
Instructions
Follow the steps carefully to get the output easily.
- Install python on your IDE(Any of your favorite IDE).
- Copy the snippet using the 'copy' and paste it in your IDE.
- Run the file to generate the output.
I hope you found this useful.
I found this code snippet by searching for 'Compare two dictionary with more value in one' in kandi. You can try any such use case!
Environment Tested
I tested this solution in the following versions. Be mindful of changes when working with other versions.
- The solution is created in PyCharm 2021.3.
- The solution is tested on Python 3.9.7.
- Pandas version-v1.5.2.
Using this solution, we are able to compare two dictionaries with more values in one in python with simple steps. This process also facilities an easy way to use, hassle-free method to create a hands-on working version of code which would help us to compare two dictionaries with more values in one in python.
FAQ
1.How can I compare two dictionaries in Python?
You can loop through a dictionary's key-value pairs using the items() function. Then, check if each key is in the other dictionary using the in operator. Then, you can compare the values if the key exists and include any differences in a list.
2.What does the items() method do in Python dictionaries?
The items() method gets a view object. It has a list of (key, value) pairs from the dictionary. You can iterate over this view object to access the key-value pairs. This makes it useful for looping through a dictionary's contents. It's also good for copying the dictionary.
3.How does the in operator work with dictionaries in Python?
The in operator checks if a value (or key) is present in a container, including dictionaries. It returns True if it finds the value in the dictionary and False if it does not. This operator is efficient and used when you need to check for the existence of a key in a dictionary.
4.What is the benefit of using the items() method to iterate over dictionary elements?
You can use the items() method to access the key-value pairs of the dictionary. This makes it easy to work on each pair in a loop. Additionally, it is concise and readable. It is better for working with dictionary elements than other methods.
5.Can I update one dictionary with the contents of another in Python?
Yes, you can update one dictionary with the contents of another. Iterate over the key-value pairs of the source dictionary using the items() method. Then, update the target dictionary. This approach ensures the target dictionary reflects changes to the source.
Support
- For any support on kandi solution kits, please use the chat
- For further learning resources, visit the Open Weaver Community learning page.
A dictionary is a pre-built data structure in Python that holds a set of key-value pairs. Each key is associated with a value, and you can use the key to access the corresponding value.
In Python, dictionaries can be used inside functions like any other data type. You can also modify a dictionary inside a function by adding, removing, or updating items. It's important to note that when passing a dictionary to a function, it will be passed by reference, not by value, so any changes made to the dictionary inside the function will affect the original dictionary.
You may traverse through the key-value pairs using a "for" loop and the "items()" function.
- for loop: Python's for loop is a control structure that enables you to repeatedly iterate over a series of items, such as a list, a tuple, a string, or a range, and to run a block of code for each one of them.
- items(): The items() method is a built-in method of dictionaries in Python that returns a view object containing the key-value pairs of the dictionary as tuples.
Please check the code below to use the dictionaries inside a function in Python.
Fig : Preview of the output that you will get on running this code from your IDE.
Code
Instructions
Follow the steps carefully to get the output easily.
- Install python on your IDE(Any of your favorite IDE).
- Copy the snippet using the 'copy' and paste it in your IDE.
- Run the file to generate the output.
I hope you found this useful.
I found this code snippet by searching for 'Using dictionary inside a function in python' in kandi. You can try any such use case!
Environment Tested
I tested this solution in the following versions. Be mindful of changes when working with other versions.
- The solution is created in PyCharm 2021.3.
- The solution is tested on Python 3.9.7.
Using this solution, we are able to using dictionary inside a function in python with simple steps. This process also facilities an easy way to use, hassle-free method to create a hands-on working version of code which would help us to using dictionary inside a function in python.
Support
- For any support on kandi solution kits, please use the chat
- For further learning resources, visit the Open Weaver Community learning page.
A Python dictionary is a key-value pairs collection where each key is unique. It is like a real-life dictionary in which a word is a key, and its definition is the value.
Dictionaries are also known as associative arrays, maps, or hash maps. Python dictionaries are used for many purposes because of their efficient and easy-to-use data structure. Here are a few common use cases for dictionaries:
- Storing key-value pairs: Dictionaries are well suited for storing key-value pairs, where the key is a unique identifier.
- Representing complex data: Dictionaries can be nested inside other dictionaries or inside lists, which allows you to represent complex data structures.
- Handling large data sets: Dictionaries are implemented using a hash table, which makes them very efficient for searching, inserting, and deleting elements.
- Counting and grouping: Dictionaries are commonly used to count the item occurrences in a list or group items by certain criteria.
- Caching: Dictionaries can be used as a cache to store the results of expensive operations to quickly retrieve the next time the operation is needed.
For more information about a Python Dictionary, refer to the code below.
Fig : Preview of the output that you will get on running this code from your IDE.
Code
Instructions
Follow the steps carefully to get the output easily.
- Install python on your IDE(Any of your favorite IDE).
- Copy the snippet using the 'copy' and paste it in your IDE.
- Run the file to generate the output.
I hope you found this useful.
I found this code snippet by searching for 'How to use a Python dictionary'in kandi. You can try any such use case!
Environment Tested
I tested this solution in the following versions. Be mindful of changes when working with other versions.
- The solution is created in PyCharm 2021.3.
- The solution is tested on Python 3.9.7.
Using this solution, we are able to use a python dictionaries with simple steps. This process also facilities an easy way to use, hassle-free method to create a hands-on working version of code which would help us to use a python dictionaries.
Support
- For any support on kandi solution kits, please use the chat
- For further learning resources, visit the Open Weaver Community learning page.
Trending Discussions on Dictionary
Unknown host CPU architecture: arm64 , Android NDK SiliconM1 Apple MacBook Pro
Filter a dictionary of lists
FastAPI - GET request results in typeerror (value is not a valid dict)
Java, project panama and how to deal with Hunspell 'suggest' result
Explode pandas column of dictionary with list of tuples as value
how Julia determines index of dictionary keys?
Pythonic way to make a dictionary from lists of unequal length without padding Nones
Different access time to a value of a dictionary when mixing int and str keys
Confusion regarding the Blocking of "peer threads" when a user-level thread blocks
How can I document methods inherited from a metaclass?
QUESTION
Unknown host CPU architecture: arm64 , Android NDK SiliconM1 Apple MacBook Pro
Asked 2022-Apr-04 at 18:41I've got a project that is working fine in windows os but when I switched my laptop and opened an existing project in MacBook Pro M1. I'm unable to run an existing android project in MacBook pro M1. first I was getting
Execution failed for task ':app:kaptDevDebugKotlin'. > A failure occurred while executing org.jetbrains.kotlin.gradle.internal.KaptExecution > java.lang.reflect.InvocationTargetException (no error message)
this error was due to the Room database I applied a fix that was adding below library before Room database and also changed my JDK location from file structure from JRE to JDK.
kapt "org.xerial:sqlite-jdbc:3.34.0"
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6
after that now I'm getting an issue which is Unknown host CPU architecture: arm64
there is an SDK in my project that is using this below line.
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16
App Gradle
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23
[CXX1405] error when building with ndkBuild using /Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/Android.mk: Build command failed. Error while executing process /Users/mac/Library/Android/sdk/ndk/21.4.7075529/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/Android.mk APP_ABI=arm64-v8a NDK_ALL_ABIS=arm64-v8a NDK_DEBUG=1 APP_PLATFORM=android-21 NDK_OUT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/build/intermediates/cxx/Debug/4k4s2lc6/obj NDK_LIBS_OUT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/build/intermediates/cxx/Debug/4k4s2lc6/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n} ERROR: Unknown host CPU architecture: arm64
which is causing this issue and whenever I comment on this line
path 'Android.mk'
it starts working fine, is there any way around which will help me run this project with this piece of code without getting this NDK issue?
Update - It seems that Room got fixed in the latest updates, Therefore you may consider updating Room to latest version (2.3.0-alpha01 / 2.4.0-alpha03 or above)
ANSWER
Answered 2022-Apr-04 at 18:41To solve this on a Apple Silicon M1 I found three options
AUse NDK 24
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27
You can install it with
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28
or
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29
Depending what where sdkmanager
is located
Change your ndk-build
to use Rosetta x86. Search for your installed ndk with
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30
eg
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31
and change
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31DIR="$(cd "$(dirname "$0")" && pwd)"
32$DIR/build/ndk-build "$@"
33
to
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31DIR="$(cd "$(dirname "$0")" && pwd)"
32$DIR/build/ndk-build "$@"
33DIR="$(cd "$(dirname "$0")" && pwd)"
34arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
35
QUESTION
Filter a dictionary of lists
Asked 2022-Mar-24 at 07:56I have a dictionary of the form:
1{"level": [1, 2, 3],
2 "conf": [-1, 1, 2],
3 "text": ["here", "hel", "llo"]}
4
I want to filter the lists to remove every item at index i
where an index in the value "conf"
is not >0.
So for the above dict
, the output should be this:
1{"level": [1, 2, 3],
2 "conf": [-1, 1, 2],
3 "text": ["here", "hel", "llo"]}
4{"level": [2, 3],
5 "conf": [1, 2],
6 "text": ["hel", "llo"]}
7
As the first value of conf
was not > 0.
I have tried something like this:
1{"level": [1, 2, 3],
2 "conf": [-1, 1, 2],
3 "text": ["here", "hel", "llo"]}
4{"level": [2, 3],
5 "conf": [1, 2],
6 "text": ["hel", "llo"]}
7new_dict = {i: [a for a in j if a >= min_conf] for i, j in my_dict.items()}
8
But that would work just for one key.
ANSWER
Answered 2022-Feb-21 at 05:50I believe this will work:
For each list, we will filter the values where conf
is negative, and after that we will filter conf
itself.
1{"level": [1, 2, 3],
2 "conf": [-1, 1, 2],
3 "text": ["here", "hel", "llo"]}
4{"level": [2, 3],
5 "conf": [1, 2],
6 "text": ["hel", "llo"]}
7new_dict = {i: [a for a in j if a >= min_conf] for i, j in my_dict.items()}
8d = {"level":[1,2,3], "conf":[-1,1,2], "text":["-1","hel","llo"]}
9for key in d:
10 if key != "conf":
11 d[key] = [d[key][i] for i in range(len(d[key])) if d["conf"][i] >= 0]
12d["conf"] = [i for i in d["conf"] if i>=0]
13print(d)
14
A simpler solution will be (exactly the same but using list comprehension, so we don't need to do it separately for conf
and the rest:
1{"level": [1, 2, 3],
2 "conf": [-1, 1, 2],
3 "text": ["here", "hel", "llo"]}
4{"level": [2, 3],
5 "conf": [1, 2],
6 "text": ["hel", "llo"]}
7new_dict = {i: [a for a in j if a >= min_conf] for i, j in my_dict.items()}
8d = {"level":[1,2,3], "conf":[-1,1,2], "text":["-1","hel","llo"]}
9for key in d:
10 if key != "conf":
11 d[key] = [d[key][i] for i in range(len(d[key])) if d["conf"][i] >= 0]
12d["conf"] = [i for i in d["conf"] if i>=0]
13print(d)
14d = {"level":[1,2,3], "conf":[-1,1,2], "text":["-1","hel","llo"]}
15
16d = {i:[d[i][j] for j in range(len(d[i])) if d["conf"][j] >= 0] for i in d}
17
Output:
{'level': [2, 3], 'conf': [1, 2], 'text': ['hel', 'llo']}
QUESTION
FastAPI - GET request results in typeerror (value is not a valid dict)
Asked 2022-Mar-23 at 22:191class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6
This is my model:
1class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6class Userattribute(Base):
7 __tablename__ = "user_attribute"
8
9 name = Column(String)
10 value = Column(String)
11 user_id = Column(String)
12 id = Column(String, primary_key=True, index=True)
13
In a crud.py I define a get_attributes
method.
1class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6class Userattribute(Base):
7 __tablename__ = "user_attribute"
8
9 name = Column(String)
10 value = Column(String)
11 user_id = Column(String)
12 id = Column(String, primary_key=True, index=True)
13def get_attributes(db: Session, skip: int = 0, limit: int = 100):
14 return db.query(models.Userattribute).offset(skip).limit(limit).all()
15
This is my GET
endpoint:
1class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6class Userattribute(Base):
7 __tablename__ = "user_attribute"
8
9 name = Column(String)
10 value = Column(String)
11 user_id = Column(String)
12 id = Column(String, primary_key=True, index=True)
13def get_attributes(db: Session, skip: int = 0, limit: int = 100):
14 return db.query(models.Userattribute).offset(skip).limit(limit).all()
15@app.get("/attributes/", response_model=List[schemas.Userattribute])
16def read_attributes(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
17 users = crud.get_attributes(db, skip=skip, limit=limit)
18 print(users)
19 return users
20
The connection to the database seems to work, but a problem is the datatype:
1class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6class Userattribute(Base):
7 __tablename__ = "user_attribute"
8
9 name = Column(String)
10 value = Column(String)
11 user_id = Column(String)
12 id = Column(String, primary_key=True, index=True)
13def get_attributes(db: Session, skip: int = 0, limit: int = 100):
14 return db.query(models.Userattribute).offset(skip).limit(limit).all()
15@app.get("/attributes/", response_model=List[schemas.Userattribute])
16def read_attributes(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
17 users = crud.get_attributes(db, skip=skip, limit=limit)
18 print(users)
19 return users
20pydantic.error_wrappers.ValidationError: 7 validation errors for Userattribute
21response -> 0
22 value is not a valid dict (type=type_error.dict)
23response -> 1
24 value is not a valid dict (type=type_error.dict)
25response -> 2
26 value is not a valid dict (type=type_error.dict)
27response -> 3
28 value is not a valid dict (type=type_error.dict)
29response -> 4
30 value is not a valid dict (type=type_error.dict)
31response -> 5
32 value is not a valid dict (type=type_error.dict)
33response -> 6
34 value is not a valid dict (type=type_error.dict)
35
Why does FASTApi expect a dictionary here? I don´t really understand it, since I am not able to even print the response. How can I fix this?
ANSWER
Answered 2022-Mar-23 at 22:19SQLAlchemy does not return a dictionary, which is what pydantic expects by default. You can configure your model to also support loading from standard orm parameters (i.e. attributes on the object instead of dictionary lookups):
1class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6class Userattribute(Base):
7 __tablename__ = "user_attribute"
8
9 name = Column(String)
10 value = Column(String)
11 user_id = Column(String)
12 id = Column(String, primary_key=True, index=True)
13def get_attributes(db: Session, skip: int = 0, limit: int = 100):
14 return db.query(models.Userattribute).offset(skip).limit(limit).all()
15@app.get("/attributes/", response_model=List[schemas.Userattribute])
16def read_attributes(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
17 users = crud.get_attributes(db, skip=skip, limit=limit)
18 print(users)
19 return users
20pydantic.error_wrappers.ValidationError: 7 validation errors for Userattribute
21response -> 0
22 value is not a valid dict (type=type_error.dict)
23response -> 1
24 value is not a valid dict (type=type_error.dict)
25response -> 2
26 value is not a valid dict (type=type_error.dict)
27response -> 3
28 value is not a valid dict (type=type_error.dict)
29response -> 4
30 value is not a valid dict (type=type_error.dict)
31response -> 5
32 value is not a valid dict (type=type_error.dict)
33response -> 6
34 value is not a valid dict (type=type_error.dict)
35class Userattribute(BaseModel):
36 name: str
37 value: str
38 user_id: str
39 id: str
40
41 class Config:
42 orm_mode = True
43
You can also attach a debugger right before the call to return
to see what's being returned.
Since this answer has become slightly popular, I'd like to also mention that you can make orm_mode = True
the default for your schema classes by having a common parent class that inherits from BaseModel
:
1class Userattribute(BaseModel):
2 name: str
3 value: str
4 user_id: str
5 id: str
6class Userattribute(Base):
7 __tablename__ = "user_attribute"
8
9 name = Column(String)
10 value = Column(String)
11 user_id = Column(String)
12 id = Column(String, primary_key=True, index=True)
13def get_attributes(db: Session, skip: int = 0, limit: int = 100):
14 return db.query(models.Userattribute).offset(skip).limit(limit).all()
15@app.get("/attributes/", response_model=List[schemas.Userattribute])
16def read_attributes(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
17 users = crud.get_attributes(db, skip=skip, limit=limit)
18 print(users)
19 return users
20pydantic.error_wrappers.ValidationError: 7 validation errors for Userattribute
21response -> 0
22 value is not a valid dict (type=type_error.dict)
23response -> 1
24 value is not a valid dict (type=type_error.dict)
25response -> 2
26 value is not a valid dict (type=type_error.dict)
27response -> 3
28 value is not a valid dict (type=type_error.dict)
29response -> 4
30 value is not a valid dict (type=type_error.dict)
31response -> 5
32 value is not a valid dict (type=type_error.dict)
33response -> 6
34 value is not a valid dict (type=type_error.dict)
35class Userattribute(BaseModel):
36 name: str
37 value: str
38 user_id: str
39 id: str
40
41 class Config:
42 orm_mode = True
43class OurBaseModel(BaseModel):
44 class Config:
45 orm_mode = True
46
47
48class Userattribute(OurBaseModel):
49 name: str
50 value: str
51 user_id: str
52 id: str
53
This is useful if you want to support orm_mode
for most of your classes (and for those where you don't, inherit from the regular BaseModel
).
QUESTION
Java, project panama and how to deal with Hunspell 'suggest' result
Asked 2022-Feb-24 at 21:41I'm experimenting with Hunspell and how to interact with it using Java Project Panama (Build 19-panama+1-13 (2022/1/18)). I was able to get some initial testing done, as in creating a handle to Hunspell
and subsequently using that to perform a spell check. I'm now trying something more elaborate, letting Hunspell give me suggestions
for a word not present in the dictionary. This is the code that I have for that now:
1public class HelloHun {
2 public static void main(String[] args) {
3 MemoryAddress hunspellHandle = null;
4 try (ResourceScope scope = ResourceScope.newConfinedScope()) {
5 var allocator = SegmentAllocator.nativeAllocator(scope);
6
7 // Point it to US english dictionary and (so called) affix file
8 // Note #1: it is possible to add words to the dictionary if you like
9 // Note #2: it is possible to have separate/individual dictionaries and affix files (e.g. per user/doc type)
10 var en_US_aff = allocator.allocateUtf8String("/usr/share/hunspell/en_US.aff");
11 var en_US_dic = allocator.allocateUtf8String("/usr/share/hunspell/en_US.dic");
12
13 // Get a handle to the Hunspell shared library and load up the dictionary and affix
14 hunspellHandle = Hunspell_create(en_US_aff, en_US_dic);
15
16 // Feed it a wrong word
17 var javaWord = "koing";
18
19 // Do a simple spell check of the word
20 var word = allocator.allocateUtf8String(javaWord);
21 var spellingResult = Hunspell_spell(hunspellHandle, word);
22 System.out.println(String.format("%s is spelled %s", javaWord, (spellingResult == 0 ? "incorrect" : "correct")));
23
24 // Hunspell also supports giving suggestions for a word - which is what we do next
25 // Note #3: by testing this `koing` word in isolation - we know that there are 4 alternatives for this word
26 // Note #4: I'm still investigating how to access individual suggestions
27
28 var suggestions = allocator.allocate(10);
29 var suggestionCount = Hunspell_suggest(hunspellHandle, suggestions, word);
30
31 System.out.println(String.format("There are %d suggestions for %s", suggestionCount, javaWord));
32
33 // `suggestions` - according to the hunspell API - is a `pointer to an array of strings pointer`
34 // we know how many `strings` pointer there are, as that is the returned value from `suggest`
35 // Question: how to process `suggestions` to get individual suggestions
36
37
38 } finally {
39 if (hunspellHandle != null) {
40 Hunspell_destroy(hunspellHandle);
41 }
42 }
43 }
44}
45
What I'm seeing is that a call to Hunspell_suggest
(created from jextract
) succeeds and gives me back (4) suggestions (which I verified using Hunspell from the commandline) - so no problem there.
What is more challenging for me now is how do I unpack the suggestions
element that comes back from this call? I've been looking at various examples, but none of them seem to go into this level of detail (and even if I find examples, they seem to be using outdated panama APIs).
So in essence, here is my question:
How do I unpack a structure that reportedly consists of a pointer to an array of strings pointer using panama JDK19 APIs to their respective collection of strings?
ANSWER
Answered 2022-Feb-24 at 21:41Looking at the header here: https://github.com/hunspell/hunspell/blob/master/src/hunspell/hunspell.h#L80
1public class HelloHun {
2 public static void main(String[] args) {
3 MemoryAddress hunspellHandle = null;
4 try (ResourceScope scope = ResourceScope.newConfinedScope()) {
5 var allocator = SegmentAllocator.nativeAllocator(scope);
6
7 // Point it to US english dictionary and (so called) affix file
8 // Note #1: it is possible to add words to the dictionary if you like
9 // Note #2: it is possible to have separate/individual dictionaries and affix files (e.g. per user/doc type)
10 var en_US_aff = allocator.allocateUtf8String("/usr/share/hunspell/en_US.aff");
11 var en_US_dic = allocator.allocateUtf8String("/usr/share/hunspell/en_US.dic");
12
13 // Get a handle to the Hunspell shared library and load up the dictionary and affix
14 hunspellHandle = Hunspell_create(en_US_aff, en_US_dic);
15
16 // Feed it a wrong word
17 var javaWord = "koing";
18
19 // Do a simple spell check of the word
20 var word = allocator.allocateUtf8String(javaWord);
21 var spellingResult = Hunspell_spell(hunspellHandle, word);
22 System.out.println(String.format("%s is spelled %s", javaWord, (spellingResult == 0 ? "incorrect" : "correct")));
23
24 // Hunspell also supports giving suggestions for a word - which is what we do next
25 // Note #3: by testing this `koing` word in isolation - we know that there are 4 alternatives for this word
26 // Note #4: I'm still investigating how to access individual suggestions
27
28 var suggestions = allocator.allocate(10);
29 var suggestionCount = Hunspell_suggest(hunspellHandle, suggestions, word);
30
31 System.out.println(String.format("There are %d suggestions for %s", suggestionCount, javaWord));
32
33 // `suggestions` - according to the hunspell API - is a `pointer to an array of strings pointer`
34 // we know how many `strings` pointer there are, as that is the returned value from `suggest`
35 // Question: how to process `suggestions` to get individual suggestions
36
37
38 } finally {
39 if (hunspellHandle != null) {
40 Hunspell_destroy(hunspellHandle);
41 }
42 }
43 }
44}
45/* suggest(suggestions, word) - search suggestions
46 * input: pointer to an array of strings pointer and the (bad) word
47 * array of strings pointer (here *slst) may not be initialized
48 * output: number of suggestions in string array, and suggestions in
49 * a newly allocated array of strings (*slts will be NULL when number
50 * of suggestion equals 0.)
51 */
52LIBHUNSPELL_DLL_EXPORTED int Hunspell_suggest(Hunhandle* pHunspell,
53 char*** slst,
54 const char* word);
55
The slst
is a classic 'out' parameter. i.e. we pass a pointer to some value (in this case a char**
i.e. an array of strings), and the function will set this pointer for us, as a way to return multiple results. (the first result being the number of suggestions)
In panama you use 'out' parameters by allocating a segment with the layout of the type the parameter is a pointer of. In this case char***
is a pointer to char**
, so the layout is ADDRESS
. We then pass the created segment to the function, and finally retrieve/use the value from that segment after the function call, which will have filled in the segment contents:
1public class HelloHun {
2 public static void main(String[] args) {
3 MemoryAddress hunspellHandle = null;
4 try (ResourceScope scope = ResourceScope.newConfinedScope()) {
5 var allocator = SegmentAllocator.nativeAllocator(scope);
6
7 // Point it to US english dictionary and (so called) affix file
8 // Note #1: it is possible to add words to the dictionary if you like
9 // Note #2: it is possible to have separate/individual dictionaries and affix files (e.g. per user/doc type)
10 var en_US_aff = allocator.allocateUtf8String("/usr/share/hunspell/en_US.aff");
11 var en_US_dic = allocator.allocateUtf8String("/usr/share/hunspell/en_US.dic");
12
13 // Get a handle to the Hunspell shared library and load up the dictionary and affix
14 hunspellHandle = Hunspell_create(en_US_aff, en_US_dic);
15
16 // Feed it a wrong word
17 var javaWord = "koing";
18
19 // Do a simple spell check of the word
20 var word = allocator.allocateUtf8String(javaWord);
21 var spellingResult = Hunspell_spell(hunspellHandle, word);
22 System.out.println(String.format("%s is spelled %s", javaWord, (spellingResult == 0 ? "incorrect" : "correct")));
23
24 // Hunspell also supports giving suggestions for a word - which is what we do next
25 // Note #3: by testing this `koing` word in isolation - we know that there are 4 alternatives for this word
26 // Note #4: I'm still investigating how to access individual suggestions
27
28 var suggestions = allocator.allocate(10);
29 var suggestionCount = Hunspell_suggest(hunspellHandle, suggestions, word);
30
31 System.out.println(String.format("There are %d suggestions for %s", suggestionCount, javaWord));
32
33 // `suggestions` - according to the hunspell API - is a `pointer to an array of strings pointer`
34 // we know how many `strings` pointer there are, as that is the returned value from `suggest`
35 // Question: how to process `suggestions` to get individual suggestions
36
37
38 } finally {
39 if (hunspellHandle != null) {
40 Hunspell_destroy(hunspellHandle);
41 }
42 }
43 }
44}
45/* suggest(suggestions, word) - search suggestions
46 * input: pointer to an array of strings pointer and the (bad) word
47 * array of strings pointer (here *slst) may not be initialized
48 * output: number of suggestions in string array, and suggestions in
49 * a newly allocated array of strings (*slts will be NULL when number
50 * of suggestion equals 0.)
51 */
52LIBHUNSPELL_DLL_EXPORTED int Hunspell_suggest(Hunhandle* pHunspell,
53 char*** slst,
54 const char* word);
55// char***
56var suggestionsRef = allocator.allocate(ValueLayout.ADDRESS); // allocate space for an address
57var suggestionCount = Hunspell_suggest(hunspellHandle, suggestionsRef, word);
58// char** (the value set by the function)
59MemoryAddress suggestions = suggestionsRef.get(ValueLayout.ADDRESS, 0);
60
After that, you can iterate over the array of strings:
1public class HelloHun {
2 public static void main(String[] args) {
3 MemoryAddress hunspellHandle = null;
4 try (ResourceScope scope = ResourceScope.newConfinedScope()) {
5 var allocator = SegmentAllocator.nativeAllocator(scope);
6
7 // Point it to US english dictionary and (so called) affix file
8 // Note #1: it is possible to add words to the dictionary if you like
9 // Note #2: it is possible to have separate/individual dictionaries and affix files (e.g. per user/doc type)
10 var en_US_aff = allocator.allocateUtf8String("/usr/share/hunspell/en_US.aff");
11 var en_US_dic = allocator.allocateUtf8String("/usr/share/hunspell/en_US.dic");
12
13 // Get a handle to the Hunspell shared library and load up the dictionary and affix
14 hunspellHandle = Hunspell_create(en_US_aff, en_US_dic);
15
16 // Feed it a wrong word
17 var javaWord = "koing";
18
19 // Do a simple spell check of the word
20 var word = allocator.allocateUtf8String(javaWord);
21 var spellingResult = Hunspell_spell(hunspellHandle, word);
22 System.out.println(String.format("%s is spelled %s", javaWord, (spellingResult == 0 ? "incorrect" : "correct")));
23
24 // Hunspell also supports giving suggestions for a word - which is what we do next
25 // Note #3: by testing this `koing` word in isolation - we know that there are 4 alternatives for this word
26 // Note #4: I'm still investigating how to access individual suggestions
27
28 var suggestions = allocator.allocate(10);
29 var suggestionCount = Hunspell_suggest(hunspellHandle, suggestions, word);
30
31 System.out.println(String.format("There are %d suggestions for %s", suggestionCount, javaWord));
32
33 // `suggestions` - according to the hunspell API - is a `pointer to an array of strings pointer`
34 // we know how many `strings` pointer there are, as that is the returned value from `suggest`
35 // Question: how to process `suggestions` to get individual suggestions
36
37
38 } finally {
39 if (hunspellHandle != null) {
40 Hunspell_destroy(hunspellHandle);
41 }
42 }
43 }
44}
45/* suggest(suggestions, word) - search suggestions
46 * input: pointer to an array of strings pointer and the (bad) word
47 * array of strings pointer (here *slst) may not be initialized
48 * output: number of suggestions in string array, and suggestions in
49 * a newly allocated array of strings (*slts will be NULL when number
50 * of suggestion equals 0.)
51 */
52LIBHUNSPELL_DLL_EXPORTED int Hunspell_suggest(Hunhandle* pHunspell,
53 char*** slst,
54 const char* word);
55// char***
56var suggestionsRef = allocator.allocate(ValueLayout.ADDRESS); // allocate space for an address
57var suggestionCount = Hunspell_suggest(hunspellHandle, suggestionsRef, word);
58// char** (the value set by the function)
59MemoryAddress suggestions = suggestionsRef.get(ValueLayout.ADDRESS, 0);
60for (int i = 0; i < suggestionCount; i++) {
61 // char* (an element in the array)
62 MemoryAddress suggestion = suggestions.getAtIndex(ValueLayout.ADDRESS, i);
63 // read the string
64 String javaSuggestion = suggestion.getUtf8String(suggestion, 0);
65}
66
QUESTION
Explode pandas column of dictionary with list of tuples as value
Asked 2022-Feb-03 at 21:11I have the following dataframe where col2 is a dictionary with a list of tuples as values. The keys are consistantly 'added' and 'deleted' in the whole dataframe.
Input df
col1 | col2 |
---|---|
value1 | {'added': [(59, 'dep1_v2'), (60, 'dep2_v2')], 'deleted': [(59, 'dep1_v1'), (60, 'dep2_v1')]} |
value 2 | {'added': [(61, 'dep3_v2')], 'deleted': [(61, 'dep3_v1')]} |
Here's a copy-pasteable example dataframe:
1jsons = ["{'added': [(59, 'dep1_v2'), (60, 'dep2_v2')], 'deleted': [(59, 'dep1_v1'), (60, 'dep2_v1')]}",
2 "{'added': [(61, 'dep3_v2')], 'deleted': [(61, 'dep3_v1')]}"]
3
4df = pd.DataFrame({"col1": ["value1", "value2"], "col2": jsons})
5
edit
col2 directly comes from the diff_parsed field of pydriller output
I want to "explode" col2 so that I obtain the following result:
Desired output
col1 | number | added | deleted |
---|---|---|---|
value1 | 59 | dep1_v2 | dep1_v1 |
value1 | 60 | dep2_v2 | dep2_v1 |
value2 | 61 | dep3_v2 | dep3_v1 |
So far, I tried the following:
1jsons = ["{'added': [(59, 'dep1_v2'), (60, 'dep2_v2')], 'deleted': [(59, 'dep1_v1'), (60, 'dep2_v1')]}",
2 "{'added': [(61, 'dep3_v2')], 'deleted': [(61, 'dep3_v1')]}"]
3
4df = pd.DataFrame({"col1": ["value1", "value2"], "col2": jsons})
5df = df.join(pd.json_normalize(df.col2))
6df.drop(columns=['col2'], inplace=True)
7
The above code is simplified. I first manipulate the column to convert to proper json. It was in an attempt to first explode on 'added' and 'deleted' and then try to play around with the format to obtain what I want...but the list of tuples is not preserved and I obtain the following:
col1 | added | deleted |
---|---|---|
value1 | 59, dep1_v2, 60, dep2_v2 | 59, dep1_v1, 60, dep2_v1 |
value2 | 61, dep3_v1 | 61, dep3_v2 |
Thanks
ANSWER
Answered 2022-Feb-03 at 01:47Here's a solution. It's a little long, but it works:
1jsons = ["{'added': [(59, 'dep1_v2'), (60, 'dep2_v2')], 'deleted': [(59, 'dep1_v1'), (60, 'dep2_v1')]}",
2 "{'added': [(61, 'dep3_v2')], 'deleted': [(61, 'dep3_v1')]}"]
3
4df = pd.DataFrame({"col1": ["value1", "value2"], "col2": jsons})
5df = df.join(pd.json_normalize(df.col2))
6df.drop(columns=['col2'], inplace=True)
7tmp = pd.concat([df, pd.json_normalize(df['col2'])], axis=1).drop('col2', axis=1).explode(['added', 'deleted'])
8new_df = pd.concat([tmp.drop(['added', 'deleted'], axis=1).reset_index(drop=True), pd.DataFrame(tmp['added'].tolist()).merge(pd.DataFrame(tmp['deleted'].tolist()), on=0).set_axis(['number', 'added', 'deleted'], axis=1)], axis=1)
9
Output:
1jsons = ["{'added': [(59, 'dep1_v2'), (60, 'dep2_v2')], 'deleted': [(59, 'dep1_v1'), (60, 'dep2_v1')]}",
2 "{'added': [(61, 'dep3_v2')], 'deleted': [(61, 'dep3_v1')]}"]
3
4df = pd.DataFrame({"col1": ["value1", "value2"], "col2": jsons})
5df = df.join(pd.json_normalize(df.col2))
6df.drop(columns=['col2'], inplace=True)
7tmp = pd.concat([df, pd.json_normalize(df['col2'])], axis=1).drop('col2', axis=1).explode(['added', 'deleted'])
8new_df = pd.concat([tmp.drop(['added', 'deleted'], axis=1).reset_index(drop=True), pd.DataFrame(tmp['added'].tolist()).merge(pd.DataFrame(tmp['deleted'].tolist()), on=0).set_axis(['number', 'added', 'deleted'], axis=1)], axis=1)
9>>> new_df
10 col1 number added deleted
110 value1 59 dep1_v2 dep1_v1
121 value1 60 dep2_v2 dep2_v1
132 value2 61 dep3_v2 dep3_v1
14
QUESTION
how Julia determines index of dictionary keys?
Asked 2022-Jan-29 at 20:05I confronted strange behavior in Dictionary collection in Julia. a Dictionary can be defined in Julia like this:
1dictionary = Dict(1 => 77, 2 => 66, 3 => 1)
2
and you can access keys using keys
:
1dictionary = Dict(1 => 77, 2 => 66, 3 => 1)
2> keys(dictionary)
3
4# [output]
5KeySet for a Dict{Int64, Int64} with 3 entries. Keys:
62
73
81
9
10# now i want to make sure Julia consider above order. so i use collect and then i will call first element of it
11> collect(keys(dictionary))[1]
12
13# [output]
142
15
as you can see the order of keys in keys(dictionary)
output is so strange. seems Julia doesn't consider the order of (key=>value) in input! even it doesn't seem to be ordered ascending or descending. How Julia does indexing for keys(dictionary)
output?
Expected Output:
1dictionary = Dict(1 => 77, 2 => 66, 3 => 1)
2> keys(dictionary)
3
4# [output]
5KeySet for a Dict{Int64, Int64} with 3 entries. Keys:
62
73
81
9
10# now i want to make sure Julia consider above order. so i use collect and then i will call first element of it
11> collect(keys(dictionary))[1]
12
13# [output]
142
15> keys(dictionary)
16
17# [output]
18KeySet for a Dict{Int64, Int64} with 3 entries. Keys:
191
202
213
22
23> collect(keys(dictionary))[1]
24
25# [output]
261
27
I expect keys(dictionary)
give me the keys in the order that I entered them in defining dictionary
.
ANSWER
Answered 2022-Jan-29 at 19:41The key order in Dict
is currently undefined (this might change in the future).
If you want order to be preserved use OrderedDict
from DataStructures.jl:
1dictionary = Dict(1 => 77, 2 => 66, 3 => 1)
2> keys(dictionary)
3
4# [output]
5KeySet for a Dict{Int64, Int64} with 3 entries. Keys:
62
73
81
9
10# now i want to make sure Julia consider above order. so i use collect and then i will call first element of it
11> collect(keys(dictionary))[1]
12
13# [output]
142
15> keys(dictionary)
16
17# [output]
18KeySet for a Dict{Int64, Int64} with 3 entries. Keys:
191
202
213
22
23> collect(keys(dictionary))[1]
24
25# [output]
261
27julia> using DataStructures
28
29julia> dictionary = OrderedDict(1 => 77, 2 => 66, 3 => 1)
30OrderedDict{Int64, Int64} with 3 entries:
31 1 => 77
32 2 => 66
33 3 => 1
34
35julia> keys(dictionary)
36KeySet for a OrderedDict{Int64, Int64} with 3 entries. Keys:
37 1
38 2
39 3
40
QUESTION
Pythonic way to make a dictionary from lists of unequal length without padding Nones
Asked 2021-Dec-17 at 09:10I have a list of 'Id's' that I wish to associate with a property from another list, their 'rows'. I have found a way to do it by making smaller dictionaries and concatenating them together which works, but I wondered if there was a more pythonic way to do it?
Code
1row1 = list(range(1, 6, 1))
2row2 = list(range(6, 11, 1))
3row3 = list(range(11, 16, 1))
4row4 = list(range(16, 21, 1))
5
6row1_dict = {}
7row2_dict = {}
8row3_dict = {}
9row4_dict = {}
10
11for n in row1:
12 row1_dict[n] = 1
13for n in row2:
14 row2_dict[n] = 2
15for n in row3:
16 row3_dict[n] = 3
17for n in row4:
18 row4_dict[n] = 4
19
20id_to_row_dict = {}
21id_to_row_dict = {**row1_dict, **row2_dict, **row3_dict, **row4_dict}
22print('\n')
23for k, v in id_to_row_dict.items():
24 print(k, " : ", v)
25
Output of dictionary which I want to replicate more pythonically
1row1 = list(range(1, 6, 1))
2row2 = list(range(6, 11, 1))
3row3 = list(range(11, 16, 1))
4row4 = list(range(16, 21, 1))
5
6row1_dict = {}
7row2_dict = {}
8row3_dict = {}
9row4_dict = {}
10
11for n in row1:
12 row1_dict[n] = 1
13for n in row2:
14 row2_dict[n] = 2
15for n in row3:
16 row3_dict[n] = 3
17for n in row4:
18 row4_dict[n] = 4
19
20id_to_row_dict = {}
21id_to_row_dict = {**row1_dict, **row2_dict, **row3_dict, **row4_dict}
22print('\n')
23for k, v in id_to_row_dict.items():
24 print(k, " : ", v)
251 : 1
262 : 1
273 : 1
284 : 1
295 : 1
306 : 2
317 : 2
328 : 2
339 : 2
3410 : 2
3511 : 3
3612 : 3
3713 : 3
3814 : 3
3915 : 3
4016 : 4
4117 : 4
4218 : 4
4319 : 4
4420 : 4
45
Desired output
Same as my output above, I just want to see if there is a better way to do it?
ANSWER
Answered 2021-Dec-17 at 08:09This dict-comprehension should do it:
1row1 = list(range(1, 6, 1))
2row2 = list(range(6, 11, 1))
3row3 = list(range(11, 16, 1))
4row4 = list(range(16, 21, 1))
5
6row1_dict = {}
7row2_dict = {}
8row3_dict = {}
9row4_dict = {}
10
11for n in row1:
12 row1_dict[n] = 1
13for n in row2:
14 row2_dict[n] = 2
15for n in row3:
16 row3_dict[n] = 3
17for n in row4:
18 row4_dict[n] = 4
19
20id_to_row_dict = {}
21id_to_row_dict = {**row1_dict, **row2_dict, **row3_dict, **row4_dict}
22print('\n')
23for k, v in id_to_row_dict.items():
24 print(k, " : ", v)
251 : 1
262 : 1
273 : 1
284 : 1
295 : 1
306 : 2
317 : 2
328 : 2
339 : 2
3410 : 2
3511 : 3
3612 : 3
3713 : 3
3814 : 3
3915 : 3
4016 : 4
4117 : 4
4218 : 4
4319 : 4
4420 : 4
45rows = [row1, row2, row3, row4]
46{k: v for v, row in enumerate(rows, 1) for k in row}
47
QUESTION
Different access time to a value of a dictionary when mixing int and str keys
Asked 2021-Sep-21 at 10:20Let's say I have two dictionaries and I know want to measure the time needed to check if a key is in the dictionary. I tried to run this piece of code:
1from timeit import timeit
2
3dct1 = {str(i): 1 for i in range(10**7)}
4dct2 = {i: 1 for i in range(10**7)}
5
6print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
7print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
8
Here are the results that I get:
1from timeit import timeit
2
3dct1 = {str(i): 1 for i in range(10**7)}
4dct2 = {i: 1 for i in range(10**7)}
5
6print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
7print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
82.529034548999334
92.212983401999736
10
Now, let's say I try to mix integers and strings in both dictionaries, and measure access time again:
1from timeit import timeit
2
3dct1 = {str(i): 1 for i in range(10**7)}
4dct2 = {i: 1 for i in range(10**7)}
5
6print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
7print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
82.529034548999334
92.212983401999736
10dct1[7] = 1
11dct2["7"] = 1
12
13print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
14print(timeit('7 in dct1', setup='from __main__ import dct1', number=10**8))
15print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
16print(timeit('"7" in dct2', setup='from __main__ import dct2', number=10**8))
17
I get something weird:
1from timeit import timeit
2
3dct1 = {str(i): 1 for i in range(10**7)}
4dct2 = {i: 1 for i in range(10**7)}
5
6print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
7print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
82.529034548999334
92.212983401999736
10dct1[7] = 1
11dct2["7"] = 1
12
13print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
14print(timeit('7 in dct1', setup='from __main__ import dct1', number=10**8))
15print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
16print(timeit('"7" in dct2', setup='from __main__ import dct2', number=10**8))
173.443614432000686
182.6335261530002754
192.1873921409987815
202.272667104998618
21
The first value is much higher than what I had before (3.44 vs 2.52). However, the third value is basically the same as before (2.18 vs 2.21). Why is this happening? Can you reproduce the same thing or is this only me? Also, I can't understand the big difference between the first and the second value: it looks like it's more difficult to access a string key, but the same thing seems to apply only slightly to the second dictionary. Why?
Update
You don't even need to actually add a new key. All you need to do to see an increase in complexity is just checking if a key with different type exists!! This is much weirder than I thought. Look at the example here:
1from timeit import timeit
2
3dct1 = {str(i): 1 for i in range(10**7)}
4dct2 = {i: 1 for i in range(10**7)}
5
6print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
7print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
82.529034548999334
92.212983401999736
10dct1[7] = 1
11dct2["7"] = 1
12
13print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
14print(timeit('7 in dct1', setup='from __main__ import dct1', number=10**8))
15print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
16print(timeit('"7" in dct2', setup='from __main__ import dct2', number=10**8))
173.443614432000686
182.6335261530002754
192.1873921409987815
202.272667104998618
21from timeit import timeit
22
23dct1 = {str(i): 1 for i in range(10**7)}
24dct2 = {i: 1 for i in range(10**7)}
25
26print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
27# 2.55
28print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
29# 2.26
30
317 in dct1
32"7" in dct2
33
34print(timeit('"7" in dct1', setup='from __main__ import dct1', number=10**8))
35# 3.34
36print(timeit('7 in dct2', setup='from __main__ import dct2', number=10**8))
37# 2.35
38
ANSWER
Answered 2021-Sep-21 at 10:20Let me try to answer my own question. The dict implementation in CPython is optimised for lookups of str keys. Indeed, there are two different functions that are used to perform lookups:
lookdict
is a generic dictionary lookup function that is used with all types of keyslookdict_unicode
is a specialised lookup function used for dictionaries composed of str-only keys
Python will use the string-optimised version until a search for non-string data, after which the more general function is used.
And it looks like you cannot even reverse the behaviour of a particular dict instance: once it starts using the generic function, you can't go back to using the specialised one!
QUESTION
Confusion regarding the Blocking of "peer threads" when a user-level thread blocks
Asked 2021-Sep-07 at 10:40I was reading about differences between threads and processes, and literally everywhere online, one difference is commonly written without much explanation:
If a process gets blocked, remaining processes can continue execution. If a user level thread gets blocked, all of its peer threads also get blocked.
It doesn't make any sense to me. What would be the sense of concurrency if a scheduler cannot switch between a blocked thread and a ready/runnable thread. The reason given is that since the OS doesn't differentiate between the various threads of a given parent process, it blocks all of them at once.
I find it very unconvincing, since all modern OS have thread control blocks with a thread ID, even if it is valid only within the memory space of the parent process. Like the example given in Galvin's Operating Systems book, I wouldn't want the thread which is handling my typing to be blocked if the spell checking thread cannot connect to some online dictionary, perhaps.
Either I am understanding this concept wrong, or all these websites have just copied some old thread differences over the years. Moreover, I cannot find this statement in books, like Galvin's or maybe in William Stalling's COA book where threads have been discussed.
These are resouces where I found the statements:
ANSWER
Answered 2021-Aug-30 at 11:12There is a difference between kernel-level and user-level threads. In simple words:
- Kernel-level threads: Threads that are managed by the operating system, including scheduling. They are what is executed on the processor. That's what probably most of us think of threads.
- User-level threads: Threads that are managed by the program itself. They are also called fibers or coroutines in some contexts. In contrast to kernel-level threads, they need to "yield the execution", i.e. switching from one user-level to another user-level thread is done explicitly by the program. User-level threads are mapped to kernel-level threads.
As user-level threads need to be mapped to kernel-level threads, you need to choose a suiteable mapping. You could map each user-level to a separate kernel-level thread. You could also map many user-level to one kernel-level thread. In the latter mapping, you let multiple concurrent execution paths be executed by a single thread "as we know it". If one of those paths blocks, recall that user-level threads need to yield the execution, then the executing (kernel-level) thread blocks, which causes all other assigned paths to also be effectively blocked. I think, this is what the statement refers to. FYI: In Java, user-level threads – the multithreading you do in your programs – are mapped to kernel-level threads by the JVM, i.e. the runtime system.
Related stuff:
QUESTION
How can I document methods inherited from a metaclass?
Asked 2021-Aug-23 at 11:50Consider the following metaclass/class definitions:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10
As we know, defining a method in a metaclass means that it is inherited by the class, and can be used by the class. This means that the following code in the interactive console works fine:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12
However, one major downside of this approach is that any documentation that we might have included in the definition of the method is lost. If we type help(UsesMeta)
into the interactive console, we see that there is no reference to the method greet_user
, let alone the docstring that we put in the method definition:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12Help on class UsesMeta in module __main__:
13class UsesMeta(builtins.object)
14 | A class that uses `Meta` as its metaclass.
15 |
16 | Data descriptors defined here:
17 |
18 | __dict__
19 | dictionary for instance variables (if defined)
20 |
21 | __weakref__
22 | list of weak references to the object (if defined)
23
Now of course, the __doc__
attribute for a class is writable, so one solution would be to rewrite the metaclass/class definitions like so:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12Help on class UsesMeta in module __main__:
13class UsesMeta(builtins.object)
14 | A class that uses `Meta` as its metaclass.
15 |
16 | Data descriptors defined here:
17 |
18 | __dict__
19 | dictionary for instance variables (if defined)
20 |
21 | __weakref__
22 | list of weak references to the object (if defined)
23from pydoc import render_doc
24from functools import cache
25
26def get_documentation(func_or_cls):
27 """Get the output printed by the `help` function as a string"""
28 return '\n'.join(render_doc(func_or_cls).splitlines()[2:])
29
30
31class Meta(type):
32 """A python metaclass."""
33
34 @classmethod
35 @cache
36 def _docs(metacls) -> str:
37 """Get the documentation for all public methods and properties defined in the metaclass."""
38
39 divider = '\n\n----------------------------------------------\n\n'
40 metacls_name = metacls.__name__
41 metacls_dict = metacls.__dict__
42
43 methods_header = (
44 f'Classmethods inherited from metaclass `{metacls_name}`'
45 f'\n\n'
46 )
47
48 method_docstrings = '\n\n'.join(
49 get_documentation(method)
50 for method_name, method in metacls_dict.items()
51 if not (method_name.startswith('_') or isinstance(method, property))
52 )
53
54 properties_header = (
55 f'Classmethod properties inherited from metaclass `{metacls_name}`'
56 f'\n\n'
57 )
58
59 properties_docstrings = '\n\n'.join(
60 f'{property_name}\n{get_documentation(prop)}'
61 for property_name, prop in metacls_dict.items()
62 if isinstance(prop, property) and not property_name.startswith('_')
63 )
64
65 return ''.join((
66 divider,
67 methods_header,
68 method_docstrings,
69 divider,
70 properties_header,
71 properties_docstrings,
72 divider
73 ))
74
75
76 def __new__(metacls, cls_name, cls_bases, cls_dict):
77 """Make a new class, but tweak `.__doc__` so it includes information about the metaclass's methods."""
78
79 new = super().__new__(metacls, cls_name, cls_bases, cls_dict)
80 metacls_docs = metacls._docs()
81
82 if new.__doc__ is None:
83 new.__doc__ = metacls_docs
84 else:
85 new.__doc__ += metacls_docs
86
87 return new
88
89 def greet_user(cls):
90 """Print a friendly greeting identifying the class's name."""
91 print(f"Hello, I'm the class '{cls.__name__}'!")
92
93
94class UsesMeta(metaclass=Meta):
95 """A class that uses `Meta` as its metaclass."""
96
This "solves" the problem; if we now type help(UsesMeta)
into the interactive console, the methods inherited from Meta
are now fully documented:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12Help on class UsesMeta in module __main__:
13class UsesMeta(builtins.object)
14 | A class that uses `Meta` as its metaclass.
15 |
16 | Data descriptors defined here:
17 |
18 | __dict__
19 | dictionary for instance variables (if defined)
20 |
21 | __weakref__
22 | list of weak references to the object (if defined)
23from pydoc import render_doc
24from functools import cache
25
26def get_documentation(func_or_cls):
27 """Get the output printed by the `help` function as a string"""
28 return '\n'.join(render_doc(func_or_cls).splitlines()[2:])
29
30
31class Meta(type):
32 """A python metaclass."""
33
34 @classmethod
35 @cache
36 def _docs(metacls) -> str:
37 """Get the documentation for all public methods and properties defined in the metaclass."""
38
39 divider = '\n\n----------------------------------------------\n\n'
40 metacls_name = metacls.__name__
41 metacls_dict = metacls.__dict__
42
43 methods_header = (
44 f'Classmethods inherited from metaclass `{metacls_name}`'
45 f'\n\n'
46 )
47
48 method_docstrings = '\n\n'.join(
49 get_documentation(method)
50 for method_name, method in metacls_dict.items()
51 if not (method_name.startswith('_') or isinstance(method, property))
52 )
53
54 properties_header = (
55 f'Classmethod properties inherited from metaclass `{metacls_name}`'
56 f'\n\n'
57 )
58
59 properties_docstrings = '\n\n'.join(
60 f'{property_name}\n{get_documentation(prop)}'
61 for property_name, prop in metacls_dict.items()
62 if isinstance(prop, property) and not property_name.startswith('_')
63 )
64
65 return ''.join((
66 divider,
67 methods_header,
68 method_docstrings,
69 divider,
70 properties_header,
71 properties_docstrings,
72 divider
73 ))
74
75
76 def __new__(metacls, cls_name, cls_bases, cls_dict):
77 """Make a new class, but tweak `.__doc__` so it includes information about the metaclass's methods."""
78
79 new = super().__new__(metacls, cls_name, cls_bases, cls_dict)
80 metacls_docs = metacls._docs()
81
82 if new.__doc__ is None:
83 new.__doc__ = metacls_docs
84 else:
85 new.__doc__ += metacls_docs
86
87 return new
88
89 def greet_user(cls):
90 """Print a friendly greeting identifying the class's name."""
91 print(f"Hello, I'm the class '{cls.__name__}'!")
92
93
94class UsesMeta(metaclass=Meta):
95 """A class that uses `Meta` as its metaclass."""
96Help on class UsesMeta in module __main__:
97class UsesMeta(builtins.object)
98 | A class that uses `Meta` as its metaclass.
99 |
100 | ----------------------------------------------
101 |
102 | Classmethods inherited from metaclass `Meta`
103 |
104 | greet_user(cls)
105 | Print a friendly greeting identifying the class's name.
106 |
107 | ----------------------------------------------
108 |
109 | Classmethod properties inherited from metaclass `Meta`
110 |
111 |
112 |
113 | ----------------------------------------------
114 |
115 | Data descriptors defined here:
116 |
117 | __dict__
118 | dictionary for instance variables (if defined)
119 |
120 | __weakref__
121 | list of weak references to the object (if defined)
122
That's an awful lot of code to achieve this goal, however. Is there a better way?
How does the standard library do it?
I'm also curious about the way certain classes in the standard library manage this. If we have an Enum
definition like so:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12Help on class UsesMeta in module __main__:
13class UsesMeta(builtins.object)
14 | A class that uses `Meta` as its metaclass.
15 |
16 | Data descriptors defined here:
17 |
18 | __dict__
19 | dictionary for instance variables (if defined)
20 |
21 | __weakref__
22 | list of weak references to the object (if defined)
23from pydoc import render_doc
24from functools import cache
25
26def get_documentation(func_or_cls):
27 """Get the output printed by the `help` function as a string"""
28 return '\n'.join(render_doc(func_or_cls).splitlines()[2:])
29
30
31class Meta(type):
32 """A python metaclass."""
33
34 @classmethod
35 @cache
36 def _docs(metacls) -> str:
37 """Get the documentation for all public methods and properties defined in the metaclass."""
38
39 divider = '\n\n----------------------------------------------\n\n'
40 metacls_name = metacls.__name__
41 metacls_dict = metacls.__dict__
42
43 methods_header = (
44 f'Classmethods inherited from metaclass `{metacls_name}`'
45 f'\n\n'
46 )
47
48 method_docstrings = '\n\n'.join(
49 get_documentation(method)
50 for method_name, method in metacls_dict.items()
51 if not (method_name.startswith('_') or isinstance(method, property))
52 )
53
54 properties_header = (
55 f'Classmethod properties inherited from metaclass `{metacls_name}`'
56 f'\n\n'
57 )
58
59 properties_docstrings = '\n\n'.join(
60 f'{property_name}\n{get_documentation(prop)}'
61 for property_name, prop in metacls_dict.items()
62 if isinstance(prop, property) and not property_name.startswith('_')
63 )
64
65 return ''.join((
66 divider,
67 methods_header,
68 method_docstrings,
69 divider,
70 properties_header,
71 properties_docstrings,
72 divider
73 ))
74
75
76 def __new__(metacls, cls_name, cls_bases, cls_dict):
77 """Make a new class, but tweak `.__doc__` so it includes information about the metaclass's methods."""
78
79 new = super().__new__(metacls, cls_name, cls_bases, cls_dict)
80 metacls_docs = metacls._docs()
81
82 if new.__doc__ is None:
83 new.__doc__ = metacls_docs
84 else:
85 new.__doc__ += metacls_docs
86
87 return new
88
89 def greet_user(cls):
90 """Print a friendly greeting identifying the class's name."""
91 print(f"Hello, I'm the class '{cls.__name__}'!")
92
93
94class UsesMeta(metaclass=Meta):
95 """A class that uses `Meta` as its metaclass."""
96Help on class UsesMeta in module __main__:
97class UsesMeta(builtins.object)
98 | A class that uses `Meta` as its metaclass.
99 |
100 | ----------------------------------------------
101 |
102 | Classmethods inherited from metaclass `Meta`
103 |
104 | greet_user(cls)
105 | Print a friendly greeting identifying the class's name.
106 |
107 | ----------------------------------------------
108 |
109 | Classmethod properties inherited from metaclass `Meta`
110 |
111 |
112 |
113 | ----------------------------------------------
114 |
115 | Data descriptors defined here:
116 |
117 | __dict__
118 | dictionary for instance variables (if defined)
119 |
120 | __weakref__
121 | list of weak references to the object (if defined)
122from enum import Enum
123
124class FooEnum(Enum):
125 BAR = 1
126
Then, typing help(FooEnum)
into the interactive console includes this snippet:
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12Help on class UsesMeta in module __main__:
13class UsesMeta(builtins.object)
14 | A class that uses `Meta` as its metaclass.
15 |
16 | Data descriptors defined here:
17 |
18 | __dict__
19 | dictionary for instance variables (if defined)
20 |
21 | __weakref__
22 | list of weak references to the object (if defined)
23from pydoc import render_doc
24from functools import cache
25
26def get_documentation(func_or_cls):
27 """Get the output printed by the `help` function as a string"""
28 return '\n'.join(render_doc(func_or_cls).splitlines()[2:])
29
30
31class Meta(type):
32 """A python metaclass."""
33
34 @classmethod
35 @cache
36 def _docs(metacls) -> str:
37 """Get the documentation for all public methods and properties defined in the metaclass."""
38
39 divider = '\n\n----------------------------------------------\n\n'
40 metacls_name = metacls.__name__
41 metacls_dict = metacls.__dict__
42
43 methods_header = (
44 f'Classmethods inherited from metaclass `{metacls_name}`'
45 f'\n\n'
46 )
47
48 method_docstrings = '\n\n'.join(
49 get_documentation(method)
50 for method_name, method in metacls_dict.items()
51 if not (method_name.startswith('_') or isinstance(method, property))
52 )
53
54 properties_header = (
55 f'Classmethod properties inherited from metaclass `{metacls_name}`'
56 f'\n\n'
57 )
58
59 properties_docstrings = '\n\n'.join(
60 f'{property_name}\n{get_documentation(prop)}'
61 for property_name, prop in metacls_dict.items()
62 if isinstance(prop, property) and not property_name.startswith('_')
63 )
64
65 return ''.join((
66 divider,
67 methods_header,
68 method_docstrings,
69 divider,
70 properties_header,
71 properties_docstrings,
72 divider
73 ))
74
75
76 def __new__(metacls, cls_name, cls_bases, cls_dict):
77 """Make a new class, but tweak `.__doc__` so it includes information about the metaclass's methods."""
78
79 new = super().__new__(metacls, cls_name, cls_bases, cls_dict)
80 metacls_docs = metacls._docs()
81
82 if new.__doc__ is None:
83 new.__doc__ = metacls_docs
84 else:
85 new.__doc__ += metacls_docs
86
87 return new
88
89 def greet_user(cls):
90 """Print a friendly greeting identifying the class's name."""
91 print(f"Hello, I'm the class '{cls.__name__}'!")
92
93
94class UsesMeta(metaclass=Meta):
95 """A class that uses `Meta` as its metaclass."""
96Help on class UsesMeta in module __main__:
97class UsesMeta(builtins.object)
98 | A class that uses `Meta` as its metaclass.
99 |
100 | ----------------------------------------------
101 |
102 | Classmethods inherited from metaclass `Meta`
103 |
104 | greet_user(cls)
105 | Print a friendly greeting identifying the class's name.
106 |
107 | ----------------------------------------------
108 |
109 | Classmethod properties inherited from metaclass `Meta`
110 |
111 |
112 |
113 | ----------------------------------------------
114 |
115 | Data descriptors defined here:
116 |
117 | __dict__
118 | dictionary for instance variables (if defined)
119 |
120 | __weakref__
121 | list of weak references to the object (if defined)
122from enum import Enum
123
124class FooEnum(Enum):
125 BAR = 1
126 | ----------------------------------------------------------------------
127 | Readonly properties inherited from enum.EnumMeta:
128 |
129 | __members__
130 | Returns a mapping of member name->value.
131 |
132 | This mapping lists all enum members, including aliases. Note that this
133 | is a read-only view of the internal mapping.
134
How exactly does the enum
module achieve this?
The reason why I'm using metaclasses here, rather than just defining classmethod
s in the body of a class definition
Some methods that you might write in a metaclass, such as __iter__
, __getitem__
or __len__
, can't be written as classmethod
s, but can lead to extremely expressive code if you define them in a metaclass. The enum
module is an excellent example of this.
ANSWER
Answered 2021-Aug-22 at 18:43I haven't looked at the rest of the stdlib, but EnumMeta
accomplishes this by overriding the __dir__
method (i.e. specifying it in the EnumMeta
class):
1class Meta(type):
2 """A python metaclass."""
3 def greet_user(cls):
4 """Print a friendly greeting identifying the class's name."""
5 print(f"Hello, I'm the class '{cls.__name__}'!")
6
7
8class UsesMeta(metaclass=Meta):
9 """A class that uses `Meta` as its metaclass."""
10>>> UsesMeta.greet_user()
11Hello, I'm the class 'UsesMeta'!
12Help on class UsesMeta in module __main__:
13class UsesMeta(builtins.object)
14 | A class that uses `Meta` as its metaclass.
15 |
16 | Data descriptors defined here:
17 |
18 | __dict__
19 | dictionary for instance variables (if defined)
20 |
21 | __weakref__
22 | list of weak references to the object (if defined)
23from pydoc import render_doc
24from functools import cache
25
26def get_documentation(func_or_cls):
27 """Get the output printed by the `help` function as a string"""
28 return '\n'.join(render_doc(func_or_cls).splitlines()[2:])
29
30
31class Meta(type):
32 """A python metaclass."""
33
34 @classmethod
35 @cache
36 def _docs(metacls) -> str:
37 """Get the documentation for all public methods and properties defined in the metaclass."""
38
39 divider = '\n\n----------------------------------------------\n\n'
40 metacls_name = metacls.__name__
41 metacls_dict = metacls.__dict__
42
43 methods_header = (
44 f'Classmethods inherited from metaclass `{metacls_name}`'
45 f'\n\n'
46 )
47
48 method_docstrings = '\n\n'.join(
49 get_documentation(method)
50 for method_name, method in metacls_dict.items()
51 if not (method_name.startswith('_') or isinstance(method, property))
52 )
53
54 properties_header = (
55 f'Classmethod properties inherited from metaclass `{metacls_name}`'
56 f'\n\n'
57 )
58
59 properties_docstrings = '\n\n'.join(
60 f'{property_name}\n{get_documentation(prop)}'
61 for property_name, prop in metacls_dict.items()
62 if isinstance(prop, property) and not property_name.startswith('_')
63 )
64
65 return ''.join((
66 divider,
67 methods_header,
68 method_docstrings,
69 divider,
70 properties_header,
71 properties_docstrings,
72 divider
73 ))
74
75
76 def __new__(metacls, cls_name, cls_bases, cls_dict):
77 """Make a new class, but tweak `.__doc__` so it includes information about the metaclass's methods."""
78
79 new = super().__new__(metacls, cls_name, cls_bases, cls_dict)
80 metacls_docs = metacls._docs()
81
82 if new.__doc__ is None:
83 new.__doc__ = metacls_docs
84 else:
85 new.__doc__ += metacls_docs
86
87 return new
88
89 def greet_user(cls):
90 """Print a friendly greeting identifying the class's name."""
91 print(f"Hello, I'm the class '{cls.__name__}'!")
92
93
94class UsesMeta(metaclass=Meta):
95 """A class that uses `Meta` as its metaclass."""
96Help on class UsesMeta in module __main__:
97class UsesMeta(builtins.object)
98 | A class that uses `Meta` as its metaclass.
99 |
100 | ----------------------------------------------
101 |
102 | Classmethods inherited from metaclass `Meta`
103 |
104 | greet_user(cls)
105 | Print a friendly greeting identifying the class's name.
106 |
107 | ----------------------------------------------
108 |
109 | Classmethod properties inherited from metaclass `Meta`
110 |
111 |
112 |
113 | ----------------------------------------------
114 |
115 | Data descriptors defined here:
116 |
117 | __dict__
118 | dictionary for instance variables (if defined)
119 |
120 | __weakref__
121 | list of weak references to the object (if defined)
122from enum import Enum
123
124class FooEnum(Enum):
125 BAR = 1
126 | ----------------------------------------------------------------------
127 | Readonly properties inherited from enum.EnumMeta:
128 |
129 | __members__
130 | Returns a mapping of member name->value.
131 |
132 | This mapping lists all enum members, including aliases. Note that this
133 | is a read-only view of the internal mapping.
134class EnumMeta(type):
135 .
136 .
137 .
138 def __dir__(self):
139 return (
140 ['__class__', '__doc__', '__members__', '__module__']
141 + self._member_names_
142 )
143
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Dictionary
Tutorials and Learning Resources are not available at this moment for Dictionary