Popular New Releases in Bytecode
jadx
1.3.5
gravity
Official 0.8.3 release
Recaf
2.21.13
javassist
Javassist 3.28.0-GA
python-uncompyle6
Jed + Rocky average
Popular Libraries in Bytecode
by skylot java
29830 Apache-2.0
Dex to Java decompiler
by google go
10322 Apache-2.0
Grumpy is a Python to Go source code transcompiler and runtime.
by marcobambini c
3883 MIT
Gravity Programming Language
by Col-E java
3714 MIT
The modern Java bytecode editor
by NectarJS javascript
3192 MIT
🔱 Javascript's God Mode. No VM. No Bytecode. No GC. Just native binaries.
by jboss-javassist java
3172 NOASSERTION
Java bytecode engineering toolkit
by google python
2601 Apache-2.0
by rocky python
2467 GPL-3.0
A cross-version Python bytecode decompiler
by jbevain csharp
2055 MIT
Cecil is a library to inspect, modify and create .NET programs and libraries.
Trending New libraries in Bytecode
by GraxCode java
450 GPL-3.0
Multifunctional java deobfuscation tool suite
by vtereshkov c
389 BSD-2-Clause
Umka: a statically typed embeddable scripting language
by facebookexperimental c++
234 NOASSERTION
Instagram's experimental performance oriented greenfield implementation of Python.
by Guardsquare java
148 Apache-2.0
Library to read, write, analyze, and process java bytecode
by alibaba java
138 Apache-2.0
Java Bytecode Kit
by ymm-tech java
131 MIT
Easy-byte-coder is a non-invasive bytecode injection framework based on JVM. Java application developers can implement bytecode injection quickly, without caring about the underlying instrument principle and implementation details. Easy-byte-coder provides AOP ability in the form of plugin. Static waving (Agent) and dynamic waving (Attach) make plugin development easy.
by spaceflint7 csharp
114 MIT
Implementation of the .NET platform on top of the Java Virtual Machine
by shulieTech java
113 Apache-2.0
LinkAgent is a Java-based open-source agent designed to collect data and control Functions for Java applications through JVM bytecode, without modifying applications codes.
by Tigermouthbear kotlin
107 GPL-3.0
A Kotlin program used to analyse and discover backdoors in Minecraft Java 1.12.2 forge mods
Top Authors in Bytecode
1
4 Libraries
3252
2
4 Libraries
14
3
4 Libraries
1318
4
3 Libraries
12968
5
3 Libraries
231
6
3 Libraries
73
7
3 Libraries
3743
8
3 Libraries
139
9
3 Libraries
31
10
2 Libraries
25
1
4 Libraries
3252
2
4 Libraries
14
3
4 Libraries
1318
4
3 Libraries
12968
5
3 Libraries
231
6
3 Libraries
73
7
3 Libraries
3743
8
3 Libraries
139
9
3 Libraries
31
10
2 Libraries
25
Trending Kits in Bytecode
No Trending Kits are available at this moment for Bytecode
Trending Discussions on Bytecode
IntelliJ - Invalid source release: 17
The transaction declared chain ID 5777, but the connected node is on 1337
Lambda expressions and anonymous classes don't work when loaded as hidden classes
How is CPython implemented?
Bad request when deploying smart contract
JavaScript: V8 question: are small integers pooled?
env_parent(): different results when using default argument or when passing explicitly default argument
looping over array, performance difference between indexed and enhanced for loop
solidity TypeError: Object of type set is not JSON serializable
JDK 17: Switch statement causes java.lang.VerifyError: Bad type on operand stack
QUESTION
IntelliJ - Invalid source release: 17
Asked 2022-Mar-17 at 13:46I've created a new Java project in IntelliJ with Gradle that uses Java 17. When running my app it has the error Cause: error: invalid source release: 17
.
My Settings
I've installed openjdk-17
through IntelliJ
and set it as my Project SDK
.
The Project language level
has been set to 17 - Sealed types, always-strict floating-point semantics
.
In Modules -> Sources
I've set the Language level
to Project default (17 - Sealed types, always strict floating-point semantics)
.
In Modules -> Dependencies
I've set the Module SDK
to Project SDK openjdk-17
.
In Settings -> Build, Execution, Deployment -> Compiler -> Java Compiler
I've set the Project bytecode version
to 17
.
Gradle
1plugins {
2 id 'org.springframework.boot' version '2.5.6'
3 id 'io.spring.dependency-management' version '1.0.11.RELEASE'
4 id 'java'
5}
6
7group = 'com.app'
8version = '0.0.1-SNAPSHOT'
9sourceCompatibility = '17'
10
11repositories {
12 mavenCentral()
13}
14
15dependencies {
16 implementation 'org.springframework.boot:spring-boot-starter-web'
17 implementation 'org.springframework.boot:spring-boot-starter-websocket'
18 testImplementation 'org.springframework.boot:spring-boot-starter-test'
19 implementation 'com.fasterxml.jackson.core:jackson-core:2.13.0'
20 implementation 'com.fasterxml.jackson.core:jackson-databind:2.13.0'
21}
22
23test {
24 useJUnitPlatform()
25}
26
I've looked at all of the answers here but I can't seem to fix this. I must be missing something but I can't find it. I've not had any problems using Java 8 or 11.
How do I resolve this?
ANSWER
Answered 2021-Oct-24 at 14:23The message typically entails that your JAVA_HOME environment variable points to a different Java version.
Here are the steps to follow:
- Close IntelliJ IDEA
- Open a terminal window and check your JAVA_HOME variable value:
- *nix system:
echo $JAVA_HOME
- Windows system:
echo %JAVA_HOME%
- *nix system:
- The JAVA_HOME path should be pointing to a different path, then set it to the openjdk-17 path:
- *nix system:
export JAVA_HOME=/path/to/openjdk-17
- Windows system:
set JAVA_HOME=path\to\openjdk-17
- *nix system:
- Open your project again in IntelliJ IDEA
- Make sure to set both source and target compatibility versions (not only the
sourceCompatibility
)
You should be able to build your project.
EDIT: Gradle ToolchainYou may need also to instruct Gradle to use a different JVM than the one it uses itself by setting the Java plugin toolchain to your target version:
1plugins {
2 id 'org.springframework.boot' version '2.5.6'
3 id 'io.spring.dependency-management' version '1.0.11.RELEASE'
4 id 'java'
5}
6
7group = 'com.app'
8version = '0.0.1-SNAPSHOT'
9sourceCompatibility = '17'
10
11repositories {
12 mavenCentral()
13}
14
15dependencies {
16 implementation 'org.springframework.boot:spring-boot-starter-web'
17 implementation 'org.springframework.boot:spring-boot-starter-websocket'
18 testImplementation 'org.springframework.boot:spring-boot-starter-test'
19 implementation 'com.fasterxml.jackson.core:jackson-core:2.13.0'
20 implementation 'com.fasterxml.jackson.core:jackson-databind:2.13.0'
21}
22
23test {
24 useJUnitPlatform()
25}
26// build.gradle
27java {
28 toolchain {
29 languageVersion = JavaLanguageVersion.of(17)
30 }
31}
32
QUESTION
The transaction declared chain ID 5777, but the connected node is on 1337
Asked 2022-Mar-11 at 02:52I am trying to deploy my SimpleStorage.sol contract to a ganache local chain by making a transaction using python. It seems to have trouble connecting to the chain.
1from solcx import compile_standard
2from web3 import Web3
3import json
4import os
5from dotenv import load_dotenv
6
7load_dotenv()
8
9with open("./SimpleStorage.sol", "r") as file:
10 simple_storage_file = file.read()
11
12compiled_sol = compile_standard(
13 {
14 "language": "Solidity",
15 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
16 "settings": {
17 "outputSelection": {
18 "*": {"*": ["abi", "metadata", "evm.bytecode", "evm.sourceMap"]}
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25with open("compiled_code.json", "w") as file:
26 json.dump(compiled_sol, file)
27
28
29# get bytecode
30bytecode = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["evm"][
31 "bytecode"
32]["object"]
33
34
35# get ABI
36abi = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["abi"]
37
38# to connect to ganache blockchain
39w3 = Web3(Web3.HTTPProvider("HTTP://127.0.0.1:7545"))
40chain_id = 5777
41my_address = "0xca1EA31e644F13E3E36631382686fD471c62267A"
42private_key = os.getenv("PRIVATE_KEY")
43
44
45# create the contract in python
46
47SimpleStorage = w3.eth.contract(abi=abi, bytecode=bytecode)
48
49# get the latest transaction
50nonce = w3.eth.getTransactionCount(my_address)
51
52# 1. Build a transaction
53# 2. Sign a transaction
54# 3. Send a transaction
55
56
57transaction = SimpleStorage.constructor().buildTransaction(
58 {"chainId": chain_id, "from": my_address, "nonce": nonce}
59)
60print(transaction)
61
62
It seems to be connected to the ganache chain because it prints the nonce, but when I build and try to print the transaction here is the entire traceback call I am receiving
1from solcx import compile_standard
2from web3 import Web3
3import json
4import os
5from dotenv import load_dotenv
6
7load_dotenv()
8
9with open("./SimpleStorage.sol", "r") as file:
10 simple_storage_file = file.read()
11
12compiled_sol = compile_standard(
13 {
14 "language": "Solidity",
15 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
16 "settings": {
17 "outputSelection": {
18 "*": {"*": ["abi", "metadata", "evm.bytecode", "evm.sourceMap"]}
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25with open("compiled_code.json", "w") as file:
26 json.dump(compiled_sol, file)
27
28
29# get bytecode
30bytecode = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["evm"][
31 "bytecode"
32]["object"]
33
34
35# get ABI
36abi = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["abi"]
37
38# to connect to ganache blockchain
39w3 = Web3(Web3.HTTPProvider("HTTP://127.0.0.1:7545"))
40chain_id = 5777
41my_address = "0xca1EA31e644F13E3E36631382686fD471c62267A"
42private_key = os.getenv("PRIVATE_KEY")
43
44
45# create the contract in python
46
47SimpleStorage = w3.eth.contract(abi=abi, bytecode=bytecode)
48
49# get the latest transaction
50nonce = w3.eth.getTransactionCount(my_address)
51
52# 1. Build a transaction
53# 2. Sign a transaction
54# 3. Send a transaction
55
56
57transaction = SimpleStorage.constructor().buildTransaction(
58 {"chainId": chain_id, "from": my_address, "nonce": nonce}
59)
60print(transaction)
61
62Traceback (most recent call last):
63File "C:\Users\evens\demos\web3_py_simple_storage\deploy.py", line
6452, in <module>
65transaction = SimpleStorage.constructor().buildTransaction(
66File "C:\Python310\lib\site-packages\eth_utils\decorators.py", line
6718, in _wrapper
68return self.method(obj, *args, **kwargs)
69File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
70packages\web3\contract.py", line 684, in buildTransaction
71return fill_transaction_defaults(self.web3, built_transaction)
72File "cytoolz/functoolz.pyx", line 250, in
73cytoolz.functoolz.curry.__call__
74return self.func(*args, **kwargs)
75File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
76packages\web3\_utils\transactions.py", line 114, in
77fill_transaction_defaults
78default_val = default_getter(web3, transaction)
79File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
80packages\web3\_utils\transactions.py", line 60, in <lambda>
81'gas': lambda web3, tx: web3.eth.estimate_gas(tx),
82File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
83packages\web3\eth.py", line 820, in estimate_gas
84return self._estimate_gas(transaction, block_identifier)
85File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
86packages\web3\module.py", line 57, in caller
87result = w3.manager.request_blocking(method_str,
88File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
89packages\web3\manager.py", line 197, in request_blocking
90response = self._make_request(method, params)
91File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
92packages\web3\manager.py", line 150, in _make_request
93return request_func(method, params)
94File "cytoolz/functoolz.pyx", line 250, in
95cytoolz.functoolz.curry.__call__
96return self.func(*args, **kwargs)
97File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
98packages\web3\middleware\formatting.py", line 76, in
99apply_formatters
100response = make_request(method, params)
101File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
102packages\web3\middleware\gas_price_strategy.py", line 90, in
103middleware
104return make_request(method, params)
105File "cytoolz/functoolz.pyx", line 250, in
106cytoolz.functoolz.curry.__call__
107return self.func(*args, **kwargs)
108File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
109packages\web3\middleware\formatting.py", line 74, in
110apply_formatters
111response = make_request(method, formatted_params)
112File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
113packages\web3\middleware\attrdict.py", line 33, in middleware
114response = make_request(method, params)
115File "cytoolz/functoolz.pyx", line 250, in
116cytoolz.functoolz.curry.__call__
117return self.func(*args, **kwargs)
118File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
119packages\web3\middleware\formatting.py", line 74, in
120apply_formatters
121response = make_request(method, formatted_params)
122File "cytoolz/functoolz.pyx", line 250, in
123cytoolz.functoolz.curry.__call__
124return self.func(*args, **kwargs)
125File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
126packages\web3\middleware\formatting.py", line 73, in
127apply_formatters
128formatted_params = formatter(params)
129File "cytoolz/functoolz.pyx", line 503, in
130cytoolz.functoolz.Compose.__call__
131ret = PyObject_Call(self.first, args, kwargs)
132File "cytoolz/functoolz.pyx", line 250, in
133cytoolz.functoolz.curry.__call__
134return self.func(*args, **kwargs)
135File "C:\Python310\lib\site-packages\eth_utils\decorators.py", line
13691, in wrapper
137return ReturnType(result) # type: ignore
138File "C:\Python310\lib\site-packages\eth_utils\applicators.py", line
13922, in apply_formatter_at_index
140yield formatter(item)
141File "cytoolz/functoolz.pyx", line 250, in
142cytoolz.functoolz.curry.__call__
143File "cytoolz/functoolz.pyx", line 250, in
144cytoolz.functoolz.curry.__call__
145return self.func(*args, **kwargs)
146File "C:\Python310\lib\site-packages\eth_utils\applicators.py", line
14772, in apply_formatter_if
148return formatter(value)
149File "cytoolz/functoolz.pyx", line 250, in
150cytoolz.functoolz.curry.__call__
151return self.func(*args, **kwargs)
152File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
153packages\web3\middleware\validation.py", line 57, in
154validate_chain_id
155raise ValidationError(
156web3.exceptions.ValidationError: The transaction declared chain ID
1575777, but the connected node is on 1337
158
ANSWER
Answered 2022-Jan-17 at 18:17Had this issue myself, apparently it's some sort of Ganache CLI error but the simplest fix I could find was to change the network id in Ganache through settings>server to 1337. It restarts the session so you'd then need to change the address and private key variable.
If it's the same tutorial I'm doing, you're likely to come unstuck after this... the code for transaction should be:
1from solcx import compile_standard
2from web3 import Web3
3import json
4import os
5from dotenv import load_dotenv
6
7load_dotenv()
8
9with open("./SimpleStorage.sol", "r") as file:
10 simple_storage_file = file.read()
11
12compiled_sol = compile_standard(
13 {
14 "language": "Solidity",
15 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
16 "settings": {
17 "outputSelection": {
18 "*": {"*": ["abi", "metadata", "evm.bytecode", "evm.sourceMap"]}
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25with open("compiled_code.json", "w") as file:
26 json.dump(compiled_sol, file)
27
28
29# get bytecode
30bytecode = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["evm"][
31 "bytecode"
32]["object"]
33
34
35# get ABI
36abi = compiled_sol["contracts"]["SimpleStorage.sol"]["SimpleStorage"]["abi"]
37
38# to connect to ganache blockchain
39w3 = Web3(Web3.HTTPProvider("HTTP://127.0.0.1:7545"))
40chain_id = 5777
41my_address = "0xca1EA31e644F13E3E36631382686fD471c62267A"
42private_key = os.getenv("PRIVATE_KEY")
43
44
45# create the contract in python
46
47SimpleStorage = w3.eth.contract(abi=abi, bytecode=bytecode)
48
49# get the latest transaction
50nonce = w3.eth.getTransactionCount(my_address)
51
52# 1. Build a transaction
53# 2. Sign a transaction
54# 3. Send a transaction
55
56
57transaction = SimpleStorage.constructor().buildTransaction(
58 {"chainId": chain_id, "from": my_address, "nonce": nonce}
59)
60print(transaction)
61
62Traceback (most recent call last):
63File "C:\Users\evens\demos\web3_py_simple_storage\deploy.py", line
6452, in <module>
65transaction = SimpleStorage.constructor().buildTransaction(
66File "C:\Python310\lib\site-packages\eth_utils\decorators.py", line
6718, in _wrapper
68return self.method(obj, *args, **kwargs)
69File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
70packages\web3\contract.py", line 684, in buildTransaction
71return fill_transaction_defaults(self.web3, built_transaction)
72File "cytoolz/functoolz.pyx", line 250, in
73cytoolz.functoolz.curry.__call__
74return self.func(*args, **kwargs)
75File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
76packages\web3\_utils\transactions.py", line 114, in
77fill_transaction_defaults
78default_val = default_getter(web3, transaction)
79File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
80packages\web3\_utils\transactions.py", line 60, in <lambda>
81'gas': lambda web3, tx: web3.eth.estimate_gas(tx),
82File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
83packages\web3\eth.py", line 820, in estimate_gas
84return self._estimate_gas(transaction, block_identifier)
85File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
86packages\web3\module.py", line 57, in caller
87result = w3.manager.request_blocking(method_str,
88File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
89packages\web3\manager.py", line 197, in request_blocking
90response = self._make_request(method, params)
91File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
92packages\web3\manager.py", line 150, in _make_request
93return request_func(method, params)
94File "cytoolz/functoolz.pyx", line 250, in
95cytoolz.functoolz.curry.__call__
96return self.func(*args, **kwargs)
97File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
98packages\web3\middleware\formatting.py", line 76, in
99apply_formatters
100response = make_request(method, params)
101File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
102packages\web3\middleware\gas_price_strategy.py", line 90, in
103middleware
104return make_request(method, params)
105File "cytoolz/functoolz.pyx", line 250, in
106cytoolz.functoolz.curry.__call__
107return self.func(*args, **kwargs)
108File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
109packages\web3\middleware\formatting.py", line 74, in
110apply_formatters
111response = make_request(method, formatted_params)
112File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
113packages\web3\middleware\attrdict.py", line 33, in middleware
114response = make_request(method, params)
115File "cytoolz/functoolz.pyx", line 250, in
116cytoolz.functoolz.curry.__call__
117return self.func(*args, **kwargs)
118File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
119packages\web3\middleware\formatting.py", line 74, in
120apply_formatters
121response = make_request(method, formatted_params)
122File "cytoolz/functoolz.pyx", line 250, in
123cytoolz.functoolz.curry.__call__
124return self.func(*args, **kwargs)
125File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
126packages\web3\middleware\formatting.py", line 73, in
127apply_formatters
128formatted_params = formatter(params)
129File "cytoolz/functoolz.pyx", line 503, in
130cytoolz.functoolz.Compose.__call__
131ret = PyObject_Call(self.first, args, kwargs)
132File "cytoolz/functoolz.pyx", line 250, in
133cytoolz.functoolz.curry.__call__
134return self.func(*args, **kwargs)
135File "C:\Python310\lib\site-packages\eth_utils\decorators.py", line
13691, in wrapper
137return ReturnType(result) # type: ignore
138File "C:\Python310\lib\site-packages\eth_utils\applicators.py", line
13922, in apply_formatter_at_index
140yield formatter(item)
141File "cytoolz/functoolz.pyx", line 250, in
142cytoolz.functoolz.curry.__call__
143File "cytoolz/functoolz.pyx", line 250, in
144cytoolz.functoolz.curry.__call__
145return self.func(*args, **kwargs)
146File "C:\Python310\lib\site-packages\eth_utils\applicators.py", line
14772, in apply_formatter_if
148return formatter(value)
149File "cytoolz/functoolz.pyx", line 250, in
150cytoolz.functoolz.curry.__call__
151return self.func(*args, **kwargs)
152File "C:\Users\evens\AppData\Roaming\Python\Python310\site-
153packages\web3\middleware\validation.py", line 57, in
154validate_chain_id
155raise ValidationError(
156web3.exceptions.ValidationError: The transaction declared chain ID
1575777, but the connected node is on 1337
158transaction =
159 SimpleStorage.constructor().buildTransaction( {
160 "gasPrice": w3.eth.gas_price,
161 "chainId": chain_id,
162 "from": my_address,
163 "nonce": nonce,
164})
165print(transaction)
166
Otherwise you get a value error if you don't set the gasPrice
QUESTION
Lambda expressions and anonymous classes don't work when loaded as hidden classes
Asked 2022-Feb-26 at 05:14I am trying to compile and load dynamically generated Java code during runtime. Since both ClassLoader::defineClass and Unsafe::defineAnonymousClass have serious drawbacks in this scenario, I tried using hidden classes via Lookup::defineHiddenClass instead. This works fine for all classes that I tried to load, except for those that call lambda expressions or contain anonymous classes.
Calling a lambda expression throws the following exception:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8
Executing code that instantiates an anonymous class throws the following error:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29
This is a short example that recreates the problem:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50
I've already tried compiling and running the code with different JDKs, using different ways to create new instances of the hidden class, searching for bugs at https://bugs.openjdk.java.net/, messing with the bytecode itself and several other things. I am not an expert on Java internals, so I am not sure whether I have not understood the JEP that introduced hidden classes correctly.
Am I doing something wrong, is this just impossible or is this a bug?
Edit: The JEP states
Migration should take the following into account: To invoke private nestmate instance methods from code in a hidden class, use invokevirtual or invokeinterface instead of invokespecial. Generated bytecode that uses invokespecial to invoke a private nestmate instance method will fail verification. invokespecial should only be used to invoke private nestmate constructors.
This might be the problem for the anonymous class. Is there a way to compile the code such that invokespecial is avoided in the bytecode?
ANSWER
Answered 2022-Feb-23 at 18:19You can not turn arbitrary classes into hidden classes.
The documentation of defineHiddenClass
contains the sentence
- On any attempt to resolve the entry in the run-time constant pool indicated by
this_class
, the symbolic reference is considered to be resolved toC
and resolution always succeeds immediately.
What it doesn’t spell out explicitly is that this is the only place where a type resolution ever ends up at the hidden class.
But it has been said unambiguously in bug report JDK-8222730:
For a hidden class, its specified hidden name should only be accessible through the hidden class's 'this_class' constant pool entry.
The class should not be accessible by specifying its original name in, for example, a method or field signature even within the hidden class.
Which we can check. Even a simple case like
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71
will already fail. Note that it is a special case that the attempt to resolve the original class name LambdaRunner
within the hidden class will not fail, as you used an existing class as template. So you get an IncompatibleClassChangeError
or a VerifierError
due to mismatches between the hidden class and the existing LambdaRunner
class. When you don’t use a class definition of an existing class, you’d get a NoClassDefFoundError
.
The same applies to
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80
As the cited bug report said, neither field nor methods can refer to the hidden class in their signature.
A less intuitive example is
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86
which will fail depending on the compiler and options, as when the StringConcatFactory
is used, the behavior is like an invocation of a method having all non-constant parts as parameters and returning a String
. So this is another case of having the hidden class in a method signature.
Lambda expressions are special, as a class like
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93
gets compiled similar to
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93 static class LambdaRunner implements Runnable {
94 @Override
95 public void run() {
96 Runnable runnable = LambdaRunner::lambdaBody;
97 runnable.run();
98 }
99 private static void lambdaBody() {
100 System.out.println("Success");
101 }
102 }
103
which doesn’t have the hidden class in the method signature, but has to refer to the method holding the body of the lambda expression as a MethodReference
. Within the constant pool, the description of this method refers to its declaring class using the this_class
entry. So it gets redirected to the hidden class as described in the documentation.
But the construction of the MethodType
as part of the MethodReference
does not use this information to load a Class
like a class literal would do. Instead, it tries to load the hidden class through the defining class loader, which fails with the NoClassDefFoundError
you have posted.
This seems to be related to JDK-8130087 which suggests that ordinary method resolution differs from the way, MethodType
works, which can make MethodType
fail where just invoking the method would work.
But it’s possible to demonstrate that even fixing this issue wouldn’t solve the general problem:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93 static class LambdaRunner implements Runnable {
94 @Override
95 public void run() {
96 Runnable runnable = LambdaRunner::lambdaBody;
97 runnable.run();
98 }
99 private static void lambdaBody() {
100 System.out.println("Success");
101 }
102 }
103 static class LambdaRunner implements Runnable {
104 @Override
105 public void run() {
106 var lookup = MethodHandles.lookup();
107 var noArgVoid = MethodType.methodType(void.class);
108 try {
109 MethodHandle mh = LambdaMetafactory.metafactory(lookup, "run",
110 MethodType.methodType(Runnable.class), noArgVoid,
111 lookup.findStatic(LambdaRunner.class, "lambdaBody", noArgVoid),
112 noArgVoid).getTarget();
113 System.out.println("got factory");
114 Runnable runnable = (Runnable)mh.invokeExact();
115 System.out.println("got runnable");
116 runnable.run();
117 }
118 catch(RuntimeException|Error e) {
119 throw e;
120 }
121 catch(Throwable e) {
122 throw new AssertionError(e);
123 }
124 }
125 private static void lambdaBody() {
126 System.out.println("Success");
127 }
128 }
129
This bypasses the problem described above and calls the LambdaMetafactory
manually. When being redefined as hidden class, it will print:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93 static class LambdaRunner implements Runnable {
94 @Override
95 public void run() {
96 Runnable runnable = LambdaRunner::lambdaBody;
97 runnable.run();
98 }
99 private static void lambdaBody() {
100 System.out.println("Success");
101 }
102 }
103 static class LambdaRunner implements Runnable {
104 @Override
105 public void run() {
106 var lookup = MethodHandles.lookup();
107 var noArgVoid = MethodType.methodType(void.class);
108 try {
109 MethodHandle mh = LambdaMetafactory.metafactory(lookup, "run",
110 MethodType.methodType(Runnable.class), noArgVoid,
111 lookup.findStatic(LambdaRunner.class, "lambdaBody", noArgVoid),
112 noArgVoid).getTarget();
113 System.out.println("got factory");
114 Runnable runnable = (Runnable)mh.invokeExact();
115 System.out.println("got runnable");
116 runnable.run();
117 }
118 catch(RuntimeException|Error e) {
119 throw e;
120 }
121 catch(Throwable e) {
122 throw new AssertionError(e);
123 }
124 }
125 private static void lambdaBody() {
126 System.out.println("Success");
127 }
128 }
129got factory
130got runnable
131Exception in thread "main" java.lang.NoClassDefFoundError: test/HiddenClassLambdaTest$LambdaRunner/0x0000000800c01400
132 at test/test.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:15)
133Caused by: java.lang.ClassNotFoundException: test.HiddenClassLambdaTest$LambdaRunner.0x0000000800c01400
134 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
135 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
136 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
137 ... 1 more
138
which shows that all obstacles have been circumvented, but when it comes to the actual invocation from the generated Runnable
to the method holding the lambda body, it will fail due to the fact that the target class is hidden. A JVM with eager resolution of symbolic references might fail earlier, i.e. the example might not print got runnable
then.
Unlike the old JVM anonymous classes, there is no way to link to a hidden class, not even from another hidden class.
The bottom line is, as said at the beginning, you can not turn arbitrary classes into hidden classes. Lambda expressions are not the only feature not working with hidden classes. It’s not a good idea to try and get surprised. Hidden classes should only be used in conjunction with bytecode generators carefully using only features known to work.
QUESTION
How is CPython implemented?
Asked 2022-Feb-10 at 02:37So I lately came across an explanation for Python's interpreter and compiler (CPython specifically).
Please correct me if I'm wrong. I just want to be sure I understand these specific concepts.
So CPython gets both compiled (to bytecode) and then interpreted (in the PVM)? And what does the PVM do exactly? Does it read the bytecode line by line, and translate each one to binary instructions that can be executed on a specific computer? Does this mean that a computer based on an Intel processor needs a different PVM from an AMD-based computer?
ANSWER
Answered 2022-Feb-09 at 13:49- Yes, CPython is compiled to bytecode which is then executed by the virtual machine.
- The virtual machine executes instructions one-by-one. It's written in C (but you can write it in another language) and looks like a huge
if/else
statement like "if the current instruction is this, do this; if the instruction is this, do another thing", and so on. Instructions aren't translated to binary - that's why it's called an interpreter.- You can find the list of instructions here: https://docs.python.org/3.10/library/dis.html#python-bytecode-instructions
- The implementation of the VM is available here: https://github.com/python/cpython/blob/f71a69aa9209cf67cc1060051b147d6afa379bba/Python/ceval.c#L1718
- Bytecode doesn't have a concept of "line": it's just a stream of bytes. The interpreter can read one byte at a time and use another
if/else
statement to decide what instruction it's looking at. For example:
1curr_byte = read_byte()
2if curr_byte == 0x00:
3 # Parse instruction with no arguments
4 curr_instruction = DO_THING_A;
5 args = NULL;
6elif curr_byte == 0x01:
7 another_byte = read_byte()
8 if another_byte == 0x00:
9 # Parse a two-byte instruction
10 curr_instruction = DO_THING_B;
11 args = NULL;
12 else:
13 # Parse a one-byte instruction
14 # with one argument
15 curr_instruction = DO_THING_C;
16 args = another_byte >> 1; # or whatever
17elif curr_byte == ...:
18 ... # go on and on and on
19
QUESTION
Bad request when deploying smart contract
Asked 2022-Jan-18 at 21:38So I'm currently trying to deploy a router smart contract. I've been building it through erdpy contract build
, which has been successful (I'm on rust nightly tool chain as the Smart contract needs it). And I am now trying to deploy it, but I can't manage to do it. I keep having a 400 BadRequest
from https://devnet-api.elrond.com/transaction/send
.
Here are the logs from the deployment:
1erdpy contract deploy
2INFO:accounts:Account.sync_nonce()
3INFO:accounts:Account.sync_nonce() done: 0
4INFO:cli.contracts:Contract address: erd1qqqqqqqqqqqqqpgqzqv7kk893c3ftwgaekvvv9whpqcfn4kazqxq3mud36
5INFO:transactions:Transaction.send: nonce=0
6CRITICAL:cli:Proxy request error for url [https://devnet-api.elrond.com/transaction/send]: {'statusCode': 400, 'message': 'Bad Request'}
7
8
And here is erdpy.json
used to configure the command:
1erdpy contract deploy
2INFO:accounts:Account.sync_nonce()
3INFO:accounts:Account.sync_nonce() done: 0
4INFO:cli.contracts:Contract address: erd1qqqqqqqqqqqqqpgqzqv7kk893c3ftwgaekvvv9whpqcfn4kazqxq3mud36
5INFO:transactions:Transaction.send: nonce=0
6CRITICAL:cli:Proxy request error for url [https://devnet-api.elrond.com/transaction/send]: {'statusCode': 400, 'message': 'Bad Request'}
7
8{
9 "configurations": {
10 "default": {
11 "proxy": "https://devnet-api.elrond.com",
12 "chainID": "D"
13 }
14 },
15 "contract":{
16 "deploy":{
17 "verbose": true,
18 "bytecode": "output/router.wasm",
19 "recall-nonce": true,
20 "nonce": 1,
21 "pem": "../../../wallets/owner/wallet-owner.pem",
22 "gas-limit": 600000000,
23 "send": true,
24 "outfile": "deploy-testnet.interaction.json"
25 }
26 }
27}
28
29
The contract I'm trying to deploy is the following. I've also been through the OpenAPI Spec or the documentation searching for an answer, but there is nothing about it. This route is normally returning error message, but for this specific case it is not.
Some other contract like ping-pong
are working properly with the same erdpy.json
config.
ANSWER
Answered 2022-Jan-05 at 10:47I have you tried to deploy with the argument --verbose
?
That should be something like that (not sure of the syntax because I am on phone)
erdpy --verbose contract deploy
QUESTION
JavaScript: V8 question: are small integers pooled?
Asked 2022-Jan-17 at 12:37was looking at this V8 design doc where it has a section for Constant Pool Entries
it says
Constant pools are used to store heap objects and small integers that are referenced as constants in generated bytecode. and
... Small integers and the strong referenced oddball type’s have bytecodes to load them directly and do not go into the constant pool.
So I am confused: are small integers pooled or not?
My understanding is that it is not worth it pooling small integers if sizeof(int) < sizeof(int *)
- because it is cheaper to just copy the actual integer instead of copying the pointer that points to the integer in the constant pool. Also variables that hold integers can be optimised to be stored directly in CPU registers and skip being allocated in memory first.
Also, are they located on the V8 heap or the stack? My understanding had always been that smis are just be the immediate values allocated on the stack instead of being a pointer + an integer allocated on heap. Also if you take a heap snapshot using chrome devtool you cannot find smis in the heap snapshot - only heap number such as big integers or double like 3.14 are on the heap until I saw this article https://v8.dev/blog/pointer-compression#value-tagging-in-v8
JavaScript values in V8 are represented as objects and allocated on the V8 heap, no matter if they are objects, arrays, numbers or strings. This allows us to represent any value as a pointer to an object.
Now I am just baffled - are smis also allocated on the heap?
ANSWER
Answered 2022-Jan-17 at 12:37V8 developer here.
are small integers pooled or not?
They are not (at least not right now). That said, this is a small implementation detail and could be done either way: it would totally be possible to use the constant pool for Smis. I suppose the decision to build special machinery for Smis (instead of reusing the general-purpose constant pool) was made because things turned out to be more efficient that way.
it is not worth it pooling small integers if
sizeof(int) < sizeof(int *)
The details are different (a Smi is not an int
, and constant pool slots are referenced by index rather than C++ pointer), but this reasoning does go in the right direction: avoiding indirections can save time and memory.
are smis also allocated on the heap?
Yes, everything is allocated on the heap. The stack is only useful for temporary (and sufficiently small) things; that's largely unrelated to the type of thing.
The "trick" of Smis is that they're not stored as separate objects: when you have an object that refers to a Smi, such as let foo = {smi: 42}
, then the value 42
can be smi-encoded and stored directly inside the "foo" object (whereas if the value was 42.5
, then the object would store a pointer to a separate "HeapNumber"). But since the object is on the heap, so is the Smi.
@DanielCruz
What I understand [...] is that constant small integers are pooled. Variable small integers are not.
Nope. Any literal that occurs in source code is "constant". Whether you use let
or const
for your variables has nothing to do with this.
QUESTION
env_parent(): different results when using default argument or when passing explicitly default argument
Asked 2022-Jan-15 at 21:30I have noticed different behavior when using env_parent()
from rlang
package and when using env_parent(caller_env())
, although caller_env()
is a default argument for env_parent()
first parameter:
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17
I'm not very familiar with environments and I was able to prepare MRE only based on shiny app (I have noticed this using shiny app). It is important to have app in two files - ui.R
and server.R
:
ui.R
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17library(shiny)
18library(rlang)
19
20parent <<- function() {
21 env_parent()
22}
23
24parents_default_arg_passed <<- function() {
25 env_parent(caller_env())
26}
27
28ui <- fluidPage(
29 textOutput("env_parent"),
30 textOutput("env_parents_default_arg_passed")
31)
32
server.R
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17library(shiny)
18library(rlang)
19
20parent <<- function() {
21 env_parent()
22}
23
24parents_default_arg_passed <<- function() {
25 env_parent(caller_env())
26}
27
28ui <- fluidPage(
29 textOutput("env_parent"),
30 textOutput("env_parents_default_arg_passed")
31)
32server <- function(input, output, session) {
33
34 output$env_parent <- renderPrint({
35 names(parent())
36 })
37
38 output$env_parents_default_arg_passed <- renderPrint({
39 names(parents_default_arg_passed())
40 })
41
42}
43
I see this output after running the app:
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17library(shiny)
18library(rlang)
19
20parent <<- function() {
21 env_parent()
22}
23
24parents_default_arg_passed <<- function() {
25 env_parent(caller_env())
26}
27
28ui <- fluidPage(
29 textOutput("env_parent"),
30 textOutput("env_parents_default_arg_passed")
31)
32server <- function(input, output, session) {
33
34 output$env_parent <- renderPrint({
35 names(parent())
36 })
37
38 output$env_parents_default_arg_passed <- renderPrint({
39 names(parents_default_arg_passed())
40 })
41
42}
43[1] "ui"
44[1] "~" ".__tidyeval_quosure_mask__."
45
Why is that?
ANSWER
Answered 2022-Jan-15 at 21:30The short answer to your question is, there is a difference between explicitly calling a function with an argument, compared to calling the function and letting it supply the same argument as default by itself. In the latter case, the caller environment is the execution environment of the function. In the former case the caller environment is where the function was explicitly called from.
It took me a while to find a nice visualization of what is going on, but I think I found one.
Let's create two functions similar to {rlang}'s env_parent
and caller_env
. The difference is that myCallEnv
will not only return the execution environment of the calling function, it will also show the call strack tree with lobstr::cst()
:
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17library(shiny)
18library(rlang)
19
20parent <<- function() {
21 env_parent()
22}
23
24parents_default_arg_passed <<- function() {
25 env_parent(caller_env())
26}
27
28ui <- fluidPage(
29 textOutput("env_parent"),
30 textOutput("env_parents_default_arg_passed")
31)
32server <- function(input, output, session) {
33
34 output$env_parent <- renderPrint({
35 names(parent())
36 })
37
38 output$env_parents_default_arg_passed <- renderPrint({
39 names(parents_default_arg_passed())
40 })
41
42}
43[1] "ui"
44[1] "~" ".__tidyeval_quosure_mask__."
45myCallEnv <- function() {
46 print(lobstr::cst())
47 caller_env(n = 1)
48 }
49
50myEnvParent <- function(e = myCallEnv()) {
51 env_parent(env = e)
52}
53
Now let's call myEnvParent
one time with its default argument and one time by supplying the same default argument explicitly:
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17library(shiny)
18library(rlang)
19
20parent <<- function() {
21 env_parent()
22}
23
24parents_default_arg_passed <<- function() {
25 env_parent(caller_env())
26}
27
28ui <- fluidPage(
29 textOutput("env_parent"),
30 textOutput("env_parents_default_arg_passed")
31)
32server <- function(input, output, session) {
33
34 output$env_parent <- renderPrint({
35 names(parent())
36 })
37
38 output$env_parents_default_arg_passed <- renderPrint({
39 names(parents_default_arg_passed())
40 })
41
42}
43[1] "ui"
44[1] "~" ".__tidyeval_quosure_mask__."
45myCallEnv <- function() {
46 print(lobstr::cst())
47 caller_env(n = 1)
48 }
49
50myEnvParent <- function(e = myCallEnv()) {
51 env_parent(env = e)
52}
53myEnvParent()
54# â–†
55# 1. └─global myEnvParent()
56# 2. ├─rlang::env_parent(env = e)
57# 3. │ └─rlang:::get_env_retired(env, "env_parent()")
58# 4. │ └─rlang::is_environment(x)
59# 5. └─global myCallEnv()
60# 6. ├─base::print(lobstr::cst())
61# 7. └─lobstr::cst()
62# NULL
63# <environment: R_GlobalEnv>
64
65myEnvParent(myCallEnv())
66# â–†
67# 1. ├─global myEnvParent(myCallEnv())
68# 2. │ └─rlang::env_parent(env = e)
69# 3. │ └─rlang:::get_env_retired(env, "env_parent()")
70# 4. │ └─rlang::is_environment(x)
71# 5. └─global myCallEnv()
72# 6. ├─base::print(lobstr::cst())
73# 7. └─lobstr::cst()
74# NULL
75# <environment: package:rlang>
76# attr(,"name")
77# [1] "package:rlang"
78# attr(,"path")
79# [1] # "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/rlang"
80
Comparing both outputs we can see that in the first call, myEnvParent()
, myCallEnv()
is called from the execution environment of myEnvParent()
. The parent environment is the global environment R_GlobalEnv
.
In the second call, myEnvParent(myCallEnv())
, myCallEnv()
is called from the global environment and therefore its parent is the second environment on the search path:
1library(rlang)
2env_parent
3#> function (env = caller_env(), n = 1)
4#> {
5#> env_ <- get_env_retired(env, "env_parent()")
6#> while (n > 0) {
7#> if (is_empty_env(env_)) {
8#> abort("The empty environment has no parent")
9#> }
10#> n <- n - 1
11#> env_ <- parent.env(env_)
12#> }
13#> env_
14#> }
15#> <bytecode: 0x000000001d8ad000>
16#> <environment: namespace:rlang>
17library(shiny)
18library(rlang)
19
20parent <<- function() {
21 env_parent()
22}
23
24parents_default_arg_passed <<- function() {
25 env_parent(caller_env())
26}
27
28ui <- fluidPage(
29 textOutput("env_parent"),
30 textOutput("env_parents_default_arg_passed")
31)
32server <- function(input, output, session) {
33
34 output$env_parent <- renderPrint({
35 names(parent())
36 })
37
38 output$env_parents_default_arg_passed <- renderPrint({
39 names(parents_default_arg_passed())
40 })
41
42}
43[1] "ui"
44[1] "~" ".__tidyeval_quosure_mask__."
45myCallEnv <- function() {
46 print(lobstr::cst())
47 caller_env(n = 1)
48 }
49
50myEnvParent <- function(e = myCallEnv()) {
51 env_parent(env = e)
52}
53myEnvParent()
54# â–†
55# 1. └─global myEnvParent()
56# 2. ├─rlang::env_parent(env = e)
57# 3. │ └─rlang:::get_env_retired(env, "env_parent()")
58# 4. │ └─rlang::is_environment(x)
59# 5. └─global myCallEnv()
60# 6. ├─base::print(lobstr::cst())
61# 7. └─lobstr::cst()
62# NULL
63# <environment: R_GlobalEnv>
64
65myEnvParent(myCallEnv())
66# â–†
67# 1. ├─global myEnvParent(myCallEnv())
68# 2. │ └─rlang::env_parent(env = e)
69# 3. │ └─rlang:::get_env_retired(env, "env_parent()")
70# 4. │ └─rlang::is_environment(x)
71# 5. └─global myCallEnv()
72# 6. ├─base::print(lobstr::cst())
73# 7. └─lobstr::cst()
74# NULL
75# <environment: package:rlang>
76# attr(,"name")
77# [1] "package:rlang"
78# attr(,"path")
79# [1] # "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/rlang"
80searchpaths()
81# [1] ".GlobalEnv"
82# [2] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/rlang"
83# [3] "tools:rstudio"
84# [4] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/stats"
85# [5] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/graphics"
86# [6] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/grDevices"
87# [7] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/utils"
88# [8] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/datasets"
89# [9] "/Library/Frameworks/R.framework/Versions/4.0/Resources/library/methods"
90#[10] "Autoloads"
91#[11] "/Library/Frameworks/R.framework/Resources/library/base"
92
QUESTION
looping over array, performance difference between indexed and enhanced for loop
Asked 2022-Jan-05 at 19:41The JLS states, that for arrays, "The enhanced for statement is equivalent to a basic for statement of the form". However if I check the generated bytecode for JDK8, for both variants different bytecode is generated, and if I try to measure the performance, surprisingly, the enhanced one seems to be giving better results(on jdk8)... Can someone advise why it's that? I'd guess it's because of incorrect jmh testing, so if it's that, please suggest how to fix that. (I know that JMH states not to test using loops, but I don't think this applies here, as I'm actually trying to measure the loops here)
My JMH testing was rather simple (probably too simple), but I cannot explain the results. Testing JMH code is below, typical results are:
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3
meaning typically enhanced for loop is faster, and measurement for it is more accurate than for indexed loop, so we cannot address the difference to measurement uncertainty. Principally the same results are for array initialized with random integers, or bigger arrays.
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49
ANSWER
Answered 2022-Jan-05 at 19:41TL;DR: You are observing what happens when JIT compiler cannot trust that values
are not changing inside the loop. Additionally, in the tiny benchmark like this, Blackhole.consume
costs dominate, obscuring the results.
Simplifying the test:
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
50@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
51@Fork(3)
52@BenchmarkMode(Mode.AverageTime)
53@OutputTimeUnit(TimeUnit.NANOSECONDS)
54@State(Scope.Benchmark)
55public class JdkBenchmarks {
56
57 public int[] values;
58
59 @Setup
60 public void setupArray() {
61 int count = 1000;
62 values = new int[count];
63 for(int i = 0; i < count; i++) {
64 values[i] = i;
65 }
66 }
67
68 @Benchmark
69 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
70 public void indexed(Blackhole bh) {
71 for(int i = 0; i < values.length; i++) {
72 bh.consume(values[i]);
73 }
74 }
75
76 @Benchmark
77 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
78 public void indexed_cached(Blackhole bh) {
79 int[] vs = values;
80 int length = vs.length;
81 for(int i = 0; i < length; i++) {
82 bh.consume(vs[i]);
83 }
84 }
85
86 @Benchmark
87 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
88 public void enhanced(Blackhole bh) {
89 for (int value : values) {
90 bh.consume(value);
91 }
92 }
93}
94
Running both enhanced
and indexed_cached
under -prof perfasm
reveals this hot loop (I specifically did @CompilerControl(DONT_INLINE)
to let @Benchmark
method be compiled alone, which makes an easier to digest perfasm
output):
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
50@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
51@Fork(3)
52@BenchmarkMode(Mode.AverageTime)
53@OutputTimeUnit(TimeUnit.NANOSECONDS)
54@State(Scope.Benchmark)
55public class JdkBenchmarks {
56
57 public int[] values;
58
59 @Setup
60 public void setupArray() {
61 int count = 1000;
62 values = new int[count];
63 for(int i = 0; i < count; i++) {
64 values[i] = i;
65 }
66 }
67
68 @Benchmark
69 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
70 public void indexed(Blackhole bh) {
71 for(int i = 0; i < values.length; i++) {
72 bh.consume(values[i]);
73 }
74 }
75
76 @Benchmark
77 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
78 public void indexed_cached(Blackhole bh) {
79 int[] vs = values;
80 int length = vs.length;
81 for(int i = 0; i < length; i++) {
82 bh.consume(vs[i]);
83 }
84 }
85
86 @Benchmark
87 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
88 public void enhanced(Blackhole bh) {
89 for (int value : values) {
90 bh.consume(value);
91 }
92 }
93}
94 ↗ 0x...4240: mov 0x10(%r8,%rsi,4),%r10d ; load values[i], blackhole it
95 22.68% │ 0x...4245: mov 0x14(%r8,%rsi,4),%r11d ; ... repeat 7 more times...
96 │ 0x...424a: mov 0x18(%r8,%rsi,4),%r10d ;
97 20.95% │ 0x...424f: mov 0x1c(%r8,%rsi,4),%r10d ;
98 0.02% │ 0x...4254: mov 0x20(%r8,%rsi,4),%r11d ;
99 24.73% │ 0x...4259: mov 0x24(%r8,%rsi,4),%r10d ;
100 0.24% │ 0x...425e: mov 0x28(%r8,%rsi,4),%r11d ;
101 20.04% │ 0x...4263: mov 0x2c(%r8,%rsi,4),%r10d ;
102 0.22% │ 0x...4268: add $0x8,%esi ; i += 8
103 │ 0x...426b: cmp %ebp,%esi ; compare i with length (in %ebp)
104 0.26% â•° 0x...426d: jl 0x...4240 ; circle back if 8 more elements available
105
Very efficient!
Running indexed
with -prof perfasm
reveals:
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
50@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
51@Fork(3)
52@BenchmarkMode(Mode.AverageTime)
53@OutputTimeUnit(TimeUnit.NANOSECONDS)
54@State(Scope.Benchmark)
55public class JdkBenchmarks {
56
57 public int[] values;
58
59 @Setup
60 public void setupArray() {
61 int count = 1000;
62 values = new int[count];
63 for(int i = 0; i < count; i++) {
64 values[i] = i;
65 }
66 }
67
68 @Benchmark
69 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
70 public void indexed(Blackhole bh) {
71 for(int i = 0; i < values.length; i++) {
72 bh.consume(values[i]);
73 }
74 }
75
76 @Benchmark
77 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
78 public void indexed_cached(Blackhole bh) {
79 int[] vs = values;
80 int length = vs.length;
81 for(int i = 0; i < length; i++) {
82 bh.consume(vs[i]);
83 }
84 }
85
86 @Benchmark
87 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
88 public void enhanced(Blackhole bh) {
89 for (int value : values) {
90 bh.consume(value);
91 }
92 }
93}
94 ↗ 0x...4240: mov 0x10(%r8,%rsi,4),%r10d ; load values[i], blackhole it
95 22.68% │ 0x...4245: mov 0x14(%r8,%rsi,4),%r11d ; ... repeat 7 more times...
96 │ 0x...424a: mov 0x18(%r8,%rsi,4),%r10d ;
97 20.95% │ 0x...424f: mov 0x1c(%r8,%rsi,4),%r10d ;
98 0.02% │ 0x...4254: mov 0x20(%r8,%rsi,4),%r11d ;
99 24.73% │ 0x...4259: mov 0x24(%r8,%rsi,4),%r10d ;
100 0.24% │ 0x...425e: mov 0x28(%r8,%rsi,4),%r11d ;
101 20.04% │ 0x...4263: mov 0x2c(%r8,%rsi,4),%r10d ;
102 0.22% │ 0x...4268: add $0x8,%esi ; i += 8
103 │ 0x...426b: cmp %ebp,%esi ; compare i with length (in %ebp)
104 0.26% â•° 0x...426d: jl 0x...4240 ; circle back if 8 more elements available
105 ↗ 0x...4170: mov 0xc(%r12,%r8,8),%r9d ; array bounds check, load values.length
106 3.42% │ 0x...4175: cmp %r9d,%r10d ; array bounds check, compare i
107 16.02% │ 0x...4178: jae 0x...41b1 ; failed? jump to exception handling
108 │ 0x...417a: lea (%r12,%r8,8),%r11 ; load values[i], part 1
109 0.04% │ 0x...417e: mov 0x10(%r11,%r10,4),%r11d ; load values[i], part 2
110 │ ; %r11d is blackholed
111 35.69% │ 0x...4183: mov 0xc(%rsi),%r8d ; get "values"
112 0.71% │ 0x...4187: mov 0x348(%r15),%r11 ; safepoint poll, part 1 (JVM infra)
113 4.03% │ 0x...418e: inc %r10d ; i++
114 0.12% │ 0x...4191: test %eax,(%r11) ; safepoint poll, part 2 (JVM infra)
115 27.74% │ 0x...4194: mov 0xc(%r12,%r8,8),%r9d ; load values.length
116 8.53% │ 0x...4199: cmp %r9d,%r10d ; check i < values.length
117 0.24% â•° 0x...419c: jl 0x...4170 ; circle back if more
118
This is because Blackhole.consume
call is opaque to the compiler (like many other non-inlined calls), so it has to conservatively presume that values
can change in the middle of the loop!
Which means, compiler cannot stash values
in a register, it cannot trust the array bounds check to always succeed, it cannot even guarantee the loop terminates (hence safepoint polls), and on top of that, loop unrolling does not want to multiply that per-element mess even more.
So you get the penalty like this (TR 3970X, JDK 17.0.2 EA, Linux x86_64):
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
50@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
51@Fork(3)
52@BenchmarkMode(Mode.AverageTime)
53@OutputTimeUnit(TimeUnit.NANOSECONDS)
54@State(Scope.Benchmark)
55public class JdkBenchmarks {
56
57 public int[] values;
58
59 @Setup
60 public void setupArray() {
61 int count = 1000;
62 values = new int[count];
63 for(int i = 0; i < count; i++) {
64 values[i] = i;
65 }
66 }
67
68 @Benchmark
69 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
70 public void indexed(Blackhole bh) {
71 for(int i = 0; i < values.length; i++) {
72 bh.consume(values[i]);
73 }
74 }
75
76 @Benchmark
77 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
78 public void indexed_cached(Blackhole bh) {
79 int[] vs = values;
80 int length = vs.length;
81 for(int i = 0; i < length; i++) {
82 bh.consume(vs[i]);
83 }
84 }
85
86 @Benchmark
87 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
88 public void enhanced(Blackhole bh) {
89 for (int value : values) {
90 bh.consume(value);
91 }
92 }
93}
94 ↗ 0x...4240: mov 0x10(%r8,%rsi,4),%r10d ; load values[i], blackhole it
95 22.68% │ 0x...4245: mov 0x14(%r8,%rsi,4),%r11d ; ... repeat 7 more times...
96 │ 0x...424a: mov 0x18(%r8,%rsi,4),%r10d ;
97 20.95% │ 0x...424f: mov 0x1c(%r8,%rsi,4),%r10d ;
98 0.02% │ 0x...4254: mov 0x20(%r8,%rsi,4),%r11d ;
99 24.73% │ 0x...4259: mov 0x24(%r8,%rsi,4),%r10d ;
100 0.24% │ 0x...425e: mov 0x28(%r8,%rsi,4),%r11d ;
101 20.04% │ 0x...4263: mov 0x2c(%r8,%rsi,4),%r10d ;
102 0.22% │ 0x...4268: add $0x8,%esi ; i += 8
103 │ 0x...426b: cmp %ebp,%esi ; compare i with length (in %ebp)
104 0.26% â•° 0x...426d: jl 0x...4240 ; circle back if 8 more elements available
105 ↗ 0x...4170: mov 0xc(%r12,%r8,8),%r9d ; array bounds check, load values.length
106 3.42% │ 0x...4175: cmp %r9d,%r10d ; array bounds check, compare i
107 16.02% │ 0x...4178: jae 0x...41b1 ; failed? jump to exception handling
108 │ 0x...417a: lea (%r12,%r8,8),%r11 ; load values[i], part 1
109 0.04% │ 0x...417e: mov 0x10(%r11,%r10,4),%r11d ; load values[i], part 2
110 │ ; %r11d is blackholed
111 35.69% │ 0x...4183: mov 0xc(%rsi),%r8d ; get "values"
112 0.71% │ 0x...4187: mov 0x348(%r15),%r11 ; safepoint poll, part 1 (JVM infra)
113 4.03% │ 0x...418e: inc %r10d ; i++
114 0.12% │ 0x...4191: test %eax,(%r11) ; safepoint poll, part 2 (JVM infra)
115 27.74% │ 0x...4194: mov 0xc(%r12,%r8,8),%r9d ; load values.length
116 8.53% │ 0x...4199: cmp %r9d,%r10d ; check i < values.length
117 0.24% â•° 0x...419c: jl 0x...4170 ; circle back if more
118Benchmark Mode Cnt Score Error Units
119JdkBenchmarks.enhanced avgt 5 144.962 ± 0.918 ns/op
120JdkBenchmarks.indexed avgt 5 1030.981 ± 3.775 ns/op ; + 880 ns/op!
121JdkBenchmarks.indexed_cached avgt 5 144.799 ± 0.643 ns/op ; same as enhanced
122
Additional fun part:
On most JDKs the dominating costs are the costs of calling the Blackhole.consume
in this test. Compared to the cost of array access, the cost of Java-style Blackhole
is quite bad. With JDK 17+ and JMH 1.34, the compiler Blackholes
would be used, and thus provide much more fidelity for the test.
Without compiler blackholes, the effect hides in the Blackhole
overhead nearly completely (>25x overhead means we can execute a lot of bad code preceding the Blackhole
call!):
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
50@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
51@Fork(3)
52@BenchmarkMode(Mode.AverageTime)
53@OutputTimeUnit(TimeUnit.NANOSECONDS)
54@State(Scope.Benchmark)
55public class JdkBenchmarks {
56
57 public int[] values;
58
59 @Setup
60 public void setupArray() {
61 int count = 1000;
62 values = new int[count];
63 for(int i = 0; i < count; i++) {
64 values[i] = i;
65 }
66 }
67
68 @Benchmark
69 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
70 public void indexed(Blackhole bh) {
71 for(int i = 0; i < values.length; i++) {
72 bh.consume(values[i]);
73 }
74 }
75
76 @Benchmark
77 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
78 public void indexed_cached(Blackhole bh) {
79 int[] vs = values;
80 int length = vs.length;
81 for(int i = 0; i < length; i++) {
82 bh.consume(vs[i]);
83 }
84 }
85
86 @Benchmark
87 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
88 public void enhanced(Blackhole bh) {
89 for (int value : values) {
90 bh.consume(value);
91 }
92 }
93}
94 ↗ 0x...4240: mov 0x10(%r8,%rsi,4),%r10d ; load values[i], blackhole it
95 22.68% │ 0x...4245: mov 0x14(%r8,%rsi,4),%r11d ; ... repeat 7 more times...
96 │ 0x...424a: mov 0x18(%r8,%rsi,4),%r10d ;
97 20.95% │ 0x...424f: mov 0x1c(%r8,%rsi,4),%r10d ;
98 0.02% │ 0x...4254: mov 0x20(%r8,%rsi,4),%r11d ;
99 24.73% │ 0x...4259: mov 0x24(%r8,%rsi,4),%r10d ;
100 0.24% │ 0x...425e: mov 0x28(%r8,%rsi,4),%r11d ;
101 20.04% │ 0x...4263: mov 0x2c(%r8,%rsi,4),%r10d ;
102 0.22% │ 0x...4268: add $0x8,%esi ; i += 8
103 │ 0x...426b: cmp %ebp,%esi ; compare i with length (in %ebp)
104 0.26% â•° 0x...426d: jl 0x...4240 ; circle back if 8 more elements available
105 ↗ 0x...4170: mov 0xc(%r12,%r8,8),%r9d ; array bounds check, load values.length
106 3.42% │ 0x...4175: cmp %r9d,%r10d ; array bounds check, compare i
107 16.02% │ 0x...4178: jae 0x...41b1 ; failed? jump to exception handling
108 │ 0x...417a: lea (%r12,%r8,8),%r11 ; load values[i], part 1
109 0.04% │ 0x...417e: mov 0x10(%r11,%r10,4),%r11d ; load values[i], part 2
110 │ ; %r11d is blackholed
111 35.69% │ 0x...4183: mov 0xc(%rsi),%r8d ; get "values"
112 0.71% │ 0x...4187: mov 0x348(%r15),%r11 ; safepoint poll, part 1 (JVM infra)
113 4.03% │ 0x...418e: inc %r10d ; i++
114 0.12% │ 0x...4191: test %eax,(%r11) ; safepoint poll, part 2 (JVM infra)
115 27.74% │ 0x...4194: mov 0xc(%r12,%r8,8),%r9d ; load values.length
116 8.53% │ 0x...4199: cmp %r9d,%r10d ; check i < values.length
117 0.24% â•° 0x...419c: jl 0x...4170 ; circle back if more
118Benchmark Mode Cnt Score Error Units
119JdkBenchmarks.enhanced avgt 5 144.962 ± 0.918 ns/op
120JdkBenchmarks.indexed avgt 5 1030.981 ± 3.775 ns/op ; + 880 ns/op!
121JdkBenchmarks.indexed_cached avgt 5 144.799 ± 0.643 ns/op ; same as enhanced
122Benchmark Mode Cnt Score Error Units
123JdkBenchmarks.enhanced avgt 5 4062.866 ± 4.736 ns/op
124JdkBenchmarks.indexed avgt 5 4071.620 ± 1.057 ns/op ; + 10 ns/op [whoops]
125JdkBenchmarks.indexed_cached avgt 5 4061.390 ± 0.692 ns/op ; same as enhanced
126
It would re-manifest if we drop @CompilerControl(DONT_INLINE)
, because the resulting generated code would be much messier:
1JdkBenchmarks.enhanced avgt 5 2556.281 ± 31.789 ns/op
2JdkBenchmarks.indexed avgt 5 4032.164 ± 100.121 ns/op
3public class JdkBenchmarks {
4
5 @Benchmark
6 @BenchmarkMode(AverageTime)
7 @OutputTimeUnit(NANOSECONDS)
8 public void indexed(Blackhole blackhole, TestState testState) {
9 int length = testState.values.length;
10 for(int i = 0; i < length; i++) {
11 blackhole.consume(testState.values[i]);
12 }
13 }
14
15 @Benchmark
16 @BenchmarkMode(AverageTime)
17 @OutputTimeUnit(NANOSECONDS)
18 public void enhanced(Blackhole blackhole, TestState testState) {
19 for (int value : testState.values) {
20 blackhole.consume(value);
21 }
22 }
23
24
25 @State(Scope.Benchmark)
26 public static class TestState {
27 public int[] values;
28
29 @Setup
30 public void setupArray() {
31 int count = 1000;
32 values = new int[count];
33 for(int i = 0; i < count; i++) {
34 values[i] = i;
35 }
36 }
37 }
38
39 public static void main(String[] args) throws RunnerException {
40 Options opt = new OptionsBuilder()
41 .include(JdkBenchmarks.class.getSimpleName())
42 .forks(1)
43 .build();
44
45 new Runner(opt).run();
46 }
47
48}
49@Warmup(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
50@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
51@Fork(3)
52@BenchmarkMode(Mode.AverageTime)
53@OutputTimeUnit(TimeUnit.NANOSECONDS)
54@State(Scope.Benchmark)
55public class JdkBenchmarks {
56
57 public int[] values;
58
59 @Setup
60 public void setupArray() {
61 int count = 1000;
62 values = new int[count];
63 for(int i = 0; i < count; i++) {
64 values[i] = i;
65 }
66 }
67
68 @Benchmark
69 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
70 public void indexed(Blackhole bh) {
71 for(int i = 0; i < values.length; i++) {
72 bh.consume(values[i]);
73 }
74 }
75
76 @Benchmark
77 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
78 public void indexed_cached(Blackhole bh) {
79 int[] vs = values;
80 int length = vs.length;
81 for(int i = 0; i < length; i++) {
82 bh.consume(vs[i]);
83 }
84 }
85
86 @Benchmark
87 @CompilerControl(CompilerControl.Mode.DONT_INLINE)
88 public void enhanced(Blackhole bh) {
89 for (int value : values) {
90 bh.consume(value);
91 }
92 }
93}
94 ↗ 0x...4240: mov 0x10(%r8,%rsi,4),%r10d ; load values[i], blackhole it
95 22.68% │ 0x...4245: mov 0x14(%r8,%rsi,4),%r11d ; ... repeat 7 more times...
96 │ 0x...424a: mov 0x18(%r8,%rsi,4),%r10d ;
97 20.95% │ 0x...424f: mov 0x1c(%r8,%rsi,4),%r10d ;
98 0.02% │ 0x...4254: mov 0x20(%r8,%rsi,4),%r11d ;
99 24.73% │ 0x...4259: mov 0x24(%r8,%rsi,4),%r10d ;
100 0.24% │ 0x...425e: mov 0x28(%r8,%rsi,4),%r11d ;
101 20.04% │ 0x...4263: mov 0x2c(%r8,%rsi,4),%r10d ;
102 0.22% │ 0x...4268: add $0x8,%esi ; i += 8
103 │ 0x...426b: cmp %ebp,%esi ; compare i with length (in %ebp)
104 0.26% â•° 0x...426d: jl 0x...4240 ; circle back if 8 more elements available
105 ↗ 0x...4170: mov 0xc(%r12,%r8,8),%r9d ; array bounds check, load values.length
106 3.42% │ 0x...4175: cmp %r9d,%r10d ; array bounds check, compare i
107 16.02% │ 0x...4178: jae 0x...41b1 ; failed? jump to exception handling
108 │ 0x...417a: lea (%r12,%r8,8),%r11 ; load values[i], part 1
109 0.04% │ 0x...417e: mov 0x10(%r11,%r10,4),%r11d ; load values[i], part 2
110 │ ; %r11d is blackholed
111 35.69% │ 0x...4183: mov 0xc(%rsi),%r8d ; get "values"
112 0.71% │ 0x...4187: mov 0x348(%r15),%r11 ; safepoint poll, part 1 (JVM infra)
113 4.03% │ 0x...418e: inc %r10d ; i++
114 0.12% │ 0x...4191: test %eax,(%r11) ; safepoint poll, part 2 (JVM infra)
115 27.74% │ 0x...4194: mov 0xc(%r12,%r8,8),%r9d ; load values.length
116 8.53% │ 0x...4199: cmp %r9d,%r10d ; check i < values.length
117 0.24% â•° 0x...419c: jl 0x...4170 ; circle back if more
118Benchmark Mode Cnt Score Error Units
119JdkBenchmarks.enhanced avgt 5 144.962 ± 0.918 ns/op
120JdkBenchmarks.indexed avgt 5 1030.981 ± 3.775 ns/op ; + 880 ns/op!
121JdkBenchmarks.indexed_cached avgt 5 144.799 ± 0.643 ns/op ; same as enhanced
122Benchmark Mode Cnt Score Error Units
123JdkBenchmarks.enhanced avgt 5 4062.866 ± 4.736 ns/op
124JdkBenchmarks.indexed avgt 5 4071.620 ± 1.057 ns/op ; + 10 ns/op [whoops]
125JdkBenchmarks.indexed_cached avgt 5 4061.390 ± 0.692 ns/op ; same as enhanced
126Benchmark Mode Cnt Score Error Units
127JdkBenchmarks.enhanced avgt 5 4067.118 ± 40.699 ns/op
128JdkBenchmarks.indexed avgt 5 4601.370 ± 0.632 ns/op ; + 530 ns/op
129JdkBenchmarks.indexed_cached avgt 5 4064.455 ± 1.554 ns/op ; same as enhanced
130
QUESTION
solidity TypeError: Object of type set is not JSON serializable
Asked 2021-Dec-29 at 07:57I ran the code in VSCode and got a TypeError: Object of type set is not JSON serializable
. I just start to learn to code, really don't get it, and googled it, also didn't know what does JSON serializable means.
1from solcx import compile_standard
2import json
3
4# get the contract content
5with open("./SimpleStorage.sol", "r") as file:
6 simple_storage_file = file.read()
7
8# compile the contract
9
10compiled_sol = compile_standard(
11 {
12 "language": "Solidity",
13 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
14 "settings": {
15 "outputSelection": {
16 "*": {
17 "*": {"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
18 }
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25# creat json file dump the comiled code in it to make it more readable.
26with open("compiled_code.json", "w") as file:
27 json.dump(compiled_sol, file)
28
29print(compiled_sol)
30
The full error information is below:
1from solcx import compile_standard
2import json
3
4# get the contract content
5with open("./SimpleStorage.sol", "r") as file:
6 simple_storage_file = file.read()
7
8# compile the contract
9
10compiled_sol = compile_standard(
11 {
12 "language": "Solidity",
13 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
14 "settings": {
15 "outputSelection": {
16 "*": {
17 "*": {"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
18 }
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25# creat json file dump the comiled code in it to make it more readable.
26with open("compiled_code.json", "w") as file:
27 json.dump(compiled_sol, file)
28
29print(compiled_sol)
30(env) (base) liwei@liweideMacBook-Pro practice % python3 deploy.py
31Traceback (most recent call last):
32 File "deploy.py", line 10, in <module>
33 compiled_sol = compile_standard(
34 File "/Users/liwei/Desktop/demos/practice/env/lib/python3.8/site-packages/solcx/main.py", line 375, in compile_standard
35 stdin=json.dumps(input_data),
36 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 231, in dumps
37 return _default_encoder.encode(obj)
38 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 199, in encode
39 chunks = self.iterencode(o, _one_shot=True)
40 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 257, in iterencode
41 return _iterencode(o, 0)
42 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 179, in default
43 raise TypeError(f'Object of type {o.__class__.__name__} '
44TypeError: Object of type set is not JSON serializable
45
ANSWER
Answered 2021-Dec-29 at 01:48Instead of this:
1from solcx import compile_standard
2import json
3
4# get the contract content
5with open("./SimpleStorage.sol", "r") as file:
6 simple_storage_file = file.read()
7
8# compile the contract
9
10compiled_sol = compile_standard(
11 {
12 "language": "Solidity",
13 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
14 "settings": {
15 "outputSelection": {
16 "*": {
17 "*": {"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
18 }
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25# creat json file dump the comiled code in it to make it more readable.
26with open("compiled_code.json", "w") as file:
27 json.dump(compiled_sol, file)
28
29print(compiled_sol)
30(env) (base) liwei@liweideMacBook-Pro practice % python3 deploy.py
31Traceback (most recent call last):
32 File "deploy.py", line 10, in <module>
33 compiled_sol = compile_standard(
34 File "/Users/liwei/Desktop/demos/practice/env/lib/python3.8/site-packages/solcx/main.py", line 375, in compile_standard
35 stdin=json.dumps(input_data),
36 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 231, in dumps
37 return _default_encoder.encode(obj)
38 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 199, in encode
39 chunks = self.iterencode(o, _one_shot=True)
40 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 257, in iterencode
41 return _iterencode(o, 0)
42 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 179, in default
43 raise TypeError(f'Object of type {o.__class__.__name__} '
44TypeError: Object of type set is not JSON serializable
45{"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
46
you should use this:
1from solcx import compile_standard
2import json
3
4# get the contract content
5with open("./SimpleStorage.sol", "r") as file:
6 simple_storage_file = file.read()
7
8# compile the contract
9
10compiled_sol = compile_standard(
11 {
12 "language": "Solidity",
13 "sources": {"SimpleStorage.sol": {"content": simple_storage_file}},
14 "settings": {
15 "outputSelection": {
16 "*": {
17 "*": {"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
18 }
19 }
20 },
21 },
22 solc_version="0.6.0",
23)
24
25# creat json file dump the comiled code in it to make it more readable.
26with open("compiled_code.json", "w") as file:
27 json.dump(compiled_sol, file)
28
29print(compiled_sol)
30(env) (base) liwei@liweideMacBook-Pro practice % python3 deploy.py
31Traceback (most recent call last):
32 File "deploy.py", line 10, in <module>
33 compiled_sol = compile_standard(
34 File "/Users/liwei/Desktop/demos/practice/env/lib/python3.8/site-packages/solcx/main.py", line 375, in compile_standard
35 stdin=json.dumps(input_data),
36 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 231, in dumps
37 return _default_encoder.encode(obj)
38 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 199, in encode
39 chunks = self.iterencode(o, _one_shot=True)
40 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 257, in iterencode
41 return _iterencode(o, 0)
42 File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/encoder.py", line 179, in default
43 raise TypeError(f'Object of type {o.__class__.__name__} '
44TypeError: Object of type set is not JSON serializable
45{"abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"}
46["abi", "metadata", "evm.bytecode", "evm.bytecode.sourceMap"]
47
Sets in Python aren't JSON serializable.
QUESTION
JDK 17: Switch statement causes java.lang.VerifyError: Bad type on operand stack
Asked 2021-Oct-26 at 09:25Just tried JDK17 on Eclipse 2021-09 to have it fail with a java.lang.VerifyError
, which wasn't very helpful itself. I tracked it down to a switch statement, that gets fed a value pulled out of a Map
or another generic type. If I use a local variable in the switch statement instead, everything works as intended.
Test code:
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18
This throws the following error:
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18Error: Unable to initialize main class SwitchFail
19Caused by: java.lang.VerifyError: Bad type on operand stack
20Exception Details:
21 Location:
22 SwitchFail.main([Ljava/lang/String;)V @33: invokevirtual
23 Reason:
24 Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/lang/String'
25 Current Frame:
26 bci: @33
27 flags: { }
28 locals: { '[Ljava/lang/String;', 'java/util/HashMap', 'java/lang/Object' }
29 stack: { 'java/lang/Object' }
30 Bytecode:
31 0000000: bb00 1059 b700 124c 2b04 b800 1312 19b9
32 0000010: 001b 0300 572b 04b8 0013 b900 2102 0059
33 0000020: 4db6 0025 57b1
34
35
36
37
Didn't try another JDK between 11 and 17 yet. Switches gained some new functions between those versions, so that might be it. Maybe it's a problem in the Eclipse JDT or my local JDK, so any tries to reproduce this error on another configuration or IDE would be great. Tried on OpenJDK (build 17+35-2724) for macOS.
Edit: Also happens on
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18Error: Unable to initialize main class SwitchFail
19Caused by: java.lang.VerifyError: Bad type on operand stack
20Exception Details:
21 Location:
22 SwitchFail.main([Ljava/lang/String;)V @33: invokevirtual
23 Reason:
24 Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/lang/String'
25 Current Frame:
26 bci: @33
27 flags: { }
28 locals: { '[Ljava/lang/String;', 'java/util/HashMap', 'java/lang/Object' }
29 stack: { 'java/lang/Object' }
30 Bytecode:
31 0000000: bb00 1059 b700 124c 2b04 b800 1312 19b9
32 0000010: 001b 0300 572b 04b8 0013 b900 2102 0059
33 0000020: 4db6 0025 57b1
34
35
36
37List<String> stringList = Arrays.asList("Test");
38switch(stringList.get(0)) {}
39
Most likely an issue with the new JDT for Java 17 or my local installation.
ANSWER
Answered 2021-Oct-26 at 09:25It is a problem with your Eclipse, not with Java-17 itself. Java-17 has been released only yesterday. Wait for some time until the IDEs are updated to support Java-17.
Demo:
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18Error: Unable to initialize main class SwitchFail
19Caused by: java.lang.VerifyError: Bad type on operand stack
20Exception Details:
21 Location:
22 SwitchFail.main([Ljava/lang/String;)V @33: invokevirtual
23 Reason:
24 Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/lang/String'
25 Current Frame:
26 bci: @33
27 flags: { }
28 locals: { '[Ljava/lang/String;', 'java/util/HashMap', 'java/lang/Object' }
29 stack: { 'java/lang/Object' }
30 Bytecode:
31 0000000: bb00 1059 b700 124c 2b04 b800 1312 19b9
32 0000010: 001b 0300 572b 04b8 0013 b900 2102 0059
33 0000020: 4db6 0025 57b1
34
35
36
37List<String> stringList = Arrays.asList("Test");
38switch(stringList.get(0)) {}
39import java.util.HashMap;
40import java.util.Map;
41
42public class SwitchFail {
43 public static void main(String[] args) {
44 Map<Integer, String> stringMap = new HashMap<>();
45 stringMap.put(1, "Test");
46 switch (stringMap.get(1)) {
47 default:
48 System.out.println("Hello");
49 }
50
51 String plainString = "Test";
52 switch (plainString) {
53 default:
54 System.out.println("Hi");
55 }
56 }
57}
58
A sample run:
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18Error: Unable to initialize main class SwitchFail
19Caused by: java.lang.VerifyError: Bad type on operand stack
20Exception Details:
21 Location:
22 SwitchFail.main([Ljava/lang/String;)V @33: invokevirtual
23 Reason:
24 Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/lang/String'
25 Current Frame:
26 bci: @33
27 flags: { }
28 locals: { '[Ljava/lang/String;', 'java/util/HashMap', 'java/lang/Object' }
29 stack: { 'java/lang/Object' }
30 Bytecode:
31 0000000: bb00 1059 b700 124c 2b04 b800 1312 19b9
32 0000010: 001b 0300 572b 04b8 0013 b900 2102 0059
33 0000020: 4db6 0025 57b1
34
35
36
37List<String> stringList = Arrays.asList("Test");
38switch(stringList.get(0)) {}
39import java.util.HashMap;
40import java.util.Map;
41
42public class SwitchFail {
43 public static void main(String[] args) {
44 Map<Integer, String> stringMap = new HashMap<>();
45 stringMap.put(1, "Test");
46 switch (stringMap.get(1)) {
47 default:
48 System.out.println("Hello");
49 }
50
51 String plainString = "Test";
52 switch (plainString) {
53 default:
54 System.out.println("Hi");
55 }
56 }
57}
58[~/Desktop]: java SwitchFail.java
59Hello
60Hi
61
Test with jar:
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18Error: Unable to initialize main class SwitchFail
19Caused by: java.lang.VerifyError: Bad type on operand stack
20Exception Details:
21 Location:
22 SwitchFail.main([Ljava/lang/String;)V @33: invokevirtual
23 Reason:
24 Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/lang/String'
25 Current Frame:
26 bci: @33
27 flags: { }
28 locals: { '[Ljava/lang/String;', 'java/util/HashMap', 'java/lang/Object' }
29 stack: { 'java/lang/Object' }
30 Bytecode:
31 0000000: bb00 1059 b700 124c 2b04 b800 1312 19b9
32 0000010: 001b 0300 572b 04b8 0013 b900 2102 0059
33 0000020: 4db6 0025 57b1
34
35
36
37List<String> stringList = Arrays.asList("Test");
38switch(stringList.get(0)) {}
39import java.util.HashMap;
40import java.util.Map;
41
42public class SwitchFail {
43 public static void main(String[] args) {
44 Map<Integer, String> stringMap = new HashMap<>();
45 stringMap.put(1, "Test");
46 switch (stringMap.get(1)) {
47 default:
48 System.out.println("Hello");
49 }
50
51 String plainString = "Test";
52 switch (plainString) {
53 default:
54 System.out.println("Hi");
55 }
56 }
57}
58[~/Desktop]: java SwitchFail.java
59Hello
60Hi
61[~/Desktop/java17]: javac SwitchFail.java
62[~/Desktop/java17]: jar -cvf java17test.jar .
63added manifest
64adding: SwitchFail.java(in = 379) (out= 212)(deflated 44%)
65adding: SwitchFail.class(in = 920) (out= 546)(deflated 40%)
66adding: .DS_Store(in = 6148) (out= 178)(deflated 97%)
67[~/Desktop/java17]: java -cp java17test.jar SwitchFail
68Hello
69Hi
70
The JDK on my system:
1import java.util.HashMap;
2import java.util.Map;
3
4public class SwitchFail {
5 public static void main(String[] args) {
6 //doesnt work
7 Map<Integer, String> stringMap = new HashMap<>();
8 stringMap.put(1, "Test");
9 switch(stringMap.get(1)) {
10 }
11
12 //works
13 String plainString = "Test";
14 switch(plainString) {
15 }
16 }
17}
18Error: Unable to initialize main class SwitchFail
19Caused by: java.lang.VerifyError: Bad type on operand stack
20Exception Details:
21 Location:
22 SwitchFail.main([Ljava/lang/String;)V @33: invokevirtual
23 Reason:
24 Type 'java/lang/Object' (current frame, stack[0]) is not assignable to 'java/lang/String'
25 Current Frame:
26 bci: @33
27 flags: { }
28 locals: { '[Ljava/lang/String;', 'java/util/HashMap', 'java/lang/Object' }
29 stack: { 'java/lang/Object' }
30 Bytecode:
31 0000000: bb00 1059 b700 124c 2b04 b800 1312 19b9
32 0000010: 001b 0300 572b 04b8 0013 b900 2102 0059
33 0000020: 4db6 0025 57b1
34
35
36
37List<String> stringList = Arrays.asList("Test");
38switch(stringList.get(0)) {}
39import java.util.HashMap;
40import java.util.Map;
41
42public class SwitchFail {
43 public static void main(String[] args) {
44 Map<Integer, String> stringMap = new HashMap<>();
45 stringMap.put(1, "Test");
46 switch (stringMap.get(1)) {
47 default:
48 System.out.println("Hello");
49 }
50
51 String plainString = "Test";
52 switch (plainString) {
53 default:
54 System.out.println("Hi");
55 }
56 }
57}
58[~/Desktop]: java SwitchFail.java
59Hello
60Hi
61[~/Desktop/java17]: javac SwitchFail.java
62[~/Desktop/java17]: jar -cvf java17test.jar .
63added manifest
64adding: SwitchFail.java(in = 379) (out= 212)(deflated 44%)
65adding: SwitchFail.class(in = 920) (out= 546)(deflated 40%)
66adding: .DS_Store(in = 6148) (out= 178)(deflated 97%)
67[~/Desktop/java17]: java -cp java17test.jar SwitchFail
68Hello
69Hi
70[~/Desktop/java17]: java -version
71openjdk version "17" 2021-09-14
72OpenJDK Runtime Environment (build 17+35-2724)
73OpenJDK 64-Bit Server VM (build 17+35-2724, mixed mode, sharing)
74
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Bytecode
Tutorials and Learning Resources are not available at this moment for Bytecode