Explore all Oracle open source software, libraries, packages, source code, cloud functions and APIs.
Popular New Releases in Oracle
dbeaver
22.0.3
mybatis-3
mybatis-3.5.9
Mycat-Server
Mycat-server-1.6.7.6-test
orm
2.12.1
dbal
3.3.3
Popular Libraries in Oracle
dbeaver
by dbeaver 
java
26064 Apache-2.0
Free universal database tool and SQL client
mybatis-3
by mybatis java
17074 NOASSERTION
MyBatis SQL mapper framework for Java
DataX
by alibaba java
10885 NOASSERTION
DataX是阿里云DataWorks数据集成的开源版本。
Mycat-Server
by MyCATApache java
9250 GPL-2.0
orm
by doctrine php
9195 MIT
Doctrine Object Relational Mapper (ORM)
dbal
by doctrine php
8582 MIT
Doctrine Database Abstraction Layer
usql
by xo go
6967 MIT
Universal command-line interface for SQL databases
elasticsearch-sql
by NLPchina java
6622 Apache-2.0
Use SQL to query Elasticsearch
xorm
by go-xorm go
6365 BSD-3-Clause
Simple and Powerful ORM for Go, support mysql,postgres,tidb,sqlite3,mssql,oracle, Moved to https://gitea.com/xorm/xorm
Trending New libraries in Oracle
kiss-orm
by Seb-C typescript
447 MIT
An extremely simple and explicit ORM for TypeScript
sqle
by actiontech go
412 MPL-2.0
SQLE is a SQL audit platform | SQLE 是一个支持多场景，原生支持 MySQL 审核且数据库类型可扩展的 SQL 审核工具
relly
by KOBA789 rust
389 MIT
RDBMS のしくみを学ぶための小さな RDBMS 実装
Oracle-warp
by YG-tsj shell
253 MIT
Oracle甲骨文云（支持ARM/X86的KVM架构）纯IPV4/纯IPV6，IPV4+IPV6 无损双栈WARP多功能组合脚本：开启端口脚本，升级内核脚本，root密码脚本，bbr加速脚本，奈飞检测脚本，DNS统一脚本，xray脚本等…功能增加中！！
CG-SQL
by facebookincubator html
243 MIT
CG/SQL is a compiler that converts a SQL Stored Procedure like language into C for SQLite. SQLite has no stored procedures of its own. CG/CQL can also generate other useful artifacts for testing and schema maintenance.
WOA-Drivers
by edk2-porting shell
214
Windows Drivers for Qualcomm Snapdragon platforms
centos2ol
by oracle shell
210 UPL-1.0
Script and documentation to switch CentOS Linux to Oracle Linux
dungeons-and-dragons-nft
by PatrickAlphaC javascript
184 MIT
#chainlink #nft
modmap
by btbd c
162
Module extending manual mapper
Top Authors in Oracle
1
49 Libraries
11848
2
17 Libraries
213
3
7 Libraries
20
4
6 Libraries
44
5
6 Libraries
189
6
5 Libraries
357
7
5 Libraries
290
8
5 Libraries
988
9
5 Libraries
68
10
5 Libraries
22225
1
49 Libraries
11848
2
17 Libraries
213
3
7 Libraries
20
4
6 Libraries
44
5
6 Libraries
189
6
5 Libraries
357
7
5 Libraries
290
8
5 Libraries
988
9
5 Libraries
68
10
5 Libraries
22225
Trending Kits in Oracle
No Trending Kits are available at this moment for Oracle
Trending Discussions on Oracle
Oracle 12c docker setup on Apple M1
Lambda expressions and anonymous classes don't work when loaded as hidden classes
How to configure proxy in emulators in new versions of Android Studio?
IconData takes a long time to load and does not show icons
What is 'serviceability memory category' of Native Memory Tracking?
Criteria for default garbage collector Hotspot JVM 11/17
Do parallel streams treat upstream iterators in a thread safe way?
Is it legal to use `record` as a variable name even though it's a keyword?
What is the Java grammar that allows "new int[] {0}[0] = 1;" to compile?
Oracle SQL or PL/SQL: How to identify candlestick pattern only in end of uptrend or downtrend and set a flag in column?
QUESTION
Oracle 12c docker setup on Apple M1
Asked 2022-Mar-21 at 09:52I am trying to run Oracle db in docker on M1 Mac. I have tried images from both store/oracle/database-enterprise:12.2.0.1-slim and container-registry.oracle.com/database/enterprise:12.2.0.1-slim but getting the same error.
docker run -d -it --name oracle -v $(pwd)/db/oradata:/ORCL store/oracle/database-enterprise:12.2.0.1-slim
I also tried non-slim version and by providing the --platform linux/amd64 to the docker command. Result is same.
Here's the result of docker logs -f oracle
1Setup Oracle Database
2Oracle Database 12.2.0.1 Setup
3Sat Jul 31 18:44:14 UTC 2021
4
5Check parameters ......
6log file is : /home/oracle/setup/log/paramChk.log
7paramChk.sh is done at 1 sec
8
9untar DB bits ......
10log file is : /home/oracle/setup/log/untarDB.log
11untarDB.sh is done at 29 sec
12
13config DB ......
14log file is : /home/oracle/setup/log/configDB.log
15Sat Jul 31 18:44:42 UTC 2021
16Start Docker DB configuration
17Call configDBora.sh to configure database
18Sat Jul 31 18:44:42 UTC 2021
19Configure DB as oracle user
20Setup Database directories ...
21
22SQL*Plus: Release 12.2.0.1.0 Production on Sat Jul 31 18:44:43 2021
23
24Copyright (c) 1982, 2016, Oracle. All rights reserved.
25
26ERROR:
27ORA-12547: TNS:lost contact
28
29
30Enter user-name: SP2-0306: Invalid option.
31Usage: CONN[ECT] [{logon|/|proxy} [AS {SYSDBA|SYSOPER|SYSASM|SYSBACKUP|SYSDG|SYSKM|SYSRAC}] [edition=value]]
32where <logon> ::= <username>[/<password>][@<connect_identifier>]
33 <proxy> ::= <proxyuser>[<username>][/<password>][@<connect_identifier>]
34Enter user-name: Enter password:
35ERROR:
36ORA-12547: TNS:lost contact
37
38
39SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus
40update password
41
42Enter password for SYS:
43create pdb : ORCLPDB1
44
45SQL*Plus: Release 12.2.0.1.0 Production on Sat Jul 31 18:44:43 2021
46
47Copyright (c) 1982, 2016, Oracle. All rights reserved.
48
49ERROR:
50ORA-12547: TNS:lost contact
51
52
53Enter user-name: SP2-0306: Invalid option.
54Usage: CONN[ECT] [{logon|/|proxy} [AS {SYSDBA|SYSOPER|SYSASM|SYSBACKUP|SYSDG|SYSKM|SYSRAC}] [edition=value]]
55where <logon> ::= <username>[/<password>][@<connect_identifier>]
56 <proxy> ::= <proxyuser>[<username>][/<password>][@<connect_identifier>]
57Enter user-name: SP2-0306: Invalid option.
58Usage: CONN[ECT] [{logon|/|proxy} [AS {SYSDBA|SYSOPER|SYSASM|SYSBACKUP|SYSDG|SYSKM|SYSRAC}] [edition=value]]
59where <logon> ::= <username>[/<password>][@<connect_identifier>]
60 <proxy> ::= <proxyuser>[<username>][/<password>][@<connect_identifier>]
61SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus
62Reset Database parameters
63
64SQL*Plus: Release 12.2.0.1.0 Production on Sat Jul 31 18:44:43 2021
65
66Copyright (c) 1982, 2016, Oracle. All rights reserved.
67
68ERROR:
69ORA-12547: TNS:lost contact
70
71
72Enter user-name: SP2-0306: Invalid option.
73Usage: CONN[ECT] [{logon|/|proxy} [AS {SYSDBA|SYSOPER|SYSASM|SYSBACKUP|SYSDG|SYSKM|SYSRAC}] [edition=value]]
74where <logon> ::= <username>[/<password>][@<connect_identifier>]
75 <proxy> ::= <proxyuser>[<username>][/<password>][@<connect_identifier>]
76Enter user-name: Enter password:
77ERROR:
78ORA-12547: TNS:lost contact
79
80
81SP2-0157: unable to CONNECT to ORACLE after 3 attempts, exiting SQL*Plus
82
83LSNRCTL for Linux: Version 12.2.0.1.0 - Production on 31-JUL-2021 18:44:44
84
85Copyright (c) 1991, 2016, Oracle. All rights reserved.
86
87Starting /u01/app/oracle/product/12.2.0/dbhome_1/bin/tnslsnr: please wait...
88
89TNSLSNR for Linux: Version 12.2.0.1.0 - Production
90System parameter file is /u01/app/oracle/product/12.2.0/dbhome_1/admin/ORCLCDB/listener.ora
91Log messages written to /u01/app/oracle/diag/tnslsnr/989db909d8df/listener/alert/log.xml
92Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
93Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
94
95Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=0.0.0.0)(PORT=1521)))
96STATUS of the LISTENER
97------------------------
98Alias LISTENER
99Version TNSLSNR for Linux: Version 12.2.0.1.0 - Production
100Start Date 31-JUL-2021 18:44:44
101Uptime 0 days 0 hr. 0 min. 0 sec
102Trace Level off
103Security ON: Local OS Authentication
104SNMP OFF
105Listener Parameter File /u01/app/oracle/product/12.2.0/dbhome_1/admin/ORCLCDB/listener.ora
106Listener Log File /u01/app/oracle/diag/tnslsnr/989db909d8df/listener/alert/log.xml
107Listening Endpoints Summary...
108 (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=0.0.0.0)(PORT=1521)))
109 (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=EXTPROC1521)))
110The listener supports no services
111The command completed successfully
112
113DONE!
114Remove password info
115Docker DB configuration is complete !
116ERROR : config DB failed, please check log /home/oracle/setup/log/configDB.log for details!
117OS process OFSD (ospid 188) idle for 30 seconds, exiting
1182017-08-09T10:36:20.752530+00:00
119ARCH: Archival disabled due to shutdown: 1090
120Shutting down archive processes
121Archiving is disabled
122JIT: pid 563 requesting stop
1232017-08-09T10:36:20.815635+00:00
124Stopping background process VKTM
1252017-08-09T10:36:26.581058+00:00
126Instance shutdown complete (OS id: 563)
127/usr/bin/tail: inotify cannot be used, reverting to polling: Function not implemented
128Is it the issue with M1 or I my configuration
ANSWER
Answered 2021-Aug-04 at 20:48There are two issues here:
- Oracle Database is not supported on ARM processors, only Intel. See here: https://github.com/oracle/docker-images/issues/1814
- Oracle Database Docker images are only supported with Oracle Linux 7 or Red Hat Enterprise Linux 7 as the host OS. See here: https://github.com/oracle/docker-images/tree/main/OracleDatabase/SingleInstance
Oracle Database ... is supported for Oracle Linux 7 and Red Hat Enterprise Linux (RHEL) 7. For more details please see My Oracle Support note: Oracle Support for Database Running on Docker (Doc ID 2216342.1)
The referenced My Oracle Support Doc ID goes on to say that the database binaries in their Docker image are built specifically for Oracle Linux hosts, and will also work on Red Hat. That's it.
Because Docker provides process level virtualization it still pulls kernel and other OS libraries from the underlying host OS. A Docker image built for Oracle Linux needs an Oracle Linux host; it doesn't bring the Oracle Linux OS with it. Only Oracle Linux or Red Hat Linux are supported for any Oracle database Linux installation, with or without Docker. Ubuntu, Mac OS, Debian, or any other *NIX flavor will not provide predictable reliable results, even if it is hacked into working or the processes appear to work normally.
QUESTION
Lambda expressions and anonymous classes don't work when loaded as hidden classes
Asked 2022-Feb-26 at 05:14I am trying to compile and load dynamically generated Java code during runtime. Since both ClassLoader::defineClass and Unsafe::defineAnonymousClass have serious drawbacks in this scenario, I tried using hidden classes via Lookup::defineHiddenClass instead. This works fine for all classes that I tried to load, except for those that call lambda expressions or contain anonymous classes.
Calling a lambda expression throws the following exception:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Executing code that instantiates an anonymous class throws the following error:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29This is a short example that recreates the problem:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50I've already tried compiling and running the code with different JDKs, using different ways to create new instances of the hidden class, searching for bugs at https://bugs.openjdk.java.net/, messing with the bytecode itself and several other things. I am not an expert on Java internals, so I am not sure whether I have not understood the JEP that introduced hidden classes correctly.
Am I doing something wrong, is this just impossible or is this a bug?
Edit: The JEP states
Migration should take the following into account: To invoke private nestmate instance methods from code in a hidden class, use invokevirtual or invokeinterface instead of invokespecial. Generated bytecode that uses invokespecial to invoke a private nestmate instance method will fail verification. invokespecial should only be used to invoke private nestmate constructors.
This might be the problem for the anonymous class. Is there a way to compile the code such that invokespecial is avoided in the bytecode?
ANSWER
Answered 2022-Feb-23 at 18:19You can not turn arbitrary classes into hidden classes.
The documentation of defineHiddenClass contains the sentence
- On any attempt to resolve the entry in the run-time constant pool indicated by
this_class, the symbolic reference is considered to be resolved toCand resolution always succeeds immediately.
What it doesn’t spell out explicitly is that this is the only place where a type resolution ever ends up at the hidden class.
But it has been said unambiguously in bug report JDK-8222730:
For a hidden class, its specified hidden name should only be accessible through the hidden class's 'this_class' constant pool entry.
The class should not be accessible by specifying its original name in, for example, a method or field signature even within the hidden class.
Which we can check. Even a simple case like
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71will already fail. Note that it is a special case that the attempt to resolve the original class name LambdaRunner within the hidden class will not fail, as you used an existing class as template. So you get an IncompatibleClassChangeError or a VerifierError due to mismatches between the hidden class and the existing LambdaRunner class. When you don’t use a class definition of an existing class, you’d get a NoClassDefFoundError.
The same applies to
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80As the cited bug report said, neither field nor methods can refer to the hidden class in their signature.
A less intuitive example is
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86which will fail depending on the compiler and options, as when the StringConcatFactory is used, the behavior is like an invocation of a method having all non-constant parts as parameters and returning a String. So this is another case of having the hidden class in a method signature.
Lambda expressions are special, as a class like
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93gets compiled similar to
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93 static class LambdaRunner implements Runnable {
94 @Override
95 public void run() {
96 Runnable runnable = LambdaRunner::lambdaBody;
97 runnable.run();
98 }
99 private static void lambdaBody() {
100 System.out.println("Success");
101 }
102 }
103which doesn’t have the hidden class in the method signature, but has to refer to the method holding the body of the lambda expression as a MethodReference. Within the constant pool, the description of this method refers to its declaring class using the this_class entry. So it gets redirected to the hidden class as described in the documentation.
But the construction of the MethodType as part of the MethodReference does not use this information to load a Class like a class literal would do. Instead, it tries to load the hidden class through the defining class loader, which fails with the NoClassDefFoundError you have posted.
This seems to be related to JDK-8130087 which suggests that ordinary method resolution differs from the way, MethodType works, which can make MethodType fail where just invoking the method would work.
But it’s possible to demonstrate that even fixing this issue wouldn’t solve the general problem:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93 static class LambdaRunner implements Runnable {
94 @Override
95 public void run() {
96 Runnable runnable = LambdaRunner::lambdaBody;
97 runnable.run();
98 }
99 private static void lambdaBody() {
100 System.out.println("Success");
101 }
102 }
103 static class LambdaRunner implements Runnable {
104 @Override
105 public void run() {
106 var lookup = MethodHandles.lookup();
107 var noArgVoid = MethodType.methodType(void.class);
108 try {
109 MethodHandle mh = LambdaMetafactory.metafactory(lookup, "run",
110 MethodType.methodType(Runnable.class), noArgVoid,
111 lookup.findStatic(LambdaRunner.class, "lambdaBody", noArgVoid),
112 noArgVoid).getTarget();
113 System.out.println("got factory");
114 Runnable runnable = (Runnable)mh.invokeExact();
115 System.out.println("got runnable");
116 runnable.run();
117 }
118 catch(RuntimeException|Error e) {
119 throw e;
120 }
121 catch(Throwable e) {
122 throw new AssertionError(e);
123 }
124 }
125 private static void lambdaBody() {
126 System.out.println("Success");
127 }
128 }
129This bypasses the problem described above and calls the LambdaMetafactory manually. When being redefined as hidden class, it will print:
1Exception in thread "main" java.lang.NoClassDefFoundError: tests/HiddenClassLambdaTest$LambdaRunner/0x0000000800c04400
2 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:22)
3Caused by: java.lang.ClassNotFoundException: tests.HiddenClassLambdaTest$LambdaRunner.0x0000000800c04400
4 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:636)
5 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:182)
6 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:519)
7 ... 1 more
8Exception in thread "main" java.lang.VerifyError: Bad type on operand stack
9Exception Details:
10 Location:
11 tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400.run()V @5: invokespecial
12 Reason:
13 Type 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' (current frame, stack[2]) is not assignable to 'tests/HiddenClassLambdaTest$LambdaRunner'
14 Current Frame:
15 bci: @5
16 flags: { }
17 locals: { 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
18 stack: { uninitialized 0, uninitialized 0, 'tests/HiddenClassLambdaTest$LambdaRunner+0x0000000800c00400' }
19 Bytecode:
20 0000000: bb00 1159 2ab7 0013 4cb1
21
22 at java.base/java.lang.ClassLoader.defineClass0(Native Method)
23 at java.base/java.lang.System$2.defineClass(System.java:2193)
24 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClass(MethodHandles.java:2446)
25 at java.base/java.lang.invoke.MethodHandles$Lookup$ClassDefiner.defineClassAsLookup(MethodHandles.java:2427)
26 at java.base/java.lang.invoke.MethodHandles$Lookup.defineHiddenClass(MethodHandles.java:2133)
27 at tests.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:25)
28
29import java.lang.invoke.MethodHandles;
30
31public class HiddenClassLambdaTest {
32 /** This class is to be loaded and executed as hidden class */
33 public static final class LambdaRunner implements Runnable {
34 @Override public void run() {
35 Runnable runnable = () -> System.out.println("Success");
36 runnable.run();
37 }
38 }
39
40 public static void main(String[] args) throws Throwable {
41 // Path to the class file of the nested class defined above
42 String nestedClassPath = HiddenClassLambdaTest.class.getTypeName().replace('.','/') + "$LambdaRunner.class";
43 // Class file content of the LambdaRunner class
44 byte[] classFileContents = HiddenClassLambdaTest.class.getClassLoader().getResourceAsStream(nestedClassPath).readAllBytes();
45 Class<?> lambdaRunnerClass = MethodHandles.lookup().defineHiddenClass(classFileContents, true).lookupClass();
46 Runnable lambdaRunnerInstance = (Runnable) lambdaRunnerClass.getConstructor().newInstance();
47 lambdaRunnerInstance.run();
48 }
49}
50public class HiddenClassLambdaTest {
51
52 public static void main(String[] args) throws Throwable {
53 byte[] classFileContents = HiddenClassLambdaTest.class
54 .getResourceAsStream("HiddenClassLambdaTest$LambdaRunner.class")
55 .readAllBytes();
56 var hidden = MethodHandles.lookup()
57 .defineHiddenClass(classFileContents, true, ClassOption.NESTMATE);
58 Runnable lambdaRunnerInstance = (Runnable)hidden.findConstructor(
59 hidden.lookupClass(), MethodType.methodType(void.class)).invoke();
60 lambdaRunnerInstance.run();
61 }
62
63 static class LambdaRunner implements Runnable {
64 LambdaRunner field = this;
65
66 @Override
67 public void run() {
68 }
69 }
70}
71 static class LambdaRunner implements Runnable {
72 static void method(LambdaRunner arg) {
73 }
74
75 @Override
76 public void run() {
77 method(this);
78 }
79 }
80 static class LambdaRunner implements Runnable {
81 @Override
82 public void run() {
83 System.out.println("" + this);
84 }
85 }
86 static class LambdaRunner implements Runnable {
87 @Override
88 public void run() {
89 Runnable runnable = () -> System.out.println("Success");
90 runnable.run();
91 }
92 }
93 static class LambdaRunner implements Runnable {
94 @Override
95 public void run() {
96 Runnable runnable = LambdaRunner::lambdaBody;
97 runnable.run();
98 }
99 private static void lambdaBody() {
100 System.out.println("Success");
101 }
102 }
103 static class LambdaRunner implements Runnable {
104 @Override
105 public void run() {
106 var lookup = MethodHandles.lookup();
107 var noArgVoid = MethodType.methodType(void.class);
108 try {
109 MethodHandle mh = LambdaMetafactory.metafactory(lookup, "run",
110 MethodType.methodType(Runnable.class), noArgVoid,
111 lookup.findStatic(LambdaRunner.class, "lambdaBody", noArgVoid),
112 noArgVoid).getTarget();
113 System.out.println("got factory");
114 Runnable runnable = (Runnable)mh.invokeExact();
115 System.out.println("got runnable");
116 runnable.run();
117 }
118 catch(RuntimeException|Error e) {
119 throw e;
120 }
121 catch(Throwable e) {
122 throw new AssertionError(e);
123 }
124 }
125 private static void lambdaBody() {
126 System.out.println("Success");
127 }
128 }
129got factory
130got runnable
131Exception in thread "main" java.lang.NoClassDefFoundError: test/HiddenClassLambdaTest$LambdaRunner/0x0000000800c01400
132 at test/test.HiddenClassLambdaTest.main(HiddenClassLambdaTest.java:15)
133Caused by: java.lang.ClassNotFoundException: test.HiddenClassLambdaTest$LambdaRunner.0x0000000800c01400
134 at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
135 at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
136 at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
137 ... 1 more
138which shows that all obstacles have been circumvented, but when it comes to the actual invocation from the generated Runnable to the method holding the lambda body, it will fail due to the fact that the target class is hidden. A JVM with eager resolution of symbolic references might fail earlier, i.e. the example might not print got runnable then.
Unlike the old JVM anonymous classes, there is no way to link to a hidden class, not even from another hidden class.
The bottom line is, as said at the beginning, you can not turn arbitrary classes into hidden classes. Lambda expressions are not the only feature not working with hidden classes. It’s not a good idea to try and get surprised. Hidden classes should only be used in conjunction with bytecode generators carefully using only features known to work.
QUESTION
How to configure proxy in emulators in new versions of Android Studio?
Asked 2022-Feb-23 at 14:14Problem Description
I need to configure the proxy manually in my emulator through Android Studio. From the official Android documentation, it is suggested that this change can be made in the "settings" tab of the emulator's extended controls. The problem is that it seems to me that this documentation is outdated, as this setting is no longer displayed in the "settings" tab of the Android Studio emulators' extended controls.
Documentation
1Android Studio Bumblebee | 2021.1.1
2Build #AI-211.7628.21.2111.8092744, built on January 19, 2022
3Runtime version: 11.0.11+9-b60-7590822 amd64
4VM: OpenJDK 64-Bit Server VM by Oracle Corporation
5Windows 10 10.0
6GC: G1 Young Generation, G1 Old Generation
7Memory: 1280M
8Cores: 8
9Registry: external.system.auto.import.disabled=true
10Non-Bundled Plugins: com.wakatime.intellij.plugin (13.1.10), wu.seal.tool.jsontokotlin (3.7.2), org.jetbrains.kotlin (211-1.6.10-release-923-AS7442.40), com.developerphil.adbidea (1.6.4), org.jetbrains.compose.desktop.ide (1.0.0), ru.adelf.idea.dotenv (2021.2), org.intellij.plugins.markdown (211.7142.37)
11ANSWER
Answered 2022-Feb-17 at 19:12After a while trying to find solutions to this problem, I saw that an emulator running outside android studio provides these options. To run a standalone Android Studio emulator see the official documentation or simply enter the command:
1Android Studio Bumblebee | 2021.1.1
2Build #AI-211.7628.21.2111.8092744, built on January 19, 2022
3Runtime version: 11.0.11+9-b60-7590822 amd64
4VM: OpenJDK 64-Bit Server VM by Oracle Corporation
5Windows 10 10.0
6GC: G1 Young Generation, G1 Old Generation
7Memory: 1280M
8Cores: 8
9Registry: external.system.auto.import.disabled=true
10Non-Bundled Plugins: com.wakatime.intellij.plugin (13.1.10), wu.seal.tool.jsontokotlin (3.7.2), org.jetbrains.kotlin (211-1.6.10-release-923-AS7442.40), com.developerphil.adbidea (1.6.4), org.jetbrains.compose.desktop.ide (1.0.0), ru.adelf.idea.dotenv (2021.2), org.intellij.plugins.markdown (211.7142.37)
11emulator -avd <avd_name>
12In my case I'm using an avd named PIXEL 4 API 30, so the command will be emulator -avd PIXEL_4_API_30. If you are on Windows you may have problems running this command so I suggest you see this.
The solution proposed by @Inliner also solves this problem.
QUESTION
IconData takes a long time to load and does not show icons
Asked 2022-Jan-24 at 04:27The list with icons (IconData) takes a very long time to load, about a minute. Previously, there was no such problem - the list was loaded in a couple of seconds. I tried reinstalling Flutter and Android Studio on an SSD, but it didn't help.
How can this be fixed?
1Android Studio Arctic Fox | 2020.3.1 Patch 3
2Build #AI-203.7717.56.2031.7784292, built on October 1, 2021
3Runtime version: 11.0.10+0-b96-7249189 amd64
4VM: OpenJDK 64-Bit Server VM by Oracle Corporation
5Windows 10 10.0
6GC: G1 Young Generation, G1 Old Generation
7Memory: 1280M
8Cores: 16
9Registry: external.system.auto.import.disabled=true
10Non-Bundled Plugins: com.intellij.marketplace, Dart, com.thoughtworks.gauge, org.jetbrains.kotlin, io.flutter, org.intellij.plugins.markdown
11
12Dart: 203.8452
13Flutter: 62.0.1
14ANSWER
Answered 2021-Dec-23 at 10:33The developers have fixed this bug in one of the recent updates.
QUESTION
What is 'serviceability memory category' of Native Memory Tracking?
Asked 2022-Jan-17 at 13:38I have an java app (JDK13) running in a docker container. Recently I moved the app to JDK17 (OpenJDK17) and found a gradual increase of memory usage by docker container.
During investigation I found that the 'serviceability memory category' NMT grows constantly (15mb per an hour). I checked the page https://docs.oracle.com/en/java/javase/17/troubleshoot/diagnostic-tools.html#GUID-5EF7BB07-C903-4EBD-A9C2-EC0E44048D37 but this category is not mentioned there.
Could anyone explain what this serviceability category means and what can cause such gradual increase? Also there are some additional new memory categories comparing to JDK13. Maybe someone knows where I can read details about them.
Here is the result of command jcmd 1 VM.native_memory summary
1Native Memory Tracking:
2
3(Omitting categories weighting less than 1KB)
4
5Total: reserved=4431401KB, committed=1191617KB
6- Java Heap (reserved=2097152KB, committed=479232KB)
7 (mmap: reserved=2097152KB, committed=479232KB)
8
9- Class (reserved=1052227KB, committed=22403KB)
10 (classes #29547)
11 ( instance classes #27790, array classes #1757)
12 (malloc=3651KB #79345)
13 (mmap: reserved=1048576KB, committed=18752KB)
14 ( Metadata: )
15 ( reserved=139264KB, committed=130816KB)
16 ( used=130309KB)
17 ( waste=507KB =0.39%)
18 ( Class space:)
19 ( reserved=1048576KB, committed=18752KB)
20 ( used=18149KB)
21 ( waste=603KB =3.21%)
22
23- Thread (reserved=387638KB, committed=40694KB)
24 (thread #378)
25 (stack: reserved=386548KB, committed=39604KB)
26 (malloc=650KB #2271)
27 (arena=440KB #752)
28
29- Code (reserved=253202KB, committed=76734KB)
30 (malloc=5518KB #23715)
31 (mmap: reserved=247684KB, committed=71216KB)
32
33- GC (reserved=152419KB, committed=92391KB)
34 (malloc=40783KB #34817)
35 (mmap: reserved=111636KB, committed=51608KB)
36
37- Compiler (reserved=1506KB, committed=1506KB)
38 (malloc=1342KB #2557)
39 (arena=165KB #5)
40
41- Internal (reserved=5579KB, committed=5579KB)
42 (malloc=5543KB #33822)
43 (mmap: reserved=36KB, committed=36KB)
44
45- Other (reserved=231161KB, committed=231161KB)
46 (malloc=231161KB #347)
47
48- Symbol (reserved=30558KB, committed=30558KB)
49 (malloc=28887KB #769230)
50 (arena=1670KB #1)
51
52- Native Memory Tracking (reserved=16412KB, committed=16412KB)
53 (malloc=575KB #8281)
54 (tracking overhead=15837KB)
55
56- Shared class space (reserved=12288KB, committed=12136KB)
57 (mmap: reserved=12288KB, committed=12136KB)
58
59- Arena Chunk (reserved=18743KB, committed=18743KB)
60 (malloc=18743KB)
61
62- Tracing (reserved=32KB, committed=32KB)
63 (arena=32KB #1)
64
65- Logging (reserved=7KB, committed=7KB)
66 (malloc=7KB #289)
67
68- Arguments (reserved=1KB, committed=1KB)
69 (malloc=1KB #53)
70
71- Module (reserved=1045KB, committed=1045KB)
72 (malloc=1045KB #5026)
73
74- Safepoint (reserved=8KB, committed=8KB)
75 (mmap: reserved=8KB, committed=8KB)
76
77- Synchronization (reserved=204KB, committed=204KB)
78 (malloc=204KB #2026)
79
80- Serviceability (reserved=31187KB, committed=31187KB)
81 (malloc=31187KB #49714)
82
83- Metaspace (reserved=140032KB, committed=131584KB)
84 (malloc=768KB #622)
85 (mmap: reserved=139264KB, committed=130816KB)
86
87- String Deduplication (reserved=1KB, committed=1KB)
88 (malloc=1KB #8)
89The detailed information about increasing part of memory is:
1Native Memory Tracking:
2
3(Omitting categories weighting less than 1KB)
4
5Total: reserved=4431401KB, committed=1191617KB
6- Java Heap (reserved=2097152KB, committed=479232KB)
7 (mmap: reserved=2097152KB, committed=479232KB)
8
9- Class (reserved=1052227KB, committed=22403KB)
10 (classes #29547)
11 ( instance classes #27790, array classes #1757)
12 (malloc=3651KB #79345)
13 (mmap: reserved=1048576KB, committed=18752KB)
14 ( Metadata: )
15 ( reserved=139264KB, committed=130816KB)
16 ( used=130309KB)
17 ( waste=507KB =0.39%)
18 ( Class space:)
19 ( reserved=1048576KB, committed=18752KB)
20 ( used=18149KB)
21 ( waste=603KB =3.21%)
22
23- Thread (reserved=387638KB, committed=40694KB)
24 (thread #378)
25 (stack: reserved=386548KB, committed=39604KB)
26 (malloc=650KB #2271)
27 (arena=440KB #752)
28
29- Code (reserved=253202KB, committed=76734KB)
30 (malloc=5518KB #23715)
31 (mmap: reserved=247684KB, committed=71216KB)
32
33- GC (reserved=152419KB, committed=92391KB)
34 (malloc=40783KB #34817)
35 (mmap: reserved=111636KB, committed=51608KB)
36
37- Compiler (reserved=1506KB, committed=1506KB)
38 (malloc=1342KB #2557)
39 (arena=165KB #5)
40
41- Internal (reserved=5579KB, committed=5579KB)
42 (malloc=5543KB #33822)
43 (mmap: reserved=36KB, committed=36KB)
44
45- Other (reserved=231161KB, committed=231161KB)
46 (malloc=231161KB #347)
47
48- Symbol (reserved=30558KB, committed=30558KB)
49 (malloc=28887KB #769230)
50 (arena=1670KB #1)
51
52- Native Memory Tracking (reserved=16412KB, committed=16412KB)
53 (malloc=575KB #8281)
54 (tracking overhead=15837KB)
55
56- Shared class space (reserved=12288KB, committed=12136KB)
57 (mmap: reserved=12288KB, committed=12136KB)
58
59- Arena Chunk (reserved=18743KB, committed=18743KB)
60 (malloc=18743KB)
61
62- Tracing (reserved=32KB, committed=32KB)
63 (arena=32KB #1)
64
65- Logging (reserved=7KB, committed=7KB)
66 (malloc=7KB #289)
67
68- Arguments (reserved=1KB, committed=1KB)
69 (malloc=1KB #53)
70
71- Module (reserved=1045KB, committed=1045KB)
72 (malloc=1045KB #5026)
73
74- Safepoint (reserved=8KB, committed=8KB)
75 (mmap: reserved=8KB, committed=8KB)
76
77- Synchronization (reserved=204KB, committed=204KB)
78 (malloc=204KB #2026)
79
80- Serviceability (reserved=31187KB, committed=31187KB)
81 (malloc=31187KB #49714)
82
83- Metaspace (reserved=140032KB, committed=131584KB)
84 (malloc=768KB #622)
85 (mmap: reserved=139264KB, committed=130816KB)
86
87- String Deduplication (reserved=1KB, committed=1KB)
88 (malloc=1KB #8)
89[0x00007f6ccb970cbe] OopStorage::try_add_block()+0x2e
90[0x00007f6ccb97132d] OopStorage::allocate()+0x3d
91[0x00007f6ccbb34ee8] StackFrameInfo::StackFrameInfo(javaVFrame*, bool)+0x68
92[0x00007f6ccbb35a64] ThreadStackTrace::dump_stack_at_safepoint(int)+0xe4
93 (malloc=6755KB type=Serviceability #10944)
94Update#1 from 2022-01-17:
Thanks to @Aleksey Shipilev for help! We were able to find a place which causes the issue, is related to many ThreadMXBean#.dumpAllThreads calls. Here is MCVE, Test.java:
Run with:
1Native Memory Tracking:
2
3(Omitting categories weighting less than 1KB)
4
5Total: reserved=4431401KB, committed=1191617KB
6- Java Heap (reserved=2097152KB, committed=479232KB)
7 (mmap: reserved=2097152KB, committed=479232KB)
8
9- Class (reserved=1052227KB, committed=22403KB)
10 (classes #29547)
11 ( instance classes #27790, array classes #1757)
12 (malloc=3651KB #79345)
13 (mmap: reserved=1048576KB, committed=18752KB)
14 ( Metadata: )
15 ( reserved=139264KB, committed=130816KB)
16 ( used=130309KB)
17 ( waste=507KB =0.39%)
18 ( Class space:)
19 ( reserved=1048576KB, committed=18752KB)
20 ( used=18149KB)
21 ( waste=603KB =3.21%)
22
23- Thread (reserved=387638KB, committed=40694KB)
24 (thread #378)
25 (stack: reserved=386548KB, committed=39604KB)
26 (malloc=650KB #2271)
27 (arena=440KB #752)
28
29- Code (reserved=253202KB, committed=76734KB)
30 (malloc=5518KB #23715)
31 (mmap: reserved=247684KB, committed=71216KB)
32
33- GC (reserved=152419KB, committed=92391KB)
34 (malloc=40783KB #34817)
35 (mmap: reserved=111636KB, committed=51608KB)
36
37- Compiler (reserved=1506KB, committed=1506KB)
38 (malloc=1342KB #2557)
39 (arena=165KB #5)
40
41- Internal (reserved=5579KB, committed=5579KB)
42 (malloc=5543KB #33822)
43 (mmap: reserved=36KB, committed=36KB)
44
45- Other (reserved=231161KB, committed=231161KB)
46 (malloc=231161KB #347)
47
48- Symbol (reserved=30558KB, committed=30558KB)
49 (malloc=28887KB #769230)
50 (arena=1670KB #1)
51
52- Native Memory Tracking (reserved=16412KB, committed=16412KB)
53 (malloc=575KB #8281)
54 (tracking overhead=15837KB)
55
56- Shared class space (reserved=12288KB, committed=12136KB)
57 (mmap: reserved=12288KB, committed=12136KB)
58
59- Arena Chunk (reserved=18743KB, committed=18743KB)
60 (malloc=18743KB)
61
62- Tracing (reserved=32KB, committed=32KB)
63 (arena=32KB #1)
64
65- Logging (reserved=7KB, committed=7KB)
66 (malloc=7KB #289)
67
68- Arguments (reserved=1KB, committed=1KB)
69 (malloc=1KB #53)
70
71- Module (reserved=1045KB, committed=1045KB)
72 (malloc=1045KB #5026)
73
74- Safepoint (reserved=8KB, committed=8KB)
75 (mmap: reserved=8KB, committed=8KB)
76
77- Synchronization (reserved=204KB, committed=204KB)
78 (malloc=204KB #2026)
79
80- Serviceability (reserved=31187KB, committed=31187KB)
81 (malloc=31187KB #49714)
82
83- Metaspace (reserved=140032KB, committed=131584KB)
84 (malloc=768KB #622)
85 (mmap: reserved=139264KB, committed=130816KB)
86
87- String Deduplication (reserved=1KB, committed=1KB)
88 (malloc=1KB #8)
89[0x00007f6ccb970cbe] OopStorage::try_add_block()+0x2e
90[0x00007f6ccb97132d] OopStorage::allocate()+0x3d
91[0x00007f6ccbb34ee8] StackFrameInfo::StackFrameInfo(javaVFrame*, bool)+0x68
92[0x00007f6ccbb35a64] ThreadStackTrace::dump_stack_at_safepoint(int)+0xe4
93 (malloc=6755KB type=Serviceability #10944)
94java -Xmx512M -XX:NativeMemoryTracking=detail Test.java
95and check periodically serviceability category in result of
1Native Memory Tracking:
2
3(Omitting categories weighting less than 1KB)
4
5Total: reserved=4431401KB, committed=1191617KB
6- Java Heap (reserved=2097152KB, committed=479232KB)
7 (mmap: reserved=2097152KB, committed=479232KB)
8
9- Class (reserved=1052227KB, committed=22403KB)
10 (classes #29547)
11 ( instance classes #27790, array classes #1757)
12 (malloc=3651KB #79345)
13 (mmap: reserved=1048576KB, committed=18752KB)
14 ( Metadata: )
15 ( reserved=139264KB, committed=130816KB)
16 ( used=130309KB)
17 ( waste=507KB =0.39%)
18 ( Class space:)
19 ( reserved=1048576KB, committed=18752KB)
20 ( used=18149KB)
21 ( waste=603KB =3.21%)
22
23- Thread (reserved=387638KB, committed=40694KB)
24 (thread #378)
25 (stack: reserved=386548KB, committed=39604KB)
26 (malloc=650KB #2271)
27 (arena=440KB #752)
28
29- Code (reserved=253202KB, committed=76734KB)
30 (malloc=5518KB #23715)
31 (mmap: reserved=247684KB, committed=71216KB)
32
33- GC (reserved=152419KB, committed=92391KB)
34 (malloc=40783KB #34817)
35 (mmap: reserved=111636KB, committed=51608KB)
36
37- Compiler (reserved=1506KB, committed=1506KB)
38 (malloc=1342KB #2557)
39 (arena=165KB #5)
40
41- Internal (reserved=5579KB, committed=5579KB)
42 (malloc=5543KB #33822)
43 (mmap: reserved=36KB, committed=36KB)
44
45- Other (reserved=231161KB, committed=231161KB)
46 (malloc=231161KB #347)
47
48- Symbol (reserved=30558KB, committed=30558KB)
49 (malloc=28887KB #769230)
50 (arena=1670KB #1)
51
52- Native Memory Tracking (reserved=16412KB, committed=16412KB)
53 (malloc=575KB #8281)
54 (tracking overhead=15837KB)
55
56- Shared class space (reserved=12288KB, committed=12136KB)
57 (mmap: reserved=12288KB, committed=12136KB)
58
59- Arena Chunk (reserved=18743KB, committed=18743KB)
60 (malloc=18743KB)
61
62- Tracing (reserved=32KB, committed=32KB)
63 (arena=32KB #1)
64
65- Logging (reserved=7KB, committed=7KB)
66 (malloc=7KB #289)
67
68- Arguments (reserved=1KB, committed=1KB)
69 (malloc=1KB #53)
70
71- Module (reserved=1045KB, committed=1045KB)
72 (malloc=1045KB #5026)
73
74- Safepoint (reserved=8KB, committed=8KB)
75 (mmap: reserved=8KB, committed=8KB)
76
77- Synchronization (reserved=204KB, committed=204KB)
78 (malloc=204KB #2026)
79
80- Serviceability (reserved=31187KB, committed=31187KB)
81 (malloc=31187KB #49714)
82
83- Metaspace (reserved=140032KB, committed=131584KB)
84 (malloc=768KB #622)
85 (mmap: reserved=139264KB, committed=130816KB)
86
87- String Deduplication (reserved=1KB, committed=1KB)
88 (malloc=1KB #8)
89[0x00007f6ccb970cbe] OopStorage::try_add_block()+0x2e
90[0x00007f6ccb97132d] OopStorage::allocate()+0x3d
91[0x00007f6ccbb34ee8] StackFrameInfo::StackFrameInfo(javaVFrame*, bool)+0x68
92[0x00007f6ccbb35a64] ThreadStackTrace::dump_stack_at_safepoint(int)+0xe4
93 (malloc=6755KB type=Serviceability #10944)
94java -Xmx512M -XX:NativeMemoryTracking=detail Test.java
95jcmd YOUR_PID VM.native_memory summary
96Test java:
1Native Memory Tracking:
2
3(Omitting categories weighting less than 1KB)
4
5Total: reserved=4431401KB, committed=1191617KB
6- Java Heap (reserved=2097152KB, committed=479232KB)
7 (mmap: reserved=2097152KB, committed=479232KB)
8
9- Class (reserved=1052227KB, committed=22403KB)
10 (classes #29547)
11 ( instance classes #27790, array classes #1757)
12 (malloc=3651KB #79345)
13 (mmap: reserved=1048576KB, committed=18752KB)
14 ( Metadata: )
15 ( reserved=139264KB, committed=130816KB)
16 ( used=130309KB)
17 ( waste=507KB =0.39%)
18 ( Class space:)
19 ( reserved=1048576KB, committed=18752KB)
20 ( used=18149KB)
21 ( waste=603KB =3.21%)
22
23- Thread (reserved=387638KB, committed=40694KB)
24 (thread #378)
25 (stack: reserved=386548KB, committed=39604KB)
26 (malloc=650KB #2271)
27 (arena=440KB #752)
28
29- Code (reserved=253202KB, committed=76734KB)
30 (malloc=5518KB #23715)
31 (mmap: reserved=247684KB, committed=71216KB)
32
33- GC (reserved=152419KB, committed=92391KB)
34 (malloc=40783KB #34817)
35 (mmap: reserved=111636KB, committed=51608KB)
36
37- Compiler (reserved=1506KB, committed=1506KB)
38 (malloc=1342KB #2557)
39 (arena=165KB #5)
40
41- Internal (reserved=5579KB, committed=5579KB)
42 (malloc=5543KB #33822)
43 (mmap: reserved=36KB, committed=36KB)
44
45- Other (reserved=231161KB, committed=231161KB)
46 (malloc=231161KB #347)
47
48- Symbol (reserved=30558KB, committed=30558KB)
49 (malloc=28887KB #769230)
50 (arena=1670KB #1)
51
52- Native Memory Tracking (reserved=16412KB, committed=16412KB)
53 (malloc=575KB #8281)
54 (tracking overhead=15837KB)
55
56- Shared class space (reserved=12288KB, committed=12136KB)
57 (mmap: reserved=12288KB, committed=12136KB)
58
59- Arena Chunk (reserved=18743KB, committed=18743KB)
60 (malloc=18743KB)
61
62- Tracing (reserved=32KB, committed=32KB)
63 (arena=32KB #1)
64
65- Logging (reserved=7KB, committed=7KB)
66 (malloc=7KB #289)
67
68- Arguments (reserved=1KB, committed=1KB)
69 (malloc=1KB #53)
70
71- Module (reserved=1045KB, committed=1045KB)
72 (malloc=1045KB #5026)
73
74- Safepoint (reserved=8KB, committed=8KB)
75 (mmap: reserved=8KB, committed=8KB)
76
77- Synchronization (reserved=204KB, committed=204KB)
78 (malloc=204KB #2026)
79
80- Serviceability (reserved=31187KB, committed=31187KB)
81 (malloc=31187KB #49714)
82
83- Metaspace (reserved=140032KB, committed=131584KB)
84 (malloc=768KB #622)
85 (mmap: reserved=139264KB, committed=130816KB)
86
87- String Deduplication (reserved=1KB, committed=1KB)
88 (malloc=1KB #8)
89[0x00007f6ccb970cbe] OopStorage::try_add_block()+0x2e
90[0x00007f6ccb97132d] OopStorage::allocate()+0x3d
91[0x00007f6ccbb34ee8] StackFrameInfo::StackFrameInfo(javaVFrame*, bool)+0x68
92[0x00007f6ccbb35a64] ThreadStackTrace::dump_stack_at_safepoint(int)+0xe4
93 (malloc=6755KB type=Serviceability #10944)
94java -Xmx512M -XX:NativeMemoryTracking=detail Test.java
95jcmd YOUR_PID VM.native_memory summary
96import java.lang.management.ManagementFactory;
97import java.lang.management.ThreadInfo;
98import java.lang.management.ThreadMXBean;
99import java.util.concurrent.ExecutorService;
100import java.util.concurrent.Executors;
101import java.util.concurrent.TimeUnit;
102
103public class Test {
104
105 private static final int RUNNING = 40;
106 private static final int WAITING = 460;
107
108 private final Object monitor = new Object();
109 private final ThreadMXBean threadMxBean = ManagementFactory.getThreadMXBean();
110 private final ExecutorService executorService = Executors.newFixedThreadPool(RUNNING + WAITING);
111
112 void startRunningThread() {
113 executorService.submit(() -> {
114 while (true) {
115 }
116 });
117 }
118
119 void startWaitingThread() {
120 executorService.submit(() -> {
121 try {
122 monitor.wait();
123 } catch (InterruptedException e) {
124 e.printStackTrace();
125 }
126 });
127 }
128
129 void startThreads() {
130 for (int i = 0; i < RUNNING; i++) {
131 startRunningThread();
132 }
133
134 for (int i = 0; i < WAITING; i++) {
135 startWaitingThread();
136 }
137 }
138
139 void shutdown() {
140 executorService.shutdown();
141 try {
142 executorService.awaitTermination(5, TimeUnit.SECONDS);
143 } catch (InterruptedException e) {
144 e.printStackTrace();
145 }
146 }
147
148
149 public static void main(String[] args) throws InterruptedException {
150 Test test = new Test();
151
152 Runtime.getRuntime().addShutdownHook(new Thread(test::shutdown));
153
154 test.startThreads();
155
156 for (int i = 0; i < 12000; i++) {
157 ThreadInfo[] threadInfos = test.threadMxBean.dumpAllThreads(false, false);
158 System.out.println("ThreadInfos: " + threadInfos.length);
159
160 Thread.sleep(100);
161 }
162
163 test.shutdown();
164 }
165}
166ANSWER
Answered 2022-Jan-17 at 13:38Unfortunately (?), the easiest way to know for sure what those categories map to is to look at OpenJDK source code. The NMT tag you are looking for is mtServiceability. This would show that "serviceability" are basically diagnostic interfaces in JDK/JVM: JVMTI, heap dumps, etc.
But the same kind of thing is clear from observing that stack trace sample you are showing mentions ThreadStackTrace::dump_stack_at_safepoint -- that is something that dumps the thread information, for example for jstack, heap dump, etc. If you have a suspicion for the memory leak in that code, you might try to build a MCVE demonstrating it, and submitting the bug against OpenJDK, or showing it to a fellow OpenJDK developer. You probably know better what your application is doing to cause thread dumps, focus there.
That being said, I don't see any obvious memory leaks in StackFrameInfo, neither can I reproduce any leak with stress tests, so maybe what you are seeing is "just" thread dumping over the larger and larger thread stacks. Or you capture it when thread dump is happening. Or... It is hard to say without the MCVE.
Update: After playing with MCVE, I realized that it reproduces with 17.0.1, but not with either mainline development JDK, or JDK 18 EA, or JDK 17.0.2 EA. I tested with 17.0.2 EA before, so was not seeing it, dang. Bisection between 17.0.1 and 17.0.2 EA shows it was fixed with JDK-8273902 backport. 17.0.2 releases this week, so the bug should disappear after you upgrade.
QUESTION
Criteria for default garbage collector Hotspot JVM 11/17
Asked 2022-Jan-11 at 10:26I found a source describing that the default gc used changes depending on the available resources. It seems that the jvm uses either g1gc or serial gc dependnig on hardware and os.
The serial collector is selected by default on certain hardware and operating system configurations
Can someone point out a more detailed source on what the specific criteria is and how that would apply in a dockerized/kubernetes enivronment. In other words:
Could setting resource requests of the pod in k8s to eg. 1500 mCpu make the jvm use serial gc and changing to 2 Cpu change the default gc to g1gc? Do the limits on when which gc is used change depending on jvm version (11 vs 17)?
ANSWER
Answered 2022-Jan-11 at 10:24In JDK 11 and 17 Serial collector is used when there is only one CPU available. Otherwise G1 is selected
If you limit the number of CPUS available to your container, JVM selects Serial instead of the defaultG1
1docker run --cpus=1 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
2[0.004s][info][gc] Using **Serial**
3It uses Serial
1docker run --cpus=1 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
2[0.004s][info][gc] Using **Serial**
3docker run --cpus=2 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
4[0.008s][info][gc ] Using G1
5It uses G1
1docker run --cpus=1 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
2[0.004s][info][gc] Using **Serial**
3docker run --cpus=2 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
4[0.008s][info][gc ] Using G1
5docker run --cpus=1 --rm -it eclipse-temurin:17 java -Xlog:gc* -version
6[0.004s][info][gc] Using Serial
7It uses Serial
1docker run --cpus=1 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
2[0.004s][info][gc] Using **Serial**
3docker run --cpus=2 --rm -it eclipse-temurin:11 java -Xlog:gc* -version
4[0.008s][info][gc ] Using G1
5docker run --cpus=1 --rm -it eclipse-temurin:17 java -Xlog:gc* -version
6[0.004s][info][gc] Using Serial
7docker run --cpus=2 --rm -it eclipse-temurin:17 java -Xlog:gc* -version
8[0.007s][info][gc] Using G1
9It uses G1
QUESTION
Do parallel streams treat upstream iterators in a thread safe way?
Asked 2021-Dec-13 at 17:33Today I was using a stream that was performing a parallel() operation after a map, however; the underlying source is an iterator which is not thread safe which is similar to the BufferedReader.lines implementation.
I originally thought that trySplit would be called on the created thread, however; I observed that the accesses to the iterator have come from multiple threads.
By example, the following silly iterator implementation is just setup with enough elements to cause splitting and also keeps track of the unique threads that accessed the hasNext method.
1class SillyIterator implements Iterator<String> {
2
3 private final ArrayDeque<String> src =
4 IntStream.range(1, 10000)
5 .mapToObj(Integer::toString)
6 .collect(toCollection(ArrayDeque::new));
7 private Map<String, String> ts = new ConcurrentHashMap<>();
8 public Set<String> threads() { return ts.keySet(); }
9 private String nextRecord = null;
10
11 @Override
12 public boolean hasNext() {
13 var n = Thread.currentThread().getName();
14 ts.put(n, n);
15 if (nextRecord != null) {
16 return true;
17 } else {
18 nextRecord = src.poll();
19 return nextRecord != null;
20 }
21 }
22 @Override
23 public String next() {
24 if (nextRecord != null || hasNext()) {
25 var rec = nextRecord;
26 nextRecord = null;
27 return rec;
28 }
29 throw new NoSuchElementException();
30 }
31
32}
33Using this to create a stream as follows:
1class SillyIterator implements Iterator<String> {
2
3 private final ArrayDeque<String> src =
4 IntStream.range(1, 10000)
5 .mapToObj(Integer::toString)
6 .collect(toCollection(ArrayDeque::new));
7 private Map<String, String> ts = new ConcurrentHashMap<>();
8 public Set<String> threads() { return ts.keySet(); }
9 private String nextRecord = null;
10
11 @Override
12 public boolean hasNext() {
13 var n = Thread.currentThread().getName();
14 ts.put(n, n);
15 if (nextRecord != null) {
16 return true;
17 } else {
18 nextRecord = src.poll();
19 return nextRecord != null;
20 }
21 }
22 @Override
23 public String next() {
24 if (nextRecord != null || hasNext()) {
25 var rec = nextRecord;
26 nextRecord = null;
27 return rec;
28 }
29 throw new NoSuchElementException();
30 }
31
32}
33var iter = new SillyIterator();
34StreamSupport
35 .stream(Spliterators.spliteratorUnknownSize(
36 iter, Spliterator.ORDERED | Spliterator.NONNULL
37 ), false)
38 .map(n -> "value = " + n)
39 .parallel()
40 .collect(toList());
41
42System.out.println(iter.threads());
43This on my system output the two fork join threads as well as the main thread, which kind of scared me.
1class SillyIterator implements Iterator<String> {
2
3 private final ArrayDeque<String> src =
4 IntStream.range(1, 10000)
5 .mapToObj(Integer::toString)
6 .collect(toCollection(ArrayDeque::new));
7 private Map<String, String> ts = new ConcurrentHashMap<>();
8 public Set<String> threads() { return ts.keySet(); }
9 private String nextRecord = null;
10
11 @Override
12 public boolean hasNext() {
13 var n = Thread.currentThread().getName();
14 ts.put(n, n);
15 if (nextRecord != null) {
16 return true;
17 } else {
18 nextRecord = src.poll();
19 return nextRecord != null;
20 }
21 }
22 @Override
23 public String next() {
24 if (nextRecord != null || hasNext()) {
25 var rec = nextRecord;
26 nextRecord = null;
27 return rec;
28 }
29 throw new NoSuchElementException();
30 }
31
32}
33var iter = new SillyIterator();
34StreamSupport
35 .stream(Spliterators.spliteratorUnknownSize(
36 iter, Spliterator.ORDERED | Spliterator.NONNULL
37 ), false)
38 .map(n -> "value = " + n)
39 .parallel()
40 .collect(toList());
41
42System.out.println(iter.threads());
43[ForkJoinPool.commonPool-worker-1, ForkJoinPool.commonPool-worker-2, main]
44ANSWER
Answered 2021-Dec-13 at 17:33Thread safety does not necessarily imply being accessed by only one thread. The important aspect is that there is no concurrent access, i.e. no access by more than one thread at the same time. If the access by different threads is temporally ordered and this ordering also ensures the necessary memory visibility, which is the responsibility of the caller, it still is a thread safe usage.
The Spliterator documentation says:
Despite their obvious utility in parallel algorithms, spliterators are not expected to be thread-safe; instead, implementations of parallel algorithms using spliterators should ensure that the spliterator is only used by one thread at a time. This is generally easy to attain via serial thread-confinement, which often is a natural consequence of typical parallel algorithms that work by recursive decomposition.
The spliterator doesn’t need to be confined to the same thread throughout its lifetime, but there should be a clear handover at the caller’s side ensuring that the old thread stops using it before the new thread starts using it.
But the important takeaway is, the spliterator doesn’t need to be thread safe, hence, the iterator wrapped by a spliterator also doesn’t need to be thread safe.
Note that a typical behavior is splitting and handing over before starting traversal, but since an ordinary Iterator doesn’t support splitting, the wrapping spliterator has to iterate and buffer elements to implement splitting. Therefore, the Iterator experiences traversal by different threads (but one at a time) when the traversal has not been started from the Stream implementation’s perspective.
That said, the lines() implementation of BufferedReader is a bad example which you should not follow. Since it’s centered around a single readLine() call, it would be natural to implement Spliterator directly instead of implementing a more complicated Iterator and have it wrapped via spliteratorUnknownSize(…).
Since your example is likewise centered around a single poll() call, it’s also straight-forward to implement Spliterator directly:
1class SillyIterator implements Iterator<String> {
2
3 private final ArrayDeque<String> src =
4 IntStream.range(1, 10000)
5 .mapToObj(Integer::toString)
6 .collect(toCollection(ArrayDeque::new));
7 private Map<String, String> ts = new ConcurrentHashMap<>();
8 public Set<String> threads() { return ts.keySet(); }
9 private String nextRecord = null;
10
11 @Override
12 public boolean hasNext() {
13 var n = Thread.currentThread().getName();
14 ts.put(n, n);
15 if (nextRecord != null) {
16 return true;
17 } else {
18 nextRecord = src.poll();
19 return nextRecord != null;
20 }
21 }
22 @Override
23 public String next() {
24 if (nextRecord != null || hasNext()) {
25 var rec = nextRecord;
26 nextRecord = null;
27 return rec;
28 }
29 throw new NoSuchElementException();
30 }
31
32}
33var iter = new SillyIterator();
34StreamSupport
35 .stream(Spliterators.spliteratorUnknownSize(
36 iter, Spliterator.ORDERED | Spliterator.NONNULL
37 ), false)
38 .map(n -> "value = " + n)
39 .parallel()
40 .collect(toList());
41
42System.out.println(iter.threads());
43[ForkJoinPool.commonPool-worker-1, ForkJoinPool.commonPool-worker-2, main]
44class SillySpliterator extends Spliterators.AbstractSpliterator<String> {
45 private final ArrayDeque<String> src = IntStream.range(1, 10000)
46 .mapToObj(Integer::toString).collect(toCollection(ArrayDeque::new));
47
48 SillySpliterator() {
49 super(Long.MAX_VALUE, ORDERED | NONNULL);
50 }
51
52 @Override
53 public boolean tryAdvance(Consumer<? super String> action) {
54 String nextRecord = src.poll();
55 if(nextRecord == null) return false;
56 action.accept(nextRecord);
57 return true;
58 }
59}
60Depending on your real life case, you may also pass the actual deque size to the constructor and provide the SIZED characteristic.
Then, you may use it like
1class SillyIterator implements Iterator<String> {
2
3 private final ArrayDeque<String> src =
4 IntStream.range(1, 10000)
5 .mapToObj(Integer::toString)
6 .collect(toCollection(ArrayDeque::new));
7 private Map<String, String> ts = new ConcurrentHashMap<>();
8 public Set<String> threads() { return ts.keySet(); }
9 private String nextRecord = null;
10
11 @Override
12 public boolean hasNext() {
13 var n = Thread.currentThread().getName();
14 ts.put(n, n);
15 if (nextRecord != null) {
16 return true;
17 } else {
18 nextRecord = src.poll();
19 return nextRecord != null;
20 }
21 }
22 @Override
23 public String next() {
24 if (nextRecord != null || hasNext()) {
25 var rec = nextRecord;
26 nextRecord = null;
27 return rec;
28 }
29 throw new NoSuchElementException();
30 }
31
32}
33var iter = new SillyIterator();
34StreamSupport
35 .stream(Spliterators.spliteratorUnknownSize(
36 iter, Spliterator.ORDERED | Spliterator.NONNULL
37 ), false)
38 .map(n -> "value = " + n)
39 .parallel()
40 .collect(toList());
41
42System.out.println(iter.threads());
43[ForkJoinPool.commonPool-worker-1, ForkJoinPool.commonPool-worker-2, main]
44class SillySpliterator extends Spliterators.AbstractSpliterator<String> {
45 private final ArrayDeque<String> src = IntStream.range(1, 10000)
46 .mapToObj(Integer::toString).collect(toCollection(ArrayDeque::new));
47
48 SillySpliterator() {
49 super(Long.MAX_VALUE, ORDERED | NONNULL);
50 }
51
52 @Override
53 public boolean tryAdvance(Consumer<? super String> action) {
54 String nextRecord = src.poll();
55 if(nextRecord == null) return false;
56 action.accept(nextRecord);
57 return true;
58 }
59}
60var result = StreamSupport.stream(new SillySpliterator(), true)
61 .map(n -> "value = " + n)
62 .collect(toList());
63QUESTION
Is it legal to use `record` as a variable name even though it's a keyword?
Asked 2021-Nov-19 at 11:10This came as a surprise: I am able to declare a variable with the name record even though it now has become a keyword. Have a look at this:
1public class Main {
2
3 static class Foo {
4 void bar() { System.out.println("Foo.bar"); }
5 }
6
7 record R (int a) {}
8
9 public static void main(String[] args) {
10 Foo record = new Foo();
11 record.bar();
12
13 R r = new R(5);
14 System.out.println(r);
15 }
16}
17When compiled and run with Java 17 this gives:
1public class Main {
2
3 static class Foo {
4 void bar() { System.out.println("Foo.bar"); }
5 }
6
7 record R (int a) {}
8
9 public static void main(String[] args) {
10 Foo record = new Foo();
11 record.bar();
12
13 R r = new R(5);
14 System.out.println(r);
15 }
16}
17Foo.bar
18R[a=5]
19I had expected this to result in an error as is the case when trying to declare a variable named class. As I know the Java guys, they are extremely careful not to break existing code, and so I think this might be a deliberate choice.
(You can't even declare a variable named const because that one const is a keyword in Java.)
ANSWER
Answered 2021-Nov-12 at 15:07OK, while looking up const in the JLS again, I saw that record is a contextual keyword whereas const is just a keyword. Contextual keywords are parsed according to context, and when the parser determines that it's not looking at a a record declaration, it will not complain.
QUESTION
What is the Java grammar that allows "new int[] {0}[0] = 1;" to compile?
Asked 2021-Oct-17 at 20:38I am using The Java® Language Specification Java SE 8 Edition as a reference.
Example class:
1class MyClass {
2
3 void method() {
4 new int[] {0}[0] = 1;
5 }
6
7}
8The code new int[] {0}[0] = 1 should be an Assignment, as index 0 of the created array is being assigned the value 1.
An Assignment is comprised of a LeftHandSide, AssignmentOperator, and an Expression. In this example, the LeftHandSide should be new int[] {0}[0].
A LeftHandSide is either an ExpressionName, FieldAccess, or ArrayAccess. In this example, the LeftHandSide should be an ArrayAccess.
The problem lies with ArrayAccess. ArrayAccess is defined as being either an ExpressionName (not the case for this example) or a PrimaryNoNewArray, and then an Expression between brackets.
The code new int[] {0} is an ArrayCreationExpression. A Primary expression is either an ArrayCreationExpression or a PrimaryNoNewArray. So to me, it seems like the second case for ArrayAccess should be Primary and not PrimaryNoNewArray.
I know the JLS doesn't have explicit grammar for everything, such as parenthesized Expressions or parenthesized LeftHandSides, but this seems to be an error. I checked the most recent spec (Java SE 17) and the grammar for ArrayAccess is unchanged.
ANSWER
Answered 2021-Oct-06 at 17:17It looks like strictly speaking, this doesn't conform with the grammar as specified. But at least, it does make sense for a compiler to allow it: the reason PrimaryNoNewArray is used instead of Primary here is so that an expression like new int[1][0] cannot be ambiguously parsed as either a 2D array or as an array access like (new int[1])[0]. If the array has an initializer like new int[]{0}[0] then the syntax is not ambiguous because this cannot be parsed as creating a 2D array.
That said, since the JLS doesn't specify that it's allowed, it could be treated as either an implementation detail or a bug in the compiler which allows it.
QUESTION
Oracle SQL or PL/SQL: How to identify candlestick pattern only in end of uptrend or downtrend and set a flag in column?
Asked 2021-Oct-05 at 08:38This question and related answers will be for educational or learning purpose only.
This question is much different from my other post and is not duplicate. Since it was creating confusion and as suggested by @MT0, I am posting this as a new question here.
I have below table, where I upload stock data on daily basis.
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109Below is the sample data for one stock "XYZ":
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109+-------+------------+-------+------+------+-------+--------+
110| Stock | Close Date | Open | High | Low | Close | Volume |
111+-------+------------+-------+------+------+-------+--------+
112| XYZ | 01-01-2021 | 40 | 40.5 | 38.5 | 38.8 | 83057 |
113| XYZ | 02-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 181814 |
114| XYZ | 03-01-2021 | 38 | 38.5 | 36.5 | 37 | 117378 |
115| XYZ | 04-01-2021 | 36.5 | 36.6 | 35.6 | 35.7 | 93737 |
116| XYZ | 05-01-2021 | 35.35 | 36.8 | 35.1 | 36.7 | 169106 |
117| XYZ | 06-01-2021 | 36.5 | 38.5 | 36.5 | 38 | 123179 |
118| XYZ | 07-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 282986 |
119| XYZ | 08-01-2021 | 39 | 40.5 | 38.5 | 40 | 117437 |
120| XYZ | 09-01-2021 | 39.7 | 39.8 | 39.3 | 39.4 | 873009 |
121| XYZ | 10-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 62522 |
122| XYZ | 11-01-2021 | 38 | 38.5 | 36.5 | 37 | 114826 |
123| XYZ | 12-01-2021 | 36.5 | 37.9 | 36.3 | 37.8 | 281461 |
124| XYZ | 13-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 77334 |
125| XYZ | 14-01-2021 | 39 | 40.5 | 38.5 | 40 | 321684 |
126+-------+------------+-------+------+------+-------+--------+
127Over the period of time, there will be more than thousands of records for each stock symbol and I would like to identify candlestick pattern only at the top of upmove/uptrend or at the bottom of downmove/downtrend but NOT in sideways (Since this will be false positive). Below is the sample screeshot:
Assuming today is 12th Jan 2021, below is the expected output:
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109+-------+------------+-------+------+------+-------+--------+
110| Stock | Close Date | Open | High | Low | Close | Volume |
111+-------+------------+-------+------+------+-------+--------+
112| XYZ | 01-01-2021 | 40 | 40.5 | 38.5 | 38.8 | 83057 |
113| XYZ | 02-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 181814 |
114| XYZ | 03-01-2021 | 38 | 38.5 | 36.5 | 37 | 117378 |
115| XYZ | 04-01-2021 | 36.5 | 36.6 | 35.6 | 35.7 | 93737 |
116| XYZ | 05-01-2021 | 35.35 | 36.8 | 35.1 | 36.7 | 169106 |
117| XYZ | 06-01-2021 | 36.5 | 38.5 | 36.5 | 38 | 123179 |
118| XYZ | 07-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 282986 |
119| XYZ | 08-01-2021 | 39 | 40.5 | 38.5 | 40 | 117437 |
120| XYZ | 09-01-2021 | 39.7 | 39.8 | 39.3 | 39.4 | 873009 |
121| XYZ | 10-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 62522 |
122| XYZ | 11-01-2021 | 38 | 38.5 | 36.5 | 37 | 114826 |
123| XYZ | 12-01-2021 | 36.5 | 37.9 | 36.3 | 37.8 | 281461 |
124| XYZ | 13-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 77334 |
125| XYZ | 14-01-2021 | 39 | 40.5 | 38.5 | 40 | 321684 |
126+-------+------------+-------+------+------+-------+--------+
127+-------+-------------------+------------+------------+--------------+--------+---------------+
128| Stock | Consecutive Count | Start Date | End Date | Latest Close | Volume | Pattern |
129+-------+-------------------+------------+------------+--------------+--------+---------------+
130| XYZ | 3 | 09-01-2021 | 12-01-2021 | 37.8 | 281461 | Piercing Line |
131+-------+-------------------+------------+------------+--------------+--------+---------------+
132Since the source table will have many other stocks, would like to show results on 12th Jan 2021 for other stocks as well if there is any pattern identified. I feel this is quite challenging and complex logic. Hence seeking help here. Thanks in advance.
Update: Thank you @JustinCave
Here's the formula for calculation:
For Bullish Engulfing:
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109+-------+------------+-------+------+------+-------+--------+
110| Stock | Close Date | Open | High | Low | Close | Volume |
111+-------+------------+-------+------+------+-------+--------+
112| XYZ | 01-01-2021 | 40 | 40.5 | 38.5 | 38.8 | 83057 |
113| XYZ | 02-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 181814 |
114| XYZ | 03-01-2021 | 38 | 38.5 | 36.5 | 37 | 117378 |
115| XYZ | 04-01-2021 | 36.5 | 36.6 | 35.6 | 35.7 | 93737 |
116| XYZ | 05-01-2021 | 35.35 | 36.8 | 35.1 | 36.7 | 169106 |
117| XYZ | 06-01-2021 | 36.5 | 38.5 | 36.5 | 38 | 123179 |
118| XYZ | 07-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 282986 |
119| XYZ | 08-01-2021 | 39 | 40.5 | 38.5 | 40 | 117437 |
120| XYZ | 09-01-2021 | 39.7 | 39.8 | 39.3 | 39.4 | 873009 |
121| XYZ | 10-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 62522 |
122| XYZ | 11-01-2021 | 38 | 38.5 | 36.5 | 37 | 114826 |
123| XYZ | 12-01-2021 | 36.5 | 37.9 | 36.3 | 37.8 | 281461 |
124| XYZ | 13-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 77334 |
125| XYZ | 14-01-2021 | 39 | 40.5 | 38.5 | 40 | 321684 |
126+-------+------------+-------+------+------+-------+--------+
127+-------+-------------------+------------+------------+--------------+--------+---------------+
128| Stock | Consecutive Count | Start Date | End Date | Latest Close | Volume | Pattern |
129+-------+-------------------+------------+------------+--------------+--------+---------------+
130| XYZ | 3 | 09-01-2021 | 12-01-2021 | 37.8 | 281461 | Piercing Line |
131+-------+-------------------+------------+------------+--------------+--------+---------------+
132O1 > C1 and C > O and C > H1 and O < L1
133where,
134
135O1 = Previous day Open price
136C1 = Previous day Close price
137C = Today's Close price
138O = Today's Open price
139H1 = Previous day High price
140L1 = Previous day Low price
141For Bearish Harami:
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109+-------+------------+-------+------+------+-------+--------+
110| Stock | Close Date | Open | High | Low | Close | Volume |
111+-------+------------+-------+------+------+-------+--------+
112| XYZ | 01-01-2021 | 40 | 40.5 | 38.5 | 38.8 | 83057 |
113| XYZ | 02-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 181814 |
114| XYZ | 03-01-2021 | 38 | 38.5 | 36.5 | 37 | 117378 |
115| XYZ | 04-01-2021 | 36.5 | 36.6 | 35.6 | 35.7 | 93737 |
116| XYZ | 05-01-2021 | 35.35 | 36.8 | 35.1 | 36.7 | 169106 |
117| XYZ | 06-01-2021 | 36.5 | 38.5 | 36.5 | 38 | 123179 |
118| XYZ | 07-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 282986 |
119| XYZ | 08-01-2021 | 39 | 40.5 | 38.5 | 40 | 117437 |
120| XYZ | 09-01-2021 | 39.7 | 39.8 | 39.3 | 39.4 | 873009 |
121| XYZ | 10-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 62522 |
122| XYZ | 11-01-2021 | 38 | 38.5 | 36.5 | 37 | 114826 |
123| XYZ | 12-01-2021 | 36.5 | 37.9 | 36.3 | 37.8 | 281461 |
124| XYZ | 13-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 77334 |
125| XYZ | 14-01-2021 | 39 | 40.5 | 38.5 | 40 | 321684 |
126+-------+------------+-------+------+------+-------+--------+
127+-------+-------------------+------------+------------+--------------+--------+---------------+
128| Stock | Consecutive Count | Start Date | End Date | Latest Close | Volume | Pattern |
129+-------+-------------------+------------+------------+--------------+--------+---------------+
130| XYZ | 3 | 09-01-2021 | 12-01-2021 | 37.8 | 281461 | Piercing Line |
131+-------+-------------------+------------+------------+--------------+--------+---------------+
132O1 > C1 and C > O and C > H1 and O < L1
133where,
134
135O1 = Previous day Open price
136C1 = Previous day Close price
137C = Today's Close price
138O = Today's Open price
139H1 = Previous day High price
140L1 = Previous day Low price
141(O1 < C1) and (O > C) and (O < C1) and (C > O1) and (H < H1) and (L > L1)
142where,
143 O1 = Previous day Open price
144 C1 = Previous day Close price
145 C = Today's Close price
146 O = Today's Open price
147 H1 = Previous day High price
148 L1 = Previous day Low price
149 H = Today's High price
150 L = Today's Low price
151For Piercing Line:
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109+-------+------------+-------+------+------+-------+--------+
110| Stock | Close Date | Open | High | Low | Close | Volume |
111+-------+------------+-------+------+------+-------+--------+
112| XYZ | 01-01-2021 | 40 | 40.5 | 38.5 | 38.8 | 83057 |
113| XYZ | 02-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 181814 |
114| XYZ | 03-01-2021 | 38 | 38.5 | 36.5 | 37 | 117378 |
115| XYZ | 04-01-2021 | 36.5 | 36.6 | 35.6 | 35.7 | 93737 |
116| XYZ | 05-01-2021 | 35.35 | 36.8 | 35.1 | 36.7 | 169106 |
117| XYZ | 06-01-2021 | 36.5 | 38.5 | 36.5 | 38 | 123179 |
118| XYZ | 07-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 282986 |
119| XYZ | 08-01-2021 | 39 | 40.5 | 38.5 | 40 | 117437 |
120| XYZ | 09-01-2021 | 39.7 | 39.8 | 39.3 | 39.4 | 873009 |
121| XYZ | 10-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 62522 |
122| XYZ | 11-01-2021 | 38 | 38.5 | 36.5 | 37 | 114826 |
123| XYZ | 12-01-2021 | 36.5 | 37.9 | 36.3 | 37.8 | 281461 |
124| XYZ | 13-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 77334 |
125| XYZ | 14-01-2021 | 39 | 40.5 | 38.5 | 40 | 321684 |
126+-------+------------+-------+------+------+-------+--------+
127+-------+-------------------+------------+------------+--------------+--------+---------------+
128| Stock | Consecutive Count | Start Date | End Date | Latest Close | Volume | Pattern |
129+-------+-------------------+------------+------------+--------------+--------+---------------+
130| XYZ | 3 | 09-01-2021 | 12-01-2021 | 37.8 | 281461 | Piercing Line |
131+-------+-------------------+------------+------------+--------------+--------+---------------+
132O1 > C1 and C > O and C > H1 and O < L1
133where,
134
135O1 = Previous day Open price
136C1 = Previous day Close price
137C = Today's Close price
138O = Today's Open price
139H1 = Previous day High price
140L1 = Previous day Low price
141(O1 < C1) and (O > C) and (O < C1) and (C > O1) and (H < H1) and (L > L1)
142where,
143 O1 = Previous day Open price
144 C1 = Previous day Close price
145 C = Today's Close price
146 O = Today's Open price
147 H1 = Previous day High price
148 L1 = Previous day Low price
149 H = Today's High price
150 L = Today's Low price
151(O < C) and (O1 > C1) and (C > (C1 + O1)/2) and (O < C1) and (C < O1)
152 where,
153 O1 = Previous day Open price
154 C1 = Previous day Close price
155 C = Today's Close price
156 O = Today's Open price
157ANSWER
Answered 2021-Oct-04 at 20:52Patterns in MATCH_RECOGNIZE work in a similar fashion to regular expressions; you want something like:
(Note: your PIERCING_LINE formula does not give the expected output so I have assumed you want C > (C1 + O1)/2 rather than C > C1 + (O1/2).)
1/* CREATE TABLE */
2CREATE TABLE RAW_SOURCE(
3 Stock VARCHAR(100),
4 Close_Date DATE,
5 Open NUMBER,
6 High NUMBER,
7 Low NUMBER,
8 Close NUMBER,
9 Volume NUMBER
10);
11
12/* INSERT QUERY NO: 1 */
13INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
14VALUES
15(
16'XYZ', '01/01/2021', 40, 40.5, 38.5, 38.8, 83057
17);
18
19/* INSERT QUERY NO: 2 */
20INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
21VALUES
22(
23'XYZ', '02/01/2021', 39.2, 39.2, 37.2, 37.8, 181814
24);
25
26/* INSERT QUERY NO: 3 */
27INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
28VALUES
29(
30'XYZ', '03/01/2021', 38, 38.5, 36.5, 37, 117378
31);
32
33/* INSERT QUERY NO: 4 */
34INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
35VALUES
36(
37'XYZ', '04/01/2021', 36.5, 36.6, 35.6, 35.7, 93737
38);
39
40/* INSERT QUERY NO: 5 */
41INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
42VALUES
43(
44'XYZ', '05/01/2021', 35.35, 36.8, 35.1, 36.7, 169106
45);
46
47/* INSERT QUERY NO: 6 */
48INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
49VALUES
50(
51'XYZ', '06/01/2021', 36.5, 38.5, 36.5, 38, 123179
52);
53
54/* INSERT QUERY NO: 7 */
55INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
56VALUES
57(
58'XYZ', '07/01/2021', 37.5, 39.5, 37.3, 39.4, 282986
59);
60
61/* INSERT QUERY NO: 8 */
62INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
63VALUES
64(
65'XYZ', '08/01/2021', 39, 40.5, 38.5, 40, 117437
66);
67
68/* INSERT QUERY NO: 9 */
69INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
70VALUES
71(
72'XYZ', '09/01/2021', 39.7, 39.8, 39.3, 39.4, 873009
73);
74
75/* INSERT QUERY NO: 10 */
76INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
77VALUES
78(
79'XYZ', '10/01/2021', 39.2, 39.2, 37.2, 37.8, 62522
80);
81
82/* INSERT QUERY NO: 11 */
83INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
84VALUES
85(
86'XYZ', '11/01/2021', 38, 38.5, 36.5, 37, 114826
87);
88
89/* INSERT QUERY NO: 12 */
90INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
91VALUES
92(
93'XYZ', '12/01/2021', 36.5, 37.9, 36.3, 37.8, 281461
94);
95
96/* INSERT QUERY NO: 13 */
97INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
98VALUES
99(
100'XYZ', '13/01/2021', 37.5, 39.5, 37.3, 39.4, 77334
101);
102
103/* INSERT QUERY NO: 14 */
104INSERT INTO RAW_SOURCE(Stock, Close_Date, Open, High, Low, Close, Volume)
105VALUES
106(
107'XYZ', '14/01/2021', 39, 40.5, 38.5, 40, 321684
108);
109+-------+------------+-------+------+------+-------+--------+
110| Stock | Close Date | Open | High | Low | Close | Volume |
111+-------+------------+-------+------+------+-------+--------+
112| XYZ | 01-01-2021 | 40 | 40.5 | 38.5 | 38.8 | 83057 |
113| XYZ | 02-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 181814 |
114| XYZ | 03-01-2021 | 38 | 38.5 | 36.5 | 37 | 117378 |
115| XYZ | 04-01-2021 | 36.5 | 36.6 | 35.6 | 35.7 | 93737 |
116| XYZ | 05-01-2021 | 35.35 | 36.8 | 35.1 | 36.7 | 169106 |
117| XYZ | 06-01-2021 | 36.5 | 38.5 | 36.5 | 38 | 123179 |
118| XYZ | 07-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 282986 |
119| XYZ | 08-01-2021 | 39 | 40.5 | 38.5 | 40 | 117437 |
120| XYZ | 09-01-2021 | 39.7 | 39.8 | 39.3 | 39.4 | 873009 |
121| XYZ | 10-01-2021 | 39.2 | 39.2 | 37.2 | 37.8 | 62522 |
122| XYZ | 11-01-2021 | 38 | 38.5 | 36.5 | 37 | 114826 |
123| XYZ | 12-01-2021 | 36.5 | 37.9 | 36.3 | 37.8 | 281461 |
124| XYZ | 13-01-2021 | 37.5 | 39.5 | 37.3 | 39.4 | 77334 |
125| XYZ | 14-01-2021 | 39 | 40.5 | 38.5 | 40 | 321684 |
126+-------+------------+-------+------+------+-------+--------+
127+-------+-------------------+------------+------------+--------------+--------+---------------+
128| Stock | Consecutive Count | Start Date | End Date | Latest Close | Volume | Pattern |
129+-------+-------------------+------------+------------+--------------+--------+---------------+
130| XYZ | 3 | 09-01-2021 | 12-01-2021 | 37.8 | 281461 | Piercing Line |
131+-------+-------------------+------------+------------+--------------+--------+---------------+
132O1 > C1 and C > O and C > H1 and O < L1
133where,
134
135O1 = Previous day Open price
136C1 = Previous day Close price
137C = Today's Close price
138O = Today's Open price
139H1 = Previous day High price
140L1 = Previous day Low price
141(O1 < C1) and (O > C) and (O < C1) and (C > O1) and (H < H1) and (L > L1)
142where,
143 O1 = Previous day Open price
144 C1 = Previous day Close price
145 C = Today's Close price
146 O = Today's Open price
147 H1 = Previous day High price
148 L1 = Previous day Low price
149 H = Today's High price
150 L = Today's Low price
151(O < C) and (O1 > C1) and (C > (C1 + O1)/2) and (O < C1) and (C < O1)
152 where,
153 O1 = Previous day Open price
154 C1 = Previous day Close price
155 C = Today's Close price
156 O = Today's Open price
157SELECT *
158FROM raw_source
159MATCH_RECOGNIZE (
160 PARTITION BY stock
161 ORDER BY Close_Date
162 MEASURES
163 CLASSIFIER() AS pttrn
164 ALL ROWS PER MATCH
165 PATTERN (
166 ^initial_value
167 |
168 down+ (bullish_engulfing | piercing_line | $)
169 |
170 up+ (bearish_harami | $)
171 |
172 other
173 )
174 DEFINE
175 down AS
176 PREV(open) > open
177 AND PREV(close) > close
178 AND PREV(open) > PREV(close)
179 AND open > close,
180 up AS
181 PREV(open) < open
182 AND PREV(close) < close
183 AND PREV(open) < PREV(close)
184 AND open < close,
185 bullish_engulfing AS
186 -- O1 > C1 and C > O and C > H1 and O < L1
187 PREV(open) > PREV(close)
188 AND close > open
189 AND close > PREV(high)
190 AND open < PREV(low),
191 bearish_harami AS
192 -- O1 < C1 and O > C and O < C1 and C > O1 and H < H1 and L > L1
193 PREV(open) < PREV(close)
194 AND open > close
195 AND open < PREV(close)
196 AND close > PREV(open)
197 AND high < PREV(high)
198 AND low > PREV(low),
199 piercing_line AS
200 -- O < C and O1 > C1 and C > (C1 + O1)/2 and O < C1 and C < O1
201 open < close
202 AND PREV(open) > PREV(close)
203 AND close > (PREV(close) + PREV(open))/2
204 AND open < PREV(close)
205 AND close < PREV(open)
206)
207Which outputs:
STOCK CLOSE_DATE PTTRN OPEN HIGH LOW CLOSE VOLUME XYZ 01/01/2021 INITIAL_VALUE 40 40.5 38.5 38.8 83057 XYZ 02/01/2021 DOWN 39.2 39.2 37.2 37.8 181814 XYZ 03/01/2021 DOWN 38 38.5 36.5 37 117378 XYZ 04/01/2021 DOWN 36.5 36.6 35.6 35.7 93737 XYZ 05/01/2021 BULLISH_ENGULFING 35.35 36.8 35.1 36.7 169106 XYZ 06/01/2021 UP 36.5 38.5 36.5 38 123179 XYZ 07/01/2021 UP 37.5 39.5 37.3 39.4 282986 XYZ 08/01/2021 UP 39 40.5 38.5 40 117437 XYZ 09/01/2021 BEARISH_HARAMI 39.7 39.8 39.3 39.4 873009 XYZ 10/01/2021 DOWN 39.2 39.2 37.2 37.8 62522 XYZ 11/01/2021 DOWN 38 38.5 36.5 37 114826 XYZ 12/01/2021 PIERCING_LINE 36.5 37.9 36.3 37.8 281461 XYZ 13/01/2021 UP 37.5 39.5 37.3 39.4 77334 XYZ 14/01/2021 UP 39 40.5 38.5 40 321684
db<>fiddle here
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Oracle
Tutorials and Learning Resources are not available at this moment for Oracle
Share this Page
Get latest updates on Oracle