Explore all Graphics open source software, libraries, packages, source code, cloud functions and APIs.
Popular New Releases in Graphics
three.js
r139
pixijs
v6.3.0
pixi.js
v6.0.2
tfjs
filament
v1.21.1
Popular Libraries in Graphics
three.js
by mrdoob 
javascript
80965 MIT
JavaScript 3D Library.
pixijs
by pixijs typescript
36164 MIT
The HTML5 Creation Engine: Create beautiful digital content with the fastest, most flexible 2D WebGL renderer.
pixi.js
by pixijs typescript
32471 MIT
The HTML5 Creation Engine: Create beautiful digital content with the fastest, most flexible 2D WebGL renderer.
tfjs
by tensorflow typescript
16158 Apache-2.0
A WebGL accelerated JavaScript library for training and deploying ML models.
filament
by google c++
13837 Apache-2.0
Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2
tinyrenderer
by ssloy c++
11147 NOASSERTION
A brief computer graphics / rendering course
WebGL-Fluid-Simulation
by PavelDoGreat javascript
11014 MIT
Play with fluids in your browser (works even on mobile)
3d-game-shaders-for-beginners
by lettier c++
11004 BSD-3-Clause
🎮 A step-by-step guide to implementing SSAO, depth of field, lighting, normal mapping, and more for your 3D game.
deck.gl
by visgl javascript
9725 MIT
WebGL2 powered visualization framework
Trending New libraries in Graphics
Real-Time-Person-Removal
by jasonmayes javascript
4798 Apache-2.0
Removing people from complex backgrounds in real time using TensorFlow.js in the web browser
engine
by oasis-engine typescript
2597 MIT
Oasis Engine is a web-first and mobile-first high-performance real-time development platform.
skija
by JetBrains java
2045 Apache-2.0
Skia bindings for Java
trois
by troisjs typescript
1961 MIT
✨ ThreeJS + VueJS 3 + ViteJS ⚡
Graphics
by Unity-Technologies csharp
1628 NOASSERTION
Unity Graphics - Including Scriptable Render Pipeline
X-PostProcessing-Library
by QianMo csharp
1293 MIT
Unity Post Processing Stack Library | Unity引擎的高品质后处理库
procedural-gl-js
by felixpalmer javascript
1083 MPL-2.0
Mobile-first 3D mapping engine with emphasis on user experience
aladino
by luruke javascript
771
🧞♂️ Your magic WebGL carpet
3DTilesRendererJS
by NASA-AMMOS javascript
743 Apache-2.0
Renderer for 3D Tiles in Javascript using three.js
Top Authors in Graphics
1
42 Libraries
12896
2
33 Libraries
2447
3
31 Libraries
3125
4
28 Libraries
618
5
25 Libraries
769
6
20 Libraries
3739
7
20 Libraries
71795
8
20 Libraries
278
9
16 Libraries
2071
10
15 Libraries
1407
1
42 Libraries
12896
2
33 Libraries
2447
3
31 Libraries
3125
4
28 Libraries
618
5
25 Libraries
769
6
20 Libraries
3739
7
20 Libraries
71795
8
20 Libraries
278
9
16 Libraries
2071
10
15 Libraries
1407
Trending Kits in Graphics
A React graph library is a collection of components and tools. It is designed for creating graphs and other data visualizations in applications. It provides a set of reusable and customizable building blocks. It makes it easier to build interactive and insightful data visualizations.
Many react graph libraries are available, from simple ones to complex solutions. Some libraries focus on providing various chart types and customization options. Others offer functionalities like stock charting, time series analysis, and network traffic visualization. Developers can choose the library that best fits their requirements and use cases.
React graph libraries offer a wide range of features to support data visualization. These features include support for different chart types and interactive and animated charts. It includes a responsive design for adapting to different screen sizes and server-side rendering. It supports improved performance, integration with other libraries, and extensive documentation. Some libraries may provide data manipulation, real-time updates, and integration with ML algorithms.
Choosing the right library based on your project is important when using a react graph library. Developers should consider chart types, customization options, performance, documentation, and community support. Must understand the limitations and know bundle size or rendering performance.
Creating a simple react graph library involves designing the data model. It helps in defining the required components for rendering the charts or graphs. The library should provide an intuitive API. It helps developers to pass the data and configuration options to the components. It enables them to customize the appearance and behavior of the graphs. The implementation should follow React's component principle. It allows the reuse and composition of components to create more complex visualizations.
These libraries are used in data analysis, visualization, real-time monitoring, and visual analytics. They have been instrumental in providing insightful and interactive visualizations. It helps understand complex data sets. These libraries are used in applications where visualizing model outputs or training progress. It can aid in interpreting and communicating the results.
To use the graph libraries, developers should understand the documentation and examples provided. They should understand the requirements and choose a library that aligns with them. It is important to optimize the library, considering bundle size and rendering performance. Developers should leverage the library's features, such as interactivity, animations, and responsiveness. It helps enhance the user experience and convey insights.
React graph libraries offer powerful tools for creating and visualizing data in applications. They provide reusable and customizable components, extensive documentation, and various features. It helps support various data visualization needs. By leveraging these libraries, developers can create appealing and interactive data visualizations. It helps in improving the understanding and interpretation of complex data sets.
The advantages of using these libraries include their ability to simplify the development. It provides ready-to-use components, customization flexibility, extensive documentation, community support, and compatibility. These libraries have become popular among data scientists and developers. It is because it is easy, versatility, and ability. It helps create high-quality and insightful visualizations in web and mobile applications.
recharts:
- It is a lightweight and flexible charting library for React.
- It provides a simple API for creating charts.
- It supports features like tooltips, legends, and animations.
- It is beginner friendly.
- It can create simple and elegant graphs in React applications.
chartist.js:
- It is a popular JavaScript charting library.
- It can be integrated with React.
- It offers various chart types, including bar, line, and radar charts.
- It provides a simple and intuitive API.
- It makes it suitable for beginners and experienced developers.
nivo:
- It is a powerful React-based data visualization library.
- It offers various chart types, including bar, line, and pie charts.
- It provides rich customization options and responsive design and is optimized for performance.
- It makes it suitable for creating complex and dynamic visualizations.
victory:
- It is a React-based charting library.
- It focuses on providing customizable and interactive graphs.
- It supports various chart types and offers features like animations, tooltips, and zooming.
- It is well-suited for creating stunning and interactive data visualizations.
react-vis:
- It is a React-based visualization library.
- It offers rich components for creating various graphs.
- It provides customizable and responsive graphs.
- It makes it ideal for creating appealing dashboards and reports.
dc.js:
- It is a powerful data visualization library.
- It supports various graphs and charts.
- It helps create interactive and dynamic visualizations.
- It makes it suitable for complex data analysis and data-driven applications.
viser:
- It is a React-based visualization library built on top of G2Plot and G6.
- It offers various components for creating interactive and customizable graphs.
- It supports features like animations, tooltips, and interactions.
- It makes it ideal for creating appealing and interactive data visualizations.
FAQ
1. What are the most reliable React data visualization libraries available?
Some reliable data visualization libraries include D3.js, Victory, Recharts, and Chart.js. These libraries have been used as robust and dependable solutions. It helps in creating data visualizations in React applications.
2. How do I choose a React chart library that is right for my project?
Consider chart types, customization levels, documentation, and support to choose a chart library. You should also consider its performance and rendering capabilities, and compatibility. Evaluating API and examples as they align with your development workflow and requirements.
3. Are there open-source charting libraries that make it easy to work with React?
Yes, there are several open-source charting libraries available that work with React. Some popular options include Chart.js, Recharts, and react-charts-2. These libraries provide various chart types and customization options. It is an intuitive API designed for React development.
4. What is modular charting, and how can I use it in my project?
Modular charting uses reusable and composable charting components to build data visualizations. With modular charting, you can break down complex visualizations into smaller, reusable components. It can be combined to create more sophisticated charts. This approach promotes code reusability, scalability, and maintainability in your project.
5. Are there any customizable stock charts available in React libraries?
Yes, there are customizable stock charts available in React libraries. Libraries like Highcharts and TradingView provide comprehensive stock charting functionalities. It includes interactive features, technical indicators, and drawing tools. These libraries offer extensive customization options. It allows you to tailor the stock charts to your requirements.
6. How do I create data visualization charts with React components?
You can leverage charting libraries to create data visualization charts with React components. These libraries provide various React components that encapsulate the charting logic and rendering. You can pass data and configuration options to define the type, styling, and data sources. Combining these components and customizing the properties allows data visualization charts. These libraries offer extensive documentation to guide you through creating data visualizations.
Graphics in C++ is to develop a graphic model to draw curves, lines of different colors and styles, phrases with fashionable fonts with different typefaces, colors, and sizes, and geometrical shapes like circles, rectangles, etc. Graphics programming could help us to create projects, games, animation, and more. There are two different modes in C++ by default- text mode and graphics mode. To implement the graphics, which are a two-dimensional notion, we need to use C++ programming and a few functions. The prerequisites of a graphics monitor are a graphics card like a VGA, SVGA, or EGA.
Importing the "graphics.h" library into the GCC compiler enables the production of graphics in the C++ console. C++ is used to program graphics via the terminal or command prompt or the other method you need to install the DevC++ compiler. In this mode, the output is shown as points or pixels on the computer screen. The screen is divided into tiny dots called pixels in graphics mode. For instance, the screen on a VGA display is divided into 480 rows and 640 columns of dots. The term "resolution screen" refers to the number of dots per inch. The clarity of the graphics increases with the number of pixels. The circle, line, eclipse, and other geometric forms are also traceable. The primary technique adopted is object-oriented programming. Since C++ relies on low-level programs, there are no built-in drawing methods, and APIs are used to create visuals. C++ is frequently regarded as one of the leading programming languages for computer graphics.
To operate in graphics, the C++ compiler requires.
- the "graphics.h" header file that includes built-in graphic features
- files for the Borland Graphics Interface (BGI) that has the graphics driver programs to set up the computer monitor to display graphics, and
- the "chr" extension files for the character font style.
The wide adoption of C++ by the developers of IDEs, editors, compilers, test frameworks, and other tools makes it easier to use. The best libraries are magnum, OpenSceneGraph, AtomicGameEngine, Vulkan, glbinding, LibVT, Cell, minigrafx, tinyrenderer, ripes, etc.
Check out the list below to find more popular C++ 2D Graphics libraries for your applications:
In the branch of computer science known as computer graphics, techniques for digitally synthesizing and modifying visual content are explored. 3D graphics contribute to various applications these days. Animation of 3D pictures raised the demand for CGI (Computer Generated Imagery) in movies and video games, creating images closer to reality. Visualizations on computer screens are constructed using a variety of algorithms and methods. C++ helps comprehend, process, and create graphics with a rich visual experience. 3D graphics are the contrast of two-dimensional (2D) images. Artists often generate a 3D model consisting of a wireframe and polygons, which are given color, effects, movements, texturing, and lighting and then rendered as a 3D computer picture. C++-coded visuals are used to produce effects, models, animations, and simulations in real time. The 2D system uses only two coordinates named X and Y, while 3D uses an extra coordinate called Z.
The OpenGL (Open Graphics Library) industrial standard API for creating 3D (including 2D) graphics is cross-platform, hardware-accelerated, and language-independent. Modern computers have dedicated GPUs (Graphics Processing Units), each with its memory to speed up graphics rendering. The software interface for graphics hardware is called OpenGL. OpenGL graphic rendering directives sent by your programs could be focused on and sped up by the graphics hardware. The method described here simplifies the programmatic construction of geometry within the constraints of a production-level language, C++. The system's implementation is strongly object-oriented and depends on multiple dispatching. The system can be easily expanded, and new geometric operations and primitives can be easily added. New media formats, such as music and image, could be introduced to the system.
Using these libraries, we can accomplish jobs more quickly, effectively, and with fewer lines of repetitious code than without them. It makes it simple to develop several feature modules for dynamic distribution from closely tied modules to particular features. The wide adoption of C++ by the developers of IDEs, editors, compilers, test frameworks, and other tools makes it easier to use. C++ libraries are typically used in User Interface and Graphics applications. Various C++ libraries help in scientific visualization to contribute to the entertainment, gaming, and computer-aided design sectors. A few examples of C++ libraries are - OpenMVG, Horde3D, 3d-game-shaders, s2geometry, hello-webgpu, NVISII, PixelArtShader, assimp, permafrost-engine, and Urho3D.
Check out the list below to find more popular C++ 3D Graphic libraries for your app development:
Trending Discussions on Graphics
android:exported added but still getting error Apps targeting Android 12 and higher are required to specify an explicit value for android:exported
Tensorflow setup on RStudio/ R | CentOS
Configuring compilers on Mac M1 (Big Sur, Monterey) for Rcpp and other tools
Problem with non-standard evaluation in disk.frame objects using data.table syntax
How to automate legends for a new geom in ggplot2?
Why is WSL extremely slow when compared with native Windows NPM/Yarn processing?
Window not displaying SDL2
Symbol not found in flat namespace '_FT_Done_Face' from reportlab with Python@3.9 on macOS 12
jdeps can't print-module-deps due to a MultiReleaseException
"Theming Icons" functionality crashes live wallpapers on Android 12
QUESTION
android:exported added but still getting error Apps targeting Android 12 and higher are required to specify an explicit value for android:exported
Asked 2022-Mar-24 at 15:30I have added android:exported="true" to my only activity in manifest but still getting below error after updating compile sdk and target sdk version to 31.I also tried rebuilding the project , invalidating cache and restart but that didn't helped
Error- Apps targeting Android 12 and higher are required to specify an explicit value for android:exported when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details.
1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android"
3 package="com.xyz.abc">
4
5 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
6 <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
7
8 <application
9 android:name=".framework.presentation.BaseApplication"
10 android:allowBackup="true"
11 android:icon="@mipmap/ic_launcher"
12 android:label="@string/app_name"
13 android:roundIcon="@mipmap/ic_launcher_round"
14 android:supportsRtl="true"
15 android:theme="@style/AppTheme">
16 <activity android:name="com.xyz.presentation.MainActivity"
17 android:exported="true">
18 <intent-filter>
19 <action android:name="android.intent.action.MAIN" />
20
21 <category android:name="android.intent.category.LAUNCHER" />
22 </intent-filter>
23 </activity>
24 </application>
25
26</manifest>
27
Merged manifest error
Other Manifest Files (Included in merge, but did not contribute any elements) firebase-installations:17.0.0 manifest, versionedparcelable:1.1.1 manifest, runtime:1.0.1 manifest, test:core:1.2.0 manifest, loader:1.0.0 manifest, facebook-share:11.1.0 manifest, leakcanary:leaksentry:2.0-alpha-3 manifest, material-dialogs:input:3.2.1 manifest, material-icons-extended:1.0.0 manifest, play-services-stats:17.0.0 manifest, interpolator:1.0.0 manifest, activity-compose:1.3.1 manifest, material-ripple:1.0.0 manifest, foundation:1.0.0 manifest, asynclayoutinflater:1.0.0 manifest, savedstate-ktx:1.1.0 manifest, navigation-dynamic-features-fragment:2.3.5 manifest, firebase-ui-auth:7.2.0 manifest, animation:1.0.1 manifest, animation-core:1.0.1 manifest, installreferrer:1.0 manifest, firebase-crashlytics:18.0.0 manifest, ui:1.0.1 manifest, lifecycle-viewmodel-savedstate:2.3.1 manifest, play-services-auth-base:17.0.0 manifest, hilt-android:2.35.1 manifest, material-dialogs:core:3.2.1 manifest, AndroidManifest.xml navigation file, savedstate:1.1.0 manifest, cursoradapter:1.0.0 manifest, sqlite-framework:2.0.1 manifest, room-ktx:2.1.0 manifest, leakcanary-android-core:2.0-alpha-3 manifest, AndroidManifest.xml navigation file, media:1.0.0 manifest, coordinatorlayout:1.1.0 manifest, legacy-support-core-utils:1.0.0 manifest, lifecycle-runtime:2.3.1 manifest, coil-kt:coil:1.3.1 manifest, ui-tooling-preview:1.0.0 manifest, facebook-core:11.1.0 manifest, core:1.6.0 manifest, material:1.0.0 manifest, firebase-common:20.0.0 manifest, documentfile:1.0.0 manifest, lifecycle-viewmodel-compose:2.4.0-beta01 manifest, play-services-base:17.1.0 manifest, ui-tooling-data:1.0.0 manifest, coil-base:1.3.1 manifest, firebase-analytics-ktx:19.0.0 manifest, localbroadcastmanager:1.0.0 manifest, swiperefreshlayout:1.1.0-alpha03 manifest, constraintlayout-compose:1.0.0-beta02 manifest, core-ktx:1.6.0 manifest, firebase-database-collection:18.0.0 manifest, coil-compose-base:1.3.1 manifest, activity:1.3.1 manifest, AndroidManifest.xml navigation file, facebook-messenger:11.1.0 manifest, print:1.0.0 manifest, customview:1.1.0 manifest, material-icons-core:1.0.0 manifest, play-services-measurement-sdk:19.0.0 manifest, fragment:1.3.4 manifest, firebase-appcheck-interop:16.0.0-beta01 manifest, facebook-login:11.1.0 manifest, cardview:1.0.0 manifest, runtime-rxjava2:1.0.0 manifest, viewpager2:1.0.0 manifest, play-services-ads-identifier:17.0.0 manifest, play-services-measurement-impl:19.0.0 manifest, lifecycle-livedata-core:2.3.1 manifest, play-services-safetynet:17.0.0 manifest, AndroidManifest.xml navigation file, lifecycle-viewmodel-ktx:2.3.1 manifest, transport-backend-cct:3.0.0 manifest, fragment-ktx:1.2.4 manifest, appcompat:1.3.0 manifest, transport-runtime:3.0.0 manifest, lifecycle-livedata-core-ktx:2.2.0 manifest, firebase-firestore-ktx:23.0.0 manifest, legacy-support-v4:1.0.0 manifest, play-services-basement:17.1.1 manifest, firebase-storage:20.0.0 manifest, play-services-auth-api-phone:17.4.0 manifest, leakcanary-android:2.0-alpha-3 manifest, firebase-auth-interop:20.0.0 manifest, lifecycle-viewmodel:2.3.1 manifest, browser:1.0.0 manifest, firebase-auth:21.0.1 manifest, material:1.2.1 manifest, slidingpanelayout:1.0.0 manifest, vectordrawable:1.1.0 manifest, recyclerview:1.1.0 manifest, play-services-auth:19.0.0 manifest, room-runtime:2.1.0 manifest, dagger-lint-aar:2.35.1 manifest, navigation-dynamic-features-runtime:2.3.5 manifest, play-services-measurement-api:19.0.0 manifest, firebase-encoders-json:18.0.0 manifest, sqlite:2.0.1 manifest, facebook-android-sdk:11.1.0 manifest, firebase-components:17.0.0 manifest, transport-api:3.0.0 manifest, protolite-well-known-types:18.0.0 manifest, markdown-processor:0.1.3 manifest, play-services-measurement-base:19.0.0 manifest, firebase-common-ktx:20.0.0 manifest, activity-ktx:1.3.1 manifest, firebase-crashlytics-ktx:18.0.0 manifest, coil-compose:1.3.1 manifest, multidex:2.0.1 manifest, core-runtime:2.1.0 manifest, fragment-testing:1.2.0 manifest, ui-graphics:1.0.1 manifest, AndroidManifest.xml navigation file, ui-tooling:1.0.0 manifest, grpc-android:1.28.0 manifest, ui-unit:1.0.1 manifest, play-services-measurement:19.0.0 manifest, play:core:1.9.1 manifest, annotation-experimental:1.1.0 manifest, play-services-measurement-sdk-api:19.0.0 manifest, play-services-tasks:17.0.0 manifest, firebase-analytics:19.0.0 manifest, facebook-common:11.1.0 manifest, drawerlayout:1.1.1 manifest, AndroidManifest.xml navigation file, navigation-compose:2.4.0-alpha09 manifest, facebook-gamingservices:11.1.0 manifest, firebase-firestore:23.0.0 manifest, lifecycle-livedata:2.2.0 manifest, legacy-support-core-ui:1.0.0 manifest, test:monitor:1.2.0 manifest, AndroidManifest.xml navigation file, facebook-applinks:11.1.0 manifest, viewpager:1.0.0 manifest, ui-geometry:1.0.1 manifest, lifecycle-runtime-ktx:2.3.1 manifest, constraintlayout:2.0.4 manifest, ui-text:1.0.1 manifest, AndroidManifest.xml navigation file, firebase-installations-interop:17.0.0 manifest, transition:1.3.0 manifest, foundation-layout:1.0.1 manifest, appcompat-resources:1.3.1 manifest, runtime-livedata:1.0.0 manifest, runtime-saveable:1.0.1 manifest, firebase-measurement-connector:19.0.0 manifest, vectordrawable-animated:1.1.0 manifest, main nav_graph.xml navigation file Merging Errors: Error: android:exported needs to be explicitly specified for . Apps targeting Android 12 and higher are required to specify an explicit value for
android:exportedwhen the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details. Dairy.app main manifest (this file) Error: android:exported needs to be explicitly specified for . Apps targeting Android 12 and higher are required to specify an explicit value forandroid:exportedwhen the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details. Dairy.app main manifest (this file) Error: android:exported needs to be explicitly specified for . Apps targeting Android 12 and higher are required to specify an explicit value forandroid:exportedwhen the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details. Dairy.app main manifest (this file)
ANSWER
Answered 2021-Oct-05 at 10:38After the build has failed go to AndroidManifest.xml and in the bottom click merged manifest see which activities which have intent-filter but don't have exported=true attribute. Or you can just get the activities which are giving error.
Add these activities to your App manifest with android:exported="true" and app tools:node="merge" this will add exported attribute to the activities giving error.
Example:
1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android"
3 package="com.xyz.abc">
4
5 <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
6 <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
7
8 <application
9 android:name=".framework.presentation.BaseApplication"
10 android:allowBackup="true"
11 android:icon="@mipmap/ic_launcher"
12 android:label="@string/app_name"
13 android:roundIcon="@mipmap/ic_launcher_round"
14 android:supportsRtl="true"
15 android:theme="@style/AppTheme">
16 <activity android:name="com.xyz.presentation.MainActivity"
17 android:exported="true">
18 <intent-filter>
19 <action android:name="android.intent.action.MAIN" />
20
21 <category android:name="android.intent.category.LAUNCHER" />
22 </intent-filter>
23 </activity>
24 </application>
25
26</manifest>
27 <activity
28 android:name="<activity which is giving error>"
29 android:exported="true"
30 tools:node="merge" />
31You will have to do this once, you can remove this once the library developers update their libs.
QUESTION
Tensorflow setup on RStudio/ R | CentOS
Asked 2022-Feb-11 at 09:36For the last 5 days, I am trying to make Keras/Tensorflow packages work in R. I am using RStudio for installation and have used conda, miniconda, virtualenv but it crashes each time in the end. Installing a library should not be a nightmare especially when we are talking about R (one of the best statistical languages) and TensorFlow (one of the best deep learning libraries). Can someone share a reliable way to install Keras/Tensorflow on CentOS 7?
Following are the steps I am using to install tensorflow in RStudio.
Since RStudio simply crashes each time I run tensorflow::tf_config() I have no way to check what is going wrong.
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41Update 1 The only way RStudio does not crash while installing tensorflow is by executing following steps -
First, I created a new virtual environment using conda
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44Then from within RStudio, I installed reticulate and activated the virtual environment which I earlier created using conda
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50As soon as I try to import tensorflow in RStudio, it loads the library /lib64/libstdc++.so.6 instead of /root/.conda/envs/py38/lib/libstdc++.so.6 and I get the following error -
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65Here is what inside /lib64/libstdc++.so.6
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93To resolve the library issue, I added the path of the correct libstdc++.so.6 library having GLIBCXX_3.4.20 in RStudio.
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94and, also
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95But still I get the same error ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20'. Somehow RStudio still loads /lib64/libstdc++.so.6 first instead of /root/.conda/envs/py38/lib/libstdc++.so.6
Instead of RStudio, if I execute the above steps in the R console, then also I get the exact same error.
Update 2: A solution is posted here
ANSWER
Answered 2022-Jan-16 at 00:08Perhaps my failed attempts will help someone else solve this problem; my approach:
- boot up a clean CentOS 7 vm
- install R and some dependencies
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101- Download and install Anaconda via linux installer script
- Create a new conda env
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105**From within this conda env you can import tensorflow in python without error; now to access tf via R
- install an updated gcc via devtoolset
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107- attempt to use tensorflow in R via the reticulate package
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107scl enable devtoolset-7 R
108install.packages("remotes")
109remotes::install_github('rstudio/reticulate')
110reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
111reticulate::repl_python()
112# This works as expected but the command "import tensorflow" crashes R
113# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
114
115# Also tried:
116install.packages("devtools")
117devtools::install_github('rstudio/tensorflow')
118devtools::install_github('rstudio/keras')
119library(tensorflow)
120install_tensorflow() # "successful"
121tensorflow::tf_config()
122# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
123- try older versions of tensorflow/keras
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107scl enable devtoolset-7 R
108install.packages("remotes")
109remotes::install_github('rstudio/reticulate')
110reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
111reticulate::repl_python()
112# This works as expected but the command "import tensorflow" crashes R
113# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
114
115# Also tried:
116install.packages("devtools")
117devtools::install_github('rstudio/tensorflow')
118devtools::install_github('rstudio/keras')
119library(tensorflow)
120install_tensorflow() # "successful"
121tensorflow::tf_config()
122# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
123devtools::install_github('rstudio/tensorflow@v2.4.0')
124devtools::install_github('rstudio/keras@v2.4.0')
125library(tensorflow)
126tf_config()
127# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
128- Try an updated version of R (v4.0)
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
21 [3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
22 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
23 [7] LC_PAPER=en_US.UTF-8 LC_NAME=C
24 [9] LC_ADDRESS=C LC_TELEPHONE=C
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
26
27attached base packages:
28[1] stats graphics grDevices utils datasets methods base
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000 reticulate_1.22-9000
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7 lattice_0.20-45 png_0.1-7 zeallot_0.1.0
35 [5] rappdirs_0.3.3 grid_3.6.0 R6_2.5.1 jsonlite_1.7.2
36 [9] magrittr_2.0.1 tfruns_1.5.0 rlang_0.4.12 whisker_0.4
37[13] Matrix_1.3-4 generics_0.1.1 tools_3.6.0 compiler_3.6.0
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) :
51 ImportError: Traceback (most recent call last):
52 File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53 from tensorflow.python._pywrap_tensorflow_internal import *
54 File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55 module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions. Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107scl enable devtoolset-7 R
108install.packages("remotes")
109remotes::install_github('rstudio/reticulate')
110reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
111reticulate::repl_python()
112# This works as expected but the command "import tensorflow" crashes R
113# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
114
115# Also tried:
116install.packages("devtools")
117devtools::install_github('rstudio/tensorflow')
118devtools::install_github('rstudio/keras')
119library(tensorflow)
120install_tensorflow() # "successful"
121tensorflow::tf_config()
122# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
123devtools::install_github('rstudio/tensorflow@v2.4.0')
124devtools::install_github('rstudio/keras@v2.4.0')
125library(tensorflow)
126tf_config()
127# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
128# deactivate conda
129sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
130export R_VERSION=4.0.0
131curl -O https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm
132sudo yum install R-${R_VERSION}-1-1.x86_64.rpm
133
134scl enable devtoolset-7 /opt/R/4.0.0/bin/R
135install.packages("devtools")
136devtools::install_github('rstudio/reticulate')
137reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
138reticulate::repl_python()
139# 'import tensorflow' resulted in "core dumped"
140I guess the issue is with R/CentOS, as you can import and use tensorflow via python normally, but I'm not sure what else to try.
I would also like to say that I had no issues with Ubuntu (which is specifically supported by tensorflow, along with macOS and Windows), and I came across these docs that might be some help: https://wiki.hpcc.msu.edu/display/ITH/Installing+TensorFlow+using+anaconda / https://wiki.hpcc.msu.edu/pages/viewpage.action?pageId=22709999
QUESTION
Configuring compilers on Mac M1 (Big Sur, Monterey) for Rcpp and other tools
Asked 2022-Feb-10 at 21:07I'm trying to use packages that require Rcpp in R on my M1 Mac, which I was never able to get up and running after purchasing this computer. I updated it to Monterey in the hope that this would fix some installation issues but it hasn't. I tried running the Rcpp check from this page but I get the following error:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
21> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11I get that it can't "find" gfortran. I installed this release of gfortran for Monterey. When I type which gfortran into Terminal, it returns /opt/homebrew/bin/gfortran. (Maybe this version of gfortran requires Xcode tools that are too new—it says something about 13.2 and when I run clang --version it says 13.0—but I don't see another release of gfortran for Monterey?)
I also appended /opt/homebrew/bin: to PATH in R so it looks like this now:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
121> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13Other things I checked:
- Xcode command line tools is installed (
which clangreturns/usr/bin/clang). - Files
~/.R/Makevarsand~/.Renvirondon't exist.
Here's my session info:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29ANSWER
Answered 2022-Feb-10 at 21:07Background
Currently (2022-02-05), CRAN builds R binaries for Apple silicon using Apple clang (from Command Line Tools for Xcode 12.4) and an experimental build of gfortran.
If you obtain R from CRAN (i.e., here), then you need to replicate CRAN's compiler setup on your system before building R packages that contain C/C++/Fortran code from their sources (and before using Rcpp, etc.). This requirement ensures that your package builds are compatible with R itself.
A further complication is the fact that Apple clang doesn't support OpenMP, so you need to do even more work to compile programs that make use of multithreading. You could circumvent the issue by building R itself and all R packages from sources with LLVM clang, which does support OpenMP, but this approach is onerous and "for experts only". There is another approach that has been tested by a few people, including Simon Urbanek, the maintainer of R for macOS. It is experimental and also "for experts only", but seems to work on my machine and is simpler than trying to build R yourself.
Warning: These instructions come with no warranty and could break at any time. They assume some level of familiarity with C/C++/Fortran program compilation, Makefile syntax, and Unix shells. As usual, sudo at your own risk.
I will try to address compilers and OpenMP support at the same time. I am going to assume that you are starting from nothing. Feel free to skip steps you've already taken, though you might find a fresh start helpful.
I've tested these instructions on a machine running Big Sur, and at least one person has tested them on a machine running Monterey. I would be glad to hear from others.
Download an R binary from CRAN here and install. Be sure to select the binary built for Apple silicon.
Run
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30in Terminal to install the latest release version of Apple's Command Line Tools for Xcode, which includes Apple clang. You can obtain earlier versions from your browser here. The version that you install should not be older than the one that CRAN used to build your R binary.
Download the gfortran binary recommended here and install by unpacking to root:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
33The last command updates a symlink inside of the gfortran installation so that it points to the SDK inside of your Command Line Tools installation.
Download an OpenMP runtime suitable for your Apple clang version here and install by unpacking to root. You can query your Apple clang version with clang --version. For example, I have version 1300.0.29.30, so I did:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
35After unpacking, you should find these files on your system:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
35/usr/local/lib/libomp.dylib
36/usr/local/include/ompt.h
37/usr/local/include/omp.h
38/usr/local/include/omp-tools.h
39Add the following lines to $(HOME)/.R/Makevars, creating the file if necessary.
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
35/usr/local/lib/libomp.dylib
36/usr/local/include/ompt.h
37/usr/local/include/omp.h
38/usr/local/include/omp-tools.h
39CPPFLAGS+=-I/usr/local/include -Xclang -fopenmp
40LDFLAGS+=-L/usr/local/lib -lomp
41
42FC=/opt/R/arm64/gfortran/bin/gfortran -mtune=native
43FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm
44Run R and test that you can compile a program with OpenMP support. For example:
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
35/usr/local/lib/libomp.dylib
36/usr/local/include/ompt.h
37/usr/local/include/omp.h
38/usr/local/include/omp-tools.h
39CPPFLAGS+=-I/usr/local/include -Xclang -fopenmp
40LDFLAGS+=-L/usr/local/lib -lomp
41
42FC=/opt/R/arm64/gfortran/bin/gfortran -mtune=native
43FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm
44if (!requireNamespace("RcppArmadillo", quietly = TRUE)) {
45 install.packages("RcppArmadillo")
46}
47Rcpp::sourceCpp(code = '
48#include <RcppArmadillo.h>
49#ifdef _OPENMP
50# include <omp.h>
51#endif
52
53// [[Rcpp::depends(RcppArmadillo)]]
54// [[Rcpp::export]]
55void omp_test()
56{
57#ifdef _OPENMP
58 Rprintf("OpenMP threads available: %d\\n", omp_get_max_threads());
59#else
60 Rprintf("OpenMP not supported\\n");
61#endif
62}
63')
64omp_test()
651> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include -fPIC -falign-functions=64 -Wall -g -O2 -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") :
10 Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats graphics grDevices utils datasets methods base
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1 tools_4.1.1 RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7
29$ sudo xcode-select --install
30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
35/usr/local/lib/libomp.dylib
36/usr/local/include/ompt.h
37/usr/local/include/omp.h
38/usr/local/include/omp-tools.h
39CPPFLAGS+=-I/usr/local/include -Xclang -fopenmp
40LDFLAGS+=-L/usr/local/lib -lomp
41
42FC=/opt/R/arm64/gfortran/bin/gfortran -mtune=native
43FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm
44if (!requireNamespace("RcppArmadillo", quietly = TRUE)) {
45 install.packages("RcppArmadillo")
46}
47Rcpp::sourceCpp(code = '
48#include <RcppArmadillo.h>
49#ifdef _OPENMP
50# include <omp.h>
51#endif
52
53// [[Rcpp::depends(RcppArmadillo)]]
54// [[Rcpp::export]]
55void omp_test()
56{
57#ifdef _OPENMP
58 Rprintf("OpenMP threads available: %d\\n", omp_get_max_threads());
59#else
60 Rprintf("OpenMP not supported\\n");
61#endif
62}
63')
64omp_test()
65OpenMP threads available: 8
66If the C++ code fails to compile, or if it compiles without error but you get linker warnings or you find that OpenMP is not supported, then something is likely wrong. Please report any issues.
ReferencesEverything is a bit scattered:
QUESTION
Problem with non-standard evaluation in disk.frame objects using data.table syntax
Asked 2022-Feb-01 at 17:47Problem
I'm currently trying to write a function that filters some rows of a disk.frame object using regular expressions. I, unfortunately, run into some issues with the evaluation of my search string in the filter function. My idea was to pass a regular expression as a string into a function argument (e.g. storm_name) and then pass that argument into my filtering call. I used the %like% function included in {data.table} for filtering rows.
My problem is that the storm_name object gets evaluated inside the disk.frame. However, since the storm_name is only included in the function environment, but not in the disk.frame object, I get the following error:
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3I already tried to evaluate the storm_nameobject in the parent frame using eval(sotm_name, env = parent.env()), but that also didn't help. Interestingly, this problem only occurs with {disk.frame} objects but not with {data.table} objects.
For now I found a solution using {dplyr} instead. However, I would be grateful for any ideas on how this problem could be solved with {data.table}.
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23
Session Info
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52ANSWER
Answered 2022-Jan-20 at 17:38While I don't know the exact cause of this, it has to do with environments, search path, etc. For instance, these work:
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59But this does not:
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
60grep2(storms_df, "^A")
61# Error in .checkTypos(e, names_x) :
62# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
63We can work around this with eval(substitute(..)).
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
60grep2(storms_df, "^A")
61# Error in .checkTypos(e, names_x) :
62# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
63grep3 <- function(dfr, storm_name) {
64 eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
65}
66grep3(storms_df, "^A")
67# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
68# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
69# 1: Amy 1975 6 27 0 27.5 -79.0 tropical depression -1 25 1013 NA NA
70# 2: Amy 1975 6 27 6 28.5 -79.0 tropical depression -1 25 1013 NA NA
71# 3: Amy 1975 6 27 12 29.5 -79.0 tropical depression -1 25 1013 NA NA
72# ...
73(and grep3(storms_dt, "^A") works too)
This works by changing the symbol of storm_name inside the [-expression from storm_name to the literal string. Since this is done on the unevaluated expression, there are no lookups yet, no searching through this and inherited environments to find storm_name.
If you check it manually:
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
60grep2(storms_df, "^A")
61# Error in .checkTypos(e, names_x) :
62# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
63grep3 <- function(dfr, storm_name) {
64 eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
65}
66grep3(storms_df, "^A")
67# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
68# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
69# 1: Amy 1975 6 27 0 27.5 -79.0 tropical depression -1 25 1013 NA NA
70# 2: Amy 1975 6 27 6 28.5 -79.0 tropical depression -1 25 1013 NA NA
71# 3: Amy 1975 6 27 12 29.5 -79.0 tropical depression -1 25 1013 NA NA
72# ...
73debug(grep3)
74grep3(storms_df, "^A")
75# debugging in: grep3(storms_df, "^A")
76# debug at #1: {
77# eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
78# }
79# Browse[2]>
80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
81# dfr[name %like% "^A"]
82I think it's something to do with how disk.frame is affecting the environment within [ and the calling/parent environments. Interestingly (to me), you can see that the search path for variables is not empty, it's just not what we would expect:
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
60grep2(storms_df, "^A")
61# Error in .checkTypos(e, names_x) :
62# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
63grep3 <- function(dfr, storm_name) {
64 eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
65}
66grep3(storms_df, "^A")
67# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
68# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
69# 1: Amy 1975 6 27 0 27.5 -79.0 tropical depression -1 25 1013 NA NA
70# 2: Amy 1975 6 27 6 28.5 -79.0 tropical depression -1 25 1013 NA NA
71# 3: Amy 1975 6 27 12 29.5 -79.0 tropical depression -1 25 1013 NA NA
72# ...
73debug(grep3)
74grep3(storms_df, "^A")
75# debugging in: grep3(storms_df, "^A")
76# debug at #1: {
77# eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
78# }
79# Browse[2]>
80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
81# dfr[name %like% "^A"]
82grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
83grep2(storms_df, "^A")
84# Error in .checkTypos(e, names_x) :
85# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
86
87### but let's pre-define `storm_name` outside of the function,
88### then re-define the function (no change)
89storm_name <- "^A"
90grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
91head(grep2(storms_df, "^A"), 2)
92# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
93# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
94# 1: Amy 1975 6 27 0 27.5 -79 tropical depression -1 25 1013 NA NA
95# 2: Amy 1975 6 27 6 28.5 -79 tropical depression -1 25 1013 NA NA
96This seems to work, but we can see that it's using the external version of storm_name vice the parametric version, see that name is still starting with A despite the change to "^B".
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
60grep2(storms_df, "^A")
61# Error in .checkTypos(e, names_x) :
62# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
63grep3 <- function(dfr, storm_name) {
64 eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
65}
66grep3(storms_df, "^A")
67# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
68# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
69# 1: Amy 1975 6 27 0 27.5 -79.0 tropical depression -1 25 1013 NA NA
70# 2: Amy 1975 6 27 6 28.5 -79.0 tropical depression -1 25 1013 NA NA
71# 3: Amy 1975 6 27 12 29.5 -79.0 tropical depression -1 25 1013 NA NA
72# ...
73debug(grep3)
74grep3(storms_df, "^A")
75# debugging in: grep3(storms_df, "^A")
76# debug at #1: {
77# eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
78# }
79# Browse[2]>
80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
81# dfr[name %like% "^A"]
82grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
83grep2(storms_df, "^A")
84# Error in .checkTypos(e, names_x) :
85# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
86
87### but let's pre-define `storm_name` outside of the function,
88### then re-define the function (no change)
89storm_name <- "^A"
90grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
91head(grep2(storms_df, "^A"), 2)
92# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
93# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
94# 1: Amy 1975 6 27 0 27.5 -79 tropical depression -1 25 1013 NA NA
95# 2: Amy 1975 6 27 6 28.5 -79 tropical depression -1 25 1013 NA NA
96head(grep2(storms_df, "^B"), 2)
97# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
98# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
99# 1: Amy 1975 6 27 0 27.5 -79 tropical depression -1 25 1013 NA NA
100# 2: Amy 1975 6 27 6 28.5 -79 tropical depression -1 25 1013 NA NA
101Frankly, I don't understand enough of disk.frame's internals to know if this is a bug or a necessity due to what it must do for non-standard data.table-like evaluation of a not-totally-in-memory dataset.
If you're concerned with performance (fair question), the eval(substitute(..)) method does not appear to suffer much:
1Error in .checkTypos(e, names_x) :
2 Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
3# Load packages
4library(data.table)
5library(disk.frame)
6
7# Create data table and diskframe object of storm data
8storms_df <- as.disk.frame(storms)
9storms_dt <- as.data.table(storms)
10
11# Create search function
12grep_storm_name <- function(dfr, storm_name){
13
14 dfr[name %like% storm_name]
15
16}
17
18# Check function with data.table object
19grep_storm_name(storms_dt, "^A")
20
21# Check function with diskframe object
22grep_storm_name(storms_df, "^A")
23R version 4.1.0 (2021-05-18)
24Platform: x86_64-w64-mingw32/x64 (64-bit)
25Running under: Windows 10 x64 (build 19043)
26
27Matrix products: default
28
29locale:
30[1] LC_COLLATE=English_Sweden.1252 LC_CTYPE=English_Sweden.1252 LC_MONETARY=English_Sweden.1252
31[4] LC_NUMERIC=C LC_TIME=English_Sweden.1252
32
33attached base packages:
34[1] stats graphics grDevices utils datasets methods base
35
36other attached packages:
37[1] disk.frame_0.5.0 purrr_0.3.4 dplyr_1.0.7 data.table_1.14.0
38
39loaded via a namespace (and not attached):
40 [1] Rcpp_1.0.7 benchmarkmeData_1.0.4 pryr_0.1.4 pillar_1.6.4
41 [5] compiler_4.1.0 iterators_1.0.13 tools_4.1.0 digest_0.6.27
42 [9] bit_4.0.4 jsonlite_1.7.2 lifecycle_1.0.1 tibble_3.1.6
43[13] lattice_0.20-44 pkgconfig_2.0.3 rlang_0.4.12 Matrix_1.3-3
44[17] foreach_1.5.1 rstudioapi_0.13 DBI_1.1.1 parallel_4.1.0
45[21] bigassertr_0.1.4 bigreadr_0.2.4 httr_1.4.2 stringr_1.4.0
46[25] globals_0.14.0 generics_0.1.1 fs_1.5.0 vctrs_0.3.8
47[29] bit64_4.0.5 grid_4.1.0 tidyselect_1.1.1 glue_1.6.0
48[33] listenv_0.8.0 R6_2.5.1 future.apply_1.7.0 parallelly_1.25.0
49[37] fansi_1.0.0 magrittr_2.0.1 codetools_0.2-18 ellipsis_0.3.2
50[41] fst_0.9.4 assertthat_0.2.1 future_1.21.0 benchmarkme_1.0.7
51[45] utf8_1.2.2 stringi_1.7.6 doParallel_1.0.16 crayon_1.4.2
52storms_df[name %like% "^A"]
53
54nm <- "^A"
55storms_df[name %like% nm]
56
57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
58grep1(storms_df)
59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
60grep2(storms_df, "^A")
61# Error in .checkTypos(e, names_x) :
62# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
63grep3 <- function(dfr, storm_name) {
64 eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
65}
66grep3(storms_df, "^A")
67# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
68# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
69# 1: Amy 1975 6 27 0 27.5 -79.0 tropical depression -1 25 1013 NA NA
70# 2: Amy 1975 6 27 6 28.5 -79.0 tropical depression -1 25 1013 NA NA
71# 3: Amy 1975 6 27 12 29.5 -79.0 tropical depression -1 25 1013 NA NA
72# ...
73debug(grep3)
74grep3(storms_df, "^A")
75# debugging in: grep3(storms_df, "^A")
76# debug at #1: {
77# eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
78# }
79# Browse[2]>
80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
81# dfr[name %like% "^A"]
82grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
83grep2(storms_df, "^A")
84# Error in .checkTypos(e, names_x) :
85# Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
86
87### but let's pre-define `storm_name` outside of the function,
88### then re-define the function (no change)
89storm_name <- "^A"
90grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
91head(grep2(storms_df, "^A"), 2)
92# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
93# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
94# 1: Amy 1975 6 27 0 27.5 -79 tropical depression -1 25 1013 NA NA
95# 2: Amy 1975 6 27 6 28.5 -79 tropical depression -1 25 1013 NA NA
96head(grep2(storms_df, "^B"), 2)
97# name year month day hour lat long status category wind pressure ts_diameter hu_diameter
98# <char> <num> <num> <int> <num> <num> <num> <char> <ord> <int> <int> <num> <num>
99# 1: Amy 1975 6 27 0 27.5 -79 tropical depression -1 25 1013 NA NA
100# 2: Amy 1975 6 27 6 28.5 -79 tropical depression -1 25 1013 NA NA
101bench::mark(
102 raw = dfr[name %like% "^A"],
103 subst = eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name))),
104 iterations = 1000
105)
106# # A tibble: 2 x 13
107# expression min median `itr/sec` mem_alloc `gc/sec` n_itr n_gc total_time result memory time gc
108# <bch:expr> <bch:tm> <bch:tm> <dbl> <bch:byt> <dbl> <int> <dbl> <bch:tm> <list> <list> <list> <list>
109# 1 raw 12.9ms 16.8ms 55.2 1.69MB 3.97 933 67 16.9s <data.table [990 x 13]> <Rprofmem [669 x 3]> <bench_tm [1,000]> <tibble [1,000 x 3]>
110# 2 subst 12.8ms 15.8ms 60.5 1.69MB 3.25 949 51 15.7s <data.table [990 x 13]> <Rprofmem [669 x 3]> <bench_tm [1,000]> <tibble [1,000 x 3]>
111In repeated benchmarks, I've actually seen subst slightly faster, suggesting that a portion of the performance difference is unrelated to the addition of eval(substitute(..)). This difference (55.2 to 60.5 `itr/sec`) is the worst I've seen it ... a repeat just now had 57.1 and 57.5, so I suggest that performance-degradation is not a concern.
QUESTION
How to automate legends for a new geom in ggplot2?
Asked 2022-Jan-30 at 18:08I've built this new ggplot2 geom layer I'm calling geom_triangles (see https://github.com/ctesta01/ggtriangles/) that plots isosceles triangles given aesthetics including x, y, z where z is the height of the triangle and
the base of the isosceles triangle has midpoint (x,y) on the graph.
What I want is for the geom_triangles() layer to automatically provide legend components for the height and width of the triangles, but I am not sure how to do that.
I understand based on this reference that I may need to adjust the draw_key argument in the ggproto StatTriangles object, but I'm not sure how I would do that and can't seem to find examples online of how to do it. I've been looking at the source code in ggplot2 for the draw_key functions, but I'm not sure how I would introduce multiple legend components (one for each of height and width) in a single draw_key argument in the StatTriangles ggproto.
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130
What I have been able to do is to write helper functions (draw_geom_triangles_height_legend, draw_geom_triangles_width_legend) and use the patchwork, and cowplot packages to make legend components rather manually and combining them in an appropriate grid with the original plot, but I want to make producing these legend components automatic. The following code also uses the ggrepel package to add text labels in the figure.
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223
What I'm looking for is to be able to run the code for the first plot example and get a legend with 3 components similar to the color/fill, height, and width legend components as in the second plot example.
Unfortunately the helper functions are not at all satisfactory because at present one has to rely on visually estimating whether the legend's height_scale and width_scale components look correct. This is because the lengeds produced by draw_geom_triangles_height_legend and draw_geom_triangles_width_legend are their own ggplot objects and therefore aren't necessarily on the same coordinate scaling system as the main ggplot of interest for which they are supposed to be legends.
Both of the plots I included are rendered at 7in x 8.5in using ggsave.
Here's my R sessionInfo()
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246ANSWER
Answered 2022-Jan-30 at 18:08I think you might be slightly overcomplicating things. Ideally, you'd just want a single key drawing method for the whole layer. However, because you're using a Stat to do the majority of calculations, this becomes hairy to implement. In my answer, I'm avoiding this.
Let's say I'd want to use a geom-only implementation of such a layer. I can make the following (simplified) class/constructor pair. Below, I haven't bothered width_scale or height_scale parameters, just for simplicity.
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246library(ggplot2)
247
248GeomTriangles <- ggproto(
249 "GeomTriangles", GeomPoint,
250 default_aes = aes(
251 colour = "black", fill = "black", size = 0.5, linetype = 1,
252 alpha = 1, angle = 0, width = 0.5, height = 0.5
253 ),
254
255 draw_panel = function(
256 data, panel_params, coord, na.rm = FALSE
257 ) {
258 # Apply coordinate transform
259 df <- coord$transform(data, panel_params)
260
261 # Repeat every row 3x
262 idx <- rep(seq_len(nrow(df)), each = 3)
263 rep_df <- df[idx, ]
264 # Calculate offsets from origin
265 x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
266 y_off <- as.vector(outer(c(0, 1, 0), df$height))
267
268 # Rotate offsets
269 ang <- rep_df$angle * (pi / 180)
270 x_new <- x_off * cos(ang) - y_off * sin(ang)
271 y_new <- x_off * sin(ang) + y_off * cos(ang)
272
273 # Combine offsets with origin
274 x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
275 y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
276
277 grid::polygonGrob(
278 x = x, y = y, id = idx,
279 gp = grid::gpar(
280 col = alpha(df$colour, df$alpha),
281 fill = alpha(df$fill, df$alpha),
282 lwd = df$size * .pt,
283 lty = df$linetype
284 )
285 )
286 }
287)
288
Constructor
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246library(ggplot2)
247
248GeomTriangles <- ggproto(
249 "GeomTriangles", GeomPoint,
250 default_aes = aes(
251 colour = "black", fill = "black", size = 0.5, linetype = 1,
252 alpha = 1, angle = 0, width = 0.5, height = 0.5
253 ),
254
255 draw_panel = function(
256 data, panel_params, coord, na.rm = FALSE
257 ) {
258 # Apply coordinate transform
259 df <- coord$transform(data, panel_params)
260
261 # Repeat every row 3x
262 idx <- rep(seq_len(nrow(df)), each = 3)
263 rep_df <- df[idx, ]
264 # Calculate offsets from origin
265 x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
266 y_off <- as.vector(outer(c(0, 1, 0), df$height))
267
268 # Rotate offsets
269 ang <- rep_df$angle * (pi / 180)
270 x_new <- x_off * cos(ang) - y_off * sin(ang)
271 y_new <- x_off * sin(ang) + y_off * cos(ang)
272
273 # Combine offsets with origin
274 x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
275 y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
276
277 grid::polygonGrob(
278 x = x, y = y, id = idx,
279 gp = grid::gpar(
280 col = alpha(df$colour, df$alpha),
281 fill = alpha(df$fill, df$alpha),
282 lwd = df$size * .pt,
283 lty = df$linetype
284 )
285 )
286 }
287)
288geom_triangles <- function(mapping = NULL, data = NULL,
289 position = "identity", na.rm = FALSE, show.legend = NA,
290 inherit.aes = TRUE, ...) {
291 layer(
292 stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
293 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
294 params = list(na.rm = na.rm, ...)
295 )
296}
297
Example
Just to show how it works without any special keys set. I'm letting a continuous scale for width and height take over the job of your width_scale and height_scale parameters, because I didn't want to focus on that here. As you can see, two legends are made automatically, but with the wrong glyphs.
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246library(ggplot2)
247
248GeomTriangles <- ggproto(
249 "GeomTriangles", GeomPoint,
250 default_aes = aes(
251 colour = "black", fill = "black", size = 0.5, linetype = 1,
252 alpha = 1, angle = 0, width = 0.5, height = 0.5
253 ),
254
255 draw_panel = function(
256 data, panel_params, coord, na.rm = FALSE
257 ) {
258 # Apply coordinate transform
259 df <- coord$transform(data, panel_params)
260
261 # Repeat every row 3x
262 idx <- rep(seq_len(nrow(df)), each = 3)
263 rep_df <- df[idx, ]
264 # Calculate offsets from origin
265 x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
266 y_off <- as.vector(outer(c(0, 1, 0), df$height))
267
268 # Rotate offsets
269 ang <- rep_df$angle * (pi / 180)
270 x_new <- x_off * cos(ang) - y_off * sin(ang)
271 y_new <- x_off * sin(ang) + y_off * cos(ang)
272
273 # Combine offsets with origin
274 x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
275 y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
276
277 grid::polygonGrob(
278 x = x, y = y, id = idx,
279 gp = grid::gpar(
280 col = alpha(df$colour, df$alpha),
281 fill = alpha(df$fill, df$alpha),
282 lwd = df$size * .pt,
283 lty = df$linetype
284 )
285 )
286 }
287)
288geom_triangles <- function(mapping = NULL, data = NULL,
289 position = "identity", na.rm = FALSE, show.legend = NA,
290 inherit.aes = TRUE, ...) {
291 layer(
292 stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
293 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
294 params = list(na.rm = na.rm, ...)
295 )
296}
297ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
298 geom_triangles() +
299 geom_point(colour = "black") +
300 continuous_scale("width", "wscale",
301 palette = scales::rescale_pal(c(0.1, 0.5))) +
302 continuous_scale("height", "hscale",
303 palette = scales::rescale_pal(c(0.1, 0.5)))
304
Writing a function to draw a glyph isn't too difficult. In this case, we do almost the same as GeomTriangles$draw_panel, but we fix the x and y positions of the origin, and don't use a coordinate transform.
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246library(ggplot2)
247
248GeomTriangles <- ggproto(
249 "GeomTriangles", GeomPoint,
250 default_aes = aes(
251 colour = "black", fill = "black", size = 0.5, linetype = 1,
252 alpha = 1, angle = 0, width = 0.5, height = 0.5
253 ),
254
255 draw_panel = function(
256 data, panel_params, coord, na.rm = FALSE
257 ) {
258 # Apply coordinate transform
259 df <- coord$transform(data, panel_params)
260
261 # Repeat every row 3x
262 idx <- rep(seq_len(nrow(df)), each = 3)
263 rep_df <- df[idx, ]
264 # Calculate offsets from origin
265 x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
266 y_off <- as.vector(outer(c(0, 1, 0), df$height))
267
268 # Rotate offsets
269 ang <- rep_df$angle * (pi / 180)
270 x_new <- x_off * cos(ang) - y_off * sin(ang)
271 y_new <- x_off * sin(ang) + y_off * cos(ang)
272
273 # Combine offsets with origin
274 x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
275 y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
276
277 grid::polygonGrob(
278 x = x, y = y, id = idx,
279 gp = grid::gpar(
280 col = alpha(df$colour, df$alpha),
281 fill = alpha(df$fill, df$alpha),
282 lwd = df$size * .pt,
283 lty = df$linetype
284 )
285 )
286 }
287)
288geom_triangles <- function(mapping = NULL, data = NULL,
289 position = "identity", na.rm = FALSE, show.legend = NA,
290 inherit.aes = TRUE, ...) {
291 layer(
292 stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
293 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
294 params = list(na.rm = na.rm, ...)
295 )
296}
297ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
298 geom_triangles() +
299 geom_point(colour = "black") +
300 continuous_scale("width", "wscale",
301 palette = scales::rescale_pal(c(0.1, 0.5))) +
302 continuous_scale("height", "hscale",
303 palette = scales::rescale_pal(c(0.1, 0.5)))
304draw_key_triangle <- function(data, params, size) {
305 # browser()
306 idx <- rep(seq_len(nrow(data)), each = 3)
307 rep_data <- data[idx, ]
308
309 x_off <- as.vector(outer(
310 c(-0.5, 0, 0.5),
311 data$width
312 ))
313
314 y_off <- as.vector(outer(
315 c(0, 1, 0),
316 data$height
317 ))
318
319 ang <- rep_data$angle * (pi / 180)
320 x_new <- x_off * cos(ang) - y_off * sin(ang)
321 y_new <- x_off * sin(ang) + y_off * cos(ang)
322
323 # Origin x and y have fixed values
324 x <- unit(0.5, "npc") + unit(x_new, "cm")
325 y <- unit(0.2, "npc") + unit(y_new, "cm")
326
327 grid::polygonGrob(
328 x = x, y = y, id = idx,
329 gp = grid::gpar(
330 col = alpha(data$colour, data$alpha),
331 fill = alpha(data$fill, data$alpha),
332 lwd = data$size * .pt,
333 lty = data$linetype
334 )
335 )
336
337}
338When we now provide this glyph drawing function to the layer, it should draw the correct legends automatically.
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246library(ggplot2)
247
248GeomTriangles <- ggproto(
249 "GeomTriangles", GeomPoint,
250 default_aes = aes(
251 colour = "black", fill = "black", size = 0.5, linetype = 1,
252 alpha = 1, angle = 0, width = 0.5, height = 0.5
253 ),
254
255 draw_panel = function(
256 data, panel_params, coord, na.rm = FALSE
257 ) {
258 # Apply coordinate transform
259 df <- coord$transform(data, panel_params)
260
261 # Repeat every row 3x
262 idx <- rep(seq_len(nrow(df)), each = 3)
263 rep_df <- df[idx, ]
264 # Calculate offsets from origin
265 x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
266 y_off <- as.vector(outer(c(0, 1, 0), df$height))
267
268 # Rotate offsets
269 ang <- rep_df$angle * (pi / 180)
270 x_new <- x_off * cos(ang) - y_off * sin(ang)
271 y_new <- x_off * sin(ang) + y_off * cos(ang)
272
273 # Combine offsets with origin
274 x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
275 y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
276
277 grid::polygonGrob(
278 x = x, y = y, id = idx,
279 gp = grid::gpar(
280 col = alpha(df$colour, df$alpha),
281 fill = alpha(df$fill, df$alpha),
282 lwd = df$size * .pt,
283 lty = df$linetype
284 )
285 )
286 }
287)
288geom_triangles <- function(mapping = NULL, data = NULL,
289 position = "identity", na.rm = FALSE, show.legend = NA,
290 inherit.aes = TRUE, ...) {
291 layer(
292 stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
293 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
294 params = list(na.rm = na.rm, ...)
295 )
296}
297ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
298 geom_triangles() +
299 geom_point(colour = "black") +
300 continuous_scale("width", "wscale",
301 palette = scales::rescale_pal(c(0.1, 0.5))) +
302 continuous_scale("height", "hscale",
303 palette = scales::rescale_pal(c(0.1, 0.5)))
304draw_key_triangle <- function(data, params, size) {
305 # browser()
306 idx <- rep(seq_len(nrow(data)), each = 3)
307 rep_data <- data[idx, ]
308
309 x_off <- as.vector(outer(
310 c(-0.5, 0, 0.5),
311 data$width
312 ))
313
314 y_off <- as.vector(outer(
315 c(0, 1, 0),
316 data$height
317 ))
318
319 ang <- rep_data$angle * (pi / 180)
320 x_new <- x_off * cos(ang) - y_off * sin(ang)
321 y_new <- x_off * sin(ang) + y_off * cos(ang)
322
323 # Origin x and y have fixed values
324 x <- unit(0.5, "npc") + unit(x_new, "cm")
325 y <- unit(0.2, "npc") + unit(y_new, "cm")
326
327 grid::polygonGrob(
328 x = x, y = y, id = idx,
329 gp = grid::gpar(
330 col = alpha(data$colour, data$alpha),
331 fill = alpha(data$fill, data$alpha),
332 lwd = data$size * .pt,
333 lty = data$linetype
334 )
335 )
336
337}
338ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
339 geom_triangles(key_glyph = draw_key_triangle) +
340 geom_point(colour = "black") +
341 continuous_scale("width", "wscale",
342 palette = scales::rescale_pal(c(0.1, 0.5))) +
343 continuous_scale("height", "hscale",
344 palette = scales::rescale_pal(c(0.1, 0.5)))
345
Created on 2022-01-30 by the reprex package (v2.0.1)
The ideal place for the glyph constructor is in the ggproto class. So a final ggproto class could look like:
1library(ggplot2)
2library(magrittr)
3library(dplyr)
4library(ggrepel)
5library(tibble)
6library(cowplot)
7library(patchwork)
8
9StatTriangles <- ggproto("StatTriangles", Stat,
10 required_aes = c('x', 'y', 'z'),
11 compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
12
13 # specify default width
14 if (is.null(data$width)) data$width <- 1
15
16 # for each row of the data, create the 3 points that will make up our
17 # triangle based on the z, width, height_scale, and width_scale given.
18 triangle_df <-
19 tibble::tibble(
20 group = 1:nrow(data),
21 point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
22 point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
23 point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
24 )
25
26 # pivot the data into a long format so that each coordinate pair (e.g. vertex)
27 # will be its own row
28 triangle_df <- triangle_df %>% tidyr::pivot_longer(
29 cols = c(point1, point2, point3),
30 names_to = 'vertex',
31 values_to = 'coordinates'
32 )
33
34 # extract the coordinates -- this must be done rowwise because
35 # coordinates is a list where each element is a c(x,y) coordinate pair
36 triangle_df <- triangle_df %>% rowwise() %>% mutate(
37 x = coordinates[[1]],
38 y = coordinates[[2]])
39
40 # save the original x and y so we can perform rotations by the
41 # given angle with reference to (orig_x, orig_y) as the fixed point
42 # of the rotation transformation
43 triangle_df$orig_x <- rep(data$x, each = 3)
44 triangle_df$orig_y <- rep(data$y, each = 3)
45
46 # i'm not sure exactly why, but if the group isn't interacted with linetype
47 # then the edges of the triangles get messed up when rendered when linetype
48 # is used in an aesthetic
49 # triangle_df$group <-
50 # paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
51
52 # fill in aesthetics to the dataframe
53 triangle_df$colour <- rep(data$colour, each = 3)
54 triangle_df$size <- rep(data$size, each = 3)
55 triangle_df$fill <- rep(data$fill, each = 3)
56 triangle_df$linetype <- rep(data$linetype, each = 3)
57 triangle_df$alpha <- rep(data$alpha, each = 3)
58 triangle_df$angle <- rep(data$angle, each = 3)
59
60 # determine scaling factor in going from y to x
61 # scale_factor <- diff(range(data$x)) / diff(range(data$y))
62 scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
63 if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
64
65 # rotate the data according to the angle by first subtracting out the
66 # (orig_x, orig_y) component, applying coordinate rotations, and then
67 # adding the (orig_x, orig_y) component back in.
68 new_coords <- triangle_df %>% mutate(
69 x_diff = x - orig_x,
70 y_diff = (y - orig_y) * scale_factor,
71 x_new = x_diff * cos(angle) - y_diff * sin(angle),
72 y_new = x_diff * sin(angle) + y_diff * cos(angle),
73 x_new = orig_x + x_new*scale_factor,
74 y_new = (orig_y + y_new)
75 )
76
77 # overwrite the x,y coordinates with the newly computed coordinates
78 triangle_df$x <- new_coords$x_new
79 triangle_df$y <- new_coords$y_new
80
81 triangle_df
82 }
83)
84
85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
86 position = "identity", na.rm = FALSE, show.legend = NA,
87 inherit.aes = TRUE, ...) {
88 layer(
89 stat = StatTriangles, data = data, mapping = mapping, geom = geom,
90 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
91 params = list(na.rm = na.rm, ...)
92 )
93}
94
95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
96 default_aes = aes(
97 color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
98 )
99)
100
101geom_triangles <- function(mapping = NULL, data = NULL,
102 position = "identity", na.rm = FALSE, show.legend = NA,
103 inherit.aes = TRUE, ...) {
104 layer(
105 stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
106 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
107 params = list(na.rm = na.rm, ...)
108 )
109}
110
111# here's an example using mtcars
112
113plt_orig <- mtcars %>%
114 tibble::rownames_to_column('name') %>%
115 ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
116 geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
117 geom_point(color = 'black', size = 1) +
118 ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
119 scale_fill_viridis_c(end = .6) +
120 scale_color_viridis_c(end = .6) +
121 xlab("miles per gallon") +
122 ylab("engine displacement (cu. in.)") +
123 labs(fill = 'horsepower', color = 'horsepower') +
124 ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
125 "Cylinders shown in height, weight in width, horsepower in color") +
126 theme_bw() +
127 theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
128
129plt_orig
130draw_geom_triangles_height_legend <- function(
131 width = 1,
132 width_scale = .1,
133 height_scale = .1,
134 z_values = 1:3,
135 n.breaks = 3,
136 labels = c("low", "medium", "high"),
137 color = 'black',
138 fill = 'black'
139) {
140 ggplot(
141 data = data.frame(x = rep(0, times = n.breaks),
142 y = seq(1,n.breaks),
143 z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
144 width = width,
145 label = labels,
146 color = color,
147 fill = fill
148 ),
149 mapping = aes(x = x, y = y, z = z, label = label, width = width)
150 ) +
151 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
152 geom_text(mapping = aes(x = x + .5), size = 3) +
153 expand_limits(x = c(-.25, 3/4)) +
154 theme_void() +
155 theme(plot.title = element_text(size = 10, hjust = .5))
156}
157
158draw_geom_triangles_width_legend <- function(
159 width = 1:3,
160 width_scale = .1,
161 height_scale = .1,
162 z_values = 1,
163 n.breaks = 3,
164 labels = c("low", "medium", "high"),
165 color = 'black',
166 fill = 'black'
167) {
168 ggplot(
169 data = data.frame(x = rep(0, times = n.breaks),
170 y = seq(1, n.breaks),
171 z = rep(1, n.breaks),
172 width = width,
173 label = labels,
174 color = color,
175 fill = fill
176 ),
177 mapping = aes(x = x, y = y, z = z, label = label, width = width)
178 ) +
179 geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
180 geom_text(mapping = aes(x = x + .5), size = 3) +
181 expand_limits(x = c(-.25, 3/4)) +
182 theme_void() +
183 theme(plot.title = element_text(size = 10, hjust = .5))
184}
185
186# extract the original legend - this is for the color and fill (hp)
187legend_hp <- cowplot::get_legend(plt_orig)
188
189# remove the legend from the plot
190plt <- plt_orig + theme(legend.position = 'none')
191
192# create a height legend using draw_geom_triangles_height_legend
193height_legend <-
194 draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
195 labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
196 ) +
197 ggtitle("cylinders\n")
198
199
200# create a width legend using draw_geom_triangles_width_legend
201width_legend <-
202 draw_geom_triangles_width_legend(
203 width = quantile(mtcars$wt, c(.33, .66, 1)),
204 labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
205 width_scale = .2
206 ) +
207 ggtitle("weight\n(1000 lbs)\n")
208
209blank_plot <- ggplot() + theme_void()
210
211# create a legend column layout
212#
213# whitespace is used above, below, and in-between the legend components to
214# make sure the legend column pieces don't appear too densely stacked.
215#
216legend_component <-
217 (blank_plot / cowplot::plot_grid(legend_hp) / blank_plot / height_legend / blank_plot / width_legend / blank_plot) +
218 plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
219
220# create the layout with the plot and the legend component
221(plt + legend_component) +
222 plot_layout(nrow = 1, widths = c(1, .15))
223> sessionInfo()
224R version 4.1.2 (2021-11-01)
225Platform: x86_64-apple-darwin17.0 (64-bit)
226Running under: macOS Mojave 10.14.2
227
228Matrix products: default
229BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
231
232locale:
233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
234
235attached base packages:
236[1] stats graphics grDevices utils datasets methods base
237
238other attached packages:
239[1] patchwork_1.1.1 cowplot_1.1.1 tibble_3.1.6 ggrepel_0.9.1 dplyr_1.0.7 magrittr_2.0.1 ggplot2_3.3.5 colorout_1.2-2
240
241loaded via a namespace (and not attached):
242 [1] Rcpp_1.0.7 tidyselect_1.1.1 munsell_0.5.0 viridisLite_0.4.0 colorspace_2.0-2 R6_2.5.1 rlang_0.4.12 fansi_0.5.0
243 [9] tools_4.1.2 grid_4.1.2 gtable_0.3.0 utf8_1.2.2 DBI_1.1.2 withr_2.4.3 ellipsis_0.3.2 digest_0.6.29
244[17] yaml_2.2.1 assertthat_0.2.1 lifecycle_1.0.1 crayon_1.4.2 tidyr_1.1.4 farver_2.1.0 purrr_0.3.4 vctrs_0.3.8
245[25] glue_1.6.0 labeling_0.4.2 compiler_4.1.2 pillar_1.6.4 generics_0.1.1 scales_1.1.1 pkgconfig_2.0.3
246library(ggplot2)
247
248GeomTriangles <- ggproto(
249 "GeomTriangles", GeomPoint,
250 default_aes = aes(
251 colour = "black", fill = "black", size = 0.5, linetype = 1,
252 alpha = 1, angle = 0, width = 0.5, height = 0.5
253 ),
254
255 draw_panel = function(
256 data, panel_params, coord, na.rm = FALSE
257 ) {
258 # Apply coordinate transform
259 df <- coord$transform(data, panel_params)
260
261 # Repeat every row 3x
262 idx <- rep(seq_len(nrow(df)), each = 3)
263 rep_df <- df[idx, ]
264 # Calculate offsets from origin
265 x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
266 y_off <- as.vector(outer(c(0, 1, 0), df$height))
267
268 # Rotate offsets
269 ang <- rep_df$angle * (pi / 180)
270 x_new <- x_off * cos(ang) - y_off * sin(ang)
271 y_new <- x_off * sin(ang) + y_off * cos(ang)
272
273 # Combine offsets with origin
274 x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
275 y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
276
277 grid::polygonGrob(
278 x = x, y = y, id = idx,
279 gp = grid::gpar(
280 col = alpha(df$colour, df$alpha),
281 fill = alpha(df$fill, df$alpha),
282 lwd = df$size * .pt,
283 lty = df$linetype
284 )
285 )
286 }
287)
288geom_triangles <- function(mapping = NULL, data = NULL,
289 position = "identity", na.rm = FALSE, show.legend = NA,
290 inherit.aes = TRUE, ...) {
291 layer(
292 stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
293 position = position, show.legend = show.legend, inherit.aes = inherit.aes,
294 params = list(na.rm = na.rm, ...)
295 )
296}
297ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
298 geom_triangles() +
299 geom_point(colour = "black") +
300 continuous_scale("width", "wscale",
301 palette = scales::rescale_pal(c(0.1, 0.5))) +
302 continuous_scale("height", "hscale",
303 palette = scales::rescale_pal(c(0.1, 0.5)))
304draw_key_triangle <- function(data, params, size) {
305 # browser()
306 idx <- rep(seq_len(nrow(data)), each = 3)
307 rep_data <- data[idx, ]
308
309 x_off <- as.vector(outer(
310 c(-0.5, 0, 0.5),
311 data$width
312 ))
313
314 y_off <- as.vector(outer(
315 c(0, 1, 0),
316 data$height
317 ))
318
319 ang <- rep_data$angle * (pi / 180)
320 x_new <- x_off * cos(ang) - y_off * sin(ang)
321 y_new <- x_off * sin(ang) + y_off * cos(ang)
322
323 # Origin x and y have fixed values
324 x <- unit(0.5, "npc") + unit(x_new, "cm")
325 y <- unit(0.2, "npc") + unit(y_new, "cm")
326
327 grid::polygonGrob(
328 x = x, y = y, id = idx,
329 gp = grid::gpar(
330 col = alpha(data$colour, data$alpha),
331 fill = alpha(data$fill, data$alpha),
332 lwd = data$size * .pt,
333 lty = data$linetype
334 )
335 )
336
337}
338ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
339 geom_triangles(key_glyph = draw_key_triangle) +
340 geom_point(colour = "black") +
341 continuous_scale("width", "wscale",
342 palette = scales::rescale_pal(c(0.1, 0.5))) +
343 continuous_scale("height", "hscale",
344 palette = scales::rescale_pal(c(0.1, 0.5)))
345GeomTriangles <- ggproto(
346 "GeomTriangles", GeomPoint,
347 ..., # Whatever you want to put in here
348 draw_key = draw_key_triangle
349)
350Footnote: using scales for width and height isn't generally recommended because it may affect other geoms as well.
QUESTION
Why is WSL extremely slow when compared with native Windows NPM/Yarn processing?
Asked 2022-Jan-06 at 00:43I am working with WSL a lot lately because I need some native UNIX tools (and emulators aren't good enough). I noticed that the speed difference when working with NPM/Yarn is incredible.
I conducted a simple test that confirmed my feelings. The test was running npx create-react-app my-test-app and the WSL result was Done in 287.56s. while GitBash finished with Done in 10.46s..
This is not the whole picture, because the perceived time was higher in both cases, but even based on that - there is a big issue somewhere. I just don't know where. The project I'm working on uses tens of libraries and changing even one of them takes minutes instead of seconds.
Is this something that I can fix? If so - where to look for clues?
Additional info:
my processor: Processor AMD Ryzen 7 5800H with Radeon Graphics, 3201 Mhz, 8 Core(s), 16 Logical Processors
I'm running Windows 11 with all the latest updates to both the system and the WSL. The chosen system is Ubuntu 20.04
I've seen some questions that are somewhat similar like 'npm install' extremely slow on Windows, but they don't touch WSL at all (and my pure Windows NPM works fast).
the issue is not limited to NPM, it's also for Yarn
another problem that I'm getting is that file watching is not happening (I need to restart the server with every change). In some applications I don't get any errors, sometimes I get the following:
1Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/DumpStack.log.tmp'
2Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/hiberfil.sys'
3Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/pagefile.sys'
4Watchpack Error (initial scan): Error: EACCES: permission denied, lstat '/mnt/c/swapfile.sys'
5npm start in an empty (freshly initialized) create-react-app takes ages to render something in the browser in WSL and when executed from GitBash - I can see stuff in 2-4 seconds
it is possible that's it's purely a WSL problem, but it just hurts the most when using NPM/Yarn
ANSWER
Answered 2021-Aug-29 at 15:40Since you mention executing the same files (with proper performance) from within Git Bash, I'm going to make an assumption here. Correct me if I'm wrong on this, and I'll delete the answer and look for another possibility.
This would be explained (and expected) if your files are stored on /mnt/c (a.k.a. C:, or /C under Git Bash) or any other Windows drive, as they would likely need to be to be accessed by Git Bash.
WSL2 uses the 9P protocol to access Windows drives, and it is currently known to be very slow when compared to:
- Native NTFS (obviously)
- The ext4 filesystem on the virtual disk used by WSL2
- And even the performance of WSL1 with Windows drives
I've seen a git clone of a large repo (the WSL2 Linux kernel Github) take 8 minutes on WSL2 on a Windows drive, but only seconds on the root filesystem.
Two possibilities:
If possible (and it is for most Node projects), convert your WSL to version 1 with
wsl --set-version <distroname> 1. I always recommend making a backup withwsl --exportfirst.And since you are making a backup anyway, you may as well just create a copy of the instance by
wsl --importing your backup as--version 1(as the last argument). WSL1 and WSL2 both have their uses, and you may find it helpful to keep both around.See this answer for more details on the exact syntax..
Or just move the project over to somewhere under the WSL root, such as
/home/username/src/.
QUESTION
Window not displaying SDL2
Asked 2021-Dec-28 at 20:47I am using SDL2 for the first time, and when I try to create a window, it's not displaying. The only sight of the window is an icon spawning in my dock (Image of the icon, SDLTest.out is the name of my executable file). I found out that it spawned when SDL_INIT()was called.
I tried updating the window, changing its color and adding the flag SDL_WINDOW_SHOWN, but none of these solutions worked. I even pasted a code from the Internet, but it didn't work better.
Here is the code that I pasted:
1#include <SDL2/SDL.h>
2#include <stdio.h>
3#include <stdlib.h>
4
5
6int main(int argc, char *argv[])
7{
8 SDL_Window *window = NULL;
9 SDL_Renderer *renderer = NULL;
10 int status = EXIT_FAILURE;
11 SDL_Color orange = {255, 127, 40, 255};
12
13 if(0 != SDL_Init(SDL_INIT_VIDEO))
14 {
15 fprintf(stderr, "Error SDL_Init : %s", SDL_GetError());
16 goto Quit;
17 }
18 window = SDL_CreateWindow("SDL2", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED,
19 640, 480, SDL_WINDOW_SHOWN);
20 if(NULL == window)
21 {
22 fprintf(stderr, "Error SDL_CreateWindow : %s", SDL_GetError());
23 goto Quit;
24 }
25 renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
26 if(NULL == renderer)
27 {
28 fprintf(stderr, "Error SDL_CreateRenderer : %s", SDL_GetError());
29 goto Quit;
30 }
31
32 if(0 != SDL_SetRenderDrawColor(renderer, orange.r, orange.g, orange.b, orange.a))
33 {
34 fprintf(stderr, "Error SDL_SetRenderDrawColor : %s", SDL_GetError());
35 goto Quit;
36 }
37
38 if(0 != SDL_RenderClear(renderer))
39 {
40 fprintf(stderr, "Error SDL_SetRenderDrawColor : %s", SDL_GetError());
41 goto Quit;
42 }
43
44 SDL_Delay(500);
45 SDL_RenderPresent(renderer);
46 SDL_Delay(500);
47
48 status = EXIT_SUCCESS;
49
50Quit:
51 if(NULL != renderer)
52 SDL_DestroyRenderer(renderer);
53 if(NULL != window)
54 SDL_DestroyWindow(window);
55 SDL_Quit();
56 return status;
57}
58My OS is MacOS 11.6, my compiler GCC, my computer has a graphic card Intel Iris Pro Graphics 6200, and I installed SDL2 using Homebrew.
Can someone help me?
ANSWER
Answered 2021-Dec-28 at 20:47I just needed an event loop. I added this code and it worked:
1#include <SDL2/SDL.h>
2#include <stdio.h>
3#include <stdlib.h>
4
5
6int main(int argc, char *argv[])
7{
8 SDL_Window *window = NULL;
9 SDL_Renderer *renderer = NULL;
10 int status = EXIT_FAILURE;
11 SDL_Color orange = {255, 127, 40, 255};
12
13 if(0 != SDL_Init(SDL_INIT_VIDEO))
14 {
15 fprintf(stderr, "Error SDL_Init : %s", SDL_GetError());
16 goto Quit;
17 }
18 window = SDL_CreateWindow("SDL2", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED,
19 640, 480, SDL_WINDOW_SHOWN);
20 if(NULL == window)
21 {
22 fprintf(stderr, "Error SDL_CreateWindow : %s", SDL_GetError());
23 goto Quit;
24 }
25 renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED);
26 if(NULL == renderer)
27 {
28 fprintf(stderr, "Error SDL_CreateRenderer : %s", SDL_GetError());
29 goto Quit;
30 }
31
32 if(0 != SDL_SetRenderDrawColor(renderer, orange.r, orange.g, orange.b, orange.a))
33 {
34 fprintf(stderr, "Error SDL_SetRenderDrawColor : %s", SDL_GetError());
35 goto Quit;
36 }
37
38 if(0 != SDL_RenderClear(renderer))
39 {
40 fprintf(stderr, "Error SDL_SetRenderDrawColor : %s", SDL_GetError());
41 goto Quit;
42 }
43
44 SDL_Delay(500);
45 SDL_RenderPresent(renderer);
46 SDL_Delay(500);
47
48 status = EXIT_SUCCESS;
49
50Quit:
51 if(NULL != renderer)
52 SDL_DestroyRenderer(renderer);
53 if(NULL != window)
54 SDL_DestroyWindow(window);
55 SDL_Quit();
56 return status;
57}
58 SDL_bool quit = SDL_FALSE;
59 while(!quit)
60 {
61 SDL_RenderPresent(renderer);
62 SDL_WaitEvent(&event);
63 if(event.type == SDL_QUIT)
64 quit = SDL_TRUE;
65 }
66Thank you to HolyBlackCat for your comment!
QUESTION
Symbol not found in flat namespace '_FT_Done_Face' from reportlab with Python@3.9 on macOS 12
Asked 2021-Dec-20 at 13:45I have a django project using easy-thumbnail as a dependency.
Installing all packages with pip is working as expected, but when I try to run my app I get this error:
1Invalid template library specified. ImportError raised when trying to load 'backend.templatetags.get_thumbnail': dlopen(/opt/homebrew/lib/python3.9/site-packages/reportlab/graphics/_renderPM.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_FT_Done_Face'
2The error is raised from reportlab which is a dependency of easy-thumbnail. As far as I understand, reportlab is not able to find freetype. But it is installed correctly imho.
I'm using macOS 12.0.1
I installed Python and freetype via Homebrew. pkg-config says, that freetype2 is available at the expected paths.
What am I doing wrong? How can I fix this?
Edit
I did otool -l on the failing .so-file and this is, what I get (here I'm running it again in a venv):
1Invalid template library specified. ImportError raised when trying to load 'backend.templatetags.get_thumbnail': dlopen(/opt/homebrew/lib/python3.9/site-packages/reportlab/graphics/_renderPM.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_FT_Done_Face'
2/Users/markusgerards/.pyenv/versions/myapp/lib/python3.9/site-packages/reportlab/graphics/_renderPM.cpython-39-darwin.so:
3 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
4I suspect that freetype should be listed there... right?
ANSWER
Answered 2021-Nov-15 at 14:19I reinstalled reportlab with this command:
1Invalid template library specified. ImportError raised when trying to load 'backend.templatetags.get_thumbnail': dlopen(/opt/homebrew/lib/python3.9/site-packages/reportlab/graphics/_renderPM.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '_FT_Done_Face'
2/Users/markusgerards/.pyenv/versions/myapp/lib/python3.9/site-packages/reportlab/graphics/_renderPM.cpython-39-darwin.so:
3 /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
4pip install reportlab --force-reinstall --no-cache-dir --global-option=build_ext
5This forced pip to actually build the package and now everythings works as intended!
QUESTION
jdeps can't print-module-deps due to a MultiReleaseException
Asked 2021-Dec-13 at 13:36We have a JavaFX based application which is not modularized (there are reasons, a legacy library is involved) but we build an custom runtime using jdeps and jlink.
We've recently rewritten the app and added a couple of new dependencies, as well as removing others. Now the script that is building the application suddenly stopped working during the jdeps call.
Note: This is happening on Linux – I've yet to test other OS'ses, but I don't expect another result.
When the script calls
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2the result is always
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28I couldn't find much in regards to this specific exception and everything I've found so far was not applicable to our situation.
To not wait as long as the build takes to reach this point, I've let the program print out the command before it gets executed and use it on a terminal. Then it get's a bit weirder:
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39Why of all sudden to I get a different exception? Which one is correct now? I have no clue.
Fact is, the jar containing org.slf4j only has the automatic module name.
I really have have no idea, what I should do with little information... If someone else can point to something, I'd be glad.
Thanks, Daniel
PS: The following code prints the command and executes it:
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39echo "detecting required modules"
40CMD="$JDK/bin/jdeps -q --multi-release ${JAVA_VERSION} --ignore-missing-deps --print-module-deps --class-path ${OUT_LIBS}/* ${MAIN_CLASS_FILE}"; echo "$CMD"
41detected_modules=$("$JDK"/bin/jdeps -q \
42 --multi-release ${JAVA_VERSION} \
43 --ignore-missing-deps \
44 --print-module-deps \
45 --class-path "${OUT_LIBS}/*" \
46 "${MAIN_CLASS_FILE}") || exit
47echo "detected modules: ${detected_modules}"
48They really seem to create different result... 🤷
PPS: If I remove the --multi-release part, I get a different error, that jackson is multi-release, but I need to specify what I want...
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39echo "detecting required modules"
40CMD="$JDK/bin/jdeps -q --multi-release ${JAVA_VERSION} --ignore-missing-deps --print-module-deps --class-path ${OUT_LIBS}/* ${MAIN_CLASS_FILE}"; echo "$CMD"
41detected_modules=$("$JDK"/bin/jdeps -q \
42 --multi-release ${JAVA_VERSION} \
43 --ignore-missing-deps \
44 --print-module-deps \
45 --class-path "${OUT_LIBS}/*" \
46 "${MAIN_CLASS_FILE}") || exit
47echo "detected modules: ${detected_modules}"
48Error: jackson-core-2.13.0.jar is a multi-release jar file but --multi-release option is not set
49Edit #1
In the pom file, we have the following deps
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39echo "detecting required modules"
40CMD="$JDK/bin/jdeps -q --multi-release ${JAVA_VERSION} --ignore-missing-deps --print-module-deps --class-path ${OUT_LIBS}/* ${MAIN_CLASS_FILE}"; echo "$CMD"
41detected_modules=$("$JDK"/bin/jdeps -q \
42 --multi-release ${JAVA_VERSION} \
43 --ignore-missing-deps \
44 --print-module-deps \
45 --class-path "${OUT_LIBS}/*" \
46 "${MAIN_CLASS_FILE}") || exit
47echo "detected modules: ${detected_modules}"
48Error: jackson-core-2.13.0.jar is a multi-release jar file but --multi-release option is not set
49<dependency>
50 <groupId>org.openjfx</groupId>
51 <artifactId>javafx-base</artifactId>
52 <version>${use.javafx.version}</version>
53</dependency>
54<dependency>
55 <groupId>org.openjfx</groupId>
56 <artifactId>javafx-controls</artifactId>
57 <version>${use.javafx.version}</version>
58</dependency>
59<dependency>
60 <groupId>org.openjfx</groupId>
61 <artifactId>javafx-fxml</artifactId>
62 <version>${use.javafx.version}</version>
63</dependency>
64<dependency>
65 <groupId>org.openjfx</groupId>
66 <artifactId>javafx-graphics</artifactId>
67 <version>${use.javafx.version}</version>
68</dependency>
69<dependency>
70 <groupId>org.openjfx</groupId>
71 <artifactId>javafx-web</artifactId>
72 <version>${use.javafx.version}</version>
73</dependency>
74
75<dependency>
76 <groupId>org.jetbrains.kotlin</groupId>
77 <artifactId>kotlin-stdlib-jdk8</artifactId>
78 <version>${kotlin.version}</version>
79</dependency>
80<dependency>
81 <groupId>org.jetbrains.kotlin</groupId>
82 <artifactId>kotlin-reflect</artifactId>
83 <version>${kotlin.version}</version>
84</dependency>
85<dependency>
86 <groupId>org.jetbrains.kotlin</groupId>
87 <artifactId>kotlin-test-junit</artifactId>
88 <version>${kotlin.version}</version>
89 <scope>test</scope>
90</dependency>
91<dependency>
92 <groupId>org.jetbrains.kotlinx</groupId>
93 <artifactId>kotlinx-coroutines-core</artifactId>
94 <version>${kotlinx.coroutines.version}</version>
95</dependency>
96<dependency>
97 <groupId>org.jetbrains.kotlinx</groupId>
98 <artifactId>kotlinx-coroutines-jdk8</artifactId>
99 <version>${kotlinx.coroutines.version}</version>
100</dependency>
101
102<dependency>
103 <groupId>io.insert-koin</groupId>
104 <artifactId>koin-core-jvm</artifactId>
105 <version>${koin.version}</version>
106</dependency>
107<dependency>
108 <groupId>io.insert-koin</groupId>
109 <artifactId>koin-test-jvm</artifactId>
110 <version>${koin.version}</version>
111</dependency>
112
113<dependency>
114 <groupId>org.simpleframework</groupId>
115 <artifactId>simple-xml</artifactId>
116 <version>${simplexml.version}</version>
117</dependency>
118
119<dependency>
120 <groupId>org.apache.logging.log4j</groupId>
121 <artifactId>log4j-api</artifactId>
122 <version>${log4j.version}</version>
123</dependency>
124<dependency>
125 <groupId>org.apache.logging.log4j</groupId>
126 <artifactId>log4j-core</artifactId>
127 <version>${log4j.version}</version>
128</dependency>
129<dependency>
130 <groupId>org.apache.logging.log4j</groupId>
131 <artifactId>log4j-web</artifactId>
132 <version>${log4j.version}</version>
133</dependency>
134<dependency>
135 <groupId>com.fasterxml.jackson.core</groupId>
136 <artifactId>jackson-annotations</artifactId>
137 <version>${jackson.version}</version>
138</dependency>
139<dependency>
140 <groupId>com.fasterxml.jackson.core</groupId>
141 <artifactId>jackson-core</artifactId>
142 <version>${jackson.version}</version>
143</dependency>
144<dependency>
145 <groupId>com.fasterxml.jackson.core</groupId>
146 <artifactId>jackson-databind</artifactId>
147 <version>${jackson.version}</version>
148</dependency>
149<dependency>
150 <groupId>com.fasterxml.jackson.module</groupId>
151 <artifactId>jackson-module-kotlin</artifactId>
152 <version>${jackson.version}</version>
153</dependency>
154<dependency>
155 <groupId>com.squareup.okhttp3</groupId>
156 <artifactId>okhttp</artifactId>
157 <version>${okhttp.version}</version>
158</dependency>
159<dependency>
160 <groupId>net.sf.proguard</groupId>
161 <artifactId>proguard-base</artifactId>
162 <version>${proguard.version}</version>
163 <scope>runtime</scope>
164</dependency>
165<dependency>
166 <groupId>org.junit.jupiter</groupId>
167 <artifactId>junit-jupiter-api</artifactId>
168 <version>${junit.version}</version>
169 <scope>test</scope>
170</dependency>
171<dependency>
172 <groupId>org.junit.jupiter</groupId>
173 <artifactId>junit-jupiter-engine</artifactId>
174 <version>${junit.version}</version>
175 <scope>test</scope>
176</dependency>
177
178<dependency>
179 <groupId>org.kordamp.ikonli</groupId>
180 <artifactId>ikonli-materialdesign2-pack</artifactId>
181 <version>${ikonli.mdi2.version}</version>
182</dependency>
183<dependency>
184 <groupId>org.kordamp.ikonli</groupId>
185 <artifactId>ikonli-javafx</artifactId>
186 <version>${ikonli.version}</version>
187</dependency>
188<dependency>
189 <groupId>org.controlsfx</groupId>
190 <artifactId>controlsfx</artifactId>
191 <version>${controlsfx.version}</version>
192</dependency>
193<dependency>
194 <groupId>com.dlsc</groupId>
195 <artifactId>GMapsFX</artifactId>
196 <version>${gmapfx.version}</version>
197</dependency>
198
199<dependency>
200 <groupId>org.xerial</groupId>
201 <artifactId>sqlite-jdbc</artifactId>
202 <version>${sqlite.jdbc.version}</version>
203</dependency>
204
205<!-- some custom deps from our company -->
206
207<dependency>
208 <groupId>org.conscrypt</groupId>
209 <artifactId>conscrypt-openjdk</artifactId>
210 <version>${conscrypt.version}</version>
211 <classifier>${os.detected.classifier}</classifier>
212</dependency>
213Resulting in the following JAR files (minus our own):
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39echo "detecting required modules"
40CMD="$JDK/bin/jdeps -q --multi-release ${JAVA_VERSION} --ignore-missing-deps --print-module-deps --class-path ${OUT_LIBS}/* ${MAIN_CLASS_FILE}"; echo "$CMD"
41detected_modules=$("$JDK"/bin/jdeps -q \
42 --multi-release ${JAVA_VERSION} \
43 --ignore-missing-deps \
44 --print-module-deps \
45 --class-path "${OUT_LIBS}/*" \
46 "${MAIN_CLASS_FILE}") || exit
47echo "detected modules: ${detected_modules}"
48Error: jackson-core-2.13.0.jar is a multi-release jar file but --multi-release option is not set
49<dependency>
50 <groupId>org.openjfx</groupId>
51 <artifactId>javafx-base</artifactId>
52 <version>${use.javafx.version}</version>
53</dependency>
54<dependency>
55 <groupId>org.openjfx</groupId>
56 <artifactId>javafx-controls</artifactId>
57 <version>${use.javafx.version}</version>
58</dependency>
59<dependency>
60 <groupId>org.openjfx</groupId>
61 <artifactId>javafx-fxml</artifactId>
62 <version>${use.javafx.version}</version>
63</dependency>
64<dependency>
65 <groupId>org.openjfx</groupId>
66 <artifactId>javafx-graphics</artifactId>
67 <version>${use.javafx.version}</version>
68</dependency>
69<dependency>
70 <groupId>org.openjfx</groupId>
71 <artifactId>javafx-web</artifactId>
72 <version>${use.javafx.version}</version>
73</dependency>
74
75<dependency>
76 <groupId>org.jetbrains.kotlin</groupId>
77 <artifactId>kotlin-stdlib-jdk8</artifactId>
78 <version>${kotlin.version}</version>
79</dependency>
80<dependency>
81 <groupId>org.jetbrains.kotlin</groupId>
82 <artifactId>kotlin-reflect</artifactId>
83 <version>${kotlin.version}</version>
84</dependency>
85<dependency>
86 <groupId>org.jetbrains.kotlin</groupId>
87 <artifactId>kotlin-test-junit</artifactId>
88 <version>${kotlin.version}</version>
89 <scope>test</scope>
90</dependency>
91<dependency>
92 <groupId>org.jetbrains.kotlinx</groupId>
93 <artifactId>kotlinx-coroutines-core</artifactId>
94 <version>${kotlinx.coroutines.version}</version>
95</dependency>
96<dependency>
97 <groupId>org.jetbrains.kotlinx</groupId>
98 <artifactId>kotlinx-coroutines-jdk8</artifactId>
99 <version>${kotlinx.coroutines.version}</version>
100</dependency>
101
102<dependency>
103 <groupId>io.insert-koin</groupId>
104 <artifactId>koin-core-jvm</artifactId>
105 <version>${koin.version}</version>
106</dependency>
107<dependency>
108 <groupId>io.insert-koin</groupId>
109 <artifactId>koin-test-jvm</artifactId>
110 <version>${koin.version}</version>
111</dependency>
112
113<dependency>
114 <groupId>org.simpleframework</groupId>
115 <artifactId>simple-xml</artifactId>
116 <version>${simplexml.version}</version>
117</dependency>
118
119<dependency>
120 <groupId>org.apache.logging.log4j</groupId>
121 <artifactId>log4j-api</artifactId>
122 <version>${log4j.version}</version>
123</dependency>
124<dependency>
125 <groupId>org.apache.logging.log4j</groupId>
126 <artifactId>log4j-core</artifactId>
127 <version>${log4j.version}</version>
128</dependency>
129<dependency>
130 <groupId>org.apache.logging.log4j</groupId>
131 <artifactId>log4j-web</artifactId>
132 <version>${log4j.version}</version>
133</dependency>
134<dependency>
135 <groupId>com.fasterxml.jackson.core</groupId>
136 <artifactId>jackson-annotations</artifactId>
137 <version>${jackson.version}</version>
138</dependency>
139<dependency>
140 <groupId>com.fasterxml.jackson.core</groupId>
141 <artifactId>jackson-core</artifactId>
142 <version>${jackson.version}</version>
143</dependency>
144<dependency>
145 <groupId>com.fasterxml.jackson.core</groupId>
146 <artifactId>jackson-databind</artifactId>
147 <version>${jackson.version}</version>
148</dependency>
149<dependency>
150 <groupId>com.fasterxml.jackson.module</groupId>
151 <artifactId>jackson-module-kotlin</artifactId>
152 <version>${jackson.version}</version>
153</dependency>
154<dependency>
155 <groupId>com.squareup.okhttp3</groupId>
156 <artifactId>okhttp</artifactId>
157 <version>${okhttp.version}</version>
158</dependency>
159<dependency>
160 <groupId>net.sf.proguard</groupId>
161 <artifactId>proguard-base</artifactId>
162 <version>${proguard.version}</version>
163 <scope>runtime</scope>
164</dependency>
165<dependency>
166 <groupId>org.junit.jupiter</groupId>
167 <artifactId>junit-jupiter-api</artifactId>
168 <version>${junit.version}</version>
169 <scope>test</scope>
170</dependency>
171<dependency>
172 <groupId>org.junit.jupiter</groupId>
173 <artifactId>junit-jupiter-engine</artifactId>
174 <version>${junit.version}</version>
175 <scope>test</scope>
176</dependency>
177
178<dependency>
179 <groupId>org.kordamp.ikonli</groupId>
180 <artifactId>ikonli-materialdesign2-pack</artifactId>
181 <version>${ikonli.mdi2.version}</version>
182</dependency>
183<dependency>
184 <groupId>org.kordamp.ikonli</groupId>
185 <artifactId>ikonli-javafx</artifactId>
186 <version>${ikonli.version}</version>
187</dependency>
188<dependency>
189 <groupId>org.controlsfx</groupId>
190 <artifactId>controlsfx</artifactId>
191 <version>${controlsfx.version}</version>
192</dependency>
193<dependency>
194 <groupId>com.dlsc</groupId>
195 <artifactId>GMapsFX</artifactId>
196 <version>${gmapfx.version}</version>
197</dependency>
198
199<dependency>
200 <groupId>org.xerial</groupId>
201 <artifactId>sqlite-jdbc</artifactId>
202 <version>${sqlite.jdbc.version}</version>
203</dependency>
204
205<!-- some custom deps from our company -->
206
207<dependency>
208 <groupId>org.conscrypt</groupId>
209 <artifactId>conscrypt-openjdk</artifactId>
210 <version>${conscrypt.version}</version>
211 <classifier>${os.detected.classifier}</classifier>
212</dependency>
213annotations-13.0.jar
214apiguardian-api-1.1.2.jar
215conscrypt-openjdk-2.5.2-linux-x86_64.jar
216controlsfx-11.1.0.jar
217GMapsFX-11.0.2.jar
218hamcrest-core-1.3.jar
219ikonli-core-12.2.0.jar
220ikonli-javafx-12.2.0.jar
221ikonli-materialdesign2-pack-12.2.0.jar
222jackson-annotations-2.13.0.jar
223jackson-core-2.13.0.jar
224jackson-databind-2.13.0.jar
225jackson-module-kotlin-2.13.0.jar
226jakarta.activation-1.2.2.jar
227jakarta.activation-api-1.2.2.jar
228jakarta.xml.bind-api-2.3.3.jar
229jaxb-impl-2.3.3.jar
230junit-4.12.jar
231junit-jupiter-api-5.8.1.jar
232junit-jupiter-engine-5.8.1.jar
233junit-platform-commons-1.8.1.jar
234junit-platform-engine-1.8.1.jar
235koin-core-jvm-3.1.3.jar
236koin-test-jvm-3.1.3.jar
237kotlin-reflect-1.5.31.jar
238kotlin-stdlib-1.5.31.jar
239kotlin-stdlib-common-1.5.30.jar
240kotlin-stdlib-jdk7-1.5.31.jar
241kotlin-stdlib-jdk8-1.5.31.jar
242kotlin-test-1.5.31.jar
243kotlin-test-annotations-common-1.5.30.jar
244kotlin-test-common-1.5.30.jar
245kotlin-test-junit-1.5.31.jar
246kotlinx-coroutines-core-1.5.2.jar
247kotlinx-coroutines-core-jvm-1.5.2.jar
248kotlinx-coroutines-jdk8-1.5.2.jar
249log4j-api-2.14.1.jar
250log4j-core-2.14.1.jar
251log4j-web-2.14.1.jar
252logback-classic-1.2.3.jar
253logback-core-1.2.3.jar
254okhttp-4.9.2.jar
255okio-2.8.0.jar
256opentest4j-1.2.0.jar
257proguard-base-6.2.2.jar
258simple-xml-2.7.1.jar
259slf4j-api-1.7.29.jar
260sqlite-jdbc-3.36.0.3.jar
261stax-1.2.0.jar
262stax-api-1.0.1.jar
263xpp3-1.1.3.3.jar
264The Java version in use is Bell Soft's Full Liberica JDK 17, we used all recent versions since 14 or 15 where it worked well prior to the rebuild of the UI.
edit #2:
The result of
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39echo "detecting required modules"
40CMD="$JDK/bin/jdeps -q --multi-release ${JAVA_VERSION} --ignore-missing-deps --print-module-deps --class-path ${OUT_LIBS}/* ${MAIN_CLASS_FILE}"; echo "$CMD"
41detected_modules=$("$JDK"/bin/jdeps -q \
42 --multi-release ${JAVA_VERSION} \
43 --ignore-missing-deps \
44 --print-module-deps \
45 --class-path "${OUT_LIBS}/*" \
46 "${MAIN_CLASS_FILE}") || exit
47echo "detected modules: ${detected_modules}"
48Error: jackson-core-2.13.0.jar is a multi-release jar file but --multi-release option is not set
49<dependency>
50 <groupId>org.openjfx</groupId>
51 <artifactId>javafx-base</artifactId>
52 <version>${use.javafx.version}</version>
53</dependency>
54<dependency>
55 <groupId>org.openjfx</groupId>
56 <artifactId>javafx-controls</artifactId>
57 <version>${use.javafx.version}</version>
58</dependency>
59<dependency>
60 <groupId>org.openjfx</groupId>
61 <artifactId>javafx-fxml</artifactId>
62 <version>${use.javafx.version}</version>
63</dependency>
64<dependency>
65 <groupId>org.openjfx</groupId>
66 <artifactId>javafx-graphics</artifactId>
67 <version>${use.javafx.version}</version>
68</dependency>
69<dependency>
70 <groupId>org.openjfx</groupId>
71 <artifactId>javafx-web</artifactId>
72 <version>${use.javafx.version}</version>
73</dependency>
74
75<dependency>
76 <groupId>org.jetbrains.kotlin</groupId>
77 <artifactId>kotlin-stdlib-jdk8</artifactId>
78 <version>${kotlin.version}</version>
79</dependency>
80<dependency>
81 <groupId>org.jetbrains.kotlin</groupId>
82 <artifactId>kotlin-reflect</artifactId>
83 <version>${kotlin.version}</version>
84</dependency>
85<dependency>
86 <groupId>org.jetbrains.kotlin</groupId>
87 <artifactId>kotlin-test-junit</artifactId>
88 <version>${kotlin.version}</version>
89 <scope>test</scope>
90</dependency>
91<dependency>
92 <groupId>org.jetbrains.kotlinx</groupId>
93 <artifactId>kotlinx-coroutines-core</artifactId>
94 <version>${kotlinx.coroutines.version}</version>
95</dependency>
96<dependency>
97 <groupId>org.jetbrains.kotlinx</groupId>
98 <artifactId>kotlinx-coroutines-jdk8</artifactId>
99 <version>${kotlinx.coroutines.version}</version>
100</dependency>
101
102<dependency>
103 <groupId>io.insert-koin</groupId>
104 <artifactId>koin-core-jvm</artifactId>
105 <version>${koin.version}</version>
106</dependency>
107<dependency>
108 <groupId>io.insert-koin</groupId>
109 <artifactId>koin-test-jvm</artifactId>
110 <version>${koin.version}</version>
111</dependency>
112
113<dependency>
114 <groupId>org.simpleframework</groupId>
115 <artifactId>simple-xml</artifactId>
116 <version>${simplexml.version}</version>
117</dependency>
118
119<dependency>
120 <groupId>org.apache.logging.log4j</groupId>
121 <artifactId>log4j-api</artifactId>
122 <version>${log4j.version}</version>
123</dependency>
124<dependency>
125 <groupId>org.apache.logging.log4j</groupId>
126 <artifactId>log4j-core</artifactId>
127 <version>${log4j.version}</version>
128</dependency>
129<dependency>
130 <groupId>org.apache.logging.log4j</groupId>
131 <artifactId>log4j-web</artifactId>
132 <version>${log4j.version}</version>
133</dependency>
134<dependency>
135 <groupId>com.fasterxml.jackson.core</groupId>
136 <artifactId>jackson-annotations</artifactId>
137 <version>${jackson.version}</version>
138</dependency>
139<dependency>
140 <groupId>com.fasterxml.jackson.core</groupId>
141 <artifactId>jackson-core</artifactId>
142 <version>${jackson.version}</version>
143</dependency>
144<dependency>
145 <groupId>com.fasterxml.jackson.core</groupId>
146 <artifactId>jackson-databind</artifactId>
147 <version>${jackson.version}</version>
148</dependency>
149<dependency>
150 <groupId>com.fasterxml.jackson.module</groupId>
151 <artifactId>jackson-module-kotlin</artifactId>
152 <version>${jackson.version}</version>
153</dependency>
154<dependency>
155 <groupId>com.squareup.okhttp3</groupId>
156 <artifactId>okhttp</artifactId>
157 <version>${okhttp.version}</version>
158</dependency>
159<dependency>
160 <groupId>net.sf.proguard</groupId>
161 <artifactId>proguard-base</artifactId>
162 <version>${proguard.version}</version>
163 <scope>runtime</scope>
164</dependency>
165<dependency>
166 <groupId>org.junit.jupiter</groupId>
167 <artifactId>junit-jupiter-api</artifactId>
168 <version>${junit.version}</version>
169 <scope>test</scope>
170</dependency>
171<dependency>
172 <groupId>org.junit.jupiter</groupId>
173 <artifactId>junit-jupiter-engine</artifactId>
174 <version>${junit.version}</version>
175 <scope>test</scope>
176</dependency>
177
178<dependency>
179 <groupId>org.kordamp.ikonli</groupId>
180 <artifactId>ikonli-materialdesign2-pack</artifactId>
181 <version>${ikonli.mdi2.version}</version>
182</dependency>
183<dependency>
184 <groupId>org.kordamp.ikonli</groupId>
185 <artifactId>ikonli-javafx</artifactId>
186 <version>${ikonli.version}</version>
187</dependency>
188<dependency>
189 <groupId>org.controlsfx</groupId>
190 <artifactId>controlsfx</artifactId>
191 <version>${controlsfx.version}</version>
192</dependency>
193<dependency>
194 <groupId>com.dlsc</groupId>
195 <artifactId>GMapsFX</artifactId>
196 <version>${gmapfx.version}</version>
197</dependency>
198
199<dependency>
200 <groupId>org.xerial</groupId>
201 <artifactId>sqlite-jdbc</artifactId>
202 <version>${sqlite.jdbc.version}</version>
203</dependency>
204
205<!-- some custom deps from our company -->
206
207<dependency>
208 <groupId>org.conscrypt</groupId>
209 <artifactId>conscrypt-openjdk</artifactId>
210 <version>${conscrypt.version}</version>
211 <classifier>${os.detected.classifier}</classifier>
212</dependency>
213annotations-13.0.jar
214apiguardian-api-1.1.2.jar
215conscrypt-openjdk-2.5.2-linux-x86_64.jar
216controlsfx-11.1.0.jar
217GMapsFX-11.0.2.jar
218hamcrest-core-1.3.jar
219ikonli-core-12.2.0.jar
220ikonli-javafx-12.2.0.jar
221ikonli-materialdesign2-pack-12.2.0.jar
222jackson-annotations-2.13.0.jar
223jackson-core-2.13.0.jar
224jackson-databind-2.13.0.jar
225jackson-module-kotlin-2.13.0.jar
226jakarta.activation-1.2.2.jar
227jakarta.activation-api-1.2.2.jar
228jakarta.xml.bind-api-2.3.3.jar
229jaxb-impl-2.3.3.jar
230junit-4.12.jar
231junit-jupiter-api-5.8.1.jar
232junit-jupiter-engine-5.8.1.jar
233junit-platform-commons-1.8.1.jar
234junit-platform-engine-1.8.1.jar
235koin-core-jvm-3.1.3.jar
236koin-test-jvm-3.1.3.jar
237kotlin-reflect-1.5.31.jar
238kotlin-stdlib-1.5.31.jar
239kotlin-stdlib-common-1.5.30.jar
240kotlin-stdlib-jdk7-1.5.31.jar
241kotlin-stdlib-jdk8-1.5.31.jar
242kotlin-test-1.5.31.jar
243kotlin-test-annotations-common-1.5.30.jar
244kotlin-test-common-1.5.30.jar
245kotlin-test-junit-1.5.31.jar
246kotlinx-coroutines-core-1.5.2.jar
247kotlinx-coroutines-core-jvm-1.5.2.jar
248kotlinx-coroutines-jdk8-1.5.2.jar
249log4j-api-2.14.1.jar
250log4j-core-2.14.1.jar
251log4j-web-2.14.1.jar
252logback-classic-1.2.3.jar
253logback-core-1.2.3.jar
254okhttp-4.9.2.jar
255okio-2.8.0.jar
256opentest4j-1.2.0.jar
257proguard-base-6.2.2.jar
258simple-xml-2.7.1.jar
259slf4j-api-1.7.29.jar
260sqlite-jdbc-3.36.0.3.jar
261stax-1.2.0.jar
262stax-api-1.0.1.jar
263xpp3-1.1.3.3.jar
264mvn compile org.apache.maven.plugins:maven-dependency-plugin:3.1.1:resolve -DexcludeTransitive
265is
1~/path/to/jdk/bin/jdeps -q --multi-release 11 --ignore-missing-deps --print-module-deps --class-path ~/path/to/app/target/package/libs/* target/classes/ch/cnlab/uxtest/MainKt.class
2Exception in thread "main" java.lang.Error: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
3 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:271)
4 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.parse(DependencyFinder.java:133)
5 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.transitiveArchiveDeps(DepsAnalyzer.java:217)
6 at jdk.jdeps/com.sun.tools.jdeps.DepsAnalyzer.run(DepsAnalyzer.java:138)
7 at jdk.jdeps/com.sun.tools.jdeps.ModuleExportsAnalyzer.run(ModuleExportsAnalyzer.java:74)
8 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask$ListModuleDeps.run(JdepsTask.java:1047)
9 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:574)
10 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
11 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
12Caused by: java.util.concurrent.ExecutionException: com.sun.tools.jdeps.MultiReleaseException
13 at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
14 at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
15 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.waitForTasksCompleted(DependencyFinder.java:267)
16 ... 8 more
17Caused by: com.sun.tools.jdeps.MultiReleaseException
18 at jdk.jdeps/com.sun.tools.jdeps.VersionHelper.add(VersionHelper.java:62)
19 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileReader.readClassFile(ClassFileReader.java:360)
20 at jdk.jdeps/com.sun.tools.jdeps.ClassFileReader$JarFileIterator.hasNext(ClassFileReader.java:402)
21 at jdk.jdeps/com.sun.tools.jdeps.DependencyFinder.lambda$parse$5(DependencyFinder.java:179)
22 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
24 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
25 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
26 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
27 at java.base/java.lang.Thread.run(Thread.java:833)
28Exception in thread "main" java.lang.module.FindException: Module org.slf4j not found, required by com.dlsc.gmapsfx
29 at java.base/java.lang.module.Resolver.findFail(Resolver.java:893)
30 at java.base/java.lang.module.Resolver.resolve(Resolver.java:192)
31 at java.base/java.lang.module.Resolver.resolve(Resolver.java:141)
32 at java.base/java.lang.module.Configuration.resolve(Configuration.java:421)
33 at java.base/java.lang.module.Configuration.resolve(Configuration.java:255)
34 at jdk.jdeps/com.sun.tools.jdeps.JdepsConfiguration$Builder.build(JdepsConfiguration.java:564)
35 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.buildConfig(JdepsTask.java:603)
36 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:557)
37 at jdk.jdeps/com.sun.tools.jdeps.JdepsTask.run(JdepsTask.java:533)
38 at jdk.jdeps/com.sun.tools.jdeps.Main.main(Main.java:49)
39echo "detecting required modules"
40CMD="$JDK/bin/jdeps -q --multi-release ${JAVA_VERSION} --ignore-missing-deps --print-module-deps --class-path ${OUT_LIBS}/* ${MAIN_CLASS_FILE}"; echo "$CMD"
41detected_modules=$("$JDK"/bin/jdeps -q \
42 --multi-release ${JAVA_VERSION} \
43 --ignore-missing-deps \
44 --print-module-deps \
45 --class-path "${OUT_LIBS}/*" \
46 "${MAIN_CLASS_FILE}") || exit
47echo "detected modules: ${detected_modules}"
48Error: jackson-core-2.13.0.jar is a multi-release jar file but --multi-release option is not set
49<dependency>
50 <groupId>org.openjfx</groupId>
51 <artifactId>javafx-base</artifactId>
52 <version>${use.javafx.version}</version>
53</dependency>
54<dependency>
55 <groupId>org.openjfx</groupId>
56 <artifactId>javafx-controls</artifactId>
57 <version>${use.javafx.version}</version>
58</dependency>
59<dependency>
60 <groupId>org.openjfx</groupId>
61 <artifactId>javafx-fxml</artifactId>
62 <version>${use.javafx.version}</version>
63</dependency>
64<dependency>
65 <groupId>org.openjfx</groupId>
66 <artifactId>javafx-graphics</artifactId>
67 <version>${use.javafx.version}</version>
68</dependency>
69<dependency>
70 <groupId>org.openjfx</groupId>
71 <artifactId>javafx-web</artifactId>
72 <version>${use.javafx.version}</version>
73</dependency>
74
75<dependency>
76 <groupId>org.jetbrains.kotlin</groupId>
77 <artifactId>kotlin-stdlib-jdk8</artifactId>
78 <version>${kotlin.version}</version>
79</dependency>
80<dependency>
81 <groupId>org.jetbrains.kotlin</groupId>
82 <artifactId>kotlin-reflect</artifactId>
83 <version>${kotlin.version}</version>
84</dependency>
85<dependency>
86 <groupId>org.jetbrains.kotlin</groupId>
87 <artifactId>kotlin-test-junit</artifactId>
88 <version>${kotlin.version}</version>
89 <scope>test</scope>
90</dependency>
91<dependency>
92 <groupId>org.jetbrains.kotlinx</groupId>
93 <artifactId>kotlinx-coroutines-core</artifactId>
94 <version>${kotlinx.coroutines.version}</version>
95</dependency>
96<dependency>
97 <groupId>org.jetbrains.kotlinx</groupId>
98 <artifactId>kotlinx-coroutines-jdk8</artifactId>
99 <version>${kotlinx.coroutines.version}</version>
100</dependency>
101
102<dependency>
103 <groupId>io.insert-koin</groupId>
104 <artifactId>koin-core-jvm</artifactId>
105 <version>${koin.version}</version>
106</dependency>
107<dependency>
108 <groupId>io.insert-koin</groupId>
109 <artifactId>koin-test-jvm</artifactId>
110 <version>${koin.version}</version>
111</dependency>
112
113<dependency>
114 <groupId>org.simpleframework</groupId>
115 <artifactId>simple-xml</artifactId>
116 <version>${simplexml.version}</version>
117</dependency>
118
119<dependency>
120 <groupId>org.apache.logging.log4j</groupId>
121 <artifactId>log4j-api</artifactId>
122 <version>${log4j.version}</version>
123</dependency>
124<dependency>
125 <groupId>org.apache.logging.log4j</groupId>
126 <artifactId>log4j-core</artifactId>
127 <version>${log4j.version}</version>
128</dependency>
129<dependency>
130 <groupId>org.apache.logging.log4j</groupId>
131 <artifactId>log4j-web</artifactId>
132 <version>${log4j.version}</version>
133</dependency>
134<dependency>
135 <groupId>com.fasterxml.jackson.core</groupId>
136 <artifactId>jackson-annotations</artifactId>
137 <version>${jackson.version}</version>
138</dependency>
139<dependency>
140 <groupId>com.fasterxml.jackson.core</groupId>
141 <artifactId>jackson-core</artifactId>
142 <version>${jackson.version}</version>
143</dependency>
144<dependency>
145 <groupId>com.fasterxml.jackson.core</groupId>
146 <artifactId>jackson-databind</artifactId>
147 <version>${jackson.version}</version>
148</dependency>
149<dependency>
150 <groupId>com.fasterxml.jackson.module</groupId>
151 <artifactId>jackson-module-kotlin</artifactId>
152 <version>${jackson.version}</version>
153</dependency>
154<dependency>
155 <groupId>com.squareup.okhttp3</groupId>
156 <artifactId>okhttp</artifactId>
157 <version>${okhttp.version}</version>
158</dependency>
159<dependency>
160 <groupId>net.sf.proguard</groupId>
161 <artifactId>proguard-base</artifactId>
162 <version>${proguard.version}</version>
163 <scope>runtime</scope>
164</dependency>
165<dependency>
166 <groupId>org.junit.jupiter</groupId>
167 <artifactId>junit-jupiter-api</artifactId>
168 <version>${junit.version}</version>
169 <scope>test</scope>
170</dependency>
171<dependency>
172 <groupId>org.junit.jupiter</groupId>
173 <artifactId>junit-jupiter-engine</artifactId>
174 <version>${junit.version}</version>
175 <scope>test</scope>
176</dependency>
177
178<dependency>
179 <groupId>org.kordamp.ikonli</groupId>
180 <artifactId>ikonli-materialdesign2-pack</artifactId>
181 <version>${ikonli.mdi2.version}</version>
182</dependency>
183<dependency>
184 <groupId>org.kordamp.ikonli</groupId>
185 <artifactId>ikonli-javafx</artifactId>
186 <version>${ikonli.version}</version>
187</dependency>
188<dependency>
189 <groupId>org.controlsfx</groupId>
190 <artifactId>controlsfx</artifactId>
191 <version>${controlsfx.version}</version>
192</dependency>
193<dependency>
194 <groupId>com.dlsc</groupId>
195 <artifactId>GMapsFX</artifactId>
196 <version>${gmapfx.version}</version>
197</dependency>
198
199<dependency>
200 <groupId>org.xerial</groupId>
201 <artifactId>sqlite-jdbc</artifactId>
202 <version>${sqlite.jdbc.version}</version>
203</dependency>
204
205<!-- some custom deps from our company -->
206
207<dependency>
208 <groupId>org.conscrypt</groupId>
209 <artifactId>conscrypt-openjdk</artifactId>
210 <version>${conscrypt.version}</version>
211 <classifier>${os.detected.classifier}</classifier>
212</dependency>
213annotations-13.0.jar
214apiguardian-api-1.1.2.jar
215conscrypt-openjdk-2.5.2-linux-x86_64.jar
216controlsfx-11.1.0.jar
217GMapsFX-11.0.2.jar
218hamcrest-core-1.3.jar
219ikonli-core-12.2.0.jar
220ikonli-javafx-12.2.0.jar
221ikonli-materialdesign2-pack-12.2.0.jar
222jackson-annotations-2.13.0.jar
223jackson-core-2.13.0.jar
224jackson-databind-2.13.0.jar
225jackson-module-kotlin-2.13.0.jar
226jakarta.activation-1.2.2.jar
227jakarta.activation-api-1.2.2.jar
228jakarta.xml.bind-api-2.3.3.jar
229jaxb-impl-2.3.3.jar
230junit-4.12.jar
231junit-jupiter-api-5.8.1.jar
232junit-jupiter-engine-5.8.1.jar
233junit-platform-commons-1.8.1.jar
234junit-platform-engine-1.8.1.jar
235koin-core-jvm-3.1.3.jar
236koin-test-jvm-3.1.3.jar
237kotlin-reflect-1.5.31.jar
238kotlin-stdlib-1.5.31.jar
239kotlin-stdlib-common-1.5.30.jar
240kotlin-stdlib-jdk7-1.5.31.jar
241kotlin-stdlib-jdk8-1.5.31.jar
242kotlin-test-1.5.31.jar
243kotlin-test-annotations-common-1.5.30.jar
244kotlin-test-common-1.5.30.jar
245kotlin-test-junit-1.5.31.jar
246kotlinx-coroutines-core-1.5.2.jar
247kotlinx-coroutines-core-jvm-1.5.2.jar
248kotlinx-coroutines-jdk8-1.5.2.jar
249log4j-api-2.14.1.jar
250log4j-core-2.14.1.jar
251log4j-web-2.14.1.jar
252logback-classic-1.2.3.jar
253logback-core-1.2.3.jar
254okhttp-4.9.2.jar
255okio-2.8.0.jar
256opentest4j-1.2.0.jar
257proguard-base-6.2.2.jar
258simple-xml-2.7.1.jar
259slf4j-api-1.7.29.jar
260sqlite-jdbc-3.36.0.3.jar
261stax-1.2.0.jar
262stax-api-1.0.1.jar
263xpp3-1.1.3.3.jar
264mvn compile org.apache.maven.plugins:maven-dependency-plugin:3.1.1:resolve -DexcludeTransitive
265[INFO] The following files have been resolved:
266[INFO] com.fasterxml.jackson.core:jackson-databind:jar:2.13.0:compile -- module com.fasterxml.jackson.databind
267[INFO] com.squareup.okhttp3:okhttp:jar:4.9.2:compile -- module okhttp3 [auto]
268[INFO] com.fasterxml.jackson.core:jackson-annotations:jar:2.13.0:compile -- module com.fasterxml.jackson.annotation
269[INFO] org.openjfx:javafx-fxml:jar:17.0.1:compile -- module javafx.fxmlEmpty [auto]
270[INFO] org.openjfx:javafx-web:jar:17.0.1:compile -- module javafx.webEmpty [auto]
271[INFO] org.jetbrains.kotlin:kotlin-stdlib-jdk8:jar:1.5.31:compile -- module kotlin.stdlib.jdk8
272[INFO] io.insert-koin:koin-core-jvm:jar:3.1.3:compile -- module koin.core.jvm (auto)
273[INFO] org.jetbrains.kotlin:kotlin-test-junit:jar:1.5.31:test -- module kotlin.test.junit
274[INFO] org.simpleframework:simple-xml:jar:2.7.1:compile -- module simple.xml [auto]
275[INFO] org.apache.logging.log4j:log4j-core:jar:2.14.1:compile -- module org.apache.logging.log4j.core [auto]
276[INFO] io.insert-koin:koin-test-jvm:jar:3.1.3:compile -- module koin.test.jvm (auto)
277[INFO] org.junit.jupiter:junit-jupiter-engine:jar:5.8.1:test -- module org.junit.jupiter.engine
278[INFO] org.xerial:sqlite-jdbc:jar:3.36.0.3:compile -- module org.xerial.sqlitejdbc [auto]
279[INFO] net.sf.proguard:proguard-base:jar:6.2.2:runtime -- module proguard.base (auto)
280[INFO] org.openjfx:javafx-graphics:jar:17.0.1:compile -- module javafx.graphicsEmpty [auto]
281[INFO] org.jetbrains.kotlinx:kotlinx-coroutines-core:jar:1.5.2:compile -- module kotlinx.coroutines.core (auto)
282[INFO] org.apache.logging.log4j:log4j-web:jar:2.14.1:compile -- module org.apache.logging.log4j.web [auto]
283[INFO] ch.<ourgroup>:<our-artifact>:1.8.8:compile -- module <our-lib> (auto)
284[INFO] org.openjfx:javafx-controls:jar:17.0.1:compile -- module javafx.controlsEmpty [auto]
285[INFO] org.controlsfx:controlsfx:jar:11.1.0:compile -- module org.controlsfx.controls
286[INFO] com.fasterxml.jackson.core:jackson-core:jar:2.13.0:compile -- module com.fasterxml.jackson.core
287[INFO] com.fasterxml.jackson.module:jackson-module-kotlin:jar:2.13.0:compile -- module com.fasterxml.jackson.kotlin
288[INFO] org.apache.logging.log4j:log4j-api:jar:2.14.1:compile -- module org.apache.logging.log4j
289[INFO] org.openjfx:javafx-base:jar:17.0.1:compile -- module javafx.baseEmpty [auto]
290[INFO] org.junit.jupiter:junit-jupiter-api:jar:5.8.1:test -- module org.junit.jupiter.api
291[INFO] org.kordamp.ikonli:ikonli-javafx:jar:12.2.0:compile -- module org.kordamp.ikonli.javafx
292[INFO] com.dlsc:GMapsFX:jar:11.0.2:compile -- module com.dlsc.gmapsfx
293[INFO] org.jetbrains.kotlin:kotlin-reflect:jar:1.5.31:compile -- module kotlin.reflect
294[INFO] org.jetbrains.kotlinx:kotlinx-coroutines-jdk8:jar:1.5.2:compile -- module kotlinx.coroutines.jdk8 (auto)
295[INFO] org.kordamp.ikonli:ikonli-materialdesign2-pack:jar:12.2.0:compile -- module org.kordamp.ikonli.materialdesign2
296[INFO] org.conscrypt:conscrypt-openjdk:jar:linux-x86_64:2.5.2:compile -- module org.conscrypt [auto]
297ANSWER
Answered 2021-Dec-13 at 13:36Update: These issues have been fixed, and a patched version of jdeps is available as part of the early access build for JDK 18 at: http://jdk.java.net/18/ (starting from build 26)
Turning my comments into an answer. There seem to be 3 bugs going on here:
- The
MultiReleaseExceptionseems to be becausejdepscan not handle classes in different jars that have the same name, such asmodule-info.class, but are stored in a differentMETA-INF/versions/xxxdirectory. (JDK-8277165) - The fact that this exception is sometimes suddenly not occuring seems to be the result of a race condition in the code that checks for the above; classes of the same name having multiple versions. (JDK-8277166)
- The
MultiReleaseExceptionis missing it's exception message since it's thrown as part of an asynchronous task, which wraps it in anExecutionException, which then leads tojdepsnot reporting the exception correctly. (JDK-8277123)
As for a workaround, I don't think there's a good one at this point, except maybe for editing all the jars on the class path so that they put the module-info.class in the same META-INF/versions/xxx directory (but, this might have other consequences as well, so you probably don't want to run with the edited jars, and only use them for jdeps).
QUESTION
"Theming Icons" functionality crashes live wallpapers on Android 12
Asked 2021-Dec-07 at 22:23I recently noticed that my live wallpaper apps are crashing when users try to set the newly introduced "Theming Icons" functionality on Android 12. This new functionality calculates a palette of colors from the user's current static wallpaper and uses this palette to color some of the other apps icons (a feature of the new "Material You" design). But for some reason when it operates on a live wallpaper it crashes the app with the following log:
1Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'android.graphics.Bitmap android.graphics.drawable.BitmapDrawable.getBitmap()' on a null object reference
2 at android.os.Parcel.createExceptionOrNull(Parcel.java:2443)
3 at android.os.Parcel.createException(Parcel.java:2421)
4 at android.os.Parcel.readException(Parcel.java:2404)
5 at android.os.Parcel.readException(Parcel.java:2346)
6 at android.service.wallpaper.IWallpaperConnection$Stub$Proxy.onWallpaperColorsChanged(IWallpaperConnection.java:298)
7 at android.service.wallpaper.WallpaperService$IWallpaperEngineWrapper.executeMessage(WallpaperService.java:2586)
8 at com.android.internal.os.HandlerCaller$MyHandler.handleMessage(HandlerCaller.java:44)
9 at android.os.Handler.dispatchMessage(Handler.java:106)
10 at android.os.Looper.loopOnce(Looper.java:226)
11 at android.os.Looper.loop(Looper.java:313)
12 at android.app.ActivityThread.main(ActivityThread.java:8582)
13 at java.lang.reflect.Method.invoke(Method.java)
14 at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:563)
15 at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1133)
16
17
18Caused by android.os.RemoteException: Remote stack trace:
19 at com.samsung.server.wallpaper.LegibilityColor.convertColors(LegibilityColor.java:418)
20 at com.android.server.wallpaper.WallpaperManagerService$WallpaperConnection.onWallpaperColorsChanged(WallpaperManagerService.java:2169)
21 at android.service.wallpaper.IWallpaperConnection$Stub.onTransact(IWallpaperConnection.java:158)
22 at android.os.Binder.execTransactInternal(Binder.java:1215)
23 at android.os.Binder.execTransact(Binder.java:1179)
24At this moment I can observe this crash only on Samsung Galaxy S21 / S21 Ultra phones but it will be present on many more phones as users update their system to Android 12.
Currently, I cannot reproduce this crash because I don't own a Galaxy S21 and the Android Studio emulator still misses the "Theming Icons" option (maybe because this functionality is currently still in beta).
Does anybody have any clue on how to solve this catastrophic problem? Any help is greatly appreciated!
Update November 18: I was able to test the new "Themed icon" feature on the Android Studio Emulator (on the newly introduced "SV2" release). This release is really buggy but I was not able to reproduce a similar crash. It makes me think that this crash is exclusive to Samsung phones.
I tried to test some Galaxy S21 devices from the publicly available phones at the "Galaxy Mobile - Remote Test Lab" site, but they implement a beta version of Android 12 where the Themed Icon feature is not available.
I want to generate a complete crash report (log + video) to send to Samsung but I still can't find a Galaxy S21 with Android 12 installed (the Android 12 system update started on November 15 for Galaxy S21 phones). In the meantime, the crash reports are exploding as many more users are updating their phones to Android 12...Such a shame!
ANSWER
Answered 2021-Nov-30 at 22:05For a while I was looking for a similar problem in reviews of popular live wallpapers. I found similar reviews only for one application. I updated my phone today. All my live wallpapers stopped working as expected. Then I installed "Earth & Moon" and this app works fine. It means that we are doing something wrong or, on the contrary, we are not doing something :) In the near future I will begin to investigate this problem.
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Graphics
Tutorials and Learning Resources are not available at this moment for Graphics
Share this Page
Get latest updates on Graphics