Explore all Graphics open source software, libraries, packages, source code, cloud functions and APIs.

Popular New Releases in Graphics

three.js

r139

pixijs

v6.3.0

pixi.js

v6.0.2

tfjs

filament

v1.21.1

Popular Libraries in Graphics

three.js

by mrdoob doticonjavascriptdoticon

star image 80965 doticonMIT

JavaScript 3D Library.

pixijs

by pixijs doticontypescriptdoticon

star image 36164 doticonMIT

The HTML5 Creation Engine: Create beautiful digital content with the fastest, most flexible 2D WebGL renderer.

pixi.js

by pixijs doticontypescriptdoticon

star image 32471 doticonMIT

The HTML5 Creation Engine: Create beautiful digital content with the fastest, most flexible 2D WebGL renderer.

tfjs

by tensorflow doticontypescriptdoticon

star image 16158 doticonApache-2.0

A WebGL accelerated JavaScript library for training and deploying ML models.

filament

by google doticonc++doticon

star image 13837 doticonApache-2.0

Filament is a real-time physically based rendering engine for Android, iOS, Windows, Linux, macOS, and WebGL2

tinyrenderer

by ssloy doticonc++doticon

star image 11147 doticonNOASSERTION

A brief computer graphics / rendering course

WebGL-Fluid-Simulation

by PavelDoGreat doticonjavascriptdoticon

star image 11014 doticonMIT

Play with fluids in your browser (works even on mobile)

3d-game-shaders-for-beginners

by lettier doticonc++doticon

star image 11004 doticonBSD-3-Clause

🎮 A step-by-step guide to implementing SSAO, depth of field, lighting, normal mapping, and more for your 3D game.

deck.gl

by visgl doticonjavascriptdoticon

star image 9725 doticonMIT

WebGL2 powered visualization framework

Trending New libraries in Graphics

Real-Time-Person-Removal

by jasonmayes doticonjavascriptdoticon

star image 4798 doticonApache-2.0

Removing people from complex backgrounds in real time using TensorFlow.js in the web browser

engine

by oasis-engine doticontypescriptdoticon

star image 2597 doticonMIT

Oasis Engine is a web-first and mobile-first high-performance real-time development platform.

skija

by JetBrains doticonjavadoticon

star image 2045 doticonApache-2.0

Skia bindings for Java

trois

by troisjs doticontypescriptdoticon

star image 1961 doticonMIT

✨ ThreeJS + VueJS 3 + ViteJS ⚡

Graphics

by Unity-Technologies doticoncsharpdoticon

star image 1628 doticonNOASSERTION

Unity Graphics - Including Scriptable Render Pipeline

X-PostProcessing-Library

by QianMo doticoncsharpdoticon

star image 1293 doticonMIT

Unity Post Processing Stack Library | Unity引擎的高品质后处理库

procedural-gl-js

by felixpalmer doticonjavascriptdoticon

star image 1083 doticonMPL-2.0

Mobile-first 3D mapping engine with emphasis on user experience

aladino

by luruke doticonjavascriptdoticon

star image 771 doticon

🧞‍♂️ Your magic WebGL carpet

3DTilesRendererJS

by NASA-AMMOS doticonjavascriptdoticon

star image 743 doticonApache-2.0

Renderer for 3D Tiles in Javascript using three.js

Top Authors in Graphics

1

keijiro

42 Libraries

star icon12896

2

stackgl

33 Libraries

star icon2447

3

mattdesl

31 Libraries

star icon3125

4

hughsk

28 Libraries

star icon618

5

jeromeetienne

25 Libraries

star icon769

6

spite

20 Libraries

star icon3739

7

pixijs

20 Libraries

star icon71795

8

mikolalysenko

20 Libraries

star icon278

9

glslify

16 Libraries

star icon2071

10

Jam3

15 Libraries

star icon1407

1

42 Libraries

star icon12896

2

33 Libraries

star icon2447

3

31 Libraries

star icon3125

4

28 Libraries

star icon618

5

25 Libraries

star icon769

6

20 Libraries

star icon3739

7

20 Libraries

star icon71795

8

20 Libraries

star icon278

9

16 Libraries

star icon2071

10

15 Libraries

star icon1407

Trending Kits in Graphics

A React graph library is a collection of components and tools. It is designed for creating graphs and other data visualizations in applications. It provides a set of reusable and customizable building blocks. It makes it easier to build interactive and insightful data visualizations.  

  

Many react graph libraries are available, from simple ones to complex solutions. Some libraries focus on providing various chart types and customization options. Others offer functionalities like stock charting, time series analysis, and network traffic visualization. Developers can choose the library that best fits their requirements and use cases.  

  

React graph libraries offer a wide range of features to support data visualization. These features include support for different chart types and interactive and animated charts. It includes a responsive design for adapting to different screen sizes and server-side rendering. It supports improved performance, integration with other libraries, and extensive documentation. Some libraries may provide data manipulation, real-time updates, and integration with ML algorithms.  

  

Choosing the right library based on your project is important when using a react graph library. Developers should consider chart types, customization options, performance, documentation, and community support. Must understand the limitations and know bundle size or rendering performance.  

  

Creating a simple react graph library involves designing the data model. It helps in defining the required components for rendering the charts or graphs. The library should provide an intuitive API. It helps developers to pass the data and configuration options to the components. It enables them to customize the appearance and behavior of the graphs. The implementation should follow React's component principle. It allows the reuse and composition of components to create more complex visualizations.  

  

These libraries are used in data analysis, visualization, real-time monitoring, and visual analytics. They have been instrumental in providing insightful and interactive visualizations. It helps understand complex data sets. These libraries are used in applications where visualizing model outputs or training progress. It can aid in interpreting and communicating the results.  

  

To use the graph libraries, developers should understand the documentation and examples provided. They should understand the requirements and choose a library that aligns with them. It is important to optimize the library, considering bundle size and rendering performance. Developers should leverage the library's features, such as interactivity, animations, and responsiveness. It helps enhance the user experience and convey insights.  

  

React graph libraries offer powerful tools for creating and visualizing data in applications. They provide reusable and customizable components, extensive documentation, and various features. It helps support various data visualization needs. By leveraging these libraries, developers can create appealing and interactive data visualizations. It helps in improving the understanding and interpretation of complex data sets.  

  

The advantages of using these libraries include their ability to simplify the development. It provides ready-to-use components, customization flexibility, extensive documentation, community support, and compatibility. These libraries have become popular among data scientists and developers. It is because it is easy, versatility, and ability. It helps create high-quality and insightful visualizations in web and mobile applications.  

recharts:  

  • It is a lightweight and flexible charting library for React.  
  • It provides a simple API for creating charts.  
  • It supports features like tooltips, legends, and animations.  
  • It is beginner friendly.  
  • It can create simple and elegant graphs in React applications.  

chartist.js:  

  • It is a popular JavaScript charting library.  
  • It can be integrated with React.  
  • It offers various chart types, including bar, line, and radar charts.  
  • It provides a simple and intuitive API.  
  • It makes it suitable for beginners and experienced developers.  

nivo:  

  • It is a powerful React-based data visualization library.  
  • It offers various chart types, including bar, line, and pie charts.  
  • It provides rich customization options and responsive design and is optimized for performance.  
  • It makes it suitable for creating complex and dynamic visualizations.  

victory:  

  • It is a React-based charting library.  
  • It focuses on providing customizable and interactive graphs.  
  • It supports various chart types and offers features like animations, tooltips, and zooming.  
  • It is well-suited for creating stunning and interactive data visualizations.  

react-vis:  

  • It is a React-based visualization library.  
  • It offers rich components for creating various graphs.  
  • It provides customizable and responsive graphs.  
  • It makes it ideal for creating appealing dashboards and reports.  

dc.js:  

  • It is a powerful data visualization library.  
  • It supports various graphs and charts.  
  • It helps create interactive and dynamic visualizations.  
  • It makes it suitable for complex data analysis and data-driven applications.  

viser:  

  • It is a React-based visualization library built on top of G2Plot and G6.  
  • It offers various components for creating interactive and customizable graphs.  
  • It supports features like animations, tooltips, and interactions.  
  • It makes it ideal for creating appealing and interactive data visualizations.  

FAQ  

1. What are the most reliable React data visualization libraries available?  

Some reliable data visualization libraries include D3.js, Victory, Recharts, and Chart.js. These libraries have been used as robust and dependable solutions. It helps in creating data visualizations in React applications.  

  

2. How do I choose a React chart library that is right for my project?  

Consider chart types, customization levels, documentation, and support to choose a chart library. You should also consider its performance and rendering capabilities, and compatibility. Evaluating API and examples as they align with your development workflow and requirements.  

  

3. Are there open-source charting libraries that make it easy to work with React?  

Yes, there are several open-source charting libraries available that work with React. Some popular options include Chart.js, Recharts, and react-charts-2. These libraries provide various chart types and customization options. It is an intuitive API designed for React development.  

  

4. What is modular charting, and how can I use it in my project?  

Modular charting uses reusable and composable charting components to build data visualizations. With modular charting, you can break down complex visualizations into smaller, reusable components. It can be combined to create more sophisticated charts. This approach promotes code reusability, scalability, and maintainability in your project.  

  

5. Are there any customizable stock charts available in React libraries?  

Yes, there are customizable stock charts available in React libraries. Libraries like Highcharts and TradingView provide comprehensive stock charting functionalities. It includes interactive features, technical indicators, and drawing tools. These libraries offer extensive customization options. It allows you to tailor the stock charts to your requirements.  

  

6. How do I create data visualization charts with React components?  

You can leverage charting libraries to create data visualization charts with React components. These libraries provide various React components that encapsulate the charting logic and rendering. You can pass data and configuration options to define the type, styling, and data sources. Combining these components and customizing the properties allows data visualization charts. These libraries offer extensive documentation to guide you through creating data visualizations. 

Graphics in C++ is to develop a graphic model to draw curves, lines of different colors and styles, phrases with fashionable fonts with different typefaces, colors, and sizes, and geometrical shapes like circles, rectangles, etc. Graphics programming could help us to create projects, games, animation, and more. There are two different modes in C++ by default- text mode and graphics mode. To implement the graphics, which are a two-dimensional notion, we need to use C++ programming and a few functions. The prerequisites of a graphics monitor are a graphics card like a VGA, SVGA, or EGA.

 

Importing the "graphics.h" library into the GCC compiler enables the production of graphics in the C++ console. C++ is used to program graphics via the terminal or command prompt or the other method you need to install the DevC++ compiler. In this mode, the output is shown as points or pixels on the computer screen. The screen is divided into tiny dots called pixels in graphics mode. For instance, the screen on a VGA display is divided into 480 rows and 640 columns of dots. The term "resolution screen" refers to the number of dots per inch. The clarity of the graphics increases with the number of pixels. The circle, line, eclipse, and other geometric forms are also traceable. The primary technique adopted is object-oriented programming. Since C++ relies on low-level programs, there are no built-in drawing methods, and APIs are used to create visuals. C++ is frequently regarded as one of the leading programming languages for computer graphics. 

 

To operate in graphics, the C++ compiler requires.

- the "graphics.h" header file that includes built-in graphic features

- files for the Borland Graphics Interface (BGI) that has the graphics driver programs to set up the computer monitor to display graphics, and

- the "chr" extension files for the character font style. 

 

 The wide adoption of C++ by the developers of IDEs, editors, compilers, test frameworks, and other tools makes it easier to use. The best libraries are magnum, OpenSceneGraph, AtomicGameEngine, Vulkan, glbinding, LibVT, Cell, minigrafx, tinyrenderer, ripes, etc.


Check out the list below to find more popular C++ 2D Graphics libraries for your applications: 

In the branch of computer science known as computer graphics, techniques for digitally synthesizing and modifying visual content are explored. 3D graphics contribute to various applications these days. Animation of 3D pictures raised the demand for CGI (Computer Generated Imagery) in movies and video games, creating images closer to reality. Visualizations on computer screens are constructed using a variety of algorithms and methods. C++ helps comprehend, process, and create graphics with a rich visual experience. 3D graphics are the contrast of two-dimensional (2D) images. Artists often generate a 3D model consisting of a wireframe and polygons, which are given color, effects, movements, texturing, and lighting and then rendered as a 3D computer picture. C++-coded visuals are used to produce effects, models, animations, and simulations in real time. The 2D system uses only two coordinates named X and Y, while 3D uses an extra coordinate called Z.

 

The OpenGL (Open Graphics Library) industrial standard API for creating 3D (including 2D) graphics is cross-platform, hardware-accelerated, and language-independent. Modern computers have dedicated GPUs (Graphics Processing Units), each with its memory to speed up graphics rendering. The software interface for graphics hardware is called OpenGL. OpenGL graphic rendering directives sent by your programs could be focused on and sped up by the graphics hardware. The method described here simplifies the programmatic construction of geometry within the constraints of a production-level language, C++. The system's implementation is strongly object-oriented and depends on multiple dispatching. The system can be easily expanded, and new geometric operations and primitives can be easily added. New media formats, such as music and image, could be introduced to the system.

  

Using these libraries, we can accomplish jobs more quickly, effectively, and with fewer lines of repetitious code than without them. It makes it simple to develop several feature modules for dynamic distribution from closely tied modules to particular features. The wide adoption of C++ by the developers of IDEs, editors, compilers, test frameworks, and other tools makes it easier to use. C++ libraries are typically used in User Interface and Graphics applications. Various C++ libraries help in scientific visualization to contribute to the entertainment, gaming, and computer-aided design sectors. A few examples of C++ libraries are - OpenMVG, Horde3D, 3d-game-shaders, s2geometry, hello-webgpu, NVISII, PixelArtShader, assimp, permafrost-engine, and Urho3D.


Check out the list below to find more popular C++ 3D Graphic libraries for your app development: 

Trending Discussions on Graphics

android:exported added but still getting error Apps targeting Android 12 and higher are required to specify an explicit value for android:exported

Tensorflow setup on RStudio/ R | CentOS

Configuring compilers on Mac M1 (Big Sur, Monterey) for Rcpp and other tools

Problem with non-standard evaluation in disk.frame objects using data.table syntax

How to automate legends for a new geom in ggplot2?

Why is WSL extremely slow when compared with native Windows NPM/Yarn processing?

Window not displaying SDL2

Symbol not found in flat namespace '_FT_Done_Face' from reportlab with Python@3.9 on macOS 12

jdeps can't print-module-deps due to a MultiReleaseException

"Theming Icons" functionality crashes live wallpapers on Android 12

QUESTION

android:exported added but still getting error Apps targeting Android 12 and higher are required to specify an explicit value for android:exported

Asked 2022-Mar-24 at 15:30

I have added android:exported="true" to my only activity in manifest but still getting below error after updating compile sdk and target sdk version to 31.I also tried rebuilding the project , invalidating cache and restart but that didn't helped

Error- Apps targeting Android 12 and higher are required to specify an explicit value for android:exported when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details.

AndroidManifest File
1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android"
3    package="com.xyz.abc">
4
5    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
6    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
7
8    <application
9        android:name=".framework.presentation.BaseApplication"
10        android:allowBackup="true"
11        android:icon="@mipmap/ic_launcher"
12        android:label="@string/app_name"
13        android:roundIcon="@mipmap/ic_launcher_round"
14        android:supportsRtl="true"
15        android:theme="@style/AppTheme">
16        <activity android:name="com.xyz.presentation.MainActivity"
17            android:exported="true">
18            <intent-filter>
19                <action android:name="android.intent.action.MAIN" />
20
21                <category android:name="android.intent.category.LAUNCHER" />
22            </intent-filter>
23        </activity>
24    </application>
25
26</manifest>
27
Merged manifest error

Other Manifest Files (Included in merge, but did not contribute any elements) firebase-installations:17.0.0 manifest, versionedparcelable:1.1.1 manifest, runtime:1.0.1 manifest, test:core:1.2.0 manifest, loader:1.0.0 manifest, facebook-share:11.1.0 manifest, leakcanary:leaksentry:2.0-alpha-3 manifest, material-dialogs:input:3.2.1 manifest, material-icons-extended:1.0.0 manifest, play-services-stats:17.0.0 manifest, interpolator:1.0.0 manifest, activity-compose:1.3.1 manifest, material-ripple:1.0.0 manifest, foundation:1.0.0 manifest, asynclayoutinflater:1.0.0 manifest, savedstate-ktx:1.1.0 manifest, navigation-dynamic-features-fragment:2.3.5 manifest, firebase-ui-auth:7.2.0 manifest, animation:1.0.1 manifest, animation-core:1.0.1 manifest, installreferrer:1.0 manifest, firebase-crashlytics:18.0.0 manifest, ui:1.0.1 manifest, lifecycle-viewmodel-savedstate:2.3.1 manifest, play-services-auth-base:17.0.0 manifest, hilt-android:2.35.1 manifest, material-dialogs:core:3.2.1 manifest, AndroidManifest.xml navigation file, savedstate:1.1.0 manifest, cursoradapter:1.0.0 manifest, sqlite-framework:2.0.1 manifest, room-ktx:2.1.0 manifest, leakcanary-android-core:2.0-alpha-3 manifest, AndroidManifest.xml navigation file, media:1.0.0 manifest, coordinatorlayout:1.1.0 manifest, legacy-support-core-utils:1.0.0 manifest, lifecycle-runtime:2.3.1 manifest, coil-kt:coil:1.3.1 manifest, ui-tooling-preview:1.0.0 manifest, facebook-core:11.1.0 manifest, core:1.6.0 manifest, material:1.0.0 manifest, firebase-common:20.0.0 manifest, documentfile:1.0.0 manifest, lifecycle-viewmodel-compose:2.4.0-beta01 manifest, play-services-base:17.1.0 manifest, ui-tooling-data:1.0.0 manifest, coil-base:1.3.1 manifest, firebase-analytics-ktx:19.0.0 manifest, localbroadcastmanager:1.0.0 manifest, swiperefreshlayout:1.1.0-alpha03 manifest, constraintlayout-compose:1.0.0-beta02 manifest, core-ktx:1.6.0 manifest, firebase-database-collection:18.0.0 manifest, coil-compose-base:1.3.1 manifest, activity:1.3.1 manifest, AndroidManifest.xml navigation file, facebook-messenger:11.1.0 manifest, print:1.0.0 manifest, customview:1.1.0 manifest, material-icons-core:1.0.0 manifest, play-services-measurement-sdk:19.0.0 manifest, fragment:1.3.4 manifest, firebase-appcheck-interop:16.0.0-beta01 manifest, facebook-login:11.1.0 manifest, cardview:1.0.0 manifest, runtime-rxjava2:1.0.0 manifest, viewpager2:1.0.0 manifest, play-services-ads-identifier:17.0.0 manifest, play-services-measurement-impl:19.0.0 manifest, lifecycle-livedata-core:2.3.1 manifest, play-services-safetynet:17.0.0 manifest, AndroidManifest.xml navigation file, lifecycle-viewmodel-ktx:2.3.1 manifest, transport-backend-cct:3.0.0 manifest, fragment-ktx:1.2.4 manifest, appcompat:1.3.0 manifest, transport-runtime:3.0.0 manifest, lifecycle-livedata-core-ktx:2.2.0 manifest, firebase-firestore-ktx:23.0.0 manifest, legacy-support-v4:1.0.0 manifest, play-services-basement:17.1.1 manifest, firebase-storage:20.0.0 manifest, play-services-auth-api-phone:17.4.0 manifest, leakcanary-android:2.0-alpha-3 manifest, firebase-auth-interop:20.0.0 manifest, lifecycle-viewmodel:2.3.1 manifest, browser:1.0.0 manifest, firebase-auth:21.0.1 manifest, material:1.2.1 manifest, slidingpanelayout:1.0.0 manifest, vectordrawable:1.1.0 manifest, recyclerview:1.1.0 manifest, play-services-auth:19.0.0 manifest, room-runtime:2.1.0 manifest, dagger-lint-aar:2.35.1 manifest, navigation-dynamic-features-runtime:2.3.5 manifest, play-services-measurement-api:19.0.0 manifest, firebase-encoders-json:18.0.0 manifest, sqlite:2.0.1 manifest, facebook-android-sdk:11.1.0 manifest, firebase-components:17.0.0 manifest, transport-api:3.0.0 manifest, protolite-well-known-types:18.0.0 manifest, markdown-processor:0.1.3 manifest, play-services-measurement-base:19.0.0 manifest, firebase-common-ktx:20.0.0 manifest, activity-ktx:1.3.1 manifest, firebase-crashlytics-ktx:18.0.0 manifest, coil-compose:1.3.1 manifest, multidex:2.0.1 manifest, core-runtime:2.1.0 manifest, fragment-testing:1.2.0 manifest, ui-graphics:1.0.1 manifest, AndroidManifest.xml navigation file, ui-tooling:1.0.0 manifest, grpc-android:1.28.0 manifest, ui-unit:1.0.1 manifest, play-services-measurement:19.0.0 manifest, play:core:1.9.1 manifest, annotation-experimental:1.1.0 manifest, play-services-measurement-sdk-api:19.0.0 manifest, play-services-tasks:17.0.0 manifest, firebase-analytics:19.0.0 manifest, facebook-common:11.1.0 manifest, drawerlayout:1.1.1 manifest, AndroidManifest.xml navigation file, navigation-compose:2.4.0-alpha09 manifest, facebook-gamingservices:11.1.0 manifest, firebase-firestore:23.0.0 manifest, lifecycle-livedata:2.2.0 manifest, legacy-support-core-ui:1.0.0 manifest, test:monitor:1.2.0 manifest, AndroidManifest.xml navigation file, facebook-applinks:11.1.0 manifest, viewpager:1.0.0 manifest, ui-geometry:1.0.1 manifest, lifecycle-runtime-ktx:2.3.1 manifest, constraintlayout:2.0.4 manifest, ui-text:1.0.1 manifest, AndroidManifest.xml navigation file, firebase-installations-interop:17.0.0 manifest, transition:1.3.0 manifest, foundation-layout:1.0.1 manifest, appcompat-resources:1.3.1 manifest, runtime-livedata:1.0.0 manifest, runtime-saveable:1.0.1 manifest, firebase-measurement-connector:19.0.0 manifest, vectordrawable-animated:1.1.0 manifest, main nav_graph.xml navigation file Merging Errors: Error: android:exported needs to be explicitly specified for . Apps targeting Android 12 and higher are required to specify an explicit value for android:exported when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details. Dairy.app main manifest (this file) Error: android:exported needs to be explicitly specified for . Apps targeting Android 12 and higher are required to specify an explicit value for android:exported when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details. Dairy.app main manifest (this file) Error: android:exported needs to be explicitly specified for . Apps targeting Android 12 and higher are required to specify an explicit value for android:exported when the corresponding component has an intent filter defined. See https://developer.android.com/guide/topics/manifest/activity-element#exported for details. Dairy.app main manifest (this file)

ANSWER

Answered 2021-Oct-05 at 10:38

After the build has failed go to AndroidManifest.xml and in the bottom click merged manifest see which activities which have intent-filter but don't have exported=true attribute. Or you can just get the activities which are giving error.

Add these activities to your App manifest with android:exported="true" and app tools:node="merge" this will add exported attribute to the activities giving error.

Example:

1<?xml version="1.0" encoding="utf-8"?>
2<manifest xmlns:android="http://schemas.android.com/apk/res/android"
3    package="com.xyz.abc">
4
5    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
6    <uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
7
8    <application
9        android:name=".framework.presentation.BaseApplication"
10        android:allowBackup="true"
11        android:icon="@mipmap/ic_launcher"
12        android:label="@string/app_name"
13        android:roundIcon="@mipmap/ic_launcher_round"
14        android:supportsRtl="true"
15        android:theme="@style/AppTheme">
16        <activity android:name="com.xyz.presentation.MainActivity"
17            android:exported="true">
18            <intent-filter>
19                <action android:name="android.intent.action.MAIN" />
20
21                <category android:name="android.intent.category.LAUNCHER" />
22            </intent-filter>
23        </activity>
24    </application>
25
26</manifest>
27      <activity
28            android:name="<activity which is giving error>"
29            android:exported="true"
30            tools:node="merge" />
31

You will have to do this once, you can remove this once the library developers update their libs.

Source https://stackoverflow.com/questions/69287478

QUESTION

Tensorflow setup on RStudio/ R | CentOS

Asked 2022-Feb-11 at 09:36

For the last 5 days, I am trying to make Keras/Tensorflow packages work in R. I am using RStudio for installation and have used conda, miniconda, virtualenv but it crashes each time in the end. Installing a library should not be a nightmare especially when we are talking about R (one of the best statistical languages) and TensorFlow (one of the best deep learning libraries). Can someone share a reliable way to install Keras/Tensorflow on CentOS 7?

Following are the steps I am using to install tensorflow in RStudio.

Since RStudio simply crashes each time I run tensorflow::tf_config() I have no way to check what is going wrong.

enter image description here

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41

Update 1 The only way RStudio does not crash while installing tensorflow is by executing following steps -

First, I created a new virtual environment using conda

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44

Then from within RStudio, I installed reticulate and activated the virtual environment which I earlier created using conda

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50

As soon as I try to import tensorflow in RStudio, it loads the library /lib64/libstdc++.so.6 instead of /root/.conda/envs/py38/lib/libstdc++.so.6 and I get the following error -

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65

Here is what inside /lib64/libstdc++.so.6

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93

To resolve the library issue, I added the path of the correct libstdc++.so.6 library having GLIBCXX_3.4.20 in RStudio.

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94

and, also

1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95

But still I get the same error ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20'. Somehow RStudio still loads /lib64/libstdc++.so.6 first instead of /root/.conda/envs/py38/lib/libstdc++.so.6

Instead of RStudio, if I execute the above steps in the R console, then also I get the exact same error.

Update 2: A solution is posted here

ANSWER

Answered 2022-Jan-16 at 00:08

Perhaps my failed attempts will help someone else solve this problem; my approach:

  • boot up a clean CentOS 7 vm
  • install R and some dependencies
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101
  • Download and install Anaconda via linux installer script
  • Create a new conda env
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105

**From within this conda env you can import tensorflow in python without error; now to access tf via R

  • install an updated gcc via devtoolset
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107
  • attempt to use tensorflow in R via the reticulate package
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107scl enable devtoolset-7 R
108install.packages("remotes")
109remotes::install_github('rstudio/reticulate')
110reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
111reticulate::repl_python()
112# This works as expected but the command "import tensorflow" crashes R
113# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
114
115# Also tried:
116install.packages("devtools")
117devtools::install_github('rstudio/tensorflow')
118devtools::install_github('rstudio/keras')
119library(tensorflow)
120install_tensorflow() # "successful"
121tensorflow::tf_config()
122# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
123
  • try older versions of tensorflow/keras
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107scl enable devtoolset-7 R
108install.packages("remotes")
109remotes::install_github('rstudio/reticulate')
110reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
111reticulate::repl_python()
112# This works as expected but the command "import tensorflow" crashes R
113# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
114
115# Also tried:
116install.packages("devtools")
117devtools::install_github('rstudio/tensorflow')
118devtools::install_github('rstudio/keras')
119library(tensorflow)
120install_tensorflow() # "successful"
121tensorflow::tf_config()
122# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
123devtools::install_github('rstudio/tensorflow@v2.4.0')
124devtools::install_github('rstudio/keras@v2.4.0')
125library(tensorflow)
126tf_config()
127# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
128
  • Try an updated version of R (v4.0)
1devtools::install_github("rstudio/reticulate")
2devtools::install_github("rstudio/keras") # This package also installs tensorflow
3library(reticulate)
4reticulate::install_miniconda()
5reticulate::use_miniconda("r-reticulate")
6library(tensorflow)
7tensorflow::tf_config() **# Crashes at this point**
8
9sessionInfo()
10
11
12R version 3.6.0 (2019-04-26)
13Platform: x86_64-redhat-linux-gnu (64-bit)
14Running under: CentOS Linux 7 (Core)
15
16Matrix products: default
17BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
18
19locale:
20 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
21 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
22 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
23 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
24 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
25[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
26
27attached base packages:
28[1] stats     graphics  grDevices utils     datasets  methods   base     
29
30other attached packages:
31[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 
32
33loaded via a namespace (and not attached):
34 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
35 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
36 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
37[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
38[17] base64enc_0.1-3
39
40
41conda create --name py38 python=3.8.0
42conda activate py38
43conda install tensorflow=2.4
44devtools::install_github("rstudio/reticulate")
45library(reticulate)
46reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
47reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
48reticulate::py_available(initialize = TRUE)
49ts <- reticulate::import("tensorflow")
50Error in py_module_import(module, convert = convert) : 
51  ImportError: Traceback (most recent call last):
52  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
53    from tensorflow.python._pywrap_tensorflow_internal import *
54  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
55    module = _import(
56ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
57
58
59Failed to load the native TensorFlow runtime.
60
61See https://www.tensorflow.org/install/errors
62
63for some common reasons and solutions.  Include the entire stack trace
64above this error message when asking for help.
65> strings /lib64/libstdc++.so.6 | grep GLIBC
66
67GLIBCXX_3.4
68GLIBCXX_3.4.1
69GLIBCXX_3.4.2
70GLIBCXX_3.4.3
71GLIBCXX_3.4.4
72GLIBCXX_3.4.5
73GLIBCXX_3.4.6
74GLIBCXX_3.4.7
75GLIBCXX_3.4.8
76GLIBCXX_3.4.9
77GLIBCXX_3.4.10
78GLIBCXX_3.4.11
79GLIBCXX_3.4.12
80GLIBCXX_3.4.13
81GLIBCXX_3.4.14
82GLIBCXX_3.4.15
83GLIBCXX_3.4.16
84GLIBCXX_3.4.17
85GLIBCXX_3.4.18
86GLIBCXX_3.4.19
87GLIBC_2.3
88GLIBC_2.2.5
89GLIBC_2.14
90GLIBC_2.4
91GLIBC_2.3.2
92GLIBCXX_DEBUG_MESSAGE_LENGTH
93system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')
94Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")
95sudo yum install epel-release
96sudo yum install R
97sudo yum install libxml2-devel
98sudo yum install openssl-devel
99sudo yum install libcurl-devel
100sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
101conda init
102conda create --name tf
103conda activate tf
104conda install -c conda-forge tensorflow
105sudo yum install centos-release-scl
106sudo yum install devtoolset-7-gcc*
107scl enable devtoolset-7 R
108install.packages("remotes")
109remotes::install_github('rstudio/reticulate')
110reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
111reticulate::repl_python()
112# This works as expected but the command "import tensorflow" crashes R
113# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
114
115# Also tried:
116install.packages("devtools")
117devtools::install_github('rstudio/tensorflow')
118devtools::install_github('rstudio/keras')
119library(tensorflow)
120install_tensorflow() # "successful"
121tensorflow::tf_config()
122# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
123devtools::install_github('rstudio/tensorflow@v2.4.0')
124devtools::install_github('rstudio/keras@v2.4.0')
125library(tensorflow)
126tf_config()
127# Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
128# deactivate conda
129sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm 
130export R_VERSION=4.0.0
131curl -O https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm
132sudo yum install R-${R_VERSION}-1-1.x86_64.rpm
133
134scl enable devtoolset-7 /opt/R/4.0.0/bin/R
135install.packages("devtools")
136devtools::install_github('rstudio/reticulate')
137reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
138reticulate::repl_python()
139# 'import tensorflow' resulted in "core dumped"
140

I guess the issue is with R/CentOS, as you can import and use tensorflow via python normally, but I'm not sure what else to try.

I would also like to say that I had no issues with Ubuntu (which is specifically supported by tensorflow, along with macOS and Windows), and I came across these docs that might be some help: https://wiki.hpcc.msu.edu/display/ITH/Installing+TensorFlow+using+anaconda / https://wiki.hpcc.msu.edu/pages/viewpage.action?pageId=22709999

Source https://stackoverflow.com/questions/70645074

QUESTION

Configuring compilers on Mac M1 (Big Sur, Monterey) for Rcpp and other tools

Asked 2022-Feb-10 at 21:07

I'm trying to use packages that require Rcpp in R on my M1 Mac, which I was never able to get up and running after purchasing this computer. I updated it to Monterey in the hope that this would fix some installation issues but it hasn't. I tried running the Rcpp check from this page but I get the following error:

1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
10  Error 1 occurred building shared library.
11

I get that it can't "find" gfortran. I installed this release of gfortran for Monterey. When I type which gfortran into Terminal, it returns /opt/homebrew/bin/gfortran. (Maybe this version of gfortran requires Xcode tools that are too new—it says something about 13.2 and when I run clang --version it says 13.0—but I don't see another release of gfortran for Monterey?)

I also appended /opt/homebrew/bin: to PATH in R so it looks like this now:

1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
10  Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12
1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
10  Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13

Other things I checked:

  • Xcode command line tools is installed (which clang returns /usr/bin/clang).
  • Files ~/.R/Makevars and ~/.Renviron don't exist.

Here's my session info:

1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
10  Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats     graphics  grDevices utils     datasets  methods   base     
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7        
29

ANSWER

Answered 2022-Feb-10 at 21:07
Background

Currently (2022-02-05), CRAN builds R binaries for Apple silicon using Apple clang (from Command Line Tools for Xcode 12.4) and an experimental build of gfortran.

If you obtain R from CRAN (i.e., here), then you need to replicate CRAN's compiler setup on your system before building R packages that contain C/C++/Fortran code from their sources (and before using Rcpp, etc.). This requirement ensures that your package builds are compatible with R itself.

A further complication is the fact that Apple clang doesn't support OpenMP, so you need to do even more work to compile programs that make use of multithreading. You could circumvent the issue by building R itself and all R packages from sources with LLVM clang, which does support OpenMP, but this approach is onerous and "for experts only". There is another approach that has been tested by a few people, including Simon Urbanek, the maintainer of R for macOS. It is experimental and also "for experts only", but seems to work on my machine and is simpler than trying to build R yourself.

Instructions for obtaining a working toolchain

Warning: These instructions come with no warranty and could break at any time. They assume some level of familiarity with C/C++/Fortran program compilation, Makefile syntax, and Unix shells. As usual, sudo at your own risk.

I will try to address compilers and OpenMP support at the same time. I am going to assume that you are starting from nothing. Feel free to skip steps you've already taken, though you might find a fresh start helpful.

I've tested these instructions on a machine running Big Sur, and at least one person has tested them on a machine running Monterey. I would be glad to hear from others.

  1. Download an R binary from CRAN here and install. Be sure to select the binary built for Apple silicon.

  2. Run

1> Rcpp::sourceCpp("~/github/helloworld.cpp")
2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
4ld: library not found for -lgfortran
5clang: error: linker command failed with exit code 1 (use -v to see invocation)
6make: *** [sourceCpp_4.so] Error 1
7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
10  Error 1 occurred building shared library.
11> Sys.getenv("PATH")
12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
13R version 4.1.1 (2021-08-10)
14Platform: aarch64-apple-darwin20 (64-bit)
15Running under: macOS Monterey 12.1
16
17Matrix products: default
18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
19
20locale:
21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
22
23attached base packages:
24[1] stats     graphics  grDevices utils     datasets  methods   base     
25
26loaded via a namespace (and not attached):
27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
28[4] Rcpp_1.0.7        
29$ sudo xcode-select --install
30

in Terminal to install the latest release version of Apple's Command Line Tools for Xcode, which includes Apple clang. You can obtain earlier versions from your browser here. The version that you install should not be older than the one that CRAN used to build your R binary.

  • Download the gfortran binary recommended here and install by unpacking to root:

  • 1> Rcpp::sourceCpp("~/github/helloworld.cpp")
    2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
    3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
    4ld: library not found for -lgfortran
    5clang: error: linker command failed with exit code 1 (use -v to see invocation)
    6make: *** [sourceCpp_4.so] Error 1
    7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
    8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
    9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
    10  Error 1 occurred building shared library.
    11> Sys.getenv("PATH")
    12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
    13R version 4.1.1 (2021-08-10)
    14Platform: aarch64-apple-darwin20 (64-bit)
    15Running under: macOS Monterey 12.1
    16
    17Matrix products: default
    18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
    19
    20locale:
    21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    22
    23attached base packages:
    24[1] stats     graphics  grDevices utils     datasets  methods   base     
    25
    26loaded via a namespace (and not attached):
    27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
    28[4] Rcpp_1.0.7        
    29$ sudo xcode-select --install
    30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
    31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
    32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
    33

    The last command updates a symlink inside of the gfortran installation so that it points to the SDK inside of your Command Line Tools installation.

  • Download an OpenMP runtime suitable for your Apple clang version here and install by unpacking to root. You can query your Apple clang version with clang --version. For example, I have version 1300.0.29.30, so I did:

  • 1> Rcpp::sourceCpp("~/github/helloworld.cpp")
    2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
    3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
    4ld: library not found for -lgfortran
    5clang: error: linker command failed with exit code 1 (use -v to see invocation)
    6make: *** [sourceCpp_4.so] Error 1
    7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
    8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
    9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
    10  Error 1 occurred building shared library.
    11> Sys.getenv("PATH")
    12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
    13R version 4.1.1 (2021-08-10)
    14Platform: aarch64-apple-darwin20 (64-bit)
    15Running under: macOS Monterey 12.1
    16
    17Matrix products: default
    18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
    19
    20locale:
    21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    22
    23attached base packages:
    24[1] stats     graphics  grDevices utils     datasets  methods   base     
    25
    26loaded via a namespace (and not attached):
    27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
    28[4] Rcpp_1.0.7        
    29$ sudo xcode-select --install
    30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
    31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
    32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
    33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
    34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
    35

    After unpacking, you should find these files on your system:

    1> Rcpp::sourceCpp("~/github/helloworld.cpp")
    2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
    3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
    4ld: library not found for -lgfortran
    5clang: error: linker command failed with exit code 1 (use -v to see invocation)
    6make: *** [sourceCpp_4.so] Error 1
    7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
    8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
    9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
    10  Error 1 occurred building shared library.
    11> Sys.getenv("PATH")
    12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
    13R version 4.1.1 (2021-08-10)
    14Platform: aarch64-apple-darwin20 (64-bit)
    15Running under: macOS Monterey 12.1
    16
    17Matrix products: default
    18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
    19
    20locale:
    21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    22
    23attached base packages:
    24[1] stats     graphics  grDevices utils     datasets  methods   base     
    25
    26loaded via a namespace (and not attached):
    27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
    28[4] Rcpp_1.0.7        
    29$ sudo xcode-select --install
    30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
    31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
    32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
    33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
    34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
    35/usr/local/lib/libomp.dylib
    36/usr/local/include/ompt.h
    37/usr/local/include/omp.h
    38/usr/local/include/omp-tools.h
    39
  • Add the following lines to $(HOME)/.R/Makevars, creating the file if necessary.

  • 1> Rcpp::sourceCpp("~/github/helloworld.cpp")
    2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
    3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
    4ld: library not found for -lgfortran
    5clang: error: linker command failed with exit code 1 (use -v to see invocation)
    6make: *** [sourceCpp_4.so] Error 1
    7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
    8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
    9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
    10  Error 1 occurred building shared library.
    11> Sys.getenv("PATH")
    12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
    13R version 4.1.1 (2021-08-10)
    14Platform: aarch64-apple-darwin20 (64-bit)
    15Running under: macOS Monterey 12.1
    16
    17Matrix products: default
    18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
    19
    20locale:
    21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    22
    23attached base packages:
    24[1] stats     graphics  grDevices utils     datasets  methods   base     
    25
    26loaded via a namespace (and not attached):
    27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
    28[4] Rcpp_1.0.7        
    29$ sudo xcode-select --install
    30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
    31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
    32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
    33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
    34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
    35/usr/local/lib/libomp.dylib
    36/usr/local/include/ompt.h
    37/usr/local/include/omp.h
    38/usr/local/include/omp-tools.h
    39CPPFLAGS+=-I/usr/local/include -Xclang -fopenmp
    40LDFLAGS+=-L/usr/local/lib -lomp
    41
    42FC=/opt/R/arm64/gfortran/bin/gfortran -mtune=native
    43FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm
    44
  • Run R and test that you can compile a program with OpenMP support. For example:

  • 1> Rcpp::sourceCpp("~/github/helloworld.cpp")
    2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
    3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
    4ld: library not found for -lgfortran
    5clang: error: linker command failed with exit code 1 (use -v to see invocation)
    6make: *** [sourceCpp_4.so] Error 1
    7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
    8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
    9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
    10  Error 1 occurred building shared library.
    11> Sys.getenv("PATH")
    12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
    13R version 4.1.1 (2021-08-10)
    14Platform: aarch64-apple-darwin20 (64-bit)
    15Running under: macOS Monterey 12.1
    16
    17Matrix products: default
    18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
    19
    20locale:
    21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    22
    23attached base packages:
    24[1] stats     graphics  grDevices utils     datasets  methods   base     
    25
    26loaded via a namespace (and not attached):
    27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
    28[4] Rcpp_1.0.7        
    29$ sudo xcode-select --install
    30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
    31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
    32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
    33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
    34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
    35/usr/local/lib/libomp.dylib
    36/usr/local/include/ompt.h
    37/usr/local/include/omp.h
    38/usr/local/include/omp-tools.h
    39CPPFLAGS+=-I/usr/local/include -Xclang -fopenmp
    40LDFLAGS+=-L/usr/local/lib -lomp
    41
    42FC=/opt/R/arm64/gfortran/bin/gfortran -mtune=native
    43FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm
    44if (!requireNamespace("RcppArmadillo", quietly = TRUE)) {
    45    install.packages("RcppArmadillo")
    46}
    47Rcpp::sourceCpp(code = '
    48#include <RcppArmadillo.h>
    49#ifdef _OPENMP
    50# include <omp.h>
    51#endif
    52
    53// [[Rcpp::depends(RcppArmadillo)]]
    54// [[Rcpp::export]]
    55void omp_test()
    56{
    57#ifdef _OPENMP
    58    Rprintf("OpenMP threads available: %d\\n", omp_get_max_threads());
    59#else
    60    Rprintf("OpenMP not supported\\n");
    61#endif
    62}
    63')
    64omp_test()
    65
    1> Rcpp::sourceCpp("~/github/helloworld.cpp")
    2ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0'
    3ld: warning: directory not found for option '-L/opt/R/arm64/gfortran/lib'
    4ld: library not found for -lgfortran
    5clang: error: linker command failed with exit code 1 (use -v to see invocation)
    6make: *** [sourceCpp_4.so] Error 1
    7clang++ -arch arm64 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG -I../inst/include   -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/Rcpp/include" -I"/Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/library/RcppArmadillo/include" -I"/Users/afredston/github" -I/opt/R/arm64/include   -fPIC  -falign-functions=64 -Wall -g -O2  -c helloworld.cpp -o helloworld.o
    8clang++ -arch arm64 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/opt/R/arm64/lib -o sourceCpp_4.so helloworld.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
    9Error in Rcpp::sourceCpp("~/github/helloworld.cpp") : 
    10  Error 1 occurred building shared library.
    11> Sys.getenv("PATH")
    12[1] "/opt/homebrew/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Library/TeX/texbin:/Applications/RStudio.app/Contents/MacOS/postback"
    13R version 4.1.1 (2021-08-10)
    14Platform: aarch64-apple-darwin20 (64-bit)
    15Running under: macOS Monterey 12.1
    16
    17Matrix products: default
    18LAPACK: /Library/Frameworks/R.framework/Versions/4.1-arm64/Resources/lib/libRlapack.dylib
    19
    20locale:
    21[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    22
    23attached base packages:
    24[1] stats     graphics  grDevices utils     datasets  methods   base     
    25
    26loaded via a namespace (and not attached):
    27[1] compiler_4.1.1           tools_4.1.1              RcppArmadillo_0.10.7.5.0
    28[4] Rcpp_1.0.7        
    29$ sudo xcode-select --install
    30$ wget https://mac.r-project.org/libs-arm64/gfortran-f51f1da0-darwin20.0-arm64.tar.gz
    31$ sudo tar xvf gfortran-f51f1da0-darwin20.0-arm64.tar.gz -C /
    32$ sudo ln -sfn $(xcrun --show-sdk-path) /opt/R/arm64/gfortran/SDK
    33$ wget https://mac.r-project.org/openmp/openmp-12.0.1-darwin20-Release.tar.gz
    34$ sudo tar xvf openmp-12.0.1-darwin20-Release.tar.gz -C /
    35/usr/local/lib/libomp.dylib
    36/usr/local/include/ompt.h
    37/usr/local/include/omp.h
    38/usr/local/include/omp-tools.h
    39CPPFLAGS+=-I/usr/local/include -Xclang -fopenmp
    40LDFLAGS+=-L/usr/local/lib -lomp
    41
    42FC=/opt/R/arm64/gfortran/bin/gfortran -mtune=native
    43FLIBS=-L/opt/R/arm64/gfortran/lib/gcc/aarch64-apple-darwin20.2.0/11.0.0 -L/opt/R/arm64/gfortran/lib -lgfortran -lemutls_w -lm
    44if (!requireNamespace("RcppArmadillo", quietly = TRUE)) {
    45    install.packages("RcppArmadillo")
    46}
    47Rcpp::sourceCpp(code = '
    48#include <RcppArmadillo.h>
    49#ifdef _OPENMP
    50# include <omp.h>
    51#endif
    52
    53// [[Rcpp::depends(RcppArmadillo)]]
    54// [[Rcpp::export]]
    55void omp_test()
    56{
    57#ifdef _OPENMP
    58    Rprintf("OpenMP threads available: %d\\n", omp_get_max_threads());
    59#else
    60    Rprintf("OpenMP not supported\\n");
    61#endif
    62}
    63')
    64omp_test()
    65OpenMP threads available: 8
    66

    If the C++ code fails to compile, or if it compiles without error but you get linker warnings or you find that OpenMP is not supported, then something is likely wrong. Please report any issues.

    References

    Everything is a bit scattered:

    • R Installation and Administration manual [link]
    • R for macOS Developers page [link]

    Source https://stackoverflow.com/questions/70638118

    QUESTION

    Problem with non-standard evaluation in disk.frame objects using data.table syntax

    Asked 2022-Feb-01 at 17:47
    Problem

    I'm currently trying to write a function that filters some rows of a disk.frame object using regular expressions. I, unfortunately, run into some issues with the evaluation of my search string in the filter function. My idea was to pass a regular expression as a string into a function argument (e.g. storm_name) and then pass that argument into my filtering call. I used the %like% function included in {data.table} for filtering rows.

    My problem is that the storm_name object gets evaluated inside the disk.frame. However, since the storm_name is only included in the function environment, but not in the disk.frame object, I get the following error:

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3

    I already tried to evaluate the storm_nameobject in the parent frame using eval(sotm_name, env = parent.env()), but that also didn't help. Interestingly, this problem only occurs with {disk.frame} objects but not with {data.table} objects.

    For now I found a solution using {dplyr} instead. However, I would be grateful for any ideas on how this problem could be solved with {data.table}.

    Reproducible Example
    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23
    Session Info
    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52

    ANSWER

    Answered 2022-Jan-20 at 17:38

    While I don't know the exact cause of this, it has to do with environments, search path, etc. For instance, these work:

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59

    But this does not:

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    60grep2(storms_df, "^A")
    61# Error in .checkTypos(e, names_x) : 
    62#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    63

    We can work around this with eval(substitute(..)).

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    60grep2(storms_df, "^A")
    61# Error in .checkTypos(e, names_x) : 
    62#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    63grep3 <- function(dfr, storm_name) { 
    64  eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    65}
    66grep3(storms_df, "^A")
    67#        name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    68#      <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    69#   1:    Amy  1975     6    27     0  27.5 -79.0 tropical depression       -1    25     1013          NA          NA
    70#   2:    Amy  1975     6    27     6  28.5 -79.0 tropical depression       -1    25     1013          NA          NA
    71#   3:    Amy  1975     6    27    12  29.5 -79.0 tropical depression       -1    25     1013          NA          NA
    72# ...
    73

    (and grep3(storms_dt, "^A") works too)

    This works by changing the symbol of storm_name inside the [-expression from storm_name to the literal string. Since this is done on the unevaluated expression, there are no lookups yet, no searching through this and inherited environments to find storm_name.

    If you check it manually:

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    60grep2(storms_df, "^A")
    61# Error in .checkTypos(e, names_x) : 
    62#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    63grep3 <- function(dfr, storm_name) { 
    64  eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    65}
    66grep3(storms_df, "^A")
    67#        name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    68#      <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    69#   1:    Amy  1975     6    27     0  27.5 -79.0 tropical depression       -1    25     1013          NA          NA
    70#   2:    Amy  1975     6    27     6  28.5 -79.0 tropical depression       -1    25     1013          NA          NA
    71#   3:    Amy  1975     6    27    12  29.5 -79.0 tropical depression       -1    25     1013          NA          NA
    72# ...
    73debug(grep3)
    74grep3(storms_df, "^A")
    75# debugging in: grep3(storms_df, "^A")
    76# debug at #1: {
    77#     eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    78# }
    79# Browse[2]> 
    80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
    81# dfr[name %like% "^A"]
    82

    I think it's something to do with how disk.frame is affecting the environment within [ and the calling/parent environments. Interestingly (to me), you can see that the search path for variables is not empty, it's just not what we would expect:

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    60grep2(storms_df, "^A")
    61# Error in .checkTypos(e, names_x) : 
    62#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    63grep3 <- function(dfr, storm_name) { 
    64  eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    65}
    66grep3(storms_df, "^A")
    67#        name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    68#      <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    69#   1:    Amy  1975     6    27     0  27.5 -79.0 tropical depression       -1    25     1013          NA          NA
    70#   2:    Amy  1975     6    27     6  28.5 -79.0 tropical depression       -1    25     1013          NA          NA
    71#   3:    Amy  1975     6    27    12  29.5 -79.0 tropical depression       -1    25     1013          NA          NA
    72# ...
    73debug(grep3)
    74grep3(storms_df, "^A")
    75# debugging in: grep3(storms_df, "^A")
    76# debug at #1: {
    77#     eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    78# }
    79# Browse[2]> 
    80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
    81# dfr[name %like% "^A"]
    82grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    83grep2(storms_df, "^A")
    84# Error in .checkTypos(e, names_x) : 
    85#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    86
    87### but let's pre-define `storm_name` outside of the function,
    88### then re-define the function (no change)
    89storm_name <- "^A"
    90grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    91head(grep2(storms_df, "^A"), 2)
    92#      name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    93#    <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    94# 1:    Amy  1975     6    27     0  27.5   -79 tropical depression       -1    25     1013          NA          NA
    95# 2:    Amy  1975     6    27     6  28.5   -79 tropical depression       -1    25     1013          NA          NA
    96

    This seems to work, but we can see that it's using the external version of storm_name vice the parametric version, see that name is still starting with A despite the change to "^B".

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    60grep2(storms_df, "^A")
    61# Error in .checkTypos(e, names_x) : 
    62#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    63grep3 <- function(dfr, storm_name) { 
    64  eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    65}
    66grep3(storms_df, "^A")
    67#        name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    68#      <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    69#   1:    Amy  1975     6    27     0  27.5 -79.0 tropical depression       -1    25     1013          NA          NA
    70#   2:    Amy  1975     6    27     6  28.5 -79.0 tropical depression       -1    25     1013          NA          NA
    71#   3:    Amy  1975     6    27    12  29.5 -79.0 tropical depression       -1    25     1013          NA          NA
    72# ...
    73debug(grep3)
    74grep3(storms_df, "^A")
    75# debugging in: grep3(storms_df, "^A")
    76# debug at #1: {
    77#     eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    78# }
    79# Browse[2]> 
    80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
    81# dfr[name %like% "^A"]
    82grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    83grep2(storms_df, "^A")
    84# Error in .checkTypos(e, names_x) : 
    85#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    86
    87### but let's pre-define `storm_name` outside of the function,
    88### then re-define the function (no change)
    89storm_name <- "^A"
    90grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    91head(grep2(storms_df, "^A"), 2)
    92#      name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    93#    <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    94# 1:    Amy  1975     6    27     0  27.5   -79 tropical depression       -1    25     1013          NA          NA
    95# 2:    Amy  1975     6    27     6  28.5   -79 tropical depression       -1    25     1013          NA          NA
    96head(grep2(storms_df, "^B"), 2)
    97#      name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    98#    <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    99# 1:    Amy  1975     6    27     0  27.5   -79 tropical depression       -1    25     1013          NA          NA
    100# 2:    Amy  1975     6    27     6  28.5   -79 tropical depression       -1    25     1013          NA          NA
    101

    Frankly, I don't understand enough of disk.frame's internals to know if this is a bug or a necessity due to what it must do for non-standard data.table-like evaluation of a not-totally-in-memory dataset.


    If you're concerned with performance (fair question), the eval(substitute(..)) method does not appear to suffer much:

    1Error in .checkTypos(e, names_x) : 
    2  Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    3# Load packages
    4library(data.table)
    5library(disk.frame)
    6
    7# Create data table and diskframe object of storm data
    8storms_df <- as.disk.frame(storms)
    9storms_dt <- as.data.table(storms)
    10
    11# Create search function
    12grep_storm_name <- function(dfr, storm_name){
    13  
    14  dfr[name %like% storm_name]
    15  
    16}
    17
    18# Check function with data.table object
    19grep_storm_name(storms_dt, "^A")
    20
    21# Check function with diskframe object
    22grep_storm_name(storms_df, "^A")
    23R version 4.1.0 (2021-05-18)
    24Platform: x86_64-w64-mingw32/x64 (64-bit)
    25Running under: Windows 10 x64 (build 19043)
    26
    27Matrix products: default
    28
    29locale:
    30[1] LC_COLLATE=English_Sweden.1252  LC_CTYPE=English_Sweden.1252    LC_MONETARY=English_Sweden.1252
    31[4] LC_NUMERIC=C                    LC_TIME=English_Sweden.1252    
    32
    33attached base packages:
    34[1] stats     graphics  grDevices utils     datasets  methods   base     
    35
    36other attached packages:
    37[1] disk.frame_0.5.0  purrr_0.3.4       dplyr_1.0.7       data.table_1.14.0
    38
    39loaded via a namespace (and not attached):
    40 [1] Rcpp_1.0.7            benchmarkmeData_1.0.4 pryr_0.1.4            pillar_1.6.4         
    41 [5] compiler_4.1.0        iterators_1.0.13      tools_4.1.0           digest_0.6.27        
    42 [9] bit_4.0.4             jsonlite_1.7.2        lifecycle_1.0.1       tibble_3.1.6         
    43[13] lattice_0.20-44       pkgconfig_2.0.3       rlang_0.4.12          Matrix_1.3-3         
    44[17] foreach_1.5.1         rstudioapi_0.13       DBI_1.1.1             parallel_4.1.0       
    45[21] bigassertr_0.1.4      bigreadr_0.2.4        httr_1.4.2            stringr_1.4.0        
    46[25] globals_0.14.0        generics_0.1.1        fs_1.5.0              vctrs_0.3.8          
    47[29] bit64_4.0.5           grid_4.1.0            tidyselect_1.1.1      glue_1.6.0           
    48[33] listenv_0.8.0         R6_2.5.1              future.apply_1.7.0    parallelly_1.25.0    
    49[37] fansi_1.0.0           magrittr_2.0.1        codetools_0.2-18      ellipsis_0.3.2       
    50[41] fst_0.9.4             assertthat_0.2.1      future_1.21.0         benchmarkme_1.0.7    
    51[45] utf8_1.2.2            stringi_1.7.6         doParallel_1.0.16     crayon_1.4.2 
    52storms_df[name %like% "^A"]
    53
    54nm <- "^A"
    55storms_df[name %like% nm]
    56
    57grep1 <- function(dfr, storm_name) { dfr[name %like% "^A"]; }
    58grep1(storms_df)
    59grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    60grep2(storms_df, "^A")
    61# Error in .checkTypos(e, names_x) : 
    62#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    63grep3 <- function(dfr, storm_name) { 
    64  eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    65}
    66grep3(storms_df, "^A")
    67#        name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    68#      <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    69#   1:    Amy  1975     6    27     0  27.5 -79.0 tropical depression       -1    25     1013          NA          NA
    70#   2:    Amy  1975     6    27     6  28.5 -79.0 tropical depression       -1    25     1013          NA          NA
    71#   3:    Amy  1975     6    27    12  29.5 -79.0 tropical depression       -1    25     1013          NA          NA
    72# ...
    73debug(grep3)
    74grep3(storms_df, "^A")
    75# debugging in: grep3(storms_df, "^A")
    76# debug at #1: {
    77#     eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name)))
    78# }
    79# Browse[2]> 
    80substitute(dfr[name %like% storm_name], list(storm_name = storm_name))
    81# dfr[name %like% "^A"]
    82grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    83grep2(storms_df, "^A")
    84# Error in .checkTypos(e, names_x) : 
    85#   Object 'storm_name' not found amongst name, year, month, day, hour and 8 more
    86
    87### but let's pre-define `storm_name` outside of the function,
    88### then re-define the function (no change)
    89storm_name <- "^A"
    90grep2 <- function(dfr, storm_name) { dfr[name %like% storm_name]; }
    91head(grep2(storms_df, "^A"), 2)
    92#      name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    93#    <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    94# 1:    Amy  1975     6    27     0  27.5   -79 tropical depression       -1    25     1013          NA          NA
    95# 2:    Amy  1975     6    27     6  28.5   -79 tropical depression       -1    25     1013          NA          NA
    96head(grep2(storms_df, "^B"), 2)
    97#      name  year month   day  hour   lat  long              status category  wind pressure ts_diameter hu_diameter
    98#    <char> <num> <num> <int> <num> <num> <num>              <char>    <ord> <int>    <int>       <num>       <num>
    99# 1:    Amy  1975     6    27     0  27.5   -79 tropical depression       -1    25     1013          NA          NA
    100# 2:    Amy  1975     6    27     6  28.5   -79 tropical depression       -1    25     1013          NA          NA
    101bench::mark(
    102  raw = dfr[name %like% "^A"],
    103  subst = eval(substitute(dfr[name %like% storm_name], list(storm_name = storm_name))),
    104  iterations = 1000
    105)
    106# # A tibble: 2 x 13
    107#   expression      min   median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc total_time result                  memory               time               gc                  
    108#   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl> <int> <dbl>   <bch:tm> <list>                  <list>               <list>             <list>              
    109# 1 raw          12.9ms   16.8ms      55.2    1.69MB     3.97   933    67      16.9s <data.table [990 x 13]> <Rprofmem [669 x 3]> <bench_tm [1,000]> <tibble [1,000 x 3]>
    110# 2 subst        12.8ms   15.8ms      60.5    1.69MB     3.25   949    51      15.7s <data.table [990 x 13]> <Rprofmem [669 x 3]> <bench_tm [1,000]> <tibble [1,000 x 3]>
    111

    In repeated benchmarks, I've actually seen subst slightly faster, suggesting that a portion of the performance difference is unrelated to the addition of eval(substitute(..)). This difference (55.2 to 60.5 `itr/sec`) is the worst I've seen it ... a repeat just now had 57.1 and 57.5, so I suggest that performance-degradation is not a concern.

    Source https://stackoverflow.com/questions/70788596

    QUESTION

    How to automate legends for a new geom in ggplot2?

    Asked 2022-Jan-30 at 18:08

    I've built this new ggplot2 geom layer I'm calling geom_triangles (see https://github.com/ctesta01/ggtriangles/) that plots isosceles triangles given aesthetics including x, y, z where z is the height of the triangle and the base of the isosceles triangle has midpoint (x,y) on the graph.

    What I want is for the geom_triangles() layer to automatically provide legend components for the height and width of the triangles, but I am not sure how to do that.

    I understand based on this reference that I may need to adjust the draw_key argument in the ggproto StatTriangles object, but I'm not sure how I would do that and can't seem to find examples online of how to do it. I've been looking at the source code in ggplot2 for the draw_key functions, but I'm not sure how I would introduce multiple legend components (one for each of height and width) in a single draw_key argument in the StatTriangles ggproto.

    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130

    first plot example with mtcars, geom_triangles, and color legend

    What I have been able to do is to write helper functions (draw_geom_triangles_height_legend, draw_geom_triangles_width_legend) and use the patchwork, and cowplot packages to make legend components rather manually and combining them in an appropriate grid with the original plot, but I want to make producing these legend components automatic. The following code also uses the ggrepel package to add text labels in the figure.

    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130draw_geom_triangles_height_legend <- function(
    131  width = 1,
    132  width_scale = .1,
    133  height_scale = .1,
    134  z_values = 1:3,
    135  n.breaks = 3,
    136  labels = c("low", "medium", "high"),
    137  color = 'black',
    138  fill = 'black'
    139) {
    140  ggplot(
    141    data = data.frame(x = rep(0, times = n.breaks),
    142                      y = seq(1,n.breaks),
    143                      z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
    144                      width = width,
    145                      label = labels,
    146                      color = color,
    147                      fill = fill
    148    ),
    149    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    150  ) +
    151    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    152    geom_text(mapping = aes(x = x + .5), size = 3) +
    153    expand_limits(x = c(-.25, 3/4)) +
    154    theme_void() +
    155    theme(plot.title = element_text(size = 10, hjust = .5))
    156}
    157
    158draw_geom_triangles_width_legend <- function(
    159  width = 1:3,
    160  width_scale = .1,
    161  height_scale = .1,
    162  z_values = 1,
    163  n.breaks = 3,
    164  labels = c("low", "medium", "high"),
    165  color = 'black',
    166  fill = 'black'
    167) {
    168  ggplot(
    169    data = data.frame(x = rep(0, times = n.breaks),
    170                      y = seq(1, n.breaks),
    171                      z = rep(1, n.breaks),
    172                      width = width,
    173                      label = labels,
    174                      color = color,
    175                      fill = fill
    176    ),
    177    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    178  ) +
    179    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    180    geom_text(mapping = aes(x = x + .5), size = 3) +
    181    expand_limits(x = c(-.25, 3/4)) +
    182    theme_void() +
    183    theme(plot.title = element_text(size = 10, hjust = .5))
    184}
    185
    186# extract the original legend - this is for the color and fill (hp)
    187legend_hp <- cowplot::get_legend(plt_orig)
    188
    189# remove the legend from the plot
    190plt <- plt_orig + theme(legend.position = 'none')
    191
    192# create a height legend using draw_geom_triangles_height_legend
    193height_legend <- 
    194  draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
    195                                    labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
    196                                    ) +
    197                                    ggtitle("cylinders\n")
    198
    199
    200# create a width legend using draw_geom_triangles_width_legend
    201width_legend <- 
    202  draw_geom_triangles_width_legend(
    203  width = quantile(mtcars$wt, c(.33, .66, 1)),
    204  labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
    205  width_scale = .2
    206  ) +
    207  ggtitle("weight\n(1000 lbs)\n")
    208
    209blank_plot <- ggplot() + theme_void()
    210  
    211# create a legend column layout
    212# 
    213# whitespace is used above, below, and in-between the legend components to
    214# make sure the legend column pieces don't appear too densely stacked.
    215# 
    216legend_component <-
    217  (blank_plot /  cowplot::plot_grid(legend_hp) / blank_plot /  height_legend / blank_plot / width_legend / blank_plot) +
    218  plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
    219
    220# create the layout with the plot and the legend component
    221(plt + legend_component) + 
    222  plot_layout(nrow = 1, widths = c(1, .15))
    223

    second plot with mtcars, geom_triangles, with added legend components for height and width

    What I'm looking for is to be able to run the code for the first plot example and get a legend with 3 components similar to the color/fill, height, and width legend components as in the second plot example.

    Unfortunately the helper functions are not at all satisfactory because at present one has to rely on visually estimating whether the legend's height_scale and width_scale components look correct. This is because the lengeds produced by draw_geom_triangles_height_legend and draw_geom_triangles_width_legend are their own ggplot objects and therefore aren't necessarily on the same coordinate scaling system as the main ggplot of interest for which they are supposed to be legends.

    Both of the plots I included are rendered at 7in x 8.5in using ggsave.

    Here's my R sessionInfo()

    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130draw_geom_triangles_height_legend <- function(
    131  width = 1,
    132  width_scale = .1,
    133  height_scale = .1,
    134  z_values = 1:3,
    135  n.breaks = 3,
    136  labels = c("low", "medium", "high"),
    137  color = 'black',
    138  fill = 'black'
    139) {
    140  ggplot(
    141    data = data.frame(x = rep(0, times = n.breaks),
    142                      y = seq(1,n.breaks),
    143                      z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
    144                      width = width,
    145                      label = labels,
    146                      color = color,
    147                      fill = fill
    148    ),
    149    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    150  ) +
    151    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    152    geom_text(mapping = aes(x = x + .5), size = 3) +
    153    expand_limits(x = c(-.25, 3/4)) +
    154    theme_void() +
    155    theme(plot.title = element_text(size = 10, hjust = .5))
    156}
    157
    158draw_geom_triangles_width_legend <- function(
    159  width = 1:3,
    160  width_scale = .1,
    161  height_scale = .1,
    162  z_values = 1,
    163  n.breaks = 3,
    164  labels = c("low", "medium", "high"),
    165  color = 'black',
    166  fill = 'black'
    167) {
    168  ggplot(
    169    data = data.frame(x = rep(0, times = n.breaks),
    170                      y = seq(1, n.breaks),
    171                      z = rep(1, n.breaks),
    172                      width = width,
    173                      label = labels,
    174                      color = color,
    175                      fill = fill
    176    ),
    177    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    178  ) +
    179    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    180    geom_text(mapping = aes(x = x + .5), size = 3) +
    181    expand_limits(x = c(-.25, 3/4)) +
    182    theme_void() +
    183    theme(plot.title = element_text(size = 10, hjust = .5))
    184}
    185
    186# extract the original legend - this is for the color and fill (hp)
    187legend_hp <- cowplot::get_legend(plt_orig)
    188
    189# remove the legend from the plot
    190plt <- plt_orig + theme(legend.position = 'none')
    191
    192# create a height legend using draw_geom_triangles_height_legend
    193height_legend <- 
    194  draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
    195                                    labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
    196                                    ) +
    197                                    ggtitle("cylinders\n")
    198
    199
    200# create a width legend using draw_geom_triangles_width_legend
    201width_legend <- 
    202  draw_geom_triangles_width_legend(
    203  width = quantile(mtcars$wt, c(.33, .66, 1)),
    204  labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
    205  width_scale = .2
    206  ) +
    207  ggtitle("weight\n(1000 lbs)\n")
    208
    209blank_plot <- ggplot() + theme_void()
    210  
    211# create a legend column layout
    212# 
    213# whitespace is used above, below, and in-between the legend components to
    214# make sure the legend column pieces don't appear too densely stacked.
    215# 
    216legend_component <-
    217  (blank_plot /  cowplot::plot_grid(legend_hp) / blank_plot /  height_legend / blank_plot / width_legend / blank_plot) +
    218  plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
    219
    220# create the layout with the plot and the legend component
    221(plt + legend_component) + 
    222  plot_layout(nrow = 1, widths = c(1, .15))
    223> sessionInfo()
    224R version 4.1.2 (2021-11-01)
    225Platform: x86_64-apple-darwin17.0 (64-bit)
    226Running under: macOS Mojave 10.14.2
    227
    228Matrix products: default
    229BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
    231
    232locale:
    233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    234
    235attached base packages:
    236[1] stats     graphics  grDevices utils     datasets  methods   base     
    237
    238other attached packages:
    239[1] patchwork_1.1.1 cowplot_1.1.1   tibble_3.1.6    ggrepel_0.9.1   dplyr_1.0.7     magrittr_2.0.1  ggplot2_3.3.5   colorout_1.2-2 
    240
    241loaded via a namespace (and not attached):
    242 [1] Rcpp_1.0.7        tidyselect_1.1.1  munsell_0.5.0     viridisLite_0.4.0 colorspace_2.0-2  R6_2.5.1          rlang_0.4.12      fansi_0.5.0      
    243 [9] tools_4.1.2       grid_4.1.2        gtable_0.3.0      utf8_1.2.2        DBI_1.1.2         withr_2.4.3       ellipsis_0.3.2    digest_0.6.29    
    244[17] yaml_2.2.1        assertthat_0.2.1  lifecycle_1.0.1   crayon_1.4.2      tidyr_1.1.4       farver_2.1.0      purrr_0.3.4       vctrs_0.3.8      
    245[25] glue_1.6.0        labeling_0.4.2    compiler_4.1.2    pillar_1.6.4      generics_0.1.1    scales_1.1.1      pkgconfig_2.0.3  
    246

    ANSWER

    Answered 2022-Jan-30 at 18:08

    I think you might be slightly overcomplicating things. Ideally, you'd just want a single key drawing method for the whole layer. However, because you're using a Stat to do the majority of calculations, this becomes hairy to implement. In my answer, I'm avoiding this.

    Let's say I'd want to use a geom-only implementation of such a layer. I can make the following (simplified) class/constructor pair. Below, I haven't bothered width_scale or height_scale parameters, just for simplicity.

    Class
    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130draw_geom_triangles_height_legend <- function(
    131  width = 1,
    132  width_scale = .1,
    133  height_scale = .1,
    134  z_values = 1:3,
    135  n.breaks = 3,
    136  labels = c("low", "medium", "high"),
    137  color = 'black',
    138  fill = 'black'
    139) {
    140  ggplot(
    141    data = data.frame(x = rep(0, times = n.breaks),
    142                      y = seq(1,n.breaks),
    143                      z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
    144                      width = width,
    145                      label = labels,
    146                      color = color,
    147                      fill = fill
    148    ),
    149    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    150  ) +
    151    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    152    geom_text(mapping = aes(x = x + .5), size = 3) +
    153    expand_limits(x = c(-.25, 3/4)) +
    154    theme_void() +
    155    theme(plot.title = element_text(size = 10, hjust = .5))
    156}
    157
    158draw_geom_triangles_width_legend <- function(
    159  width = 1:3,
    160  width_scale = .1,
    161  height_scale = .1,
    162  z_values = 1,
    163  n.breaks = 3,
    164  labels = c("low", "medium", "high"),
    165  color = 'black',
    166  fill = 'black'
    167) {
    168  ggplot(
    169    data = data.frame(x = rep(0, times = n.breaks),
    170                      y = seq(1, n.breaks),
    171                      z = rep(1, n.breaks),
    172                      width = width,
    173                      label = labels,
    174                      color = color,
    175                      fill = fill
    176    ),
    177    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    178  ) +
    179    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    180    geom_text(mapping = aes(x = x + .5), size = 3) +
    181    expand_limits(x = c(-.25, 3/4)) +
    182    theme_void() +
    183    theme(plot.title = element_text(size = 10, hjust = .5))
    184}
    185
    186# extract the original legend - this is for the color and fill (hp)
    187legend_hp <- cowplot::get_legend(plt_orig)
    188
    189# remove the legend from the plot
    190plt <- plt_orig + theme(legend.position = 'none')
    191
    192# create a height legend using draw_geom_triangles_height_legend
    193height_legend <- 
    194  draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
    195                                    labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
    196                                    ) +
    197                                    ggtitle("cylinders\n")
    198
    199
    200# create a width legend using draw_geom_triangles_width_legend
    201width_legend <- 
    202  draw_geom_triangles_width_legend(
    203  width = quantile(mtcars$wt, c(.33, .66, 1)),
    204  labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
    205  width_scale = .2
    206  ) +
    207  ggtitle("weight\n(1000 lbs)\n")
    208
    209blank_plot <- ggplot() + theme_void()
    210  
    211# create a legend column layout
    212# 
    213# whitespace is used above, below, and in-between the legend components to
    214# make sure the legend column pieces don't appear too densely stacked.
    215# 
    216legend_component <-
    217  (blank_plot /  cowplot::plot_grid(legend_hp) / blank_plot /  height_legend / blank_plot / width_legend / blank_plot) +
    218  plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
    219
    220# create the layout with the plot and the legend component
    221(plt + legend_component) + 
    222  plot_layout(nrow = 1, widths = c(1, .15))
    223> sessionInfo()
    224R version 4.1.2 (2021-11-01)
    225Platform: x86_64-apple-darwin17.0 (64-bit)
    226Running under: macOS Mojave 10.14.2
    227
    228Matrix products: default
    229BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
    231
    232locale:
    233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    234
    235attached base packages:
    236[1] stats     graphics  grDevices utils     datasets  methods   base     
    237
    238other attached packages:
    239[1] patchwork_1.1.1 cowplot_1.1.1   tibble_3.1.6    ggrepel_0.9.1   dplyr_1.0.7     magrittr_2.0.1  ggplot2_3.3.5   colorout_1.2-2 
    240
    241loaded via a namespace (and not attached):
    242 [1] Rcpp_1.0.7        tidyselect_1.1.1  munsell_0.5.0     viridisLite_0.4.0 colorspace_2.0-2  R6_2.5.1          rlang_0.4.12      fansi_0.5.0      
    243 [9] tools_4.1.2       grid_4.1.2        gtable_0.3.0      utf8_1.2.2        DBI_1.1.2         withr_2.4.3       ellipsis_0.3.2    digest_0.6.29    
    244[17] yaml_2.2.1        assertthat_0.2.1  lifecycle_1.0.1   crayon_1.4.2      tidyr_1.1.4       farver_2.1.0      purrr_0.3.4       vctrs_0.3.8      
    245[25] glue_1.6.0        labeling_0.4.2    compiler_4.1.2    pillar_1.6.4      generics_0.1.1    scales_1.1.1      pkgconfig_2.0.3  
    246library(ggplot2)
    247
    248GeomTriangles <- ggproto(
    249  "GeomTriangles", GeomPoint,
    250  default_aes = aes(
    251    colour = "black", fill = "black", size = 0.5, linetype = 1, 
    252    alpha = 1, angle = 0, width = 0.5, height = 0.5
    253  ),
    254  
    255  draw_panel = function(
    256    data, panel_params, coord, na.rm = FALSE
    257  ) {
    258    # Apply coordinate transform
    259    df <- coord$transform(data, panel_params)
    260    
    261    # Repeat every row 3x
    262    idx <- rep(seq_len(nrow(df)), each = 3)
    263    rep_df <- df[idx, ]
    264    # Calculate offsets from origin
    265    x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
    266    y_off <- as.vector(outer(c(0, 1, 0), df$height))
    267    
    268    # Rotate offsets
    269    ang <- rep_df$angle * (pi / 180)
    270    x_new <- x_off * cos(ang) - y_off * sin(ang)
    271    y_new <- x_off * sin(ang) + y_off * cos(ang)
    272    
    273    # Combine offsets with origin
    274    x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
    275    y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
    276    
    277    grid::polygonGrob(
    278      x = x, y = y, id = idx,
    279      gp = grid::gpar(
    280        col  = alpha(df$colour, df$alpha),
    281        fill = alpha(df$fill, df$alpha),
    282        lwd  = df$size * .pt,
    283        lty  = df$linetype
    284      )
    285    )
    286  }
    287)
    288
    Constructor
    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130draw_geom_triangles_height_legend <- function(
    131  width = 1,
    132  width_scale = .1,
    133  height_scale = .1,
    134  z_values = 1:3,
    135  n.breaks = 3,
    136  labels = c("low", "medium", "high"),
    137  color = 'black',
    138  fill = 'black'
    139) {
    140  ggplot(
    141    data = data.frame(x = rep(0, times = n.breaks),
    142                      y = seq(1,n.breaks),
    143                      z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
    144                      width = width,
    145                      label = labels,
    146                      color = color,
    147                      fill = fill
    148    ),
    149    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    150  ) +
    151    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    152    geom_text(mapping = aes(x = x + .5), size = 3) +
    153    expand_limits(x = c(-.25, 3/4)) +
    154    theme_void() +
    155    theme(plot.title = element_text(size = 10, hjust = .5))
    156}
    157
    158draw_geom_triangles_width_legend <- function(
    159  width = 1:3,
    160  width_scale = .1,
    161  height_scale = .1,
    162  z_values = 1,
    163  n.breaks = 3,
    164  labels = c("low", "medium", "high"),
    165  color = 'black',
    166  fill = 'black'
    167) {
    168  ggplot(
    169    data = data.frame(x = rep(0, times = n.breaks),
    170                      y = seq(1, n.breaks),
    171                      z = rep(1, n.breaks),
    172                      width = width,
    173                      label = labels,
    174                      color = color,
    175                      fill = fill
    176    ),
    177    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    178  ) +
    179    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    180    geom_text(mapping = aes(x = x + .5), size = 3) +
    181    expand_limits(x = c(-.25, 3/4)) +
    182    theme_void() +
    183    theme(plot.title = element_text(size = 10, hjust = .5))
    184}
    185
    186# extract the original legend - this is for the color and fill (hp)
    187legend_hp <- cowplot::get_legend(plt_orig)
    188
    189# remove the legend from the plot
    190plt <- plt_orig + theme(legend.position = 'none')
    191
    192# create a height legend using draw_geom_triangles_height_legend
    193height_legend <- 
    194  draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
    195                                    labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
    196                                    ) +
    197                                    ggtitle("cylinders\n")
    198
    199
    200# create a width legend using draw_geom_triangles_width_legend
    201width_legend <- 
    202  draw_geom_triangles_width_legend(
    203  width = quantile(mtcars$wt, c(.33, .66, 1)),
    204  labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
    205  width_scale = .2
    206  ) +
    207  ggtitle("weight\n(1000 lbs)\n")
    208
    209blank_plot <- ggplot() + theme_void()
    210  
    211# create a legend column layout
    212# 
    213# whitespace is used above, below, and in-between the legend components to
    214# make sure the legend column pieces don't appear too densely stacked.
    215# 
    216legend_component <-
    217  (blank_plot /  cowplot::plot_grid(legend_hp) / blank_plot /  height_legend / blank_plot / width_legend / blank_plot) +
    218  plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
    219
    220# create the layout with the plot and the legend component
    221(plt + legend_component) + 
    222  plot_layout(nrow = 1, widths = c(1, .15))
    223> sessionInfo()
    224R version 4.1.2 (2021-11-01)
    225Platform: x86_64-apple-darwin17.0 (64-bit)
    226Running under: macOS Mojave 10.14.2
    227
    228Matrix products: default
    229BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
    231
    232locale:
    233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    234
    235attached base packages:
    236[1] stats     graphics  grDevices utils     datasets  methods   base     
    237
    238other attached packages:
    239[1] patchwork_1.1.1 cowplot_1.1.1   tibble_3.1.6    ggrepel_0.9.1   dplyr_1.0.7     magrittr_2.0.1  ggplot2_3.3.5   colorout_1.2-2 
    240
    241loaded via a namespace (and not attached):
    242 [1] Rcpp_1.0.7        tidyselect_1.1.1  munsell_0.5.0     viridisLite_0.4.0 colorspace_2.0-2  R6_2.5.1          rlang_0.4.12      fansi_0.5.0      
    243 [9] tools_4.1.2       grid_4.1.2        gtable_0.3.0      utf8_1.2.2        DBI_1.1.2         withr_2.4.3       ellipsis_0.3.2    digest_0.6.29    
    244[17] yaml_2.2.1        assertthat_0.2.1  lifecycle_1.0.1   crayon_1.4.2      tidyr_1.1.4       farver_2.1.0      purrr_0.3.4       vctrs_0.3.8      
    245[25] glue_1.6.0        labeling_0.4.2    compiler_4.1.2    pillar_1.6.4      generics_0.1.1    scales_1.1.1      pkgconfig_2.0.3  
    246library(ggplot2)
    247
    248GeomTriangles <- ggproto(
    249  "GeomTriangles", GeomPoint,
    250  default_aes = aes(
    251    colour = "black", fill = "black", size = 0.5, linetype = 1, 
    252    alpha = 1, angle = 0, width = 0.5, height = 0.5
    253  ),
    254  
    255  draw_panel = function(
    256    data, panel_params, coord, na.rm = FALSE
    257  ) {
    258    # Apply coordinate transform
    259    df <- coord$transform(data, panel_params)
    260    
    261    # Repeat every row 3x
    262    idx <- rep(seq_len(nrow(df)), each = 3)
    263    rep_df <- df[idx, ]
    264    # Calculate offsets from origin
    265    x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
    266    y_off <- as.vector(outer(c(0, 1, 0), df$height))
    267    
    268    # Rotate offsets
    269    ang <- rep_df$angle * (pi / 180)
    270    x_new <- x_off * cos(ang) - y_off * sin(ang)
    271    y_new <- x_off * sin(ang) + y_off * cos(ang)
    272    
    273    # Combine offsets with origin
    274    x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
    275    y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
    276    
    277    grid::polygonGrob(
    278      x = x, y = y, id = idx,
    279      gp = grid::gpar(
    280        col  = alpha(df$colour, df$alpha),
    281        fill = alpha(df$fill, df$alpha),
    282        lwd  = df$size * .pt,
    283        lty  = df$linetype
    284      )
    285    )
    286  }
    287)
    288geom_triangles <- function(mapping = NULL, data = NULL,
    289                           position = "identity", na.rm = FALSE, show.legend = NA,
    290                           inherit.aes = TRUE, ...) {
    291  layer(
    292    stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
    293    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    294    params = list(na.rm = na.rm, ...)
    295  )
    296}
    297
    Example

    Just to show how it works without any special keys set. I'm letting a continuous scale for width and height take over the job of your width_scale and height_scale parameters, because I didn't want to focus on that here. As you can see, two legends are made automatically, but with the wrong glyphs.

    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130draw_geom_triangles_height_legend <- function(
    131  width = 1,
    132  width_scale = .1,
    133  height_scale = .1,
    134  z_values = 1:3,
    135  n.breaks = 3,
    136  labels = c("low", "medium", "high"),
    137  color = 'black',
    138  fill = 'black'
    139) {
    140  ggplot(
    141    data = data.frame(x = rep(0, times = n.breaks),
    142                      y = seq(1,n.breaks),
    143                      z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
    144                      width = width,
    145                      label = labels,
    146                      color = color,
    147                      fill = fill
    148    ),
    149    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    150  ) +
    151    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    152    geom_text(mapping = aes(x = x + .5), size = 3) +
    153    expand_limits(x = c(-.25, 3/4)) +
    154    theme_void() +
    155    theme(plot.title = element_text(size = 10, hjust = .5))
    156}
    157
    158draw_geom_triangles_width_legend <- function(
    159  width = 1:3,
    160  width_scale = .1,
    161  height_scale = .1,
    162  z_values = 1,
    163  n.breaks = 3,
    164  labels = c("low", "medium", "high"),
    165  color = 'black',
    166  fill = 'black'
    167) {
    168  ggplot(
    169    data = data.frame(x = rep(0, times = n.breaks),
    170                      y = seq(1, n.breaks),
    171                      z = rep(1, n.breaks),
    172                      width = width,
    173                      label = labels,
    174                      color = color,
    175                      fill = fill
    176    ),
    177    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    178  ) +
    179    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    180    geom_text(mapping = aes(x = x + .5), size = 3) +
    181    expand_limits(x = c(-.25, 3/4)) +
    182    theme_void() +
    183    theme(plot.title = element_text(size = 10, hjust = .5))
    184}
    185
    186# extract the original legend - this is for the color and fill (hp)
    187legend_hp <- cowplot::get_legend(plt_orig)
    188
    189# remove the legend from the plot
    190plt <- plt_orig + theme(legend.position = 'none')
    191
    192# create a height legend using draw_geom_triangles_height_legend
    193height_legend <- 
    194  draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
    195                                    labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
    196                                    ) +
    197                                    ggtitle("cylinders\n")
    198
    199
    200# create a width legend using draw_geom_triangles_width_legend
    201width_legend <- 
    202  draw_geom_triangles_width_legend(
    203  width = quantile(mtcars$wt, c(.33, .66, 1)),
    204  labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
    205  width_scale = .2
    206  ) +
    207  ggtitle("weight\n(1000 lbs)\n")
    208
    209blank_plot <- ggplot() + theme_void()
    210  
    211# create a legend column layout
    212# 
    213# whitespace is used above, below, and in-between the legend components to
    214# make sure the legend column pieces don't appear too densely stacked.
    215# 
    216legend_component <-
    217  (blank_plot /  cowplot::plot_grid(legend_hp) / blank_plot /  height_legend / blank_plot / width_legend / blank_plot) +
    218  plot_layout(heights = c(1, 1, .5, 1, .5, 1, 1))
    219
    220# create the layout with the plot and the legend component
    221(plt + legend_component) + 
    222  plot_layout(nrow = 1, widths = c(1, .15))
    223> sessionInfo()
    224R version 4.1.2 (2021-11-01)
    225Platform: x86_64-apple-darwin17.0 (64-bit)
    226Running under: macOS Mojave 10.14.2
    227
    228Matrix products: default
    229BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
    230LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
    231
    232locale:
    233[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
    234
    235attached base packages:
    236[1] stats     graphics  grDevices utils     datasets  methods   base     
    237
    238other attached packages:
    239[1] patchwork_1.1.1 cowplot_1.1.1   tibble_3.1.6    ggrepel_0.9.1   dplyr_1.0.7     magrittr_2.0.1  ggplot2_3.3.5   colorout_1.2-2 
    240
    241loaded via a namespace (and not attached):
    242 [1] Rcpp_1.0.7        tidyselect_1.1.1  munsell_0.5.0     viridisLite_0.4.0 colorspace_2.0-2  R6_2.5.1          rlang_0.4.12      fansi_0.5.0      
    243 [9] tools_4.1.2       grid_4.1.2        gtable_0.3.0      utf8_1.2.2        DBI_1.1.2         withr_2.4.3       ellipsis_0.3.2    digest_0.6.29    
    244[17] yaml_2.2.1        assertthat_0.2.1  lifecycle_1.0.1   crayon_1.4.2      tidyr_1.1.4       farver_2.1.0      purrr_0.3.4       vctrs_0.3.8      
    245[25] glue_1.6.0        labeling_0.4.2    compiler_4.1.2    pillar_1.6.4      generics_0.1.1    scales_1.1.1      pkgconfig_2.0.3  
    246library(ggplot2)
    247
    248GeomTriangles <- ggproto(
    249  "GeomTriangles", GeomPoint,
    250  default_aes = aes(
    251    colour = "black", fill = "black", size = 0.5, linetype = 1, 
    252    alpha = 1, angle = 0, width = 0.5, height = 0.5
    253  ),
    254  
    255  draw_panel = function(
    256    data, panel_params, coord, na.rm = FALSE
    257  ) {
    258    # Apply coordinate transform
    259    df <- coord$transform(data, panel_params)
    260    
    261    # Repeat every row 3x
    262    idx <- rep(seq_len(nrow(df)), each = 3)
    263    rep_df <- df[idx, ]
    264    # Calculate offsets from origin
    265    x_off <- as.vector(outer(c(-0.5, 0, 0.5), df$width))
    266    y_off <- as.vector(outer(c(0, 1, 0), df$height))
    267    
    268    # Rotate offsets
    269    ang <- rep_df$angle * (pi / 180)
    270    x_new <- x_off * cos(ang) - y_off * sin(ang)
    271    y_new <- x_off * sin(ang) + y_off * cos(ang)
    272    
    273    # Combine offsets with origin
    274    x <- unit(rep_df$x, "npc") + unit(x_new, "cm")
    275    y <- unit(rep_df$y, "npc") + unit(y_new, "cm")
    276    
    277    grid::polygonGrob(
    278      x = x, y = y, id = idx,
    279      gp = grid::gpar(
    280        col  = alpha(df$colour, df$alpha),
    281        fill = alpha(df$fill, df$alpha),
    282        lwd  = df$size * .pt,
    283        lty  = df$linetype
    284      )
    285    )
    286  }
    287)
    288geom_triangles <- function(mapping = NULL, data = NULL,
    289                           position = "identity", na.rm = FALSE, show.legend = NA,
    290                           inherit.aes = TRUE, ...) {
    291  layer(
    292    stat = "identity", geom = GeomTriangles, data = data, mapping = mapping,
    293    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    294    params = list(na.rm = na.rm, ...)
    295  )
    296}
    297ggplot(mtcars, aes(mpg, disp, height = cyl, width = wt, colour = hp, fill = hp)) +
    298  geom_triangles() +
    299  geom_point(colour = "black") +
    300  continuous_scale("width", "wscale",  
    301                   palette = scales::rescale_pal(c(0.1, 0.5))) +
    302  continuous_scale("height", "hscale", 
    303                   palette = scales::rescale_pal(c(0.1, 0.5)))
    304

    Glyphs

    Writing a function to draw a glyph isn't too difficult. In this case, we do almost the same as GeomTriangles$draw_panel, but we fix the x and y positions of the origin, and don't use a coordinate transform.

    1library(ggplot2)
    2library(magrittr)
    3library(dplyr)
    4library(ggrepel)
    5library(tibble)
    6library(cowplot)
    7library(patchwork)
    8
    9StatTriangles <- ggproto("StatTriangles", Stat,
    10  required_aes = c('x', 'y', 'z'),
    11  compute_group = function(data, scales, params, width = 1, height_scale = .05, width_scale = .05, angle = 0) {
    12
    13    # specify default width
    14    if (is.null(data$width)) data$width <- 1
    15
    16    # for each row of the data, create the 3 points that will make up our
    17    # triangle based on the z, width, height_scale, and width_scale given.
    18        triangle_df <-
    19            tibble::tibble(
    20                group = 1:nrow(data),
    21                point1 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] - width[[i]]/2*width_scale, y[[i]]))}),
    22                point2 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]] + width[[i]]/2*width_scale, y[[i]]))}),
    23                point3 = lapply(1:nrow(data), function(i) {with(data, c(x[[i]], y[[i]] + z[[i]]*height_scale))})
    24            )
    25
    26        # pivot the data into a long format so that each coordinate pair (e.g. vertex)
    27        # will be its own row
    28        triangle_df <- triangle_df %>% tidyr::pivot_longer(
    29            cols = c(point1, point2, point3),
    30            names_to = 'vertex',
    31            values_to = 'coordinates'
    32        )
    33
    34        # extract the coordinates -- this must be done rowwise because
    35        # coordinates is a list where each element is a c(x,y) coordinate pair
    36        triangle_df <- triangle_df %>% rowwise() %>% mutate(
    37            x = coordinates[[1]],
    38            y = coordinates[[2]])
    39
    40        # save the original x and y so we can perform rotations by the
    41        # given angle with reference to (orig_x, orig_y) as the fixed point
    42        # of the rotation transformation
    43    triangle_df$orig_x <- rep(data$x, each = 3)
    44    triangle_df$orig_y <- rep(data$y, each = 3)
    45
    46    # i'm not sure exactly why, but if the group isn't interacted with linetype
    47    # then the edges of the triangles get messed up when rendered when linetype
    48    # is used in an aesthetic
    49    # triangle_df$group <-
    50    #   paste0(triangle_df$orig_x, triangle_df$orig_y, triangle_df$group, rep(data$group, each = 3))
    51
    52        # fill in aesthetics to the dataframe
    53    triangle_df$colour <- rep(data$colour, each = 3)
    54    triangle_df$size <- rep(data$size, each = 3)
    55    triangle_df$fill <- rep(data$fill, each = 3)
    56    triangle_df$linetype <- rep(data$linetype, each = 3)
    57    triangle_df$alpha <- rep(data$alpha, each = 3)
    58    triangle_df$angle <- rep(data$angle, each = 3)
    59
    60    # determine scaling factor in going from y to x
    61    # scale_factor <- diff(range(data$x)) / diff(range(data$y))
    62    scale_factor <- diff(scales$x$get_limits()) / diff(scales$y$get_limits())
    63    if (! is.finite(scale_factor) | is.na(scale_factor)) scale_factor <- 1
    64
    65    # rotate the data according to the angle by first subtracting out the
    66    # (orig_x, orig_y) component, applying coordinate rotations, and then
    67    # adding the (orig_x, orig_y) component back in.
    68        new_coords <- triangle_df %>% mutate(
    69      x_diff = x - orig_x,
    70      y_diff = (y - orig_y) * scale_factor,
    71      x_new = x_diff * cos(angle) - y_diff * sin(angle),
    72      y_new = x_diff * sin(angle) + y_diff * cos(angle),
    73      x_new = orig_x + x_new*scale_factor,
    74      y_new = (orig_y + y_new)
    75        )
    76
    77        # overwrite the x,y coordinates with the newly computed coordinates
    78        triangle_df$x <- new_coords$x_new
    79        triangle_df$y <- new_coords$y_new
    80
    81    triangle_df
    82  }
    83)
    84
    85stat_triangles <- function(mapping = NULL, data = NULL, geom = "polygon",
    86                       position = "identity", na.rm = FALSE, show.legend = NA,
    87                       inherit.aes = TRUE, ...) {
    88  layer(
    89    stat = StatTriangles, data = data, mapping = mapping, geom = geom,
    90    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    91    params = list(na.rm = na.rm, ...)
    92  )
    93}
    94
    95GeomTriangles <- ggproto("GeomTriangles", GeomPolygon,
    96    default_aes = aes(
    97            color = 'black', fill = "black", size = 0.5, linetype = 1, alpha = 1, angle = 0, width = 1
    98        )
    99)
    100
    101geom_triangles <- function(mapping = NULL, data = NULL,
    102                       position = "identity", na.rm = FALSE, show.legend = NA,
    103                       inherit.aes = TRUE, ...) {
    104  layer(
    105    stat = StatTriangles, geom = GeomTriangles, data = data, mapping = mapping,
    106    position = position, show.legend = show.legend, inherit.aes = inherit.aes,
    107    params = list(na.rm = na.rm, ...)
    108  )
    109}
    110
    111# here's an example using mtcars 
    112
    113plt_orig <- mtcars %>%
    114  tibble::rownames_to_column('name') %>%
    115  ggplot(aes(x = mpg, y = disp, z = cyl, width = wt, color = hp, fill = hp, label = name)) +
    116  geom_triangles(width_scale = 10, height_scale = 15, alpha = .7) +
    117  geom_point(color = 'black', size = 1) +
    118  ggrepel::geom_text_repel(color = 'black', size = 2, nudge_y = -10) +
    119  scale_fill_viridis_c(end = .6) +
    120  scale_color_viridis_c(end = .6) +
    121  xlab("miles per gallon") +
    122  ylab("engine displacement (cu. in.)") +
    123  labs(fill = 'horsepower', color = 'horsepower') +
    124  ggtitle("MPG, Engine Displacement, # of Cylinders, Weight, and Horsepower of Cars from the 1974 Motor Trends Magazine",
    125  "Cylinders shown in height, weight in width, horsepower in color") +
    126  theme_bw() +
    127  theme(plot.title = element_text(size = 10), plot.subtitle = element_text(size = 8), legend.title = element_text(size = 10))
    128
    129plt_orig
    130draw_geom_triangles_height_legend <- function(
    131  width = 1,
    132  width_scale = .1,
    133  height_scale = .1,
    134  z_values = 1:3,
    135  n.breaks = 3,
    136  labels = c("low", "medium", "high"),
    137  color = 'black',
    138  fill = 'black'
    139) {
    140  ggplot(
    141    data = data.frame(x = rep(0, times = n.breaks),
    142                      y = seq(1,n.breaks),
    143                      z = quantile(z_values, seq(0, 1, length.out = n.breaks)) %>% as.vector(),
    144                      width = width,
    145                      label = labels,
    146                      color = color,
    147                      fill = fill
    148    ),
    149    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    150  ) +
    151    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    152    geom_text(mapping = aes(x = x + .5), size = 3) +
    153    expand_limits(x = c(-.25, 3/4)) +
    154    theme_void() +
    155    theme(plot.title = element_text(size = 10, hjust = .5))
    156}
    157
    158draw_geom_triangles_width_legend <- function(
    159  width = 1:3,
    160  width_scale = .1,
    161  height_scale = .1,
    162  z_values = 1,
    163  n.breaks = 3,
    164  labels = c("low", "medium", "high"),
    165  color = 'black',
    166  fill = 'black'
    167) {
    168  ggplot(
    169    data = data.frame(x = rep(0, times = n.breaks),
    170                      y = seq(1, n.breaks),
    171                      z = rep(1, n.breaks),
    172                      width = width,
    173                      label = labels,
    174                      color = color,
    175                      fill = fill
    176    ),
    177    mapping = aes(x = x, y = y, z = z, label = label, width = width)
    178  ) +
    179    geom_triangles(width_scale = width_scale, height_scale = height_scale, color = color, fill = fill) +
    180    geom_text(mapping = aes(x = x + .5), size = 3) +
    181    expand_limits(x = c(-.25, 3/4)) +
    182    theme_void() +
    183    theme(plot.title = element_text(size = 10, hjust = .5))
    184}
    185
    186# extract the original legend - this is for the color and fill (hp)
    187legend_hp <- cowplot::get_legend(plt_orig)
    188
    189# remove the legend from the plot
    190plt <- plt_orig + theme(legend.position = 'none')
    191
    192# create a height legend using draw_geom_triangles_height_legend
    193height_legend <- 
    194  draw_geom_triangles_height_legend(z_values = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl)),
    195                                    labels = c(min(mtcars$cyl), median(mtcars$cyl), max(mtcars$cyl))
    196                                    ) +
    197                                    ggtitle("cylinders\n")
    198
    199
    200# create a width legend using draw_geom_triangles_width_legend
    201width_legend <- 
    202  draw_geom_triangles_width_legend(
    203  width = quantile(mtcars$wt, c(.33, .66, 1)),
    204  labels = round(quantile(mtcars$wt, c(.33, .66, 1)), 2),
    205  width_scale = .2
    206  ) +
    207  ggtitle("weight\n(1000 lbs)\n")
    208
    209blank_plot <- ggplot() + theme_void()
    210  
    211# create a legend column layout
    212# 
    213# whitespace is used above, below, and in-between the legend components to
    214# make sure the legend column pieces don't appear too densely stacked.