Explore all Architecture open source software, libraries, packages, source code, cloud functions and APIs.

Popular New Releases in Architecture

hacker-laws

v0.1.0

nsq

1.2.1

diagrams

v0.21.1

disruptor

Release 4.0.0 Release Candidate 1

orleans

v3.6.2

Popular Libraries in Architecture

system-design-primer

by donnemartin doticonpythondoticon

star image 143449 doticonNOASSERTION

Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.

java-design-patterns

by iluwatar doticonjavadoticon

star image 74302 doticonNOASSERTION

Design patterns implemented in Java

clean-code-javascript

by ryanmcdermott doticonjavascriptdoticon

star image 56019 doticonMIT

:bathtub: Clean Code concepts adapted for JavaScript

architecture-samples

by android doticonkotlindoticon

star image 40272 doticonApache-2.0

A collection of samples to discuss and showcase different architectural tools and patterns for Android apps.

python-patterns

by faif doticonpythondoticon

star image 29134 doticon

A collection of design patterns/idioms in Python

hacker-laws

by dwmkerr doticonshelldoticon

star image 22528 doticonCC-BY-SA-4.0

💻📖 Laws, Theories, Principles and Patterns that developers will find useful. #hackerlaws

Hystrix

by Netflix doticonjavadoticon

star image 21923 doticonApache-2.0

Hystrix is a latency and fault tolerance library designed to isolate points of access to remote systems, services and 3rd party libraries, stop cascading failure and enable resilience in complex distributed systems where failure is inevitable.

architecture-components-samples

by android doticonkotlindoticon

star image 21518 doticonApache-2.0

Samples for Android Architecture Components.

nsq

by nsqio doticongodoticon

star image 20200 doticonMIT

A realtime distributed messaging platform

Trending New libraries in Architecture

diagrams

by mingrammer doticonpythondoticon

star image 16552 doticonMIT

:art: Diagram as Code for prototyping cloud system architectures

dtm

by yedf doticongodoticon

star image 4009 doticonBSD-3-Clause

🔥A cross-language distributed transaction manager. Support xa, tcc, saga, transactional messages. 跨语言分布式事务管理器

domain-driven-hexagon

by Sairyss doticontypescriptdoticon

star image 2500 doticonMIT

Guide on Domain-Driven Design, Hexagonal architecture, best practices etc.

design-patterns-explained-with-food

by wesdoyle doticoncsharpdoticon

star image 2063 doticon

GoF Design Patterns with "Real-World" examples involving Food-Related Businesses and mock dependencies

wild-workouts-go-ddd-example

by ThreeDotsLabs doticongodoticon

star image 1601 doticonMIT

Go DDD example application. Complete project to show how to apply DDD, Clean Architecture, and CQRS by practical refactoring.

tinykv

by tidb-incubator doticongodoticon

star image 1504 doticon

A course to build distributed key-value service based on TiKV model

tasarim-desenleri-turkce-kaynak

by yusufyilmazfr doticonhtmldoticon

star image 1389 doticon

Türkçe kaynağa destek olması amacıyla oluşturulmuş bir kaynaktır. Konu anlatımın yanı sıra C# ve Java gibi birçok dilde tasarım desenlerinin uygulamasını içermektedir.

miette

by zkat doticonrustdoticon

star image 913 doticonNOASSERTION

Fancy upgrade to std::error::Error.

clean-react

by rmanguinho doticontypescriptdoticon

star image 769 doticonGPL-3.0

Sistema em ReactJs usando Hooks, Typescript, TDD, Clean Architecture, Design Patterns e SOLID principles

Top Authors in Architecture

1

PacktPublishing

43 Libraries

star icon2173

2

maoberlehner

19 Libraries

star icon297

3

OrleansContrib

16 Libraries

star icon377

4

takaakit

13 Libraries

star icon121

5

compute-io

12 Libraries

star icon54

6

tidwall

12 Libraries

star icon1265

7

LLNL

11 Libraries

star icon445

8

ivanpaulovich

11 Libraries

star icon4232

9

hashicorp

10 Libraries

star icon11115

10

CodelyTV

10 Libraries

star icon2989

1

43 Libraries

star icon2173

2

19 Libraries

star icon297

3

16 Libraries

star icon377

4

13 Libraries

star icon121

5

12 Libraries

star icon54

6

12 Libraries

star icon1265

7

11 Libraries

star icon445

8

11 Libraries

star icon4232

9

10 Libraries

star icon11115

10

10 Libraries

star icon2989

Trending Kits in Architecture

JavaScript architecture libraries help you build better, reusable and testable code by providing you with an interface to various aspects of a computer system. They provide abstractions that allow you to build powerful apps without having to understand all the details of how they work underneath. The JavaScript architecture libraries clean-code-javascript and generator-jhipster help you write more maintainable JavaScript code. They also allow you to use more advanced features of the language. clean-code-javascript is a Javascript framework that helps developers write code that is easy to read, easy to maintain and easy to test. It provides abstractions on top of various parts of your application such as models, views, controllers, services etc. The library is used by thousands of developers across the world and has been downloaded more than 5 million times from its website. generator is a library for building custom generators for ES6+ syntax (including async/await). Jhipster is a powerful & extensible JavaScript library for working with HTML elements & CSS properties directly from your codebase. With Jhipster, you can write more efficient code by generating DOM elements and changes as you write them so you can focus on business logic instead of DOM manipulation. Generator-jhipster is a library for building custom generators for ES6+ syntax (including async/await). Popular open source JavaScript architecture libraries for developers include

Architecture is a design pattern that describes the way a system should be organized to meet required functionalities. C++ architecture libraries like lager, thunder, occa, khepri are used to model and implement the architectural patterns in a C++ program. The main purpose of using these libraries is to provide a structure for the application programmers so that they can easily understand how an application is structured and what components are involved in its design. C++ provides various ways to implement an application's architecture. Lager is a C++ library that provides a high-level interface to many types of relational database systems and other data sources. The goal of lager is to create a complete implementation that can be used for developing applications for any kind of database system. Thunder is an advanced C++ template library for creating high-performance networked applications. Thunder includes support for TCP/IP sockets and Transmission Control Protocol (TCP)/User Datagram Protocol (UDP) sockets, IPv4 and IPv6 unicast address support; stream I/O on UDP streams, server sockets, message queues, semaphores and shared memory objects. Many developers depend on the following open source C++ architecture libraries

The use of Ruby Architecture libraries like trailblazer, samson, granite, etc., contributed to the success of the framework. These libraries are used extensively in the framework for their ease of use and flexibility. The Ruby Architecture libraries are a set of tools that help you build applications with a familiar, object-oriented style. They provide a more reactive programming model and simplify the process of building web applications. Trailblazer is an open-source framework for building Ruby on Rails applications. It is ideal for any project where you have to build a complete application from scratch, but don't have the time or resources to do it by hand. It simplifies all the repetitive tasks that come with developing an application, so you can concentrate on what matters most: your business logic. Samson is a microframework for creating lightweight Rails applications in Ruby 2.4+. It's intended for small projects where performance and simplicity matter more than features like authentication or session management. Granite is a collection of classes and tools for working with relational databases in Ruby on Rails applications. It includes support for PostgreSQL as well as MySQL and SQLite3 databases, which mean that you can use any database technology with Ruby on Rails without worrying about how it works behind the scenes or what code you need to write to make it work properly. There are several popular open source Ruby Architecture libraries available for developers

The use of architecture libraries is a great way to provide structure to your code and make it easier to maintain. It also helps you document your code and provide guidelines for new developers. If you just start writing code without any kind of structure, it's going to be hard for others to understand what you're doing. You can use architecture libraries like clean-architecture-manga, MCW-serverless architecture, N-tier architecture as a reference on how things should be done in your project. C# architecture libraries are useful for implementing architectures like n-tier architecture, clean architecture and many more. Clean Architecture Manga is a clean architecture framework that can be used to design, document and implement Agile projects. It was designed with simplicity in mind and is fully customizable according to the project requirements. MCW Serverless Architecture is a lightweight and extensible framework for building modern cloud applications using AWS Lambda functions and Amazon API Gateway. This library helps you build scalable microservices for your serverless architectures. N-Tier architecture framework helps developers to develop and deploy multi-tier applications on the cloud from one central place. It provides support for multiple tiers like UI, business logic & database as well as integration between them all. Popular open source C# architecture libraries include

Python Architecture libraries have a lot of great features. They can be used to easily build a complete application from scratch or to augment existing code. The biggest advantage is that these libraries abstract away some of the details of the underlying architecture and make it easier to write applications that take advantage of all of their features. The best part about these libraries is that they are open source, so anyone can contribute if they want to improve them or make them better. The Python Architecture libraries offer a set of common building blocks for machine learning and AI applications. These libraries are used to implement the architecture of many different types of algorithms in the field of artificial intelligence. The ENAS-pytorch library is a Python interface to the Semantic Segmentation and Detection (ENAS) framework. This library allows users to segment images into different classes by using a pre-trained model trained on TensorFlow objects. Developers tend to use some of the following open source Python Architecture libraries

Go architecture libraries are a great way to jump start your serverless application development. They provide the scaffolding for creating a robust serverless application and help you avoid the common pitfalls of developing with Go. Go architecture libraries allow you to create and manage your services, components, and resources in a consistent manner that is easy to scale. These libraries handle all the details of deploying your application or service so that you can focus on what matters most: writing code. When using these libraries, it’s important to understand how they work and what they can do for you. Most libraries are designed to simplify common tasks and allow developers to focus on writing code rather than dealing with all of the details of coding. Go-clean-arch is a Go Architecture library for writing clean architecture in Go. Polaris is a Go Architecture library for writing clean architecture in Go. Go-carbon is a Go Architecture library for writing clean architecture in Go. Draft is a working draft of the Go Architecture Standard, meant to be a reference for the other libraries and patterns. Popular open source Go architecture libraries among developers include

PHP Architecture libraries like deptrac, phpat, arkitect are a great solution for developers who want to create reusable code. All of the libraries are designed in a way that it is easy to integrate them into your project. One of the most popular PHP Architecture libraries is Deptrac. It is used for developing complex applications with a large number of classes and interfaces. These libraries can make the job of writing code simpler and easier. They allow developers to write code faster and more efficiently. There are some libraries that are extremely popular and widely used by many developers around the world. Deptrac is an architecture library for PHP5/6 written in C++ which provides a set of tools for developing web applications with high performance, scalability and reliability. Phpat is a set of tools for developing web applications with high performance, scalability and reliability on different platforms such as Linux, Windows, Mac OS X etc., The arkitect project aims at providing a complete solution for building scalable architectures using modern technologies like service-oriented architecture (SOA), object-oriented design concepts like inheritance, polymorphism etc., Full list of the best open-source PHP Architecture libraries are below

There are many well-known java UI design libraries available today which help you in creating cool and engaging user interfaces using Java. These are the most popular Java UI Design libraries. They are very useful and easy to use. jitwatch is a jQuery plugin that gives you a lot of features. You can add it to your project and start using it right away. sublimepicker is a Sublime Text plugin for picking objects from anywhere on the screen, without leaving the editor window. It’s also great for picking text from within an editor or running your own custom commands. AirMapView is an open source library that provides an easy-to-use interface for manipulating maps in Java applications. It was developed by Mapbox, Inc., which makes beautiful maps for mobile apps at scale, powering millions of downloads every day. Nifty-gui is an open source UI design library written in Kotlin with Vuejs 2 (Vuex) integration. It supports creating web apps natively in Android Studio 3+, IntelliJ IDEA Ultimate 2018, WebStorm 2018 and other IDEs that can run Kotlin code with JetBrains Rider IDE. Developers tend to use some of the following open source java UI design libraries

We have a large selection of JavaScript UI Design libraries that make it easy to create beautiful and functional web interfaces. Our JavaScript UI Design library is the perfect choice when you need a web application or website built quickly. JavaScript UI Design libraries are very useful in creating dynamic and interactive web applications. These tools help you to build user interfaces for mobile, tablet, desktop and even for TV screens. UI design libraries are used mainly for developing the front-end of site, which is the part of a website that the users interact with instead of the back-end. These are libraries that help you build web interfaces, for example with CSS and HTML. They also help you create layouts, components and objects. Pencil is a JavaScript UI design library that allows you to rapidly prototype complex interfaces. It’s based on the principles of object-oriented programming and has a strong focus on component-based design. Stylo is a JavaScript UI design library used for creating web apps and websites. It allows you to quickly prototype and share designs with others without having to worry about front-end development specifics like CSS or HTML5. There are several popular open source JavaScript UI Design libraries available for developers

Ruby UI Design libraries are a set of reusable design components, which can be used to build beautiful interfaces. Ruby UI Design libraries have been created to help you write your tests in a more readable way. They are also helpful in making your tests more maintainable because they provide you with visual elements that will help you describe the behavior of your application. These libraries allow you to focus on the content, rather than getting bogged down in the details of how it is rendered. These libraries provide a wide range of functions and methods to build user interfaces. Active_Scaffold is a toolkit for building Ruby web apps that uses Active Record and Sequel to model your database. It provides a simple way to create new models, scaffold out views, and generate models based on your data. The Docker Registry Browser is a browser-based REST API for browsing and managing Docker images. It is based on the Docker Registry API and is built as a single-page application that can be embedded in any application. cli-ui is a command-line interface framework for creating interactive applications. Using cli-ui you can quickly build complex command line interfaces (CLIs) with a few lines of code. It supports many different programming languages, including Ruby, Python and javascript. Many developers depend on the following open source Ruby UI Design libraries

The use of C# UI Design libraries like maui, avalonia, ReactiveUI, gui.cs is a good practice in creating beautiful and easy to use applications. These libraries make it easier for developers to create applications with the help of markup, custom components and declarative data binding. It is important to note that these libraries are not just limited to the creation of user interfaces but can also be used for building web applications as well. C# UI Design libraries are a great way to get started with UI development in C#. These libraries provide a set of components that you can use to build your UI. They also help you save time and effort by providing base classes for common tasks, such as buttons and textboxes. Maui is an open source library that provides support for building UIs using the Model-View-Controller (MVC) design pattern. It contains various controls like TextBox, DropDownList, CheckBox, RadioButton etc. Avalonia is an open source library that provides support for building UIs using MVVM design pattern. It contains various controls like Button, ButtonGroup, CheckBoxGroup etc. ReactiveUI is another popular C# UI library that supports reactive programming style. It has many features such as data binding, animations and more. Popular open source C# UI Design libraries among developers include

The Python UI Design libraries pix2code, remi, pywinauto and pygubu allow you to create an interface in code and then have it converted into a GUI that can be used on your computer. Pix2code is a Python library for drawing UIs in the C++ programming language. pix2code is a simple but effective tool to generate a user interface from images. It does not require any backend or backend development. You can use it for both local and remote deployment. You just need to supply an image file as input and get an attractive output in minutes. It can be used to generate XAML files from Python code. Remi is a Python library for creating menus and toolbars that looks like native Windows menus and toolbars. Pywinauto is a Python library for creating UIs with no dependencies on third-party libraries such as PyQt or PySide . It relies heavily on the PyGTK graphical toolkit and also includes X11 (the default windowing system) support. It allows you to build your own widgets from scratch by extending the base Widget class. Pygubu is a command line tool (written in Python) that converts UI designs into native executables so that they can be run on any platform. Popular open source Python UI Design libraries include

When you're creating an app, you want it to be as easy as possible for your users to get what they need out of it. And that means having a consistent look and feel throughout all of your apps — not just the ones that are built using the same framework. The Go UI Design toolkit is a collection of packages and tools that make it easy to build high-quality UIs in the Go programming language. Go UI Design is built on top of the Go tools and libraries, so it’s fully compatible with all existing Go code. This means you can use Go UI Design to create projects alongside your existing codebase, without having to rewrite anything or worry about package dependencies. These libraries help you create components that are clean and easy to use. fyne is a Go interface toolkit that provides an easy way to structure your code and manage data. It also includes a templating system, so you can easily create reusable templates. boringproxy is a library for organizing and managing HTTP requests in Go. This is particularly useful when working with APIs that return large datasets. winc is a web UI design toolkit for the Go language. It helps you build interactive web interfaces without writing any code at all! Some of the most popular Go UI Design libraries among developers are

PHP UI Design libraries like xhprof, opensourcepos, viewi, LxdMosaic are the best way to enhance the user experience of your website. These libraries can be used for a variety of purposes such as performance measurement and analysis. The use of PHP UI Design libraries is a very good idea for a number of reasons. First and foremost, it allows you to create the design without having to rely on the programming skills of your developer. Secondly, it will save you money by making it easier for your developer to build the interface. Xhprof is an open source PHP profiler that can be used for profiling any application and collecting data from it. It monitors the performance of your code and shows you how it performs under different conditions. Use xhprof to profile your code and see what is taking the most time. You can also use it to see whether certain parts of your code are being run too often. Viewi is an Open Source JS framework which helps in building interactive websites using jQuery and HTML5 Canvas API. You can use this framework to build interactive websites without writing any code yourself. viewi is a popular online tool for designing interfaces, especially for mobile apps. It lets you create mockups quickly by dragging around different elements from their library and connecting them together like Lego blocks with wireframes. You can also add interactivity between different elements such as buttons or links by dragging them onto certain elements in the interface design editor window. Opensourcepos is a very nice tool for designing interfaces using HTML and CSS. It's based on Bootstrap and has many features like responsive layout and animations. The following is a list of the most popular PHP UI Design libraries

The JavaScript UX libraries that you can use to create a beautiful and interactive user experience for your website. When you're developing a web app that uses JavaScript, it's important to make sure your code is as performant as possible. This can be achieved by using the right libraries and frameworks. Intro.js is a JavaScript library for creating beautiful UI mockups in Photoshop or Illustrator. Zap-desktop is a JavaScript library that allows you to easily create fully interactive prototypes with native look and feel. Procedural-gl-js is an open source library that makes it possible to implement procedural animations in web applications by using the WebGL API. GDLauncher is an open source tool that helps developers launches their web apps in the browser, without having to code any server code or manipulate files on disk. Many developers depend on the following open source JavaScript UX libraries

The use of Java UX libraries like BlueOcean-Plugin, QuickSand, AccelerationExplorer, zap-android is one of the topmost trends in the Android Development. These libraries provide user interface elements that can be used to create a fast and responsive UI. BlueOcean is a library that provides the ability to create custom widgets and view pages. It's a good way to get started on the Android platform without having to write everything from scratch. QuickSand is an extension for Android Studio that lets you preview your design in real time. AccelerationExplorer is an extension for Android Studio that allows you to quickly see what your app will look like when it's running on various devices and displays. You can also change the orientation of your phone or tablet by rotating it. Popular open source Java UX libraries include

The Python UX library is a collection of tools for building user interfaces in Python. It provides easy-to-use modules for common user interface tasks like displaying data, controlling actions, and building dialogs. The library also includes classes for customizing the look and feel of your application, as well as an object-oriented API for creating your own widgets. Wagtail is a Django-based content management system with support for administration, taxonomy, user management and more. mathesar is a lightweight static site generator with a focus on simplicity and user experience. Experimenter is an open source toolkit for building interactive web applications in JavaScript. It's built on top of ReactJS, Redux, and Webpack. django-formset supports the Django ORM's formsets through JSON serialization (so it can be used with any other framework) which makes it flexible and extensible. Popular open source Python UX libraries among developers include

Ruby on Rails is a mature and robust framework that can be used for both small applications as well as large-scale projects. Ruby on Rails is an object-oriented web application framework written in Ruby. The codebase of the framework is maintained by the community, and the implementation of this codebase has been done in several different ways. React_on_rails is a gem that allows you to use the hot reloading feature of Rails. It allows you to write code in JavaScript, and has a plugin that allows you to run it on the server. ActiveAdmin_addons is a library that integrates with ActiveAdmin, allowing easy access to all of its features from within your Rails app. Full list of the best open-source Ruby UX libraries below.

The main advantage of using C++ UX libraries is that they can be used to build complex web applications. The user interface design can be easily incorporated into the application as it is built using these libraries. The most important thing to remember when building a web application with C++ is that it should be well documented. This will help you in debugging and maintaining your codebase in the future. Another important aspect of the development process is making sure that all the data required for the application are stored in a centralized location so that it can be accessed by other parts of the application at any time. The libraries are designed to be as simple as possible, yet powerful enough to handle almost any use case you might want to throw at them. Skift is a set of libraries for parsing and manipulating JSON in C++. The JUX library is a set of utility classes for working with JSON in Java. An easy to use library for parsing and validating argument lists for C++ applications. The CPP-Argument-Parser C++ library implements the same API as the Java version, but uses standard C++ idioms. A flexible parser framework for parsing command-line arguments in C++ programs. It is a generic argument parser framework based on the concept of "typechecking" and "type inference", which uses type deduction to infer types from arguments passed to functions (in particular, function calls). Some of the most widely used open source C++ UX libraries among developers include

Use of C# UI libraries like Hover-UI-Kit, VRCMods, Comet, Vacaro allows you to create a rich user experience with minimal code. This is especially useful when you want to create a game or application that has a user interface that is similar to other games and applications in the market. This is possible because they provide the necessary tools that allow you to create your desired UI. They also provide easy access to all the elements you need to build a complex UI. If there are any issues with your design, these libraries will help you correct them quickly without wasting time on searching for information online. The main reason behind this is that these libraries allow you to create UIs in a very short time and also allows you to explore new ideas. The Hover-UI-Kit is one such library that enables you to create a beautiful UI for your app in no time. You can use various different UI elements to make your app look attractive and professional. VRCMods is another library which provides support for VR headsets such as Oculus Rift and HTC Vive. This allows users to experience Virtual Reality while using different apps on their devices. Comet is another great library which helps developers build better user interfaces using drag and drop functionality. You can easily create beautiful UIs using this tool without any prior experience or knowledge on how things work in terms of web development. Some of the most popular C# UI libraries among developers are:

Trending Discussions on Architecture

Is there a higher performing implementation of powf(10,floorf(log10f(x)))

Unknown host CPU architecture: arm64 , Android NDK SiliconM1 Apple MacBook Pro

remark: Incremental compilation has been disabled: it is not compatible with whole module optimization

How to replace the bundled Dart SDK in Flutter to run natively on Apple Silicon (ARM64)?

How do I resolve error message: "Inheritance from an interface with '@JvmDefault' members is only allowed with -Xjvm-default option"

How does an instruction decoder tell the difference between a prefix and a primary opcode?

Flag provided but not defined: -rpc

Saving model on Tensorflow 2.7.0 with data augmentation layer

React: hiding vs removing components

M1 Mac - GDAL Wrong Architecture Error [Django]

QUESTION

Is there a higher performing implementation of powf(10,floorf(log10f(x)))

Asked 2022-Apr-16 at 01:04

I have a need to truncate a float to the nearest power of 10. For example, 1.1 would truncate to 1.0 and 4.7e3 would truncate to 1e3. I am currently doing it with the seemingly complicated powf(10,floorf(log10f(x))). I am wondering whether there is a better performing (as in faster execution speed) solution? My target CPU architecture is both x86-64 and arm64.

1#include <stdio.h>
2#include <math.h>
3
4int main()
5{
6  float x = 1.1e5f;
7  while (x > 1e-6f)
8  {
9    float y = powf(10,floorf(log10f(x)));
10    printf("%e ==> %g\n", x, y); 
11    x /= 5.0f;
12  }
13}
14

when run, this produces

1#include <stdio.h>
2#include <math.h>
3
4int main()
5{
6  float x = 1.1e5f;
7  while (x > 1e-6f)
8  {
9    float y = powf(10,floorf(log10f(x)));
10    printf("%e ==> %g\n", x, y); 
11    x /= 5.0f;
12  }
13}
141.100000e+05 ==> 100000
152.200000e+04 ==> 10000
164.400000e+03 ==> 1000
178.800000e+02 ==> 100
181.760000e+02 ==> 100
193.520000e+01 ==> 10
207.040000e+00 ==> 1
211.408000e+00 ==> 1
222.816000e-01 ==> 0.1
235.632000e-02 ==> 0.01
241.126400e-02 ==> 0.01
252.252800e-03 ==> 0.001
264.505600e-04 ==> 0.0001
279.011199e-05 ==> 1e-05
281.802240e-05 ==> 1e-05
293.604480e-06 ==> 1e-06
30

ANSWER

Answered 2022-Apr-15 at 16:36

I would say don't sweat it. Unless the program is spending a large proportion of its time doing this truncation, it's not worth optimising what is probably super-fast anyway. But if you wanted to optimise for your common cases (1e-2 <= x <= 10), then you might try using 32-bit integer arithmetic to compare with the binary representations of 1e-2, 1e-1, 1, and 10 (for instance, 1e-1 is 0x3dcccccd) ; if it's outside that range, you can fall back on the floating point version. Only experimentation will determine if this actually runs faster.

Source https://stackoverflow.com/questions/71886042

QUESTION

Unknown host CPU architecture: arm64 , Android NDK SiliconM1 Apple MacBook Pro

Asked 2022-Apr-04 at 18:41

I've got a project that is working fine in windows os but when I switched my laptop and opened an existing project in MacBook Pro M1. I'm unable to run an existing android project in MacBook pro M1. first I was getting

Execution failed for task ':app:kaptDevDebugKotlin'. > A failure occurred while executing org.jetbrains.kotlin.gradle.internal.KaptExecution > java.lang.reflect.InvocationTargetException (no error message)

this error was due to the Room database I applied a fix that was adding below library before Room database and also changed my JDK location from file structure from JRE to JDK.

kapt "org.xerial:sqlite-jdbc:3.34.0"

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6

after that now I'm getting an issue which is Unknown host CPU architecture: arm64

there is an SDK in my project that is using this below line.

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16

App Gradle

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23

[CXX1405] error when building with ndkBuild using /Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/Android.mk: Build command failed. Error while executing process /Users/mac/Library/Android/sdk/ndk/21.4.7075529/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/Android.mk APP_ABI=arm64-v8a NDK_ALL_ABIS=arm64-v8a NDK_DEBUG=1 APP_PLATFORM=android-21 NDK_OUT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/build/intermediates/cxx/Debug/4k4s2lc6/obj NDK_LIBS_OUT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/build/intermediates/cxx/Debug/4k4s2lc6/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n} ERROR: Unknown host CPU architecture: arm64

which is causing this issue and whenever I comment on this line

path 'Android.mk'

it starts working fine, is there any way around which will help me run this project with this piece of code without getting this NDK issue?

Update - It seems that Room got fixed in the latest updates, Therefore you may consider updating Room to latest version (2.3.0-alpha01 / 2.4.0-alpha03 or above)

GitHub Issue Tracker

ANSWER

Answered 2022-Apr-04 at 18:41

To solve this on a Apple Silicon M1 I found three options

A

Use NDK 24

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27

You can install it with

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27echo &quot;y&quot; | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28

or

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27echo &quot;y&quot; | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo &quot;y&quot; | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29

Depending what where sdkmanager is located enter image description here

B

Change your ndk-build to use Rosetta x86. Search for your installed ndk with

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27echo &quot;y&quot; | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo &quot;y&quot; | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2&gt;/dev/null
30

eg

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27echo &quot;y&quot; | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo &quot;y&quot; | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2&gt;/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31

and change

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27echo &quot;y&quot; | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo &quot;y&quot; | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2&gt;/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31DIR=&quot;$(cd &quot;$(dirname &quot;$0&quot;)&quot; &amp;&amp; pwd)&quot;
32$DIR/build/ndk-build &quot;$@&quot;
33

to

1   //Room components
2    kapt &quot;org.xerial:sqlite-jdbc:3.34.0&quot;
3    implementation &quot;androidx.room:room-ktx:$rootProject.roomVersion&quot;
4    kapt &quot;androidx.room:room-compiler:$rootProject.roomVersion&quot;
5    androidTestImplementation &quot;androidx.room:room-testing:$rootProject.roomVersion&quot;
6android {
7    externalNativeBuild {
8        ndkBuild {
9           path 'Android.mk'
10        }
11    }
12    ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17        cmake {
18            path &quot;src/main/cpp/CMakeLists.txt&quot;
19            version &quot;3.18.1&quot;
20            //version &quot;3.10.2&quot;
21        }
22    }
23android {
24    ndkVersion &quot;24.0.8215888&quot;
25    ...
26}
27echo &quot;y&quot; | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo &quot;y&quot; | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2&gt;/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31DIR=&quot;$(cd &quot;$(dirname &quot;$0&quot;)&quot; &amp;&amp; pwd)&quot;
32$DIR/build/ndk-build &quot;$@&quot;
33DIR=&quot;$(cd &quot;$(dirname &quot;$0&quot;)&quot; &amp;&amp; pwd)&quot;
34arch -x86_64 /bin/bash $DIR/build/ndk-build &quot;$@&quot;
35

enter image description here

C

convert your ndk-build into a cmake build

Source https://stackoverflow.com/questions/69541831

QUESTION

remark: Incremental compilation has been disabled: it is not compatible with whole module optimization

Asked 2022-Mar-31 at 03:05

Updated to the Xcode 13 beta, now I can't build the project due to an error in several pods

Older Xcode launches an app on iOS 15 too long

I use M1, maybe this is due to architecture

ANSWER

Answered 2021-Aug-25 at 21:20

That remark isn't an actual error, just a warning. The real error can be found in the nested Compile Swift source files under the top level section with the same name. Expand the logs for this command and you should see the actual error.

Source https://stackoverflow.com/questions/68801998

QUESTION

How to replace the bundled Dart SDK in Flutter to run natively on Apple Silicon (ARM64)?

Asked 2022-Mar-21 at 16:27

Dart SDK officially supports ARM64 and as of now, 2.14.2 is the latest (stable) Dart SDK that has support for ARM64. Though it was the same version that was bundled in my Flutter setup, it seemed to run on Intel architecture (Activity monitor shows dart processes running on Intel).

I manually tried replacing the dart SDK on my flutter installation bu replacing flutter-directory/bin/cache/dart-sdk/ with the contents of a zip file of the Dart SDK made for ARM64, downloaded from dart.dev archive. But trying to run an app on an Android emulator (which runs on ARM64 and was working on my old Flutter setup), throws this error:

1Launching lib/main.dart on sdk gphone64 arm64 in debug mode...
2lib/main.dart:1
3Snapshot not compatible with the current VM configuration: the snapshot requires 'release no-code_comments
4no-dwarf_stack_traces_mode lazy_async_stacks lazy_dispatchers 
5use_bare_instructions no-dedup_instructions
6no-&quot;asserts&quot; &quot;use_field_guards&quot; &quot;use_osr&quot; x64-sysv no-null-safety' but the VM has 'release no-code_comments no-
7dwarf_stack_traces_mode lazy_async_stacks 
8lazy_dispatchers use_bare_instructions no-dedup_instructions no-&quot;asserts&quot; &quot;use_field_guards&quot; &quot;use_osr&quot; arm64-sysv no-null-safety'
92
10the Dart compiler exited unexpectedly.
11
12

Is there any other way to do a completely ARM64 Flutter setup on M1 devices?

Flutter version 2.5.1

Dart version 2.14.2

Device: MacBook Air (M1, 2020)

ANSWER

Answered 2021-Sep-29 at 17:46

It seems it can't be used with Flutter yet, as seen in:

Apple Silicon support in the Dart SDK

[...] Note that the Dart SDK bundled in the Flutter SDK doesn’t have these improvements yet.

https://medium.com/dartlang/announcing-dart-2-14-b48b9bb2fb67

[Announcing Dart 2.14][ScreenShot]: https://i.stack.imgur.com/N8Qcc.png

And:

Get the Dart SDK

[...] As of Flutter 1.21, the Flutter SDK includes the full Dart SDK. So if you have Flutter installed, you might not need to explicitly download the Dart SDK. Consider downloading the Dart SDK if any of the following are true:

  • You don’t use Flutter.
  • You use a pre-1.21 version of Flutter.
  • You want to reduce disk space requirements or download time, and your use case doesn’t require Flutter. For example, you might have a continuous integration (CI) setup that requires Dart but not Flutter.

https://dart.dev/get-dart

[Get the Dart SDK][ScreenShot]: https://i.stack.imgur.com/rawJV.png

Source https://stackoverflow.com/questions/69274086

QUESTION

How do I resolve error message: &quot;Inheritance from an interface with '@JvmDefault' members is only allowed with -Xjvm-default option&quot;

Asked 2022-Mar-19 at 21:08

I'm new to Android development and I'm currently building my first real app. I'm trying to implement a MVVM architecture and because of that I'm having a viewModel for each fragment and each viewModel has a viewModelFactory. At least, this is how I understood it has to be.

I use the boilerplate code everyone seems to use for the factory:

1class ExampleViewModelFactory(private val exampleDao: ExampleDao) : ViewModelProvider.Factory {
2    override fun &lt;T : ViewModel&gt; create(modelClass: Class&lt;T&gt;): T {
3        if (modelClass.isAssignableFrom(exampleViewModel::class.java)) {
4            @Suppress(&quot;UNCHECKED_CAST&quot;)
5            return ExampleViewModel(exampleDao) as T
6        }
7        throw IllegalArgumentException(&quot;Unknown ViewModel class&quot;)
8    }
9}
10

Now the problem is, that the compiler is giving me the following error:

e: C:\Users\ ...\ExampleViewModel.kt: (64, 7): Inheritance from an interface with '@JvmDefault' members is only allowed with -Xjvm-default option

And this error is produced by the viewModelFactory class I have implemented in the viewModel. I really can't tell what this means and I cant find anything helpful or even related to my specific problem. I basically followed some basic tutorials about creating your first app, but I keep on running into errors like this. In most cases, I was able to fix the problem by myself, but this time it's different.

I know that a lot of you have a lot of experience and knowledge, so I hope that some of you find the time to help me and give me a hint what I can do to fix this.

ANSWER

Answered 2022-Feb-25 at 16:53

It seems like you are either directly or indirectly (through some other library) depending on Lifecycle 2.5.0-alpha01.

As per this issue:

You need to temporarily add following to your build.gradle:

1class ExampleViewModelFactory(private val exampleDao: ExampleDao) : ViewModelProvider.Factory {
2    override fun &lt;T : ViewModel&gt; create(modelClass: Class&lt;T&gt;): T {
3        if (modelClass.isAssignableFrom(exampleViewModel::class.java)) {
4            @Suppress(&quot;UNCHECKED_CAST&quot;)
5            return ExampleViewModel(exampleDao) as T
6        }
7        throw IllegalArgumentException(&quot;Unknown ViewModel class&quot;)
8    }
9}
10tasks.withType(KotlinCompile).configureEach {
11    kotlinOptions {
12        freeCompilerArgs += [
13                &quot;-Xjvm-default=all&quot;,
14        ]
15    }
16}
17

Note that in certain circumstances you may want to use all-compatibility instead of all, learn more about that in Jebrain's blogpost.

Starting with kotlin 1.6.20 you won't need touch build.gradle for more information see KT-47000

Source https://stackoverflow.com/questions/70992947

QUESTION

How does an instruction decoder tell the difference between a prefix and a primary opcode?

Asked 2022-Feb-23 at 02:47

I'm trying to wrap my head around the x86 instruction encoding format. All the sources that I read still make the subject confusing. I'm starting to understand it a little bit but one thing that I'm having trouble with understanding is how the CPU instruction decoder differentiates an opcode prefix from an opcode.

I'm aware that the whole format of the instruction basically depends on the opcode (with extra bit fields defined in the opcode of course). Sometimes the instruction doesn't have a prefix and the opcode is the first byte. How would the decoder know?

I'm assuming that the instruction decoder would be able to tell the difference because opcode bytes and prefix bytes would not share the same binary values. So the decoder can tell if the unique binary number in the byte is an instruction or a prefix. For example (In this example we will stick to single byte opcodes) a REX or LOCK prefix would not share the same byte value as any opcode in the architecture's instruction set.

ANSWER

Answered 2022-Feb-23 at 02:47

Traditional (single-byte) prefixes are different from opcode bytes like you said, so a state machine can just remember which prefixes it's seen until it gets to an opcode byte.

The 0f escape byte for 2-byte opcodes is not really a prefix. It has to be contiguous with the 2nd opcode byte. Thus, following a 0f, any byte is an opcode, even if it's something like f2 that would otherwise be a prefix. (This also applies following 0f 3a or 0f 38 2-byte escapes for SSSE3 and later, or VEX/EVEX prefixes that encode one of those escape sequences).

If you look at an opcode map, there are no entries that are ambiguous between single-byte prefix and opcode. (e.g. http://ref.x86asm.net/coder64.html, and notice how the 2-byte 0F .. opcodes are listed separately).


The decoders do have to know the current mode for this (and other things); for example x86-64 removed the 1-byte inc/dec reg opcodes for use as REX prefixes. (x86 32 bit opcodes that differ in x86-x64 or entirely removed). We can even use this difference to write polyglot machine code that runs differently when decoded in 32-bit vs. 64-bit mode, or even distinguish all 3 mode sizes.

x86 machine code is a byte stream that's not self-synchronizing (e.g. a ModRM or an immediate can be any byte). The CPU always knows where to start decoding from, either a jump target or the byte after the end of a previous instruction. That's the start of the instruction (including prefixes).

Bytes in memory are just bytes, only becoming instructions when they're decoded by the CPU. (Although in normal programs, simply disassembling from the top of the .text section does give you the program's instructions. Self-modifying and obfuscated code are not normal.)

AVX / AVX-512: multi-byte prefixes that overlap with opcodes

Multi-byte VEX and EVEX prefixes aren't that simple in 32-bit mode. For example VEX prefixes overlap with invalid encodings of LES and LDS in modes other than 64-bit. (The c4 and c5 opcodes for LES and LDS are always invalid in 64-bit mode, except as VEX prefixes.) https://wiki.osdev.org/X86-64_Instruction_Encoding#VEX.2FXOP_opcodes

In legacy / compat modes, there weren't any free bytes left that weren't already opcodes or prefixes when AVX (VEX prefixes) and AVX-512 (EVEX prefix), so the only room for extensions was as encodings for opcodes that are only valid with a limited set of ModRM bytes. (e.g. LES / LDS require a memory source, not register - this is why some bits are inverted in VEX prefixes, so the top 2 bits of the byte after c4 or c5 will always be 1 in 32-bit mode instead of 0. That's the "mode" field in ModRM, and 11 means register).

(Fun fact: VEX prefixes are not recognized in 16-bit real mode, apparently because some software used the same invalid encodings of LES / LDS as intentional traps, to be sorted out in the #UD exception handler. VEX prefixes are recognized in 16-bit protected mode, though.)


AMD64 freed up several bytes by removing instructions like AAM, as well as LES/LDS (and the one-byte inc/dec reg encodings for use as REX prefixes), but CPU vendors have continued to care about 32-bit mode and not added any extensions that are only available in 64-bit mode which could simply take advantage of those free opcode bytes. This means finding ways to cram new instruction encodings into increasingly small gaps in 32-bit machine code. (Often via mandatory prefixes, e.g. rep bsr = lzcnt on CPUs with that feature, which gives different results.)

So the decoders in modern CPUs that support AVX / BMI1/2 have to look at multiple bytes to decide whether this is a prefix for a valid AVX or other VEX-encoded instruction, or in 32-bit mode if it should decode as LES or LDS. (And I guess look at the rest of the instruction to decide if it should #UD).

But modern CPUs are looking at 16 or 32 bytes at a time anyway to find instruction boundaries in parallel. (And then later feed those groups of instruction bytes to actual decoders, again in parallel.) https://www.realworldtech.com/sandy-bridge/4/

Same goes for the prefix scheme used by AMD XOP, which is a lot like VEX.

Agner Fog's blog article Stop the instruction set war from 2009 (soon after AVX was announced, before the first hardware supporting it) has a table of remaining unused coding space for future extensions, and some notes about it being "assigned" to AMD, Intel, or Via.

Related / examples
Machine code tricks: decoding the same byte multiple ways

(This is not really related to prefixes, but in general seeing how the rules apply to weird cases can help understand exactly things work.)

A software disassembler does need to know a start point. This can be problematic if obfuscated code mixes code and data, and actual execution jumps to places you wouldn't get if you just assume that you can decode in order without following jumps.

Fortunately compiler-generated code doesn't do that so naive static disassembly (e.g. by objdump -d or ndisasm, as opposed to IDA) finds the same instruction boundaries that actually running the program will.

This is not a problem for running obfuscated machine code; the CPU just does what it's told, and never cares about bytes before the place you tell it to jump to. Disassembling without running / single-stepping the program is the hard thing, especially with the possibility of self-modifying code and jumps to what a naive disassembler would think was the middle of an earlier instruction.

Obfuscated machine code can even have an instruction decode one way, then jump back into what was the middle of that instruction, for a later byte to be the opcode (or prefix + opcode). Modern CPUs with uop caches or that mark instruction boundaries in I-cache run slow (but correctly) if you do this, so it's more of a fun code-golf trick (extreme code-size optimization at the expense of speed) or obfuscation technique.

For an example of this, see my codegolf.SE x86 machine code answer to Golf a Custom Fibonacci Sequence. I'll excerpt the disassembly that lines up with what the CPU sees after looping back to cfib.loop, but note that the first iteration decodes differently. So I'm using just 1 byte outside the loop instead of 2 to effectively jump into the middle for the start of the first iteration. See the linked answer for a full description and the other disassembly.

10000000000401070 &lt;cfib&gt;:
2  401070:       eb                      .byte 0xeb      # jmp rel8 consuming the 01 add opcode as a rel8
30000000000401071 &lt;cfib.loop&gt;:
4  401071:       01 d0                   add    eax,edx
5# loop entry point on first iteration, jumping over the ModRM byte (D0) of the ADD
6    (entry on first iteration):
7  401073:       92                      xchg   edx,eax
8  401074:       e2 fb                   loop   401071 &lt;cfib.loop&gt;
9  401076:       c3                      ret 
10

You can do this with opcodes that consume more later bytes, like 3D <dword> cmp eax, imm32. When the CPU sees a 3D opcode byte, it will grab the next 4 bytes as the immediate. If you later jump into those 4 bytes, they'll be considered as prefix/opcodes and everything will work (except for performance problems) the same regardless of how those bytes had previously been decoded as a different part of an instruction. The CPU has to maintain the illusion of decoding and executing 1 instruction at a time, other than performance.

I learned of this trick from @Ira Baxter's answer on Can assembled ASM code result in more than a single possible way (except for offset values)?

Source https://stackoverflow.com/questions/68898858

QUESTION

Flag provided but not defined: -rpc

Asked 2022-Feb-19 at 20:07

Am starting off in blockchain development using the book Mastering Blockchain - A deep dive into distributed ledgers, consensus protocols, smart contracts, DApps, cryptocurrencies, Ethereum,

Am using WSL with geth version 1.10.9.

1$geth version
2Geth
3Version: 1.10.9-stable
4Git Commit: eae3b1946a276ac099e0018fc792d9e8c3bfda6d
5Architecture: amd64
6Go Version: go1.17
7Operating System: linux
8GOPATH=
9GOROOT=go
10

Am trying to start geth, but am getting the error that the --rpc flag is not defined.

This is the command I am trying to run:

geth --datadir ~/etherprivate/ --networkid 786 --rpc --rpcapi 'web3,eth,net,debug,personal' --rpccorsdomain '*'

Any help on how I can solve it?

ANSWER

Answered 2021-Oct-11 at 23:20

It appears 1.10.9-stable version has a problem and is returning a -rpc error. GETH 1.10.8-stable version works fine when running the geth command with --rpc

Source https://stackoverflow.com/questions/69463898

QUESTION

Saving model on Tensorflow 2.7.0 with data augmentation layer

Asked 2022-Feb-04 at 17:25

I am getting an error when trying to save a model with data augmentation layers with Tensorflow version 2.7.0.

Here is the code of data augmentation:

1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3  [ 
4    layers.RandomFlip(&quot;horizontal&quot;),
5    layers.RandomFlip(&quot;vertical&quot;),
6    layers.RandomRotation(0.5),
7    layers.RandomZoom(0.5),
8    layers.RandomContrast(0.5),
9    RandomColorDistortion(name='random_contrast_brightness/none'),
10  ]
11)
12

Now I build my model like this:

1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3  [ 
4    layers.RandomFlip(&quot;horizontal&quot;),
5    layers.RandomFlip(&quot;vertical&quot;),
6    layers.RandomRotation(0.5),
7    layers.RandomZoom(0.5),
8    layers.RandomContrast(0.5),
9    RandomColorDistortion(name='random_contrast_brightness/none'),
10  ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16  layers.Input(input_shape),
17  data_augmentation_rgb,
18  layers.Rescaling((1./255)),
19
20  layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
21     data_format='channels_last'),
22  layers.MaxPooling2D(),
23  layers.BatchNormalization(),
24
25  layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26  layers.MaxPooling2D(),
27  layers.BatchNormalization(),
28
29  layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30  layers.MaxPooling2D(),
31  layers.BatchNormalization(),
32
33  layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34  layers.MaxPooling2D(),
35  layers.BatchNormalization(),
36
37  layers.Flatten(),
38  layers.Dense(128, activation='relu'), # best 1
39  layers.Dropout(0.1),
40  layers.Dense(128, activation='relu'), # best 1
41  layers.Dropout(0.1),
42  layers.Dense(64, activation='relu'), # best 1
43  layers.Dropout(0.1),
44  layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49

Then after the training is done I just make:

1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3  [ 
4    layers.RandomFlip(&quot;horizontal&quot;),
5    layers.RandomFlip(&quot;vertical&quot;),
6    layers.RandomRotation(0.5),
7    layers.RandomZoom(0.5),
8    layers.RandomContrast(0.5),
9    RandomColorDistortion(name='random_contrast_brightness/none'),
10  ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16  layers.Input(input_shape),
17  data_augmentation_rgb,
18  layers.Rescaling((1./255)),
19
20  layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
21     data_format='channels_last'),
22  layers.MaxPooling2D(),
23  layers.BatchNormalization(),
24
25  layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26  layers.MaxPooling2D(),
27  layers.BatchNormalization(),
28
29  layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30  layers.MaxPooling2D(),
31  layers.BatchNormalization(),
32
33  layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34  layers.MaxPooling2D(),
35  layers.BatchNormalization(),
36
37  layers.Flatten(),
38  layers.Dense(128, activation='relu'), # best 1
39  layers.Dropout(0.1),
40  layers.Dense(128, activation='relu'), # best 1
41  layers.Dropout(0.1),
42  layers.Dense(64, activation='relu'), # best 1
43  layers.Dropout(0.1),
44  layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save(&quot;./&quot;)
50

And I'm getting this error:

1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3  [ 
4    layers.RandomFlip(&quot;horizontal&quot;),
5    layers.RandomFlip(&quot;vertical&quot;),
6    layers.RandomRotation(0.5),
7    layers.RandomZoom(0.5),
8    layers.RandomContrast(0.5),
9    RandomColorDistortion(name='random_contrast_brightness/none'),
10  ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16  layers.Input(input_shape),
17  data_augmentation_rgb,
18  layers.Rescaling((1./255)),
19
20  layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
21     data_format='channels_last'),
22  layers.MaxPooling2D(),
23  layers.BatchNormalization(),
24
25  layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26  layers.MaxPooling2D(),
27  layers.BatchNormalization(),
28
29  layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30  layers.MaxPooling2D(),
31  layers.BatchNormalization(),
32
33  layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34  layers.MaxPooling2D(),
35  layers.BatchNormalization(),
36
37  layers.Flatten(),
38  layers.Dense(128, activation='relu'), # best 1
39  layers.Dropout(0.1),
40  layers.Dense(128, activation='relu'), # best 1
41  layers.Dropout(0.1),
42  layers.Dense(64, activation='relu'), # best 1
43  layers.Dropout(0.1),
44  layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save(&quot;./&quot;)
50---------------------------------------------------------------------------
51KeyError                                  Traceback (most recent call last)
52&lt;ipython-input-84-87d3f09f8bee&gt; in &lt;module&gt;()
53----&gt; 1 model.save(&quot;./&quot;)
54
55
56/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in 
57 error_handler(*args, **kwargs)
58 65     except Exception as e:  # pylint: disable=broad-except
59 66       filtered_tb = _process_traceback_frames(e.__traceback__)
60 ---&gt; 67       raise e.with_traceback(filtered_tb) from None
61 68     finally:
62 69       del filtered_tb
63
64 /usr/local/lib/python3.7/dist- 
65 packages/tensorflow/python/saved_model/function_serialization.py in 
66 serialize_concrete_function(concrete_function, node_ids, coder)
67 66   except KeyError:
68 67     raise KeyError(
69 ---&gt; 68         f&quot;Failed to add concrete function '{concrete_function.name}' to 
70 object-&quot;
71 69         f&quot;based SavedModel as it captures tensor {capture!r} which is 
72 unsupported&quot;
73 70         &quot; or not reachable from root. &quot;
74
75 KeyError: &quot;Failed to add concrete function 
76 'b'__inference_sequential_46_layer_call_fn_662953'' to object-based SavedModel as it 
77 captures tensor &lt;tf.Tensor: shape=(), dtype=resource, value=&lt;Resource Tensor&gt;&gt; which 
78 is unsupported or not reachable from root. One reason could be that a stateful 
79 object or a variable that the function depends on is not assigned to an attribute of 
80 the serialized trackable object (see SaveTest.test_captures_unreachable_variable).&quot;
81

I inspected the reason of getting this error by changing the architecture of my model and I just found that reason came from the data_augmentation layer since the RandomFlip and RandomRotation and others are changed from layers.experimental.prepocessing.RandomFlip to layers.RandomFlip, but still the error appears.

ANSWER

Answered 2022-Feb-04 at 17:25

This seems to be a bug in Tensorflow 2.7 when using model.save combined with the parameter save_format="tf", which is set by default. The layers RandomFlip, RandomRotation, RandomZoom, and RandomContrast are causing the problems, since they are not serializable. Interestingly, the Rescaling layer can be saved without any problems. A workaround would be to simply save your model with the older Keras H5 format model.save("test", save_format='h5'):

1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3  [ 
4    layers.RandomFlip(&quot;horizontal&quot;),
5    layers.RandomFlip(&quot;vertical&quot;),
6    layers.RandomRotation(0.5),
7    layers.RandomZoom(0.5),
8    layers.RandomContrast(0.5),
9    RandomColorDistortion(name='random_contrast_brightness/none'),
10  ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16  layers.Input(input_shape),
17  data_augmentation_rgb,
18  layers.Rescaling((1./255)),
19
20  layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
21     data_format='channels_last'),
22  layers.MaxPooling2D(),
23  layers.BatchNormalization(),
24
25  layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26  layers.MaxPooling2D(),
27  layers.BatchNormalization(),
28
29  layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30  layers.MaxPooling2D(),
31  layers.BatchNormalization(),
32
33  layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34  layers.MaxPooling2D(),
35  layers.BatchNormalization(),
36
37  layers.Flatten(),
38  layers.Dense(128, activation='relu'), # best 1
39  layers.Dropout(0.1),
40  layers.Dense(128, activation='relu'), # best 1
41  layers.Dropout(0.1),
42  layers.Dense(64, activation='relu'), # best 1
43  layers.Dropout(0.1),
44  layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save(&quot;./&quot;)
50---------------------------------------------------------------------------
51KeyError                                  Traceback (most recent call last)
52&lt;ipython-input-84-87d3f09f8bee&gt; in &lt;module&gt;()
53----&gt; 1 model.save(&quot;./&quot;)
54
55
56/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in 
57 error_handler(*args, **kwargs)
58 65     except Exception as e:  # pylint: disable=broad-except
59 66       filtered_tb = _process_traceback_frames(e.__traceback__)
60 ---&gt; 67       raise e.with_traceback(filtered_tb) from None
61 68     finally:
62 69       del filtered_tb
63
64 /usr/local/lib/python3.7/dist- 
65 packages/tensorflow/python/saved_model/function_serialization.py in 
66 serialize_concrete_function(concrete_function, node_ids, coder)
67 66   except KeyError:
68 67     raise KeyError(
69 ---&gt; 68         f&quot;Failed to add concrete function '{concrete_function.name}' to 
70 object-&quot;
71 69         f&quot;based SavedModel as it captures tensor {capture!r} which is 
72 unsupported&quot;
73 70         &quot; or not reachable from root. &quot;
74
75 KeyError: &quot;Failed to add concrete function 
76 'b'__inference_sequential_46_layer_call_fn_662953'' to object-based SavedModel as it 
77 captures tensor &lt;tf.Tensor: shape=(), dtype=resource, value=&lt;Resource Tensor&gt;&gt; which 
78 is unsupported or not reachable from root. One reason could be that a stateful 
79 object or a variable that the function depends on is not assigned to an attribute of 
80 the serialized trackable object (see SaveTest.test_captures_unreachable_variable).&quot;
81import tensorflow as tf
82import numpy as np
83
84class RandomColorDistortion(tf.keras.layers.Layer):
85    def __init__(self, contrast_range=[0.5, 1.5], 
86                 brightness_delta=[-0.2, 0.2], **kwargs):
87        super(RandomColorDistortion, self).__init__(**kwargs)
88        self.contrast_range = contrast_range
89        self.brightness_delta = brightness_delta
90    
91    def call(self, images, training=None):
92        if not training:
93            return images
94        contrast = np.random.uniform(
95            self.contrast_range[0], self.contrast_range[1])
96        brightness = np.random.uniform(
97            self.brightness_delta[0], self.brightness_delta[1])
98        
99        images = tf.image.adjust_contrast(images, contrast)
100        images = tf.image.adjust_brightness(images, brightness)
101        images = tf.clip_by_value(images, 0, 1)
102        return images
103    
104    def get_config(self):
105        config = super(RandomColorDistortion, self).get_config()
106        config.update({&quot;contrast_range&quot;: self.contrast_range, &quot;brightness_delta&quot;: self.brightness_delta})
107        return config
108        
109input_shape_rgb = (256, 256, 3)
110data_augmentation_rgb = tf.keras.Sequential(
111  [ 
112    tf.keras.layers.RandomFlip(&quot;horizontal&quot;),
113    tf.keras.layers.RandomFlip(&quot;vertical&quot;),
114    tf.keras.layers.RandomRotation(0.5),
115    tf.keras.layers.RandomZoom(0.5),
116    tf.keras.layers.RandomContrast(0.5),
117    RandomColorDistortion(name='random_contrast_brightness/none'),
118  ]
119)
120input_shape = (256, 256, 3)
121padding = 'same'
122kernel_size = 3
123model = tf.keras.Sequential([
124  tf.keras.layers.Input(input_shape),
125  data_augmentation_rgb,
126  tf.keras.layers.Rescaling((1./255)),
127  tf.keras.layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
128     data_format='channels_last'),
129  tf.keras.layers.MaxPooling2D(),
130  tf.keras.layers.BatchNormalization(),
131
132  tf.keras.layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
133  tf.keras.layers.MaxPooling2D(),
134  tf.keras.layers.BatchNormalization(),
135
136  tf.keras.layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
137  tf.keras.layers.MaxPooling2D(),
138  tf.keras.layers.BatchNormalization(),
139
140  tf.keras.layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
141  tf.keras.layers.MaxPooling2D(),
142  tf.keras.layers.BatchNormalization(),
143
144  tf.keras.layers.Flatten(),
145  tf.keras.layers.Dense(128, activation='relu'), # best 1
146  tf.keras.layers.Dropout(0.1),
147  tf.keras.layers.Dense(128, activation='relu'), # best 1
148  tf.keras.layers.Dropout(0.1),
149  tf.keras.layers.Dense(64, activation='relu'), # best 1
150  tf.keras.layers.Dropout(0.1),
151  tf.keras.layers.Dense(5, activation = 'softmax')
152 ])
153
154model.compile(loss='categorical_crossentropy', optimizer='adam')
155model.summary()
156model.save(&quot;test&quot;, save_format='h5')
157

Loading your model with your custom layer would look like this then:

1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3  [ 
4    layers.RandomFlip(&quot;horizontal&quot;),
5    layers.RandomFlip(&quot;vertical&quot;),
6    layers.RandomRotation(0.5),
7    layers.RandomZoom(0.5),
8    layers.RandomContrast(0.5),
9    RandomColorDistortion(name='random_contrast_brightness/none'),
10  ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16  layers.Input(input_shape),
17  data_augmentation_rgb,
18  layers.Rescaling((1./255)),
19
20  layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
21     data_format='channels_last'),
22  layers.MaxPooling2D(),
23  layers.BatchNormalization(),
24
25  layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26  layers.MaxPooling2D(),
27  layers.BatchNormalization(),
28
29  layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30  layers.MaxPooling2D(),
31  layers.BatchNormalization(),
32
33  layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34  layers.MaxPooling2D(),
35  layers.BatchNormalization(),
36
37  layers.Flatten(),
38  layers.Dense(128, activation='relu'), # best 1
39  layers.Dropout(0.1),
40  layers.Dense(128, activation='relu'), # best 1
41  layers.Dropout(0.1),
42  layers.Dense(64, activation='relu'), # best 1
43  layers.Dropout(0.1),
44  layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save(&quot;./&quot;)
50---------------------------------------------------------------------------
51KeyError                                  Traceback (most recent call last)
52&lt;ipython-input-84-87d3f09f8bee&gt; in &lt;module&gt;()
53----&gt; 1 model.save(&quot;./&quot;)
54
55
56/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in 
57 error_handler(*args, **kwargs)
58 65     except Exception as e:  # pylint: disable=broad-except
59 66       filtered_tb = _process_traceback_frames(e.__traceback__)
60 ---&gt; 67       raise e.with_traceback(filtered_tb) from None
61 68     finally:
62 69       del filtered_tb
63
64 /usr/local/lib/python3.7/dist- 
65 packages/tensorflow/python/saved_model/function_serialization.py in 
66 serialize_concrete_function(concrete_function, node_ids, coder)
67 66   except KeyError:
68 67     raise KeyError(
69 ---&gt; 68         f&quot;Failed to add concrete function '{concrete_function.name}' to 
70 object-&quot;
71 69         f&quot;based SavedModel as it captures tensor {capture!r} which is 
72 unsupported&quot;
73 70         &quot; or not reachable from root. &quot;
74
75 KeyError: &quot;Failed to add concrete function 
76 'b'__inference_sequential_46_layer_call_fn_662953'' to object-based SavedModel as it 
77 captures tensor &lt;tf.Tensor: shape=(), dtype=resource, value=&lt;Resource Tensor&gt;&gt; which 
78 is unsupported or not reachable from root. One reason could be that a stateful 
79 object or a variable that the function depends on is not assigned to an attribute of 
80 the serialized trackable object (see SaveTest.test_captures_unreachable_variable).&quot;
81import tensorflow as tf
82import numpy as np
83
84class RandomColorDistortion(tf.keras.layers.Layer):
85    def __init__(self, contrast_range=[0.5, 1.5], 
86                 brightness_delta=[-0.2, 0.2], **kwargs):
87        super(RandomColorDistortion, self).__init__(**kwargs)
88        self.contrast_range = contrast_range
89        self.brightness_delta = brightness_delta
90    
91    def call(self, images, training=None):
92        if not training:
93            return images
94        contrast = np.random.uniform(
95            self.contrast_range[0], self.contrast_range[1])
96        brightness = np.random.uniform(
97            self.brightness_delta[0], self.brightness_delta[1])
98        
99        images = tf.image.adjust_contrast(images, contrast)
100        images = tf.image.adjust_brightness(images, brightness)
101        images = tf.clip_by_value(images, 0, 1)
102        return images
103    
104    def get_config(self):
105        config = super(RandomColorDistortion, self).get_config()
106        config.update({&quot;contrast_range&quot;: self.contrast_range, &quot;brightness_delta&quot;: self.brightness_delta})
107        return config
108        
109input_shape_rgb = (256, 256, 3)
110data_augmentation_rgb = tf.keras.Sequential(
111  [ 
112    tf.keras.layers.RandomFlip(&quot;horizontal&quot;),
113    tf.keras.layers.RandomFlip(&quot;vertical&quot;),
114    tf.keras.layers.RandomRotation(0.5),
115    tf.keras.layers.RandomZoom(0.5),
116    tf.keras.layers.RandomContrast(0.5),
117    RandomColorDistortion(name='random_contrast_brightness/none'),
118  ]
119)
120input_shape = (256, 256, 3)
121padding = 'same'
122kernel_size = 3
123model = tf.keras.Sequential([
124  tf.keras.layers.Input(input_shape),
125  data_augmentation_rgb,
126  tf.keras.layers.Rescaling((1./255)),
127  tf.keras.layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1, 
128     data_format='channels_last'),
129  tf.keras.layers.MaxPooling2D(),
130  tf.keras.layers.BatchNormalization(),
131
132  tf.keras.layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
133  tf.keras.layers.MaxPooling2D(),
134  tf.keras.layers.BatchNormalization(),
135
136  tf.keras.layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
137  tf.keras.layers.MaxPooling2D(),
138  tf.keras.layers.BatchNormalization(),
139
140  tf.keras.layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
141  tf.keras.layers.MaxPooling2D(),
142  tf.keras.layers.BatchNormalization(),
143
144  tf.keras.layers.Flatten(),
145  tf.keras.layers.Dense(128, activation='relu'), # best 1
146  tf.keras.layers.Dropout(0.1),
147  tf.keras.layers.Dense(128, activation='relu'), # best 1
148  tf.keras.layers.Dropout(0.1),
149  tf.keras.layers.Dense(64, activation='relu'), # best 1
150  tf.keras.layers.Dropout(0.1),
151  tf.keras.layers.Dense(5, activation = 'softmax')
152 ])
153
154model.compile(loss='categorical_crossentropy', optimizer='adam')
155model.summary()
156model.save(&quot;test&quot;, save_format='h5')
157model = tf.keras.models.load_model('test.h5', custom_objects={'RandomColorDistortion': RandomColorDistortion})
158

where RandomColorDistortion is the name of your custom layer.

Source https://stackoverflow.com/questions/69955838

QUESTION

React: hiding vs removing components

Asked 2022-Feb-03 at 09:53

This question is about architecture more than coding.

Here's the case. In React sometimes we want to hide components. For example, when user opens new page in SPA, when some toast is closed, etc. We can hide them with adding display: none. Or we can remove them from the virtual DOM.

1// Hidden div
2&lt;div style={{ display: 'none' }}/&gt;
3
4// Removed div
5{false &amp;&amp; &lt;div/&gt;}
6

And some of our seniors othen use the first variant. Even if they hide entire page. That's what they say about this method: "This case React prerenders needed content, so when the content must appear, it takes less time".

But in this case we can't use lifecycle hooks, because even when component hides, it's not removed. But the main problem as I think, is that real DOM becomes gigantic. And that's brings slowness, isn't it?

So, what is better?


I didn't found any conversations about this question. Maybe you could help me.


EDIT 1: Despite the fact there are some answers I'd like to know more opinions. So, I decided to open up a bounty

ANSWER

Answered 2021-Sep-01 at 07:22

Well if you want to use lifecycles there are workarounds for that as well. if you are using functional components then you can manage the rerenders using the dependency props.

Its true dom size can slow you down if you use it excessively https://web.dev/dom-size/ But is better if those components are constantly being updated rather then rendering a new component on demand.

If its a list of items and its gigantic i suggest you to take a look at https://react-window.vercel.app/#/examples/list/fixed-size or https://bvaughn.github.io/react-virtualized/#/components/List

Source https://stackoverflow.com/questions/69009266

QUESTION

M1 Mac - GDAL Wrong Architecture Error [Django]

Asked 2022-Jan-23 at 19:12

I'm trying to get a django project up and running, which depends on GDAL library. I'm working on a M1 based mac.

Following the instructions on official Django docs, I've installed the necessary packages via brew

1$ brew install postgresql
2$ brew install postgis
3$ brew install gdal
4$ brew install libgeoip
5

gdalinfo --version runs fine and shows the version as 3.3.1

gdal-config --libs returns this path: -L/opt/homebrew/Cellar/gdal/3.3.1_2/lib -lgdal

a symlink is also placed on the homebrew's lib directory, which is in my path env variable.

When I try to run django without specifying the path to gdal library, it complains that it cannot find the GDAL package (even though the library is reachable, as a symlink to it is available through path env variable).

When I try to specify the path to the GDAL library using GDAL_LIBRARY_PATH, I get this error:

1$ brew install postgresql
2$ brew install postgis
3$ brew install gdal
4$ brew install libgeoip
5OSError: dlopen(/opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib, 6): no suitable image found.  Did find:
6    /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib: mach-o, but wrong architecture
7    /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.29.dylib: mach-o, but wrong architecture
8

P.s. I've already seen this answer, but it didn't help.

Isn't that strange when I try to run gdalinfo it runs fine but when django tries to run it throws me this error? What am I doing wrong?

ANSWER

Answered 2021-Nov-23 at 07:35

Try using the new arm version of python!

1$ brew install postgresql
2$ brew install postgis
3$ brew install gdal
4$ brew install libgeoip
5OSError: dlopen(/opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib, 6): no suitable image found.  Did find:
6    /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib: mach-o, but wrong architecture
7    /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.29.dylib: mach-o, but wrong architecture
8brew install --cask miniforge
9conda init zsh
10conda activate
11conda install numpy scipy scikit-learn
12

Source https://stackoverflow.com/questions/68561453

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Architecture

Tutorials and Learning Resources are not available at this moment for Architecture

Share this Page

share link

Get latest updates on Architecture