Popular New Releases in Architecture
ant-design
4.20.0-alpha.1
mall
v1.0.1
spring-framework
v5.3.19
istio
Istio 1.13.3
ant-design-pro
Popular Libraries in Architecture
by donnemartin python
143449 NOASSERTION
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
by ant-design typescript
79578 MIT
An enterprise-class UI design language and React UI library
by iluwatar java
74302 NOASSERTION
Design patterns implemented in Java
by doocs java
57101 CC-BY-SA-4.0
😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲:涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识
by ryanmcdermott javascript
56019 MIT
:bathtub: Clean Code concepts adapted for JavaScript
by macrozheng java
52180 Apache-2.0
mall项目是一套电商系统,包括前台商城系统及后台管理系统,基于SpringBoot+MyBatis实现,采用Docker容器化部署。 前台商城系统包含首页门户、商品推荐、商品搜索、商品展示、购物车、订单流程、会员中心、客户服务、帮助中心等模块。 后台管理系统包含商品管理、订单管理、会员管理、促销管理、运营管理、内容管理、统计报表、财务管理、权限管理、设置等模块。
by spring-projects java
47171 Apache-2.0
Spring Framework
by android kotlin
40272 Apache-2.0
A collection of samples to discuss and showcase different architectural tools and patterns for Android apps.
by istio go
30043 Apache-2.0
Connect, secure, control, and observe services.
Trending New libraries in Architecture
by mingrammer python
16552 MIT
:art: Diagram as Code for prototyping cloud system architectures
by dromara java
8497 Apache-2.0
这可能是史上功能最全的Java权限认证框架!目前已集成——登录认证、权限认证、分布式Session会话、微服务网关鉴权、单点登录、OAuth2.0、踢人下线、Redis集成、前后台分离、记住我模式、模拟他人账号、临时身份切换、账号封禁、多账号认证体系、注解式鉴权、路由拦截式鉴权、花式token生成、自动续签、同端互斥登录、会话治理、密码加密、jwt集成、Spring集成、WebFlux集成...
by skydoves kotlin
5211 Apache-2.0
🗡️ Pokedex demonstrates modern Android development with Hilt, Material Motion, Coroutines, Flow, Jetpack (Room, ViewModel) based on MVVM architecture.
by megaease go
4243 Apache-2.0
A Cloud Native traffic orchestration system
by yedf go
4009 BSD-3-Clause
🔥A cross-language distributed transaction manager. Support xa, tcc, saga, transactional messages. 跨语言分布式事务管理器
by cloudwego go
3958 Apache-2.0
A high-performance and strong-extensibility Golang RPC framework that helps developers build microservices.
by douyu go
3691 Apache-2.0
Jupiter是斗鱼开源的面向服务治理的Golang微服务框架
by dromara java
3542 Apache-2.0
这可能是史上功能最全的Java权限认证框架!目前已集成——登录认证、权限认证、分布式Session会话、微服务网关鉴权、单点登录、OAuth2.0、踢人下线、Redis集成、前后台分离、记住我模式、模拟他人账号、临时身份切换、账号封禁、多账号认证体系、注解式鉴权、路由拦截式鉴权、花式token生成、自动续签、同端互斥登录、会话治理、密码加密、jwt集成、Spring集成、WebFlux集成...
by LandGrey java
2694
SpringBoot 相关漏洞学习资料,利用方法和技巧合集,黑盒安全评估 check list
Top Authors in Architecture
1
115 Libraries
4459
2
44 Libraries
212
3
42 Libraries
1994
4
26 Libraries
2155
5
26 Libraries
1557
6
26 Libraries
1483
7
26 Libraries
118069
8
25 Libraries
158
9
25 Libraries
16873
10
24 Libraries
1086
1
115 Libraries
4459
2
44 Libraries
212
3
42 Libraries
1994
4
26 Libraries
2155
5
26 Libraries
1557
6
26 Libraries
1483
7
26 Libraries
118069
8
25 Libraries
158
9
25 Libraries
16873
10
24 Libraries
1086
Trending Kits in Architecture
Here are some of the famous C# Microservice Libraries. C# Microservice Libraries' use cases include Data Access, Business Logic, Security, Automation, Integration, and Monitoring.
C# Microservice Libraries are packages of code that allow developers to rapidly create, deploy, and manage microservices in the C# programming language. They provide a foundation of reusable code and components, such as authentication, logging, and messaging, that can be used to create and deploy microservices quickly. Additionally, they often provide tools and frameworks to assist in developing, deploying, and managing microservices.
Let us have a look at these libraries in detail below.
eShopOnContainers
- Built-in support for distributed application architectures and stateful microservices.
- Fully container-based architecture.
- Support for cloud-native development.
CAP
- Provides a unified API for building distributed, event-driven microservices.
- Provides a lightweight, cloud-native infrastructure for running microservices.
- Provides a complete set of tools and libraries for developing, testing, and deploying microservices.
tye
- Allows developers to create and manage multiple services with a single command.
- Automatically detect and manage configuration changes across services.
- Allows developers to iterate and test their code with a single command quickly.
surging
- Supports Dependency Injection (DI), which makes it easier for developers to manage and maintain their code.
- Provides API gateways that allow for better control over API traffic.
- Provides service discovery, allowing developers to locate network services easily.
coolstore-microservices
- Includes a comprehensive logging and reporting system.
- Supports a wide range of service discovery and monitoring tools.
- Provides a modern CI/CD pipeline for automated deployment of applications and services.
run-aspnetcore-microservices
- End-to-end monitoring and logging of service calls and performance data.
- Intuitive and extensible service-oriented architecture (SOA) framework.
- Support for advanced message-based communication between services.
microdot
- Built on the Actor Model, providing a robust framework for distributed, concurrent, fault-tolerant, and scalable applications.
- Built-in dependency injection system, allowing developers to quickly and easily inject dependencies into their services.
- Built-in support for load-balancing, meaning that services can be distributed across multiple nodes in the cluster.
Micronetes
- Provides a simple way to package, deploy and manage microservices with Docker containers.
- Powerful integration layer that allows developers to connect microservices easily with each other.
- Supports service discovery, which allows services to be discovered and connected automatically.
awesome-dotnet-tips
- Wide selection of tips and tricks for developing more efficient and maintainable C# microservices.
- Easy-to-use command line interface.
- Built-in support for the latest versions of popular C# frameworks.
Convey
- Provides various features, such as distributed tracing, service discovery, and resilience.
- Offers an opinionated approach to microservices architecture.
- Provides a set of tools for testing and monitoring microservices
Java Redis libraries often include connection pooling for efficiently managing connections to the Redis server. It often includes built-in support for deserializing and serializing Java objects to and from Redis data structures like lists and hashes.
It supports transactions that will allow multiple operations to be performed atomically. Redis libraries support Redis sub/pub messaging, allowing clients to subscribe to channels and receive real time messages. It allows Lua scripts to be executed on the Redis server by offering added performance and flexibility. It offers support for the Redis cluster, allowing clients to connect and interact with it.
Here is the list of the top 10 Java Redis Libraries handpicked for helping developers:
JavaGuide:
- Offers an easy-to-use API to interact with Redis, like simple commands for setting and getting values, incrementing or decrementing values, and working with Redis lists, sets, and hashes.
- Is designed to be highly performant and optimized for speed, which uses the Jedis client library for communicating with Redis, which is optimized for performance.
- Supports Redis transactions, allowing multiple Redis commands to be executed as a single transaction, ensuring consistency and atomicity.
mall:
- Has a small footprint and is designed to be lightweight, making it suitable for use in resource-constrained environments.
- Includes connection pooling for minimizing the overhead of creating and destroying Redis connections.
- Is thread-safe, allowing multiple threads to access Redis concurrently without requiring external synchronization.
redisson:
- Offers several distributed data structures like queues, locks, maps, and sets, which can be used for multiple JVM instances or servers.
- Supports reactive programming through reactive systems, allowing developers to work with Redis using non-blocking APIs.
- Offers integration with the Spring Framework, allowing developers to use Redisson with Spring-based applications.
eladmin:
- Is an open source library that offers a framework for building web-based administration panels.
- Is designed to be modular, allowing developers to add or remove features as required easily.
- Offers a simple and intuitive API for performing CRUD operations on database entities.
zheng:
- Offers different methods for assessing the quality of scRNA-seq data, like evaluating the number of genes detected, the percentage of mitochondrial reads, and the number of genes detected.
- Provides many clustering methods to group cells depending on gene expression similarity.
- Offers methods for inferring developmental trajectories from scRNA-seq using graph-based approaches.
springboot-learning-example:
- Offers examples of using different Spring Boot functionalities and features for building web applications, microservices, and RESTful services.
- Contains different examples from simple Hello World applications to more complex microservices with API documentation and database integration.
- Offers examples of how to implement basic and advanced security features in your Spring Boot applications using Spring Security.
SpringBoot-learning:
- Offers pre-configured Spring Boot projects which can be easily cloned and started with the help of a few commands.
- Includes examples of how to write integration and unit tests for your Spring Boot application using Mockito and JUnit.
- Offers examples of deploying Spring Boot applications on platforms like Kubernetes, Heroku, and Cloud Foundry.
flink-learning:
- Is a collection of examples and tutorials which demonstrate how to use Apache Flink to build real time stream processing applications.
- Contains various examples, from simple word count applications to more complex projects demonstrating how to use Flink with other technologies like HBase, Cassandra, and Kafka.
- Is a valuable resource for developers who want to learn how to use Flink to build scalable and fault-tolerant stream processing applications.
mall-learning:
- Contains examples from simple e-commerce applications to more complex projects that demonstrate how to use Spring Boot with other technologies like Redis, RabbitMQ, and Elasticsearch.
- Offers various examples and tutorials covering different aspects of e-commerce application development, making it easier for developers to start building their own e-commerce applications.
- Includes examples of writing unit and integration tests for your e-commerce application using Mockito and JUnit.
source-code-hunter:
- Offers a powerful search engine that will allow developers to quickly search for source code snippets from popular programming languages like Python, JavaScript, and Java.
- Offers different sorting and filtering options for helping developers narrow search results based on repository stars, code languages, and repository forks.
- Offers syntax highlighting for code snippets, which makes it easier for developers to read and understand code examples.
Microservices break down complex applications into smaller and independent services. Microservice architecture provides solutions to difficulties of using Monolith architecture, such as scalability and numerous lines of code for applications. The benefits are typically divided by business domain, allowing developers to focus on a specific part of the application and iterate on it quickly. Applications can be changed, deleted, and replaced without affecting the entire application, making application maintenance simpler.
Java Microservices Libraries are a set of open-source libraries that empower developers to quickly and easily create microservices-based applications. They provide tools and frameworks for creating and managing microservices, including APIs, distributed systems, and cloud-native architectures. Each service performs a specific task, such as handling user authentication, providing data storage, or processing payments. They help developers easily create distributed applications without having to write complex code. They give us a way to quickly identify and debug any issues that may arise in a distributed system. By using microservices in Java, new technology and process adoption become easier.
Java Microservices rely on a communication protocol called Remote Procedure Call (RPC) to facilitate communication between services. RPC is a robust protocol that allows for the remote execution of procedures or functions without the need for shared memory or database connections between services. This makes it an ideal solution for distributed systems, as it allows for efficient communication between multiple services without requiring them to be co-located. RPC also allows easy integration of new services into an existing system without requiring changes to the current system.
Each Java Microservices Library has its unique set of features and functionalities. light-4j, piggymetrics, Activiti, microservices-spring-boot, mycollab, and microservice-istio are libraries that help us to reuse the code. Micronaut is a full-stack framework for creating JVM-based applications. Apache Camel is an open-source integration framework that provides components for routing messages. Quarkus is a Kubernetes-native Java stack designed to make applications more efficient. Helidon and KumuluzEE are lightweight frameworks for creating serverless applications.
Check out the below list to find the best top 10 Java Microservices Libraries for your app development.
Here are the best open-source .NET distributed computing libraries for your applications. This comprehensive set of tools can be used to build various applications, including distributed computing.
Distributed computing uses multiple computers or nodes working together to solve a problem or execute a task. .NET offers a range of libraries and tools that make it easy to develop distributed computing applications. One of the key libraries for distributed computing in .NET is the Windows Communication Foundation (WCF), which offers a unified programming model for building distributed applications. WCF allows developers to build services that can be consumed by different clients using various protocols such as TCP, HTTP, and message queuing. Also, some libraries allow developers to create distributed applications by exchanging objects between processes on different machines. They support multiple communication channels such as HTTP, TCP, and named pipes and can be used to build client-server and peer-to-peer distributed applications.
In summary, .NET provides rich libraries and tools for building distributed computing applications. Whether you are building a client-server application, a peer-to-peer network, or a workflow-based system, .NET has the tools to build robust and scalable distributed applications. We have handpicked the top and trending open-source .NET distributed computing libraries for your next application development project.
Windows Communication Foundation (WCF):
- Used in Server, Runtime environment applications, etc.
- WCF is a unified programming model for building distributed applications.
- Provides a set of APIs and tools for building a variety of services.
- Supports different protocols such as TCP, HTTP, and message queuing.
.NET Remoting Framework:
- Used in User Interface, Frontend Framework applications, etc.
- Offers a remote control and remote scripting solution built with .NET 5.
- Enables developers to create distributed applications by exchanging objects between processes on different machines.
- Supports multiple communication channels such as HTTP, TCP, and named pipes.
Windows Workflow Foundation (WF):
- Used in Server, Runtime environment applications, etc.
- It is a set of libraries and tools for building workflow-based applications.
- Enables developers to create workflows that can be distributed across multiple machines.
- Provides a powerful way to build distributed applications that involve complex business processes.
MassTransit:
- Used in Messaging, Pub-Sub, RabbitMQ applications, etc.
- Offers an open-source message bus framework for building distributed applications.
- Provides a lightweight, scalable, and extensible way to build event-driven systems using message-based communication.
- Supports all major message brokers, including RabbitMQ, Azure Service Bus, and Amazon SQS.
NServiceBus:
- Used typically in Messaging applications.
- It’s the most popular open-source message bus framework for building distributed applications.
- Provides a reliable and scalable way to build event-driven systems using message-based communication.
- Supports various technologies, including MSMQ, RabbitMQ, SQL Server, Azure Queue Storage, Azure Service Bus, and Amazon SQS.
Akka.NET:
- Used in Architecture, Microservice applications, etc.
- Offers a toolkit and runtime for building highly concurrent, distributed, and fault-tolerant systems.
- Provides actor frameworks, which are lightweight and independent units of computation that communicate with each other using message passing.
- It is a.NET Foundation project.
Orleans:
- Used as a cross-platform framework for building robust, scalable distributed applications.
- It’s a framework for building distributed applications using the actor model.
- Offers a Cloud Native application framework for .NET.
- Provides a scalable and fault-tolerant way to build highly concurrent applications.
ServiceStack:
- Used in Utilities applications for building distributed applications.
- It’s a high-performance, open-source web services framework.
- Provides a set of APIs and tools for building RESTful services.
- Supports different protocols such as HTTP, TCP, and message queuing.
SignalR:
- Used in Networking, Websocket applications, etc.
- Offers a real-time web communication library for building distributed applications.
- Enables building real-time, event-driven systems using websockets or long polling.
- Makes it simple to add real-time web functionality to your applications.
Hazelcast.NET:
- Used typically in Big Data applications.
- It’s an in-memory data grid for building distributed applications.
- Provides a scalable and fault-tolerant way to store and retrieve data across multiple machines.
- Offers enterprise-level security through SSL and mutual authentication.
JavaScript architecture libraries help you build better, reusable and testable code by providing you with an interface to various aspects of a computer system. They provide abstractions that allow you to build powerful apps without having to understand all the details of how they work underneath. The JavaScript architecture libraries clean-code-javascript and generator-jhipster help you write more maintainable JavaScript code. They also allow you to use more advanced features of the language. clean-code-javascript is a Javascript framework that helps developers write code that is easy to read, easy to maintain and easy to test. It provides abstractions on top of various parts of your application such as models, views, controllers, services etc. The library is used by thousands of developers across the world and has been downloaded more than 5 million times from its website. generator is a library for building custom generators for ES6+ syntax (including async/await). Jhipster is a powerful & extensible JavaScript library for working with HTML elements & CSS properties directly from your codebase. With Jhipster, you can write more efficient code by generating DOM elements and changes as you write them so you can focus on business logic instead of DOM manipulation. Generator-jhipster is a library for building custom generators for ES6+ syntax (including async/await). Popular open source JavaScript architecture libraries for developers include
Architecture is a design pattern that describes the way a system should be organized to meet required functionalities. C++ architecture libraries like lager, thunder, occa, khepri are used to model and implement the architectural patterns in a C++ program. The main purpose of using these libraries is to provide a structure for the application programmers so that they can easily understand how an application is structured and what components are involved in its design. C++ provides various ways to implement an application's architecture. Lager is a C++ library that provides a high-level interface to many types of relational database systems and other data sources. The goal of lager is to create a complete implementation that can be used for developing applications for any kind of database system. Thunder is an advanced C++ template library for creating high-performance networked applications. Thunder includes support for TCP/IP sockets and Transmission Control Protocol (TCP)/User Datagram Protocol (UDP) sockets, IPv4 and IPv6 unicast address support; stream I/O on UDP streams, server sockets, message queues, semaphores and shared memory objects. Many developers depend on the following open source C++ architecture libraries
The use of Ruby Architecture libraries like trailblazer, samson, granite, etc., contributed to the success of the framework. These libraries are used extensively in the framework for their ease of use and flexibility. The Ruby Architecture libraries are a set of tools that help you build applications with a familiar, object-oriented style. They provide a more reactive programming model and simplify the process of building web applications. Trailblazer is an open-source framework for building Ruby on Rails applications. It is ideal for any project where you have to build a complete application from scratch, but don't have the time or resources to do it by hand. It simplifies all the repetitive tasks that come with developing an application, so you can concentrate on what matters most: your business logic. Samson is a microframework for creating lightweight Rails applications in Ruby 2.4+. It's intended for small projects where performance and simplicity matter more than features like authentication or session management. Granite is a collection of classes and tools for working with relational databases in Ruby on Rails applications. It includes support for PostgreSQL as well as MySQL and SQLite3 databases, which mean that you can use any database technology with Ruby on Rails without worrying about how it works behind the scenes or what code you need to write to make it work properly. There are several popular open source Ruby Architecture libraries available for developers
The use of architecture libraries is a great way to provide structure to your code and make it easier to maintain. It also helps you document your code and provide guidelines for new developers. If you just start writing code without any kind of structure, it's going to be hard for others to understand what you're doing. You can use architecture libraries like clean-architecture-manga, MCW-serverless architecture, N-tier architecture as a reference on how things should be done in your project. C# architecture libraries are useful for implementing architectures like n-tier architecture, clean architecture and many more. Clean Architecture Manga is a clean architecture framework that can be used to design, document and implement Agile projects. It was designed with simplicity in mind and is fully customizable according to the project requirements. MCW Serverless Architecture is a lightweight and extensible framework for building modern cloud applications using AWS Lambda functions and Amazon API Gateway. This library helps you build scalable microservices for your serverless architectures. N-Tier architecture framework helps developers to develop and deploy multi-tier applications on the cloud from one central place. It provides support for multiple tiers like UI, business logic & database as well as integration between them all. Popular open source C# architecture libraries include
Python Architecture libraries have a lot of great features. They can be used to easily build a complete application from scratch or to augment existing code. The biggest advantage is that these libraries abstract away some of the details of the underlying architecture and make it easier to write applications that take advantage of all of their features. The best part about these libraries is that they are open source, so anyone can contribute if they want to improve them or make them better. The Python Architecture libraries offer a set of common building blocks for machine learning and AI applications. These libraries are used to implement the architecture of many different types of algorithms in the field of artificial intelligence. The ENAS-pytorch library is a Python interface to the Semantic Segmentation and Detection (ENAS) framework. This library allows users to segment images into different classes by using a pre-trained model trained on TensorFlow objects. Developers tend to use some of the following open source Python Architecture libraries
Go architecture libraries are a great way to jump start your serverless application development. They provide the scaffolding for creating a robust serverless application and help you avoid the common pitfalls of developing with Go. Go architecture libraries allow you to create and manage your services, components, and resources in a consistent manner that is easy to scale. These libraries handle all the details of deploying your application or service so that you can focus on what matters most: writing code. When using these libraries, it’s important to understand how they work and what they can do for you. Most libraries are designed to simplify common tasks and allow developers to focus on writing code rather than dealing with all of the details of coding. Go-clean-arch is a Go Architecture library for writing clean architecture in Go. Polaris is a Go Architecture library for writing clean architecture in Go. Go-carbon is a Go Architecture library for writing clean architecture in Go. Draft is a working draft of the Go Architecture Standard, meant to be a reference for the other libraries and patterns. Popular open source Go architecture libraries among developers include
PHP Architecture libraries like deptrac, phpat, arkitect are a great solution for developers who want to create reusable code. All of the libraries are designed in a way that it is easy to integrate them into your project. One of the most popular PHP Architecture libraries is Deptrac. It is used for developing complex applications with a large number of classes and interfaces. These libraries can make the job of writing code simpler and easier. They allow developers to write code faster and more efficiently. There are some libraries that are extremely popular and widely used by many developers around the world. Deptrac is an architecture library for PHP5/6 written in C++ which provides a set of tools for developing web applications with high performance, scalability and reliability. Phpat is a set of tools for developing web applications with high performance, scalability and reliability on different platforms such as Linux, Windows, Mac OS X etc., The arkitect project aims at providing a complete solution for building scalable architectures using modern technologies like service-oriented architecture (SOA), object-oriented design concepts like inheritance, polymorphism etc., Full list of the best open-source PHP Architecture libraries are below
Java MVC libraries are a design pattern for developing web applications. It is a popular approach to building web applications and has become the most popular choice over the last decade. Java MVC libraries like blade, nutz, spring-mvc-showcase have been developed to make development easy and flexible. Blade is an open source, lightweight and easy-to-use MVC framework. It is known as a minimalist, highly modular and flexible framework. Nutz is a lightweight, modular and flexible Java MVC web application framework. It is one of the best frameworks for building modern, high performance web applications with a clean architecture and ease of use. Spring MVC is a popular open source Java web application framework for developing Java EE 5/6 applications. It provides support for RESTful web services, session state management and Ajax features along with other features like reactive programming model and Model View Controller (MVC) architecture. The following is a list of the most popular Java MVC libraries.
The use of Python MVC libraries like pythonik, fastapi-mvc-template, oddo-learning, etc. is becoming more and more popular in the world of AI development. The reason behind this trend is that these libraries provide a great deal of flexibility to developers who want to build web applications with modern features. Pythonik is one such library that enables you to build web applications quickly and easily. It supports both Django and Pyramid frameworks with many other features like authentication, authorization, session management, etc. These libraries also help us to handle data and perform operations on it. We can also add new features to our application by using third-party packages which are available in the market. The following is a list of the most popular open source Python MVC libraries:
There are several Ruby MVC libraries that you can use to develop web applications. These libraries help you write code in a modular manner, which makes it easier to maintain and extend the application. Serve is a library for serving HTML, CSS and JavaScript content through Rack. It is a Rack middleware that manages the process of rendering HTML and executing JavaScript on the client side. It can be used in conjunction with rackup or RJS handlers. served is one of the most popular libraries in Ruby community. dry-view is a lightweight, high performance implementation of view helpers in Rails 3 and 4 that provides the same interface as the original ActiveRecord::ViewHelpers but with simple syntax. It supports multiple backends (Rack, Nunjucks, Haml) and plugins like head tag support. dry-view is designed for Rails 3 or higher only but it's still usable as a gem in Rails 2 as well. Check out the list of free, open source Ruby MVC libraries to help you with your projects.
JavaScript MVC libraries are great for building web applications. They help you organize your code into components, separate concerns and create a clear separation of the different parts of your app. JavaScript MVC frameworks are used to build complex applications. They are used for designing, developing and building applications. JavaScript MVC libraries like sails, spine, swarm etc. are used to create web applications using JavaScript. Sails is an open-source framework for building dynamic websites with Node.js. It uses a model-view controller (MVC) pattern, which separates the logic from the view. The framework is built on top of Express and Koa, two popular web application frameworks for Node.js. Sails is one of the most popular JavaScript MVC libraries for Rails apps. It provides a simple way to build full-stack web applications with CoffeeScript and supports many other languages. Spine is another popular JavaScript MVC library which has been used by Facebook and Twitter to build backend services like GraphQL and Node-RED. Swarm is a new open source JavaScript MVC library that has been created by Walmart Labs and NPM (Node Package Manager). It's easy to use, features an intuitive API, and lets you build real-time applications quickly and efficiently. The entire list of open source JavaScript MVC libraries is provided below.
The use of C++ MVC libraries like treefog-framework, CPPWebFramework, qt-mvvm, libevmvc is a great choice for rapid development of web applications. These frameworks are based on the concept of MVC (Model, View and Controller). These frameworks make it easy to develop applications quickly without having to worry about creating your own UI framework or writing your own data access layer. The frameworks handle much of the boilerplate work for you so that you can focus on writing code that solves your problem. TreeFog is an open source multi-platform framework for developing web applications with C++ and Qt5. It is built on top of WebCore library and provides a set of handy web development tools for rapid development of small to medium size web applications. CPPWebFramework is a high performance web framework engine written in C++ that makes it easy to write fast and scalable web applications. The framework is designed to use modern best practices in order to achieve high performance at low costs. Qt-MVVM is based on the MVVM pattern which allows you to decouple your application's UI from its business logic through the use of ViewModels and ViewControllers. This allows you to separate your business logic from your view code so that it can be reused more easily. Full list of the best open source C++ MVC libraries below
The C# MVC library is a framework which helps in creating MVC applications with data binding and dependency injection. It has many built-in components like SignalR, Web API and others. It also provides a whole bunch of utilities to simplify development. Using the C# MVC libraries like Electron.NET, Chromely, serenity brings a lot of benefits to the application development life cycle. The main reason for this is that it requires no third-party dependencies and provides an easy way to build cross-platform apps. Electron is an open-source framework that allows you to build cross-platform desktop apps using HTML5, JavaScript, and CSS3 technology. In short, Electron makes it possible to develop desktop applications using web technologies. Chromely is an open source library which can be used to build cross-platform desktop apps on the basis of Electron. The main advantage of Chromely is that it allows you to write your code once and deploy it on various platforms like Windows, MacOS, Linux etc. The following is a comprehensive list of the best open source C# MVC libraries.
The Go webapp library is a collection of Go MVC libraries that provide a high-level, web-based user interface for developers to build applications faster. The use of Go MVC libraries, such as iris, utron, goku and gowebapp is a great way to get started with web development. Iris is a popular open source framework for developing web applications in Go. It has built-in support for routing, authentication, and other key features. The utron library is an open source toolkit for building modern web applications in Go. It features a simple but powerful template system using HTML5 syntax and CSS styling conventions. Goku is an opinionated MVC framework built on top of play/2.0. It provides a set of tools to build RESTful APIs and also provides support for pub/sub messaging mechanisms such as Kafka or Redis Publish-Subscribe (pubsub) systems like RabbitMQ or ZeroMQ. Gowebapp is a full featured native mobile app development stack that runs on iOS and Android platforms. It includes everything needed to build all aspects of your app from start to finish, including models, views and controllers, routing logic and more! Popular open source Go MVC libraries among developers include
The use of MVC frameworks for PHP development is a matter of choice. There are many ways to approach the problem and each one has its own merits. The use of MVC makes it easy to create multiple applications that share a common set of functionality without having to rewrite them from scratch each time you want to add new features, allowing developers to maintain all their applications in one place. MVC is a common architecture for crafting web applications, but it can be applied in many other scenarios as well. In fact, there are several PHP MVC libraries available for you to use in your next project. WP-MVC is the most popular PHP MVC library around. It's also the first one on this list! WP-MVC comes with a lot of features, like routing, templating and more. PHP-MVC is a great alternative to wp-mvc if you're looking for something more lightweight or want to use some features that wp-mvc doesn't have natively (like routing). This library also supports Laravel 5.* out of the box! Zend_MVC is another great alternative if you're looking for something more lightweight or want to use some features that wp-mvc doesn't have natively (like routing). Many developers depend on the following open source PHP MVC libraries
It is very difficult to maintain a large monolithic application. It is also difficult to release new features and bug fixes. But using the Java Microservice libraries like apollo, nacos, armeria can easily solve these problems. The microservice approach has been around for a while now and is being used by many organizations to improve their efficiency, increase the speed of delivery and enhance the quality of their products. Apollo is an open source framework that provides efficient support for building fast and scalable distributed applications with a unified programming model. Apollo uses HTTP as its primary protocol and provides an HTTP client implementation that supports all of the major web browsers (including IE6) as well as other clients such as curl and wget. Apollo also supports streaming data using either Netty or GZIP compression for efficient high-volume data transfers over low bandwidth connections. Nacos is a lightweight library for building reactive asynchronous systems in Java SE 9+. The main purpose of NACOS is to provide a simple yet robust way to create asynchronous applications using Java SE 9+ functional interfaces with low overhead on thread creation overhead and/or blocking calls at the expense of some performance impact due to the need to manage threads yourself. Some of the most popular among developers are:
Trending Discussions on Architecture
Is there a higher performing implementation of powf(10,floorf(log10f(x)))
Unknown host CPU architecture: arm64 , Android NDK SiliconM1 Apple MacBook Pro
remark: Incremental compilation has been disabled: it is not compatible with whole module optimization
How to replace the bundled Dart SDK in Flutter to run natively on Apple Silicon (ARM64)?
How do I resolve error message: "Inheritance from an interface with '@JvmDefault' members is only allowed with -Xjvm-default option"
How does an instruction decoder tell the difference between a prefix and a primary opcode?
Flag provided but not defined: -rpc
Saving model on Tensorflow 2.7.0 with data augmentation layer
React: hiding vs removing components
M1 Mac - GDAL Wrong Architecture Error [Django]
QUESTION
Is there a higher performing implementation of powf(10,floorf(log10f(x)))
Asked 2022-Apr-16 at 01:04I have a need to truncate a float to the nearest power of 10. For example, 1.1 would truncate to 1.0 and 4.7e3 would truncate to 1e3. I am currently doing it with the seemingly complicated powf(10,floorf(log10f(x)))
. I am wondering whether there is a better performing (as in faster execution speed) solution? My target CPU architecture is both x86-64 and arm64.
1#include <stdio.h>
2#include <math.h>
3
4int main()
5{
6 float x = 1.1e5f;
7 while (x > 1e-6f)
8 {
9 float y = powf(10,floorf(log10f(x)));
10 printf("%e ==> %g\n", x, y);
11 x /= 5.0f;
12 }
13}
14
when run, this produces
1#include <stdio.h>
2#include <math.h>
3
4int main()
5{
6 float x = 1.1e5f;
7 while (x > 1e-6f)
8 {
9 float y = powf(10,floorf(log10f(x)));
10 printf("%e ==> %g\n", x, y);
11 x /= 5.0f;
12 }
13}
141.100000e+05 ==> 100000
152.200000e+04 ==> 10000
164.400000e+03 ==> 1000
178.800000e+02 ==> 100
181.760000e+02 ==> 100
193.520000e+01 ==> 10
207.040000e+00 ==> 1
211.408000e+00 ==> 1
222.816000e-01 ==> 0.1
235.632000e-02 ==> 0.01
241.126400e-02 ==> 0.01
252.252800e-03 ==> 0.001
264.505600e-04 ==> 0.0001
279.011199e-05 ==> 1e-05
281.802240e-05 ==> 1e-05
293.604480e-06 ==> 1e-06
30
ANSWER
Answered 2022-Apr-15 at 16:36I would say don't sweat it. Unless the program is spending a large proportion of its time doing this truncation, it's not worth optimising what is probably super-fast anyway. But if you wanted to optimise for your common cases (1e-2 <= x <= 10), then you might try using 32-bit integer arithmetic to compare with the binary representations of 1e-2, 1e-1, 1, and 10 (for instance, 1e-1 is 0x3dcccccd) ; if it's outside that range, you can fall back on the floating point version. Only experimentation will determine if this actually runs faster.
QUESTION
Unknown host CPU architecture: arm64 , Android NDK SiliconM1 Apple MacBook Pro
Asked 2022-Apr-04 at 18:41I've got a project that is working fine in windows os but when I switched my laptop and opened an existing project in MacBook Pro M1. I'm unable to run an existing android project in MacBook pro M1. first I was getting
Execution failed for task ':app:kaptDevDebugKotlin'. > A failure occurred while executing org.jetbrains.kotlin.gradle.internal.KaptExecution > java.lang.reflect.InvocationTargetException (no error message)
this error was due to the Room database I applied a fix that was adding below library before Room database and also changed my JDK location from file structure from JRE to JDK.
kapt "org.xerial:sqlite-jdbc:3.34.0"
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6
after that now I'm getting an issue which is Unknown host CPU architecture: arm64
there is an SDK in my project that is using this below line.
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16
App Gradle
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23
[CXX1405] error when building with ndkBuild using /Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/Android.mk: Build command failed. Error while executing process /Users/mac/Library/Android/sdk/ndk/21.4.7075529/ndk-build with arguments {NDK_PROJECT_PATH=null APP_BUILD_SCRIPT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/Android.mk APP_ABI=arm64-v8a NDK_ALL_ABIS=arm64-v8a NDK_DEBUG=1 APP_PLATFORM=android-21 NDK_OUT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/build/intermediates/cxx/Debug/4k4s2lc6/obj NDK_LIBS_OUT=/Users/mac/Desktop/Consumer-Android/ime/dictionaries/jnidictionaryv2/build/intermediates/cxx/Debug/4k4s2lc6/lib APP_SHORT_COMMANDS=false LOCAL_SHORT_COMMANDS=false -B -n} ERROR: Unknown host CPU architecture: arm64
which is causing this issue and whenever I comment on this line
path 'Android.mk'
it starts working fine, is there any way around which will help me run this project with this piece of code without getting this NDK issue?
Update - It seems that Room got fixed in the latest updates, Therefore you may consider updating Room to latest version (2.3.0-alpha01 / 2.4.0-alpha03 or above)
ANSWER
Answered 2022-Apr-04 at 18:41To solve this on a Apple Silicon M1 I found three options
AUse NDK 24
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27
You can install it with
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28
or
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29
Depending what where sdkmanager
is located
Change your ndk-build
to use Rosetta x86. Search for your installed ndk with
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30
eg
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31
and change
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31DIR="$(cd "$(dirname "$0")" && pwd)"
32$DIR/build/ndk-build "$@"
33
to
1 //Room components
2 kapt "org.xerial:sqlite-jdbc:3.34.0"
3 implementation "androidx.room:room-ktx:$rootProject.roomVersion"
4 kapt "androidx.room:room-compiler:$rootProject.roomVersion"
5 androidTestImplementation "androidx.room:room-testing:$rootProject.roomVersion"
6android {
7 externalNativeBuild {
8 ndkBuild {
9 path 'Android.mk'
10 }
11 }
12 ndkVersion '21.4.7075529'
13
14
15}
16 externalNativeBuild {
17 cmake {
18 path "src/main/cpp/CMakeLists.txt"
19 version "3.18.1"
20 //version "3.10.2"
21 }
22 }
23android {
24 ndkVersion "24.0.8215888"
25 ...
26}
27echo "y" | sudo ${ANDROID_HOME}/tools/bin/sdkmanager --install 'ndk;24.0.8215888'
28echo "y" | sudo ${ANDROID_HOME}/sdk/cmdline-tools/latest/bin/sdkmanager --install 'ndk;24.0.8215888'
29find ~ -name ndk-build 2>/dev/null
30vi ~/Library/Android/sdk/ndk/22.1.7171670/ndk-build
31DIR="$(cd "$(dirname "$0")" && pwd)"
32$DIR/build/ndk-build "$@"
33DIR="$(cd "$(dirname "$0")" && pwd)"
34arch -x86_64 /bin/bash $DIR/build/ndk-build "$@"
35
QUESTION
remark: Incremental compilation has been disabled: it is not compatible with whole module optimization
Asked 2022-Mar-31 at 03:05Updated to the Xcode 13 beta, now I can't build the project due to an error in several pods
Older Xcode launches an app on iOS 15 too long
I use M1, maybe this is due to architecture
ANSWER
Answered 2021-Aug-25 at 21:20That remark isn't an actual error, just a warning. The real error can be found in the nested Compile Swift source files
under the top level section with the same name. Expand the logs for this command and you should see the actual error.
QUESTION
How to replace the bundled Dart SDK in Flutter to run natively on Apple Silicon (ARM64)?
Asked 2022-Mar-21 at 16:27Dart SDK officially supports ARM64 and as of now, 2.14.2 is the latest (stable) Dart SDK that has support for ARM64. Though it was the same version that was bundled in my Flutter setup, it seemed to run on Intel architecture (Activity monitor shows dart processes running on Intel).
I manually tried replacing the dart SDK on my flutter installation bu replacing flutter-directory/bin/cache/dart-sdk/
with the contents of a zip file of the Dart SDK made for ARM64, downloaded from dart.dev archive. But trying to run an app on an Android emulator (which runs on ARM64 and was working on my old Flutter setup), throws this error:
1Launching lib/main.dart on sdk gphone64 arm64 in debug mode...
2lib/main.dart:1
3Snapshot not compatible with the current VM configuration: the snapshot requires 'release no-code_comments
4no-dwarf_stack_traces_mode lazy_async_stacks lazy_dispatchers
5use_bare_instructions no-dedup_instructions
6no-"asserts" "use_field_guards" "use_osr" x64-sysv no-null-safety' but the VM has 'release no-code_comments no-
7dwarf_stack_traces_mode lazy_async_stacks
8lazy_dispatchers use_bare_instructions no-dedup_instructions no-"asserts" "use_field_guards" "use_osr" arm64-sysv no-null-safety'
92
10the Dart compiler exited unexpectedly.
11
12
Is there any other way to do a completely ARM64 Flutter setup on M1 devices?
Flutter version 2.5.1
Dart version 2.14.2
Device: MacBook Air (M1, 2020)
ANSWER
Answered 2021-Sep-29 at 17:46It seems it can't be used with Flutter yet, as seen in:
Apple Silicon support in the Dart SDK
[...] Note that the Dart SDK bundled in the Flutter SDK doesn’t have these improvements yet.
https://medium.com/dartlang/announcing-dart-2-14-b48b9bb2fb67
[Announcing Dart 2.14][ScreenShot]: https://i.stack.imgur.com/N8Qcc.png
And:
Get the Dart SDK
[...] As of Flutter 1.21, the Flutter SDK includes the full Dart SDK. So if you have Flutter installed, you might not need to explicitly download the Dart SDK. Consider downloading the Dart SDK if any of the following are true:
- You don’t use Flutter.
- You use a pre-1.21 version of Flutter.
- You want to reduce disk space requirements or download time, and your use case doesn’t require Flutter. For example, you might have a continuous integration (CI) setup that requires Dart but not Flutter.
[Get the Dart SDK][ScreenShot]: https://i.stack.imgur.com/rawJV.png
QUESTION
How do I resolve error message: "Inheritance from an interface with '@JvmDefault' members is only allowed with -Xjvm-default option"
Asked 2022-Mar-19 at 21:08I'm new to Android development and I'm currently building my first real app. I'm trying to implement a MVVM architecture and because of that I'm having a viewModel for each fragment and each viewModel has a viewModelFactory. At least, this is how I understood it has to be.
I use the boilerplate code everyone seems to use for the factory:
1class ExampleViewModelFactory(private val exampleDao: ExampleDao) : ViewModelProvider.Factory {
2 override fun <T : ViewModel> create(modelClass: Class<T>): T {
3 if (modelClass.isAssignableFrom(exampleViewModel::class.java)) {
4 @Suppress("UNCHECKED_CAST")
5 return ExampleViewModel(exampleDao) as T
6 }
7 throw IllegalArgumentException("Unknown ViewModel class")
8 }
9}
10
Now the problem is, that the compiler is giving me the following error:
e: C:\Users\ ...\ExampleViewModel.kt: (64, 7): Inheritance from an interface with '@JvmDefault' members is only allowed with -Xjvm-default option
And this error is produced by the viewModelFactory class I have implemented in the viewModel. I really can't tell what this means and I cant find anything helpful or even related to my specific problem. I basically followed some basic tutorials about creating your first app, but I keep on running into errors like this. In most cases, I was able to fix the problem by myself, but this time it's different.
I know that a lot of you have a lot of experience and knowledge, so I hope that some of you find the time to help me and give me a hint what I can do to fix this.
ANSWER
Answered 2022-Feb-25 at 16:53It seems like you are either directly or indirectly (through some other library) depending on Lifecycle 2.5.0-alpha01
.
As per this issue:
You need to temporarily add following to your
build.gradle
:
1class ExampleViewModelFactory(private val exampleDao: ExampleDao) : ViewModelProvider.Factory {
2 override fun <T : ViewModel> create(modelClass: Class<T>): T {
3 if (modelClass.isAssignableFrom(exampleViewModel::class.java)) {
4 @Suppress("UNCHECKED_CAST")
5 return ExampleViewModel(exampleDao) as T
6 }
7 throw IllegalArgumentException("Unknown ViewModel class")
8 }
9}
10tasks.withType(KotlinCompile).configureEach {
11 kotlinOptions {
12 freeCompilerArgs += [
13 "-Xjvm-default=all",
14 ]
15 }
16}
17
Note that in certain circumstances you may want to use all-compatibility
instead of all
, learn more about that in Jebrain's blogpost.
Starting with kotlin 1.6.20 you won't need touch build.gradle for more information see KT-47000
QUESTION
How does an instruction decoder tell the difference between a prefix and a primary opcode?
Asked 2022-Feb-23 at 02:47I'm trying to wrap my head around the x86 instruction encoding format. All the sources that I read still make the subject confusing. I'm starting to understand it a little bit but one thing that I'm having trouble with understanding is how the CPU instruction decoder differentiates an opcode prefix from an opcode.
I'm aware that the whole format of the instruction basically depends on the opcode (with extra bit fields defined in the opcode of course). Sometimes the instruction doesn't have a prefix and the opcode is the first byte. How would the decoder know?
I'm assuming that the instruction decoder would be able to tell the difference because opcode bytes and prefix bytes would not share the same binary values. So the decoder can tell if the unique binary number in the byte is an instruction or a prefix. For example (In this example we will stick to single byte opcodes) a REX or LOCK prefix would not share the same byte value as any opcode in the architecture's instruction set.
ANSWER
Answered 2022-Feb-23 at 02:47Traditional (single-byte) prefixes are different from opcode bytes like you said, so a state machine can just remember which prefixes it's seen until it gets to an opcode byte.
The 0f
escape byte for 2-byte opcodes is not really a prefix. It has to be contiguous with the 2nd opcode byte. Thus, following a 0f
, any byte is an opcode, even if it's something like f2
that would otherwise be a prefix. (This also applies following 0f 3a
or 0f 38
2-byte escapes for SSSE3 and later, or VEX/EVEX prefixes that encode one of those escape sequences).
If you look at an opcode map, there are no entries that are ambiguous between single-byte prefix and opcode. (e.g. http://ref.x86asm.net/coder64.html, and notice how the 2-byte 0F .. opcodes are listed separately).
The decoders do have to know the current mode for this (and other things); for example x86-64 removed the 1-byte inc/dec reg
opcodes for use as REX prefixes. (x86 32 bit opcodes that differ in x86-x64 or entirely removed). We can even use this difference to write polyglot machine code that runs differently when decoded in 32-bit vs. 64-bit mode, or even distinguish all 3 mode sizes.
x86 machine code is a byte stream that's not self-synchronizing (e.g. a ModRM or an immediate can be any byte). The CPU always knows where to start decoding from, either a jump target or the byte after the end of a previous instruction. That's the start of the instruction (including prefixes).
Bytes in memory are just bytes, only becoming instructions when they're decoded by the CPU. (Although in normal programs, simply disassembling from the top of the .text
section does give you the program's instructions. Self-modifying and obfuscated code are not normal.)
Multi-byte VEX and EVEX prefixes aren't that simple in 32-bit mode. For example VEX prefixes overlap with invalid encodings of LES and LDS in modes other than 64-bit. (The c4
and c5
opcodes for LES and LDS are always invalid in 64-bit mode, except as VEX prefixes.) https://wiki.osdev.org/X86-64_Instruction_Encoding#VEX.2FXOP_opcodes
In legacy / compat modes, there weren't any free bytes left that weren't already opcodes or prefixes when AVX (VEX prefixes) and AVX-512 (EVEX prefix), so the only room for extensions was as encodings for opcodes that are only valid with a limited set of ModRM bytes. (e.g. LES / LDS require a memory source, not register - this is why some bits are inverted in VEX prefixes, so the top 2 bits of the byte after c4
or c5
will always be 1
in 32-bit mode instead of 0
.
That's the "mode" field in ModRM, and 11
means register).
(Fun fact: VEX prefixes are not recognized in 16-bit real mode, apparently because some software used the same invalid encodings of LES / LDS as intentional traps, to be sorted out in the #UD exception handler. VEX prefixes are recognized in 16-bit protected mode, though.)
AMD64 freed up several bytes by removing instructions like AAM, as well as LES/LDS (and the one-byte inc
/dec reg
encodings for use as REX prefixes), but CPU vendors have continued to care about 32-bit mode and not added any extensions that are only available in 64-bit mode which could simply take advantage of those free opcode bytes. This means finding ways to cram new instruction encodings into increasingly small gaps in 32-bit machine code. (Often via mandatory prefixes, e.g. rep bsr
= lzcnt
on CPUs with that feature, which gives different results.)
So the decoders in modern CPUs that support AVX / BMI1/2 have to look at multiple bytes to decide whether this is a prefix for a valid AVX or other VEX-encoded instruction, or in 32-bit mode if it should decode as LES or LDS. (And I guess look at the rest of the instruction to decide if it should #UD).
But modern CPUs are looking at 16 or 32 bytes at a time anyway to find instruction boundaries in parallel. (And then later feed those groups of instruction bytes to actual decoders, again in parallel.) https://www.realworldtech.com/sandy-bridge/4/
Same goes for the prefix scheme used by AMD XOP, which is a lot like VEX.
Agner Fog's blog article Stop the instruction set war from 2009 (soon after AVX was announced, before the first hardware supporting it) has a table of remaining unused coding space for future extensions, and some notes about it being "assigned" to AMD, Intel, or Via.
Related / examples- How to tell the length of an x86 instruction? (including my answer) has some more details about x86 machine code.
- https://codegolf.stackexchange.com/questions/133486/find-an-illegal-string/133622#133622 (on codegolf.SE - the shortest sequence of bytes that will definitely #UD fault if it's not jumped over. It has to be long enough that it can't be consumed by the CPU as the immediate for a
mov r64, imm64
for example.) - Why does x/i on gdb give different results then disassemble? - an example of starting decode in the wrong place and decoding the middle of another instruction as something else.
Machine code tricks: decoding the same byte multiple ways
(This is not really related to prefixes, but in general seeing how the rules apply to weird cases can help understand exactly things work.)
A software disassembler does need to know a start point. This can be problematic if obfuscated code mixes code and data, and actual execution jumps to places you wouldn't get if you just assume that you can decode in order without following jumps.
Fortunately compiler-generated code doesn't do that so naive static disassembly (e.g. by objdump -d
or ndisasm
, as opposed to IDA) finds the same instruction boundaries that actually running the program will.
This is not a problem for running obfuscated machine code; the CPU just does what it's told, and never cares about bytes before the place you tell it to jump to. Disassembling without running / single-stepping the program is the hard thing, especially with the possibility of self-modifying code and jumps to what a naive disassembler would think was the middle of an earlier instruction.
Obfuscated machine code can even have an instruction decode one way, then jump back into what was the middle of that instruction, for a later byte to be the opcode (or prefix + opcode). Modern CPUs with uop caches or that mark instruction boundaries in I-cache run slow (but correctly) if you do this, so it's more of a fun code-golf trick (extreme code-size optimization at the expense of speed) or obfuscation technique.
For an example of this, see my codegolf.SE x86 machine code answer to Golf a Custom Fibonacci Sequence. I'll excerpt the disassembly that lines up with what the CPU sees after looping back to cfib.loop
, but note that the first iteration decodes differently. So I'm using just 1 byte outside the loop instead of 2 to effectively jump into the middle for the start of the first iteration. See the linked answer for a full description and the other disassembly.
10000000000401070 <cfib>:
2 401070: eb .byte 0xeb # jmp rel8 consuming the 01 add opcode as a rel8
30000000000401071 <cfib.loop>:
4 401071: 01 d0 add eax,edx
5# loop entry point on first iteration, jumping over the ModRM byte (D0) of the ADD
6 (entry on first iteration):
7 401073: 92 xchg edx,eax
8 401074: e2 fb loop 401071 <cfib.loop>
9 401076: c3 ret
10
You can do this with opcodes that consume more later bytes, like 3D <dword> cmp eax, imm32
. When the CPU sees a 3D
opcode byte, it will grab the next 4 bytes as the immediate. If you later jump into those 4 bytes, they'll be considered as prefix/opcodes and everything will work (except for performance problems) the same regardless of how those bytes had previously been decoded as a different part of an instruction. The CPU has to maintain the illusion of decoding and executing 1 instruction at a time, other than performance.
I learned of this trick from @Ira Baxter's answer on Can assembled ASM code result in more than a single possible way (except for offset values)?
QUESTION
Flag provided but not defined: -rpc
Asked 2022-Feb-19 at 20:07Am starting off in blockchain development using the book Mastering Blockchain - A deep dive into distributed ledgers, consensus protocols, smart contracts, DApps, cryptocurrencies, Ethereum,
Am using WSL with geth version 1.10.9.
1$geth version
2Geth
3Version: 1.10.9-stable
4Git Commit: eae3b1946a276ac099e0018fc792d9e8c3bfda6d
5Architecture: amd64
6Go Version: go1.17
7Operating System: linux
8GOPATH=
9GOROOT=go
10
Am trying to start geth, but am getting the error that the --rpc flag is not defined.
This is the command I am trying to run:
geth --datadir ~/etherprivate/ --networkid 786 --rpc --rpcapi 'web3,eth,net,debug,personal' --rpccorsdomain '*'
Any help on how I can solve it?
ANSWER
Answered 2021-Oct-11 at 23:20It appears 1.10.9-stable version has a problem and is returning a -rpc error. GETH 1.10.8-stable version works fine when running the geth command with --rpc
QUESTION
Saving model on Tensorflow 2.7.0 with data augmentation layer
Asked 2022-Feb-04 at 17:25I am getting an error when trying to save a model with data augmentation layers with Tensorflow version 2.7.0.
Here is the code of data augmentation:
1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3 [
4 layers.RandomFlip("horizontal"),
5 layers.RandomFlip("vertical"),
6 layers.RandomRotation(0.5),
7 layers.RandomZoom(0.5),
8 layers.RandomContrast(0.5),
9 RandomColorDistortion(name='random_contrast_brightness/none'),
10 ]
11)
12
Now I build my model like this:
1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3 [
4 layers.RandomFlip("horizontal"),
5 layers.RandomFlip("vertical"),
6 layers.RandomRotation(0.5),
7 layers.RandomZoom(0.5),
8 layers.RandomContrast(0.5),
9 RandomColorDistortion(name='random_contrast_brightness/none'),
10 ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16 layers.Input(input_shape),
17 data_augmentation_rgb,
18 layers.Rescaling((1./255)),
19
20 layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
21 data_format='channels_last'),
22 layers.MaxPooling2D(),
23 layers.BatchNormalization(),
24
25 layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26 layers.MaxPooling2D(),
27 layers.BatchNormalization(),
28
29 layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30 layers.MaxPooling2D(),
31 layers.BatchNormalization(),
32
33 layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34 layers.MaxPooling2D(),
35 layers.BatchNormalization(),
36
37 layers.Flatten(),
38 layers.Dense(128, activation='relu'), # best 1
39 layers.Dropout(0.1),
40 layers.Dense(128, activation='relu'), # best 1
41 layers.Dropout(0.1),
42 layers.Dense(64, activation='relu'), # best 1
43 layers.Dropout(0.1),
44 layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49
Then after the training is done I just make:
1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3 [
4 layers.RandomFlip("horizontal"),
5 layers.RandomFlip("vertical"),
6 layers.RandomRotation(0.5),
7 layers.RandomZoom(0.5),
8 layers.RandomContrast(0.5),
9 RandomColorDistortion(name='random_contrast_brightness/none'),
10 ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16 layers.Input(input_shape),
17 data_augmentation_rgb,
18 layers.Rescaling((1./255)),
19
20 layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
21 data_format='channels_last'),
22 layers.MaxPooling2D(),
23 layers.BatchNormalization(),
24
25 layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26 layers.MaxPooling2D(),
27 layers.BatchNormalization(),
28
29 layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30 layers.MaxPooling2D(),
31 layers.BatchNormalization(),
32
33 layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34 layers.MaxPooling2D(),
35 layers.BatchNormalization(),
36
37 layers.Flatten(),
38 layers.Dense(128, activation='relu'), # best 1
39 layers.Dropout(0.1),
40 layers.Dense(128, activation='relu'), # best 1
41 layers.Dropout(0.1),
42 layers.Dense(64, activation='relu'), # best 1
43 layers.Dropout(0.1),
44 layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save("./")
50
And I'm getting this error:
1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3 [
4 layers.RandomFlip("horizontal"),
5 layers.RandomFlip("vertical"),
6 layers.RandomRotation(0.5),
7 layers.RandomZoom(0.5),
8 layers.RandomContrast(0.5),
9 RandomColorDistortion(name='random_contrast_brightness/none'),
10 ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16 layers.Input(input_shape),
17 data_augmentation_rgb,
18 layers.Rescaling((1./255)),
19
20 layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
21 data_format='channels_last'),
22 layers.MaxPooling2D(),
23 layers.BatchNormalization(),
24
25 layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26 layers.MaxPooling2D(),
27 layers.BatchNormalization(),
28
29 layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30 layers.MaxPooling2D(),
31 layers.BatchNormalization(),
32
33 layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34 layers.MaxPooling2D(),
35 layers.BatchNormalization(),
36
37 layers.Flatten(),
38 layers.Dense(128, activation='relu'), # best 1
39 layers.Dropout(0.1),
40 layers.Dense(128, activation='relu'), # best 1
41 layers.Dropout(0.1),
42 layers.Dense(64, activation='relu'), # best 1
43 layers.Dropout(0.1),
44 layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save("./")
50---------------------------------------------------------------------------
51KeyError Traceback (most recent call last)
52<ipython-input-84-87d3f09f8bee> in <module>()
53----> 1 model.save("./")
54
55
56/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in
57 error_handler(*args, **kwargs)
58 65 except Exception as e: # pylint: disable=broad-except
59 66 filtered_tb = _process_traceback_frames(e.__traceback__)
60 ---> 67 raise e.with_traceback(filtered_tb) from None
61 68 finally:
62 69 del filtered_tb
63
64 /usr/local/lib/python3.7/dist-
65 packages/tensorflow/python/saved_model/function_serialization.py in
66 serialize_concrete_function(concrete_function, node_ids, coder)
67 66 except KeyError:
68 67 raise KeyError(
69 ---> 68 f"Failed to add concrete function '{concrete_function.name}' to
70 object-"
71 69 f"based SavedModel as it captures tensor {capture!r} which is
72 unsupported"
73 70 " or not reachable from root. "
74
75 KeyError: "Failed to add concrete function
76 'b'__inference_sequential_46_layer_call_fn_662953'' to object-based SavedModel as it
77 captures tensor <tf.Tensor: shape=(), dtype=resource, value=<Resource Tensor>> which
78 is unsupported or not reachable from root. One reason could be that a stateful
79 object or a variable that the function depends on is not assigned to an attribute of
80 the serialized trackable object (see SaveTest.test_captures_unreachable_variable)."
81
I inspected the reason of getting this error by changing the architecture of my model and I just found that reason came from the data_augmentation layer since the RandomFlip
and RandomRotation
and others are changed from layers.experimental.prepocessing.RandomFlip
to layers.RandomFlip
, but still the error appears.
ANSWER
Answered 2022-Feb-04 at 17:25This seems to be a bug in Tensorflow 2.7 when using model.save
combined with the parameter save_format="tf"
, which is set by default. The layers RandomFlip
, RandomRotation
, RandomZoom
, and RandomContrast
are causing the problems, since they are not serializable. Interestingly, the Rescaling
layer can be saved without any problems. A workaround would be to simply save your model with the older Keras H5 format model.save("test", save_format='h5')
:
1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3 [
4 layers.RandomFlip("horizontal"),
5 layers.RandomFlip("vertical"),
6 layers.RandomRotation(0.5),
7 layers.RandomZoom(0.5),
8 layers.RandomContrast(0.5),
9 RandomColorDistortion(name='random_contrast_brightness/none'),
10 ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16 layers.Input(input_shape),
17 data_augmentation_rgb,
18 layers.Rescaling((1./255)),
19
20 layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
21 data_format='channels_last'),
22 layers.MaxPooling2D(),
23 layers.BatchNormalization(),
24
25 layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26 layers.MaxPooling2D(),
27 layers.BatchNormalization(),
28
29 layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30 layers.MaxPooling2D(),
31 layers.BatchNormalization(),
32
33 layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34 layers.MaxPooling2D(),
35 layers.BatchNormalization(),
36
37 layers.Flatten(),
38 layers.Dense(128, activation='relu'), # best 1
39 layers.Dropout(0.1),
40 layers.Dense(128, activation='relu'), # best 1
41 layers.Dropout(0.1),
42 layers.Dense(64, activation='relu'), # best 1
43 layers.Dropout(0.1),
44 layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save("./")
50---------------------------------------------------------------------------
51KeyError Traceback (most recent call last)
52<ipython-input-84-87d3f09f8bee> in <module>()
53----> 1 model.save("./")
54
55
56/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in
57 error_handler(*args, **kwargs)
58 65 except Exception as e: # pylint: disable=broad-except
59 66 filtered_tb = _process_traceback_frames(e.__traceback__)
60 ---> 67 raise e.with_traceback(filtered_tb) from None
61 68 finally:
62 69 del filtered_tb
63
64 /usr/local/lib/python3.7/dist-
65 packages/tensorflow/python/saved_model/function_serialization.py in
66 serialize_concrete_function(concrete_function, node_ids, coder)
67 66 except KeyError:
68 67 raise KeyError(
69 ---> 68 f"Failed to add concrete function '{concrete_function.name}' to
70 object-"
71 69 f"based SavedModel as it captures tensor {capture!r} which is
72 unsupported"
73 70 " or not reachable from root. "
74
75 KeyError: "Failed to add concrete function
76 'b'__inference_sequential_46_layer_call_fn_662953'' to object-based SavedModel as it
77 captures tensor <tf.Tensor: shape=(), dtype=resource, value=<Resource Tensor>> which
78 is unsupported or not reachable from root. One reason could be that a stateful
79 object or a variable that the function depends on is not assigned to an attribute of
80 the serialized trackable object (see SaveTest.test_captures_unreachable_variable)."
81import tensorflow as tf
82import numpy as np
83
84class RandomColorDistortion(tf.keras.layers.Layer):
85 def __init__(self, contrast_range=[0.5, 1.5],
86 brightness_delta=[-0.2, 0.2], **kwargs):
87 super(RandomColorDistortion, self).__init__(**kwargs)
88 self.contrast_range = contrast_range
89 self.brightness_delta = brightness_delta
90
91 def call(self, images, training=None):
92 if not training:
93 return images
94 contrast = np.random.uniform(
95 self.contrast_range[0], self.contrast_range[1])
96 brightness = np.random.uniform(
97 self.brightness_delta[0], self.brightness_delta[1])
98
99 images = tf.image.adjust_contrast(images, contrast)
100 images = tf.image.adjust_brightness(images, brightness)
101 images = tf.clip_by_value(images, 0, 1)
102 return images
103
104 def get_config(self):
105 config = super(RandomColorDistortion, self).get_config()
106 config.update({"contrast_range": self.contrast_range, "brightness_delta": self.brightness_delta})
107 return config
108
109input_shape_rgb = (256, 256, 3)
110data_augmentation_rgb = tf.keras.Sequential(
111 [
112 tf.keras.layers.RandomFlip("horizontal"),
113 tf.keras.layers.RandomFlip("vertical"),
114 tf.keras.layers.RandomRotation(0.5),
115 tf.keras.layers.RandomZoom(0.5),
116 tf.keras.layers.RandomContrast(0.5),
117 RandomColorDistortion(name='random_contrast_brightness/none'),
118 ]
119)
120input_shape = (256, 256, 3)
121padding = 'same'
122kernel_size = 3
123model = tf.keras.Sequential([
124 tf.keras.layers.Input(input_shape),
125 data_augmentation_rgb,
126 tf.keras.layers.Rescaling((1./255)),
127 tf.keras.layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
128 data_format='channels_last'),
129 tf.keras.layers.MaxPooling2D(),
130 tf.keras.layers.BatchNormalization(),
131
132 tf.keras.layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
133 tf.keras.layers.MaxPooling2D(),
134 tf.keras.layers.BatchNormalization(),
135
136 tf.keras.layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
137 tf.keras.layers.MaxPooling2D(),
138 tf.keras.layers.BatchNormalization(),
139
140 tf.keras.layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
141 tf.keras.layers.MaxPooling2D(),
142 tf.keras.layers.BatchNormalization(),
143
144 tf.keras.layers.Flatten(),
145 tf.keras.layers.Dense(128, activation='relu'), # best 1
146 tf.keras.layers.Dropout(0.1),
147 tf.keras.layers.Dense(128, activation='relu'), # best 1
148 tf.keras.layers.Dropout(0.1),
149 tf.keras.layers.Dense(64, activation='relu'), # best 1
150 tf.keras.layers.Dropout(0.1),
151 tf.keras.layers.Dense(5, activation = 'softmax')
152 ])
153
154model.compile(loss='categorical_crossentropy', optimizer='adam')
155model.summary()
156model.save("test", save_format='h5')
157
Loading your model with your custom layer would look like this then:
1input_shape_rgb = (img_height, img_width, 3)
2data_augmentation_rgb = tf.keras.Sequential(
3 [
4 layers.RandomFlip("horizontal"),
5 layers.RandomFlip("vertical"),
6 layers.RandomRotation(0.5),
7 layers.RandomZoom(0.5),
8 layers.RandomContrast(0.5),
9 RandomColorDistortion(name='random_contrast_brightness/none'),
10 ]
11)
12# Build the model
13input_shape = (img_height, img_width, 3)
14
15model = Sequential([
16 layers.Input(input_shape),
17 data_augmentation_rgb,
18 layers.Rescaling((1./255)),
19
20 layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
21 data_format='channels_last'),
22 layers.MaxPooling2D(),
23 layers.BatchNormalization(),
24
25 layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
26 layers.MaxPooling2D(),
27 layers.BatchNormalization(),
28
29 layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
30 layers.MaxPooling2D(),
31 layers.BatchNormalization(),
32
33 layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
34 layers.MaxPooling2D(),
35 layers.BatchNormalization(),
36
37 layers.Flatten(),
38 layers.Dense(128, activation='relu'), # best 1
39 layers.Dropout(0.1),
40 layers.Dense(128, activation='relu'), # best 1
41 layers.Dropout(0.1),
42 layers.Dense(64, activation='relu'), # best 1
43 layers.Dropout(0.1),
44 layers.Dense(num_classes, activation = 'softmax')
45 ])
46
47 model.compile(loss='categorical_crossentropy', optimizer='adam',metrics=metrics)
48 model.summary()
49model.save("./")
50---------------------------------------------------------------------------
51KeyError Traceback (most recent call last)
52<ipython-input-84-87d3f09f8bee> in <module>()
53----> 1 model.save("./")
54
55
56/usr/local/lib/python3.7/dist-packages/keras/utils/traceback_utils.py in
57 error_handler(*args, **kwargs)
58 65 except Exception as e: # pylint: disable=broad-except
59 66 filtered_tb = _process_traceback_frames(e.__traceback__)
60 ---> 67 raise e.with_traceback(filtered_tb) from None
61 68 finally:
62 69 del filtered_tb
63
64 /usr/local/lib/python3.7/dist-
65 packages/tensorflow/python/saved_model/function_serialization.py in
66 serialize_concrete_function(concrete_function, node_ids, coder)
67 66 except KeyError:
68 67 raise KeyError(
69 ---> 68 f"Failed to add concrete function '{concrete_function.name}' to
70 object-"
71 69 f"based SavedModel as it captures tensor {capture!r} which is
72 unsupported"
73 70 " or not reachable from root. "
74
75 KeyError: "Failed to add concrete function
76 'b'__inference_sequential_46_layer_call_fn_662953'' to object-based SavedModel as it
77 captures tensor <tf.Tensor: shape=(), dtype=resource, value=<Resource Tensor>> which
78 is unsupported or not reachable from root. One reason could be that a stateful
79 object or a variable that the function depends on is not assigned to an attribute of
80 the serialized trackable object (see SaveTest.test_captures_unreachable_variable)."
81import tensorflow as tf
82import numpy as np
83
84class RandomColorDistortion(tf.keras.layers.Layer):
85 def __init__(self, contrast_range=[0.5, 1.5],
86 brightness_delta=[-0.2, 0.2], **kwargs):
87 super(RandomColorDistortion, self).__init__(**kwargs)
88 self.contrast_range = contrast_range
89 self.brightness_delta = brightness_delta
90
91 def call(self, images, training=None):
92 if not training:
93 return images
94 contrast = np.random.uniform(
95 self.contrast_range[0], self.contrast_range[1])
96 brightness = np.random.uniform(
97 self.brightness_delta[0], self.brightness_delta[1])
98
99 images = tf.image.adjust_contrast(images, contrast)
100 images = tf.image.adjust_brightness(images, brightness)
101 images = tf.clip_by_value(images, 0, 1)
102 return images
103
104 def get_config(self):
105 config = super(RandomColorDistortion, self).get_config()
106 config.update({"contrast_range": self.contrast_range, "brightness_delta": self.brightness_delta})
107 return config
108
109input_shape_rgb = (256, 256, 3)
110data_augmentation_rgb = tf.keras.Sequential(
111 [
112 tf.keras.layers.RandomFlip("horizontal"),
113 tf.keras.layers.RandomFlip("vertical"),
114 tf.keras.layers.RandomRotation(0.5),
115 tf.keras.layers.RandomZoom(0.5),
116 tf.keras.layers.RandomContrast(0.5),
117 RandomColorDistortion(name='random_contrast_brightness/none'),
118 ]
119)
120input_shape = (256, 256, 3)
121padding = 'same'
122kernel_size = 3
123model = tf.keras.Sequential([
124 tf.keras.layers.Input(input_shape),
125 data_augmentation_rgb,
126 tf.keras.layers.Rescaling((1./255)),
127 tf.keras.layers.Conv2D(16, kernel_size, padding=padding, activation='relu', strides=1,
128 data_format='channels_last'),
129 tf.keras.layers.MaxPooling2D(),
130 tf.keras.layers.BatchNormalization(),
131
132 tf.keras.layers.Conv2D(32, kernel_size, padding=padding, activation='relu'), # best 4
133 tf.keras.layers.MaxPooling2D(),
134 tf.keras.layers.BatchNormalization(),
135
136 tf.keras.layers.Conv2D(64, kernel_size, padding=padding, activation='relu'), # best 3
137 tf.keras.layers.MaxPooling2D(),
138 tf.keras.layers.BatchNormalization(),
139
140 tf.keras.layers.Conv2D(128, kernel_size, padding=padding, activation='relu'), # best 3
141 tf.keras.layers.MaxPooling2D(),
142 tf.keras.layers.BatchNormalization(),
143
144 tf.keras.layers.Flatten(),
145 tf.keras.layers.Dense(128, activation='relu'), # best 1
146 tf.keras.layers.Dropout(0.1),
147 tf.keras.layers.Dense(128, activation='relu'), # best 1
148 tf.keras.layers.Dropout(0.1),
149 tf.keras.layers.Dense(64, activation='relu'), # best 1
150 tf.keras.layers.Dropout(0.1),
151 tf.keras.layers.Dense(5, activation = 'softmax')
152 ])
153
154model.compile(loss='categorical_crossentropy', optimizer='adam')
155model.summary()
156model.save("test", save_format='h5')
157model = tf.keras.models.load_model('test.h5', custom_objects={'RandomColorDistortion': RandomColorDistortion})
158
where RandomColorDistortion
is the name of your custom layer.
QUESTION
React: hiding vs removing components
Asked 2022-Feb-03 at 09:53This question is about architecture more than coding.
Here's the case. In React sometimes we want to hide components. For example, when user opens new page in SPA, when some toast is closed, etc. We can hide them with adding display: none
. Or we can remove them from the virtual DOM.
1// Hidden div
2<div style={{ display: 'none' }}/>
3
4// Removed div
5{false && <div/>}
6
And some of our seniors othen use the first variant. Even if they hide entire page. That's what they say about this method: "This case React prerenders needed content, so when the content must appear, it takes less time".
But in this case we can't use lifecycle hooks, because even when component hides, it's not removed. But the main problem as I think, is that real DOM becomes gigantic. And that's brings slowness, isn't it?
So, what is better?
I didn't found any conversations about this question. Maybe you could help me.
EDIT 1: Despite the fact there are some answers I'd like to know more opinions. So, I decided to open up a bounty
ANSWER
Answered 2021-Sep-01 at 07:22Well if you want to use lifecycles there are workarounds for that as well. if you are using functional components then you can manage the rerenders using the dependency props.
Its true dom size can slow you down if you use it excessively https://web.dev/dom-size/ But is better if those components are constantly being updated rather then rendering a new component on demand.
If its a list of items and its gigantic i suggest you to take a look at https://react-window.vercel.app/#/examples/list/fixed-size or https://bvaughn.github.io/react-virtualized/#/components/List
QUESTION
M1 Mac - GDAL Wrong Architecture Error [Django]
Asked 2022-Jan-23 at 19:12I'm trying to get a django project up and running, which depends on GDAL library. I'm working on a M1 based mac.
Following the instructions on official Django docs, I've installed the necessary packages via brew
1$ brew install postgresql
2$ brew install postgis
3$ brew install gdal
4$ brew install libgeoip
5
gdalinfo --version
runs fine and shows the version as 3.3.1
gdal-config --libs
returns this path: -L/opt/homebrew/Cellar/gdal/3.3.1_2/lib -lgdal
a symlink is also placed on the homebrew's lib directory, which is in my path env variable.
When I try to run django without specifying the path to gdal library, it complains that it cannot find the GDAL package (even though the library is reachable, as a symlink to it is available through path env variable).
When I try to specify the path to the GDAL library using GDAL_LIBRARY_PATH
, I get this error:
1$ brew install postgresql
2$ brew install postgis
3$ brew install gdal
4$ brew install libgeoip
5OSError: dlopen(/opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib, 6): no suitable image found. Did find:
6 /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib: mach-o, but wrong architecture
7 /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.29.dylib: mach-o, but wrong architecture
8
P.s. I've already seen this answer, but it didn't help.
Isn't that strange when I try to run gdalinfo it runs fine but when django tries to run it throws me this error? What am I doing wrong?
ANSWER
Answered 2021-Nov-23 at 07:35Try using the new arm version of python!
1$ brew install postgresql
2$ brew install postgis
3$ brew install gdal
4$ brew install libgeoip
5OSError: dlopen(/opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib, 6): no suitable image found. Did find:
6 /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.dylib: mach-o, but wrong architecture
7 /opt/homebrew/Cellar/gdal/3.3.1_2/lib/libgdal.29.dylib: mach-o, but wrong architecture
8brew install --cask miniforge
9conda init zsh
10conda activate
11conda install numpy scipy scikit-learn
12
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Architecture
Tutorials and Learning Resources are not available at this moment for Architecture