Go is a static, compiled language created by Google. Go has advanced memory safety, garbage collection, structural typing, and CSP-style concurrency. Presently, Go is the favorite language of young developers.

Popular New Releases in Go

kubernetes

Kubernetes v1.24.0-rc.0

moby

v20.10.12

hugo

v0.97.3

gin

Release v1.7.7

frp

v0.41.0

Popular Libraries in Go

go

by golang doticongodoticon

star image 97672 doticonNOASSERTION

The Go programming language

kubernetes

by kubernetes doticongodoticon

star image 87661 doticonApache-2.0

Production-Grade Container Scheduling and Management

awesome-go

by avelino doticongodoticon

star image 76582 doticonMIT

A curated list of awesome Go frameworks, libraries and software

moby

by moby doticongodoticon

star image 62382 doticonApache-2.0

Moby Project - a collaborative project for the container ecosystem to assemble container-based systems

hugo

by gohugoio doticongodoticon

star image 58416 doticonApache-2.0

The world’s fastest framework for building websites.

gin

by gin-gonic doticongodoticon

star image 56154 doticonMIT

Gin is a HTTP web framework written in Go (Golang). It features a Martini-like API with much better performance -- up to 40 times faster. If you need smashing performance, get yourself some Gin.

frp

by fatedier doticongodoticon

star image 55464 doticonApache-2.0

A fast reverse proxy to help you expose a local server behind a NAT or firewall to the internet.

syncthing

by syncthing doticongodoticon

star image 44131 doticonMPL-2.0

Open Source Continuous File Synchronization

fzf

by junegunn doticongodoticon

star image 43649 doticonMIT

:cherry_blossom: A command-line fuzzy finder

Trending New libraries in Go

fiber

by gofiber doticongodoticon

star image 19407 doticonMIT

⚡️ Express inspired web framework written in Go

go-zero

by zeromicro doticongodoticon

star image 16273 doticonMIT

A web and RPC framework written in Go. It's born to ensure the stability of the busy sites with resilient design. Builtin goctl greatly improves the development productivity.

algorithm-pattern

by greyireland doticongodoticon

star image 12016 doticon

算法模板,最科学的刷题方式,最快速的刷题路径,你值得拥有~

go-zero

by tal-tech doticongodoticon

star image 10720 doticonMIT

go-zero is a web and rpc framework written in Go. It's born to ensure the stability of the busy sites with resilient design. Builtin goctl greatly improves the development productivity.

bubbletea

by charmbracelet doticongodoticon

star image 9726 doticonMIT

A powerful little TUI framework 🏗

duf

by muesli doticongodoticon

star image 8271 doticonNOASSERTION

Disk Usage/Free Utility - a better 'df' alternative

Xray-core

by XTLS doticongodoticon

star image 8099 doticonMPL-2.0

Xray, Penetrates Everything. Also the best v2ray-core, with XTLS support. Fully compatible configuration.

nuclei

by projectdiscovery doticongodoticon

star image 7947 doticonMIT

Fast and customizable vulnerability scanner based on simple YAML based DSL.

lima

by lima-vm doticongodoticon

star image 7768 doticonApache-2.0

Linux virtual machines, typically on macOS, for running containerd

Top Authors in Go

1

mattn

321 Libraries

star icon21651

2

NP-Go

316 Libraries

star icon0

3

hashicorp

268 Libraries

star icon183422

4

google

235 Libraries

star icon180804

5

openshift

233 Libraries

star icon18842

6

cloudfoundry

212 Libraries

star icon7560

7

unixpickle

198 Libraries

star icon2566

8

rohankumardubey

195 Libraries

star icon0

9

giantswarm

188 Libraries

star icon1559

10

michelia

159 Libraries

star icon13

1

321 Libraries

star icon21651

2

316 Libraries

star icon0

3

268 Libraries

star icon183422

4

235 Libraries

star icon180804

5

233 Libraries

star icon18842

6

212 Libraries

star icon7560

7

198 Libraries

star icon2566

8

195 Libraries

star icon0

9

188 Libraries

star icon1559

10

159 Libraries

star icon13

Trending Kits in Go

Web Proxy libraries are a way to access the contents of a website without actually accessing the website itself. Caddy is a simple and lightweight proxy server for the browser.

The web proxy acts as an intermediary between your computer and the website, fetching the content for you and displaying it on your screen. Caddy is a simple and lightweight proxy server for the browser. Betwixt is a high-performance proxy server that supports both HTTP and HTTPS protocols. It is written in JavaScript and provides a secure connection between your application and the client. Mockserver is a mock web server for testing HTTP requests in unit tests with Node.js and other evented I/O based servers such as Twisted and Gevent. Some of the most widely used open-source Web Proxy libraries among developers include

caddy:  

  • It is a modern, open-source web server with automatic HTTPS.  
  • It is popular for its simplicity and ease of use, making it a popular choice for developers.  
  • Its automatic HTTPS feature sets it apart, providing secure connections without manual configuration.

betwixt:  

  • It is a web debugging proxy tool that allows users to intercept, analyze, and change HTTP.  
  • It offers an interface for inspecting network requests and responses.  
  • It is often used for debugging web apps by capturing and examining HTTP traffic.

mockserver:  

  • It is a versatile tool for mocking and testing APIs. It enables developers to simulate server responses.  
  • It supports request matching, response templating, and powerful expectations for testing.  
  • It is particularly useful in scenarios where real API interactions are impractical.

lightproxy:  

  • It is a lightweight, cross-platform proxy tool designed for simplicity and performance.  
  • It supports HTTP interception and provides an interface for managing proxy settings.  
  • It is suitable for developers seeking a straightforward proxy solution without unnecessary complexities.

proxy.py:  

  • It is a pure Python proxy server that supports HTTP and HTTPS traffic interception.  
  • Its simplicity makes it easy to set up and use for various testing and debugging purposes.  
  • You can configure it to offer flexibility for customization.

shuttle:  

  • It is a transparent proxy server for macOS that intercepts and logs HTTP/HTTPS traffic.  
  • It simplifies the process of capturing network requests and responses for analysis.  
  • Developers and network administrators use it on macOS. It helps with debugging and monitoring purposes.

titanium-web-proxy:  

  • It is a .NET library for creating HTTP proxies with a focus on simplicity and flexibility.  
  • It allows developers to intercept and change HTTP requests and responses.  
  • It is suitable for various proxy-related tasks in .NET applications.

squid:  

  • It is a widely used open-source proxy server that supports HTTP, HTTPS, and FTP.  
  • It is popular for its caching capabilities, improving performance by storing accessed content.  
  • It is often deployed in large-scale environments to enhance web content delivery.

Titanium-Web-Proxy:  

  • It creates HTTP proxies with a focus on simplicity and flexibility.  
  • It allows developers to intercept and change HTTP requests and responses.  
  • It is suitable for various proxy-related tasks in .NET applications.  

miniproxy:  

  • It is a lightweight, simple HTTP proxy server written in Python.  
  • It helps with quick setup and usage. This makes it suitable for small-scale testing and debugging.  
  • It is a straightforward choice for scenarios where a minimalistic solution is enough. 

stealth:  

  • It is a command-line tool for creating simple HTTP proxies with a focus on stealthiness.  
  • It aims to be discreet and efficient, suitable for tasks that desire low-profile proxy.  
  • It is a minimalistic option for users who prefer command-line tools.

awslambdaproxy:  

  • It is a serverless proxy solution that leverages AWS Lambda for on-demand scalability.  
  • It allows users to create serverless proxy functions to handle HTTP requests.  
  • It is particularly useful for scenarios that prefer serverless architecture for operations.

FAQ  

1. What is a web proxy library?  

It is a set of functions and tools. It enables developers to install and manage web proxies in their applications. It allows intercepting, modifying, and controlling HTTP requests and responses.  

  

2. Why use a web proxy library?  

It helps in various purposes, such as debugging, security, and performance optimization. They enable developers to inspect and manipulate network traffic. It clears the network traffic between a client and a server.  

  

3. How does a web proxy work?  

A web proxy is an intermediary between a client and a server. It catches requests from the client and forwards them to the server. It will be the vice versa for responses. This interception allows for monitoring and modification of the traffic.  

  

4. What programming languages does the library support?  

Check the documentation to confirm which programming languages this library supports. Common languages include Python, Java, C#, and JavaScript.  

  

5. Is it possible to handle HTTPS traffic with the web proxy library?  

Many web proxy libraries support HTTPS traffic. This allows the interception and decryption of SSL/TLS-encrypted connections. Achieve this by generating and using certificates.

HTTP Security libraries allow you to set HTTP headers on your API requests that help make your app more secure. These headers include things like CORS and authentication tokens.

You can also use them to detect things like CSRF attacks. The Helmet module provides a handy utility that allows you to protect your Express apps from many common security problems. The Helmet module will automatically configure many of the common HTTP headers that are important for securing Express apps. Go-http-tunnel is a Go package that provides a middleware for transparently tunneling and/or proxying arbitrary TCP connections over HTTP. Go-http-tunnel is most commonly used to tunnel SSH connections, but can be used to create arbitrary tunnels between your network and the public internet. Many developers depend on the following open source HTTP Security libraries

helmet:  

  • Adds extra protection to websites by securing HTTP headers.  
  • Shields against common web vulnerabilities.  
  • Makes it easy to set up security-related HTTP headers.  

st2:  

  • Automates security tasks and coordinates with various security tools.  
  • Enables automatic responses to security incidents.  
  • Enhances overall security by streamlining processes.

hetty:  

  • Acts as a proxy for analyzing and securing HTTP/HTTPS traffic.  
  • Automatically detects and reports vulnerabilities.  
  • Provides a user-friendly web interface for interactive inspection.

Responder:  

  • Fast API framework for Python.  
  • Automatically validates and serializes data.  
  • Supports modular design through dependency injection. 

kore:  

  • Asynchronous web framework designed for efficient handling of concurrent connections.  
  • Built-in support for web technologies like HTTP/2 and WebSocket's.  
  • Facilitates high-performance web applications. 

go-http-tunnel:  

  • Provides secure and encrypted tunneling for HTTP traffic.  
  • Allows bypassing network restrictions for improved accessibility.  
  • Lightweight implementation in Go ensures efficiency.  

secure:  

  • Collection of utility functions for security-related tasks.  
  • Simplifies encryption, hashing, and secure password handling.  
  • Provides essential tools for maintaining a secure application. 

Meteor-Files:  

  • Meteor package designed for secure handling of files.  
  • Simplifies secure file uploads for web applications.  
  • Supports server-side file processing, enhancing flexibility 

FAQ 

1. Why should I use a Helmet in my web application?  

A helmet is essential for enhancing your web application’s security by 

  • automatically setting HTTP headers, 
  • mitigating common vulnerabilities and 
  • simplifying the implementation of security-related headers.  

   

2. What is st2, and how can it benefit my organization’s security practices?  

st2 is a powerful security automation and orchestration platform. It integrates with various security tools. This allows for automated incident response and improved overall security posture.  

   

3. How does Hetty contribute to web security analysis?  

Hetty serves as an HTTP/HTTPS proxy designed for security analysis. It offers automated vulnerability detection and a web-based interface for interactive inspection.  

   

4. What sets Responder apart as a Python API framework?  

Responder stands out with its fast performance, automatic data validation, and serialization. It also supports dependency injection, promoting a modular design for building robust APIs.  

   

5. Why consider using go-http-tunnel for HTTP traffic?  

go-http-tunnel provides secure and encrypted tunneling, enabling the bypassing of network restrictions. Its lightweight Go implementation ensures efficient, secure HTTP traffic handling. 

Here are some of the famous Go Telegram Bot Libraries. These Libraries are used for Building custom bots for customer service, Developing bots for e-commerce and Building bots for entertainment.


Go Telegram Bot Libraries are libraries written in the programming language Go which provide an interface to the Telegram Bot API. They allow developers to create and interact with Telegram bots in their own applications. They provide a range of features such as easy setup, custom commands, webhooks, and more.


Let us have a look at these libraries in detail below.

Telegram

  • Provides a highly customizable environment to create bots.
  • Provides a Botstore, which is a library of bots created by other developers.
  • Provides excellent security for bots, with end-to-end encryption and access control.

telegram-bot-api

  • Provides webhooks and inline keyboards for interaction with users.
  • Supports deep linking to external websites and content within the Telegram app.
  • Offers a unique set of features such as group chats, channels, and sticker sets.

TelegramBots

  • Offers advanced features such as game playing, natural language processing, and automated responses.
  • Supports webhooks which allow you to receive messages and updates from other services and systems.
  • Allows you to customize your bots with a variety of useful features such as message templates

Telebot

  • Offers secure bot creation and hosting with built-in features to protect user data.
  • Designed to deliver reliable performance even when handling large volumes of requests.
  • Offers advanced features such as inline keyboards, custom commands and user profiles.

telegram-bot-sdk

  • Supports multiple programming languages, such as JavaScript, Python, and Java.
  • Included in Telegram-bot-sdk is well-documented and easy to understand.
  • Enables developers to quickly create and customize powerful bots to fit their needs.

go-tgbot

  • Provides a powerful command line tool that helps developers to quickly create and manage their Telegram bots.
  • Supports all the standard Telegram API features and adds several unique features such as custom keyboards, inline bot queries and more.
  • Supports a wide range of databases, including MongoDB, Redis, BoltDB and more.

telegram-bot-go

  • Quickly build and deploy a Telegram bot with minimal effort.
  • Comes with comprehensive documentation and tutorials.
  • Offers advanced features such as custom commands, inline commands, and support for multiple languages.

gotelebot

  • Provides a convenient way to create custom commands for your bot.
  • Supports webhooks, allowing you to create more interactive bots quickly and easily.
  • Support for all the major Telegram Bot API features and extras.

Go Database libraries offer a generic SQL database interface. It supports databases like MySQL, SQLite, PostgreSQL, and many more. This library offers an effortless way to connect various databases and manage those connections. It supports the execution of SQL queries and prepared statements, allowing easy and secure access to databases.  


Go Database libraries offer a simple and intuitive way to manage database transactions, allowing the execution of different SQL statements as a single unit of work. This library offers detailed error handling, providing developers with more information on what went wrong in case of any error. This library supports prepared statements, which will help prevent SQL injection attacks and make it a secure way to interact with databases. It is a generic SQL database interface and supports multiple database drivers making it a flexible and versatile choice for building database-driven applications.  


Here is the list of the 13 best Go Database libraries that are handpicked to help developers: 

prometheus: 

  • Is designed for collecting metrics from various sources, storing, and querying them to provide insights into a system's health and performance.  
  • Offers a set of tools to build applications that exposes metrics in a format that prometheus can understand.  
  • Supports several metrics like gauges, summaries, counters, and histograms.  

tidb: 

  • Is an open source, MySQL-compatible, distributed RDBMS built for scalability and performance. 
  • Offers a way for executing SQL queries against TiDB databases. 
  • Offers a way to create, modify, and delete database schemas, indexes, and tables.  

cockroach: 

  • Is an open source, distributed SQL DBMS designed to provide reliable, scalable, and universally available transactional data storage.  
  • Offers a way for executing SQL queries against CockroachDB databases.  
  • Offers a simple and intuitive way to manage transactions in CockroachDB databases, allowing you to execute multiple SQL statements as a single unit of work. 

influxdb: 

  • Offers an easy-to-use interface to perform common database operations like creating and deleting databases, reading, and writing data, and managing users.  
  • Is available in various programming languages like Java, JavaScript, Go, and Python.  
  • Every library implementation offers a set of classes and functions which can be used for interacting with InfluxDB. 

dgraph: 

  • Is a distributed graph database that supports a flexible query and schema and is optimized for handling large-scale graph data.  
  • Offers an easy-to-use interface to perform common database operations like creating and deleting edges and nodes, reading, and writing data, and managing indexes. 
  • Includes writing data to the database, querying data from the database, and managing schema and indexes. 

milvus: 

  • Is an open source vector database designed for storing and searching large-scale vector data like videos, images, and audio.  
  • Supports similarity search, allowing users to search for similar vectors in a dataset.  
  • Is a software library that will allow developers to interact with Milvus from within their programs.  

sqlx: 

  • Is an open source library that offers an enhanced interface for working with SQL databases in Rust.  
  • Offers a simple and ergonomic API for interacting with databases, making it easier to write robust and efficient database code and reducing boilerplate code.  
  • Supports Rust’s async/await syntax, allowing non-blocking I/O operations and efficient use of system resources. 

teleport: 

  • Is an open source library that offers secure access to computing resources like containers, Kubernetes clusters, and servers.  
  • Offers a unified access layer that can be used for authorized and authenticated users, managing access to resources, and auditing user activity. 
  • Can be integrated into existing applications and supports different authentication like MFA and SSO.  

rqlite: 

  • Is an open source distributed SQL database designed for use in high-throughput and low-latency environments.  
  • Offers consistent performance, fault tolerance, and scalability by distributing SQL queries and data across different nodes in a cluster.  
  • Can make it easier for developers to build applications that use rqlite as their data store.  

immudb: 

  • Is a key-value database, open source transactional with built-in cryptographic verification.  
  • Offers tamper-evident storage, is immutable, and supports ACID transactions.  
  • Includes executing transactions that modify the database, configuring the cryptographic verification settings, and monitoring database health and performance.  

db: 

  • Is a productive data access layer for the Go programming language, which offers agnostic tools to work with various data sources.  
  • Provides tools for common operations with databases and stays out of the way with advanced cases.  
  • Offers a common interface for developers to work with different NoSQL and SQL database engines.  

cayley: 

  • Allows developers to store and query graph data using various query languages like SPARQL, Gremlin, and GraphQL.  
  • Is a software library that will allow developers to interact with Cayley from within their programs.  
  • Can make it easier for developers to build applications that use Cayley as their data store.  

vitess: 

  • Offers horizontal resharding, scaling, and failover capabilities for MySQL databases. 
  • Is a software library that will allow developers to interact with Vitess from within their programming.  
  • Developers can focus on building their application's logic and let the library handle the low-level details of communicating with the Vitess database cluster.  

Go Math Libraries offer various features, making working with mathematical computations and analysis easy. These libraries offer various mathematical functions which can be used for performing common calculations. These functions include logarithmic functions, trigonometric functions, and many more. 


The Go Math programming language is known for offering high performance, so the math libraries built with Go are often fast and efficient. It makes it ideal for larger datasets or when performing complex calculations. They also offer various data structures which can be used for storing and manipulating mathematical data like matrices, vectors, and different data types. It also offers tools for visualizing mathematical data like charts and graphs. These tools will help make complex mathematical concepts easy to understand. 


Here is the list of the top 6 Go Math Libraries handpicked to help developers perform mathematical computations and analysis easily and efficiently. 

gonum: 

  • Offers a package for performing linear algebra operations like vector operations, matrix operations, and solving linear systems of equations. 
  • Provides a package for optimization that includes constrained, unconstrained, and gradient-based optimization algorithms. 
  • Offers a package for working with graphs, like algorithms for shortest path finding, graph traversal, and many more.  

stats: 

  • Offers functions for calculating basic statistics like median, variance, standard deviation, mean, and mode. 
  • Offers functions for working with a probability distribution, like calculating probability density functions, generating random numbers from different distributions, and cumulative distribution functions. 
  • Offers functions for conducting hypothesis tests like chi-squared tests, ANOVA tests, and t-tests. 

gosl: 

  • Provides various numerical methods like numerical integration, optimization algorithms, and root finding. 
  • Provides a package for linear algebra operations like vector and matrix operations, eigenvalue and eigenvector calculations, linear systems solve, and more. 
  • Offers a package for solving differential equations, including partial differential equations and ordinary differential equations. 

mathgl: 

  • Provides various tools for creating 2D and 3D plots of mathematical data like line plots, scatter plots, contour plots, and more. 
  • Offers functions for visualizing vector fields, including vector fields and streamline plots. 
  • Offers a package for symbolic math operations like integration, simplification, and differentiation of mathematical equations.  

fixed: 

  • Offers functions for performing arithmetic operations like subtraction, addition, division, and multiplication. 
  • Provides functions for calculating trigonometric functions like cosine, sine, and tangent. 
  • Offers functions for calculating logarithmic and exponential functions like the base 2 and natural logarithm.  

gomatrix: 

  • Provides various matrix operations like matrix addition, subtraction, multiplication, division, and many more.  
  • Offers functions for linear algebra operations like matrix decomposition, eigenvector, and eigenvalue calculations, inversion, singular value decomposition, and many more.  
  • Offers functions for vector operations like cross and dot product and normalization.  

Facebook's Metaverse is vaporware as of today, but it has people talking and looking at new opportunities. Especially the developers: with virtual reality all set to raid the mainstream, application development that can support 3-dimensional experiences do display certain developmental complications and require experimentation. In fact, Statista reports that virtual reality is a market the size of $4.8 billion (2021). That being so, there is immense potential and a bright future for developers honing their skills in the niche.


Metaverse isn’t a single entity, as opposed to common belief: it is merely a “status” or a platform where people can gather, kind of like social media. In fact, tech-biggies like Microsoft and Nvidia have their own purpose-driven versions of metaverse platforms where users can log on and accomplish certain goals. Commercial biggie metaverses apart, there are quite a few open-source metaverses in the realm – like that of Mozilla i.e. Mozilla Hub.


Developers take note: virtual reality is soon coming into the mainstream, and the development of applications that can bridge the real and the virtual is going to be in demand. Open source holds quite a leverage in the development of metaverse components, being universal and readily implementable. Much like the 1900s saw a boom in two-dimensional internet, 2020 and onwards is the era of three-dimensional experiences in virtual reality. There is a high opportunity and potential for the developers here.

Docker is a platform for developing, shipping and running applications in containers. When you use Docker, you use different tools and libraries to make, control, and use containers.  


These libraries help with managing and developing containers. The libraries have tools for Docker. They have modular architecture and real-time monitoring. They also include workflow automation and curated cheat sheets.  


Here are the 14 essential Docker libraries:  

moby  

  • Moby has a modular architecture. Developers can pick and assemble only the components they need. It's for their specific use case.  
  • Moby offers essential Docker parts for users who want to make their own containers.  
  • Moby helps developers collaborate on container projects and experiment with containerization technologies.  

traefik   

  • Traefik can find and set up routes to backend services as they change in size.  
  • The system can handle different types of traffic like HTTP, TCP, and UDP.  
  • Traefik works well with container orchestration platforms such as Docker Swarm and Kubernetes. This makes it easier to handle ingress and load balancing for containerized applications.  

gogs  

  • Gogs is a lightweight Git service. It is self-hosted and has a simple, user-friendly interface for managing repositories.  
  • The software has tools for tracking issues and collaborating with others. It's an all-in-one solution for managing source code and issues.  
  • Users can customize Gogs to fit their needs and preferences.  

dive  

  • Dive is a tool for exploring container images layer by layer. It helps users understand and optimize image size and composition.  
  • Visual Diffs show changes between image layers, aiding troubleshooting and image optimization.  
  • Dive has an interactive CLI. It helps explore container images and analyze Docker images.  

docker_practice 

  • This library helps beginners learn Docker through hands-on exercises.  
  • The exercises begin with simple ideas, like making and using images. They become harder as the user gets better.  
  • The library has real-world examples of Docker, like deploying a web app and running CI.  

docker-development-youtube-series  

  • This library has video tutorials on Docker development. They cover Dockerfile basics, image management, and container orchestration.  
  • Each video tutorial has a code sample that users can use to follow.  
  • The library has a Discord server. Users can ask questions and get help there.  

docker-ce  

  • This is Docker distribution from Docker, Inc.  
  • The complete set of Docker features includes image building, container running, and orchestration.  
  • Docker tools and plugins widely support it.  

portainer  

  • This is a website where you can manage Docker containers and images.  
  • It has many features for managing Docker, like creating, managing, and monitoring containers.  
  • It is easy to use, even for users unfamiliar with Docker.  

Here are other Docker libraries with different programming languages.  

uptime-kuma  

  • Uptime-Kuma can check if services and protocols are available and performing well.  
  • Flexible Alerting helps users receive notifications when services encounter issues or become unavailable.  
  • Uptime-Kuma has a web interface that is easy to use. It lets you configure and see monitoring results. Many people can use it.  

awesome-cheatsheets  

  • The website Awesome-Cheatsheets has cheat sheets and references for programming languages and technologies.  
  • We encourage the community to contribute to keep the cheat sheets up to date and complete.  
  • Developers like the organized and user-friendly repository. It has helpful references and tips.  

n8n  

  • n8n is a tool for automating workflows. It lets users create and manage workflows visually. This makes it easy for both technical and non-technical users to use.  
  • Extensive Integration makes it easier to automate complex tasks with many pre-built integrations.  
  • Since n8n is open-source, you can host it on your server to manage your automation workflows and data.  

appwrite  

  • Appwrite is a platform for building web and mobile apps. It handles the backend. It provides many services, including authentication, storage, and databases.  
  • Appwrite is perfect for creating chat apps and document editors with real-time collaboration.  
  • It works on many platforms and programming languages so developers can use it for any app.

netdata  

  • Netdata is great for monitoring performance in real time. It has a dashboard to see system metrics and application performance.  
  • Users can customize their experience by adding plugins and integrations. These can monitor specific metrics and applications. This allows for greater flexibility and customization.  
  • Netdata has automatic anomaly detection. The system tells users when strange things happen and helps them find problems early.  

devops-exercises 

  • DevOps-Exercises offers exercises and resources to help people and teams practice DevOps principles.  
  • Gain practical experience with DevOps tools and methods through hands-on exercises.  
  • DevOps-Exercises values teamwork and welcomes contributions, which makes it valuable for DevOps enthusiasts.

Go is a programming language that makes it easy to build simple, reliable, and efficient software. Go is a programming language developed by Google in 2009. In recent years, it has gained popularity among developers due to its easy-to-learn and use syntax and the fact that it is an open source language. Go has been used for building server applications, websites, games and also mobile apps. There are many free Go coding assessment libraries available for developers to use. These libraries help in improving the performance of the application and make it more user friendly. Go can be used to create high performing applications that are easy to maintain. Some of the most popular Go Coding Assessment Open Source libraries among developers are: kube-scan - kubescan: Octarine k8s cluster risk assessment tool; recursebuster - rapid content discovery tool for recursively querying webservers; paf-credentials-checker - high performing offline tool to easily assess.

Trending Discussions on Go

Why is Rust NLL not working for multiple borrows in the same statement?

Docker push to AWS ECR hangs immediately and times out

How does Java know which overloaded method to call with lambda expressions? (Supplier, Consumer, Callable, ...)

pip-compile raising AssertionError on its logging handler

Is if(A | B) always faster than if(A || B)?

What is the proper evaluation order when assigning a value in a map?

Unable to specify `edition2021` in order to use unstable packages in Rust

Android Studio strange code sub-windows after upgrade to Arctic Fox (2020.3.1)

ellipsis ... as function in substitute?

Python 3.10 pattern matching (PEP 634) - wildcard in string

QUESTION

Why is Rust NLL not working for multiple borrows in the same statement?

Asked 2022-Apr-12 at 00:43

First, I tried something like this:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3

It can't be compiled because:

error[E0502]: cannot borrow `vec` as immutable because it is also borrowed as mutable

I thought that the Rust borrow checker could be smarter than this, so I found something called NLL, and it should solve this problem.

I tried the sample:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5

It could work, but why is it not working with rotate_right? Both of them take a &mut self. What's going on?

ANSWER

Answered 2022-Apr-12 at 00:43

It is definitely an interesting one.

They are similar - but not quite the same. resize() is a member of Vec. rotate_right(), on the other hand, is a method of slices.

Vec<T> derefs to [T], so most of the time this does not matter. But actually, while this call:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6

Desugars to something like:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6&lt;Vec&lt;i32&gt;&gt;::resize(&amp;mut vec, &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec), 0);
7

This call:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6&lt;Vec&lt;i32&gt;&gt;::resize(&amp;mut vec, &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec), 0);
7vec.rotate_right(vec.len());
8

Is more like:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6&lt;Vec&lt;i32&gt;&gt;::resize(&amp;mut vec, &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec), 0);
7vec.rotate_right(vec.len());
8&lt;[i32]&gt;::rotate_right(
9    &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(&amp;mut vec),
10    &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec),
11);
12

But in what order?

This is the MIR for rotate_right() (simplified a lot):

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6&lt;Vec&lt;i32&gt;&gt;::resize(&amp;mut vec, &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec), 0);
7vec.rotate_right(vec.len());
8&lt;[i32]&gt;::rotate_right(
9    &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(&amp;mut vec),
10    &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec),
11);
12fn foo() -&gt; () {
13    _4 = &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(move _5);
14    _6 = Vec::&lt;i32&gt;::len(move _7);
15    _2 = core::slice::&lt;impl [i32]&gt;::rotate_right(move _3, move _6);
16}
17

And this is the MIR for resize() (again, simplified a lot):

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6&lt;Vec&lt;i32&gt;&gt;::resize(&amp;mut vec, &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec), 0);
7vec.rotate_right(vec.len());
8&lt;[i32]&gt;::rotate_right(
9    &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(&amp;mut vec),
10    &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec),
11);
12fn foo() -&gt; () {
13    _4 = &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(move _5);
14    _6 = Vec::&lt;i32&gt;::len(move _7);
15    _2 = core::slice::&lt;impl [i32]&gt;::rotate_right(move _3, move _6);
16}
17fn foo() -&gt; () {
18    _4 = Vec::&lt;i32&gt;::len(move _5);
19    _2 = Vec::&lt;i32&gt;::resize(move _3, move _4, const 0_i32);
20}
21

In the resize() example, we first call Vec::len() with a reference to vec. This returns usize. Then we call Vec::resize(), when we have no outstanding references to vec, so mutably borrowing it is fine!

However, with rotate_right(), first we call <Vec<i32> as DerefMut>::deref_mut(&mut vec). This returns &mut [i32], with its lifetime tied to vec. That is, as long as this reference (mutable reference!) is alive, we are not allowed to use have any other reference to vec. But then we try to borrow vec in order to pass the (shared, but it doesn't matter) reference to Vec::len(), while we still need to use the mutable reference from deref_mut() later, in the call to <[i32]>::rotate_right()! This is an error.

This is because Rust defines an evaluation order for operands:

Expressions taking multiple operands are evaluated left to right as written in the source code.

Because vec.resize() is actually (&mut *vec).rotate_right(), we first evaluate the dereference+reference, then the arguments:

1let mut vec = vec![0];
2vec.rotate_right(vec.len());
3let mut vec = vec![0];
4vec.resize(vec.len(), 0);
5vec.resize(vec.len(), 0);
6&lt;Vec&lt;i32&gt;&gt;::resize(&amp;mut vec, &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec), 0);
7vec.rotate_right(vec.len());
8&lt;[i32]&gt;::rotate_right(
9    &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(&amp;mut vec),
10    &lt;Vec&lt;i32&gt;&gt;::len(&amp;vec),
11);
12fn foo() -&gt; () {
13    _4 = &lt;Vec&lt;i32&gt; as DerefMut&gt;::deref_mut(move _5);
14    _6 = Vec::&lt;i32&gt;::len(move _7);
15    _2 = core::slice::&lt;impl [i32]&gt;::rotate_right(move _3, move _6);
16}
17fn foo() -&gt; () {
18    _4 = Vec::&lt;i32&gt;::len(move _5);
19    _2 = Vec::&lt;i32&gt;::resize(move _3, move _4, const 0_i32);
20}
21let dereferenced_vec = &amp;mut *vec;
22let len = vec.len();
23dereferencec_vec.rotate_right(len);
24

Which is obviously a violation of the borrow rules.

On the other hand, vec.resize(vec.len()) has no work to do on the callee (vec), and so we first evaluate vec.len(), and then the call itself.

Solving this is as easy as extracting the vec.len() to a new line (new statement, to be precise), and the compiler also suggests that.

Source https://stackoverflow.com/questions/71823410

QUESTION

Docker push to AWS ECR hangs immediately and times out

Asked 2022-Mar-30 at 07:53

I'm trying to push my first docker image to ECR. I've followed the steps provided by AWS and things seem to be going smoothly until the final push which immediately times out. Specifically, I pass my aws ecr credentials to docker and get a "login succeeded" message. I then tag the image which also works. pushing to the ecr repo I get no error message, just the following:

1The push refers to repository [xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame]
2714c1b96dd83: Retrying in 1 second 
3d2cdc77dd068: Retrying in 1 second 
430aad807caf5: Retrying in 1 second 
50559774c4ea2: Retrying in 1 second 
6285b8616682f: Retrying in 1 second 
74aeea0ec2b15: Waiting 
81b1312f842d8: Waiting 
9c310009e0ef3: Waiting 
10a48777e566d3: Waiting 
112a0c9f28029a: Waiting 
12EOF
13

It tries a bunch of times and then exits with no message. Any idea what's wrong?

ANSWER

Answered 2022-Jan-02 at 14:23

I figured out my issue. I wasn't using the correct credentials. I had a personal AWS account as my default credentials and needed to add my work profile to my credentials.

EDIT
If you have multiple aws profiles, you can mention the profile name at the docker login as below (assuming you have done aws configure --profile someprofile at earlier day),

1The push refers to repository [xxxxxxxxxxx.dkr.ecr.ca-central-1.amazonaws.com/reponame]
2714c1b96dd83: Retrying in 1 second 
3d2cdc77dd068: Retrying in 1 second 
430aad807caf5: Retrying in 1 second 
50559774c4ea2: Retrying in 1 second 
6285b8616682f: Retrying in 1 second 
74aeea0ec2b15: Waiting 
81b1312f842d8: Waiting 
9c310009e0ef3: Waiting 
10a48777e566d3: Waiting 
112a0c9f28029a: Waiting 
12EOF
13aws ecr get-login-password --region us-east-1 --profile someprofile | docker login ....
14

Source https://stackoverflow.com/questions/70452836

QUESTION

How does Java know which overloaded method to call with lambda expressions? (Supplier, Consumer, Callable, ...)

Asked 2022-Mar-17 at 08:29

First off, I have no idea how to decently phrase the question, so this is up for suggestions.

Lets say we have following overloaded methods:

1void execute(Callable&lt;Void&gt; callable) {
2    try {
3        callable.call();
4    } catch (Exception e) {
5        e.printStackTrace();
6    }
7}
8
9&lt;T&gt; T execute(Supplier&lt;T&gt; supplier) {
10    return supplier.get();
11}
12
13void execute(Runnable runnable) {
14    runnable.run();
15}
16

Going off from this table, I got from another SO question

1void execute(Callable&lt;Void&gt; callable) {
2    try {
3        callable.call();
4    } catch (Exception e) {
5        e.printStackTrace();
6    }
7}
8
9&lt;T&gt; T execute(Supplier&lt;T&gt; supplier) {
10    return supplier.get();
11}
12
13void execute(Runnable runnable) {
14    runnable.run();
15}
16Supplier       ()    -&gt; x
17Consumer       x     -&gt; ()
18BiConsumer     x, y  -&gt; ()
19Callable       ()    -&gt; x throws ex
20Runnable       ()    -&gt; ()
21Function       x     -&gt; y
22BiFunction     x,y   -&gt; z
23Predicate      x     -&gt; boolean
24UnaryOperator  x1    -&gt; x2
25BinaryOperator x1,x2 -&gt; x3
26

These are the results I get locally:

1void execute(Callable&lt;Void&gt; callable) {
2    try {
3        callable.call();
4    } catch (Exception e) {
5        e.printStackTrace();
6    }
7}
8
9&lt;T&gt; T execute(Supplier&lt;T&gt; supplier) {
10    return supplier.get();
11}
12
13void execute(Runnable runnable) {
14    runnable.run();
15}
16Supplier       ()    -&gt; x
17Consumer       x     -&gt; ()
18BiConsumer     x, y  -&gt; ()
19Callable       ()    -&gt; x throws ex
20Runnable       ()    -&gt; ()
21Function       x     -&gt; y
22BiFunction     x,y   -&gt; z
23Predicate      x     -&gt; boolean
24UnaryOperator  x1    -&gt; x2
25BinaryOperator x1,x2 -&gt; x3
26// Runnable -&gt; expected as this is a plain void  
27execute(() -&gt; System.out.println()); 
28
29// Callable -&gt; why is it not a Supplier? It does not throw any exceptions..
30execute(() -&gt; null);
31
32// Supplier -&gt; this returns an Object, but how is that different from returning null?
33execute(() -&gt; new Object());
34
35// Callable -&gt; because it can throw an exception, right?
36execute(() -&gt; {throw new Exception();});
37

How does the compiler know which method to call? How does it for example make the distinction between what's a Callable and what's a Runnable?

ANSWER

Answered 2022-Mar-17 at 08:29

It all makes sense and has a simple pattern besides () -> null being a Callable I think. The Runnable is clearly different from the Supplier/Callable as it has no input and output values. The difference between Callable and Supplier is that with the Callable you have to handle exceptions.

The reason that () -> null is a Callable without an exception is the return type of your definition Callable<Void>. It requires you to return the reference to some object. The only possible reference to return for Void is null. This means that the lambda () -> null is exactly what your definition demands. It would also work for your Supplier example if you would remove the Callable definition. However, it uses Callable<Void> over Supplier<T> as the Callable has the exact type.

Callable is chosen over Supplier as it is more specific (as a comment already suggested). The Java Docs state that it chooses the most specific type if possible:

Type inference is a Java compiler's ability to look at each method invocation and corresponding declaration to determine the type argument (or arguments) that make the invocation applicable. The inference algorithm determines the types of the arguments and, if available, the type that the result is being assigned, or returned. Finally, the inference algorithm tries to find the most specific type that works with all of the arguments.

Source https://stackoverflow.com/questions/71494924

QUESTION

pip-compile raising AssertionError on its logging handler

Asked 2022-Feb-13 at 12:37

I have a dockerfile that currently only installs pip-tools

1FROM python:3.9
2
3RUN pip install --upgrade pip &amp;&amp; \
4    pip install pip-tools
5
6COPY ./ /root/project
7
8WORKDIR /root/project
9
10ENTRYPOINT [&quot;tail&quot;, &quot;-f&quot;, &quot;/dev/null&quot;]
11

I build and open a shell in the container using the following commands:

1FROM python:3.9
2
3RUN pip install --upgrade pip &amp;&amp; \
4    pip install pip-tools
5
6COPY ./ /root/project
7
8WORKDIR /root/project
9
10ENTRYPOINT [&quot;tail&quot;, &quot;-f&quot;, &quot;/dev/null&quot;]
11docker build -t brunoapi_image .
12docker run --rm -ti --name brunoapi_container --entrypoint bash brunoapi_image
13

Then, when I try to run pip-compile inside the container I get this very weird error (full traceback):

1FROM python:3.9
2
3RUN pip install --upgrade pip &amp;&amp; \
4    pip install pip-tools
5
6COPY ./ /root/project
7
8WORKDIR /root/project
9
10ENTRYPOINT [&quot;tail&quot;, &quot;-f&quot;, &quot;/dev/null&quot;]
11docker build -t brunoapi_image .
12docker run --rm -ti --name brunoapi_container --entrypoint bash brunoapi_image
13root@727f1f38f095:~/project# pip-compile
14Traceback (most recent call last):
15  File &quot;/usr/local/bin/pip-compile&quot;, line 8, in &lt;module&gt;
16    sys.exit(cli())
17  File &quot;/usr/local/lib/python3.9/site-packages/click/core.py&quot;, line 1128, in __call__
18    return self.main(*args, **kwargs)
19  File &quot;/usr/local/lib/python3.9/site-packages/click/core.py&quot;, line 1053, in main
20    rv = self.invoke(ctx)
21  File &quot;/usr/local/lib/python3.9/site-packages/click/core.py&quot;, line 1395, in invoke
22    return ctx.invoke(self.callback, **ctx.params)
23  File &quot;/usr/local/lib/python3.9/site-packages/click/core.py&quot;, line 754, in invoke
24    return __callback(*args, **kwargs)
25  File &quot;/usr/local/lib/python3.9/site-packages/click/decorators.py&quot;, line 26, in new_func
26    return f(get_current_context(), *args, **kwargs)
27  File &quot;/usr/local/lib/python3.9/site-packages/piptools/scripts/compile.py&quot;, line 342, in cli
28    repository = PyPIRepository(pip_args, cache_dir=cache_dir)
29  File &quot;/usr/local/lib/python3.9/site-packages/piptools/repositories/pypi.py&quot;, line 106, in __init__
30    self._setup_logging()
31  File &quot;/usr/local/lib/python3.9/site-packages/piptools/repositories/pypi.py&quot;, line 455, in _setup_logging
32    assert isinstance(handler, logging.StreamHandler)
33AssertionError
34

I have no clue what's going on and I've never seen this error before. Can anyone shed some light into this?

Running on macOS Monterey

ANSWER

Answered 2022-Feb-05 at 16:30

It is a bug, you can downgrade using:

pip install "pip<22"

https://github.com/jazzband/pip-tools/issues/1558

Source https://stackoverflow.com/questions/70946286

QUESTION

Is if(A | B) always faster than if(A || B)?

Asked 2022-Feb-11 at 05:03

I am reading this book by Fedor Pikus and he has some very very interesting examples which for me were a surprise.
Particularly this benchmark caught me, where the only difference is that in one of them we use || in if and in another we use |.

1void BM_misspredict(benchmark::State&amp; state)
2{
3
4    std::srand(1);
5    const unsigned int N = 10000;;
6    std::vector&lt;unsigned long&gt; v1(N), v2(N);
7    std::vector&lt;int&gt; c1(N), c2(N);
8
9    for (int i = 0; i &lt; N; ++i) 
10    {
11        v1[i] = rand();
12        v2[i] = rand();
13        c1[i] = rand() &amp; 0x1;
14        c2[i] = !c1[i];
15    }
16
17    unsigned long* p1 = v1.data();
18    unsigned long* p2 = v2.data();
19    int* b1 = c1.data();
20    int* b2 = c2.data();
21
22    for (auto _ : state)
23    {
24        unsigned long a1 = 0, a2 = 0;
25        for (size_t i = 0; i &lt; N; ++i) 
26        {
27            if (b1[i] || b2[i])  // Only difference
28            {
29                a1 += p1[i];
30            }
31            else 
32            {
33                a2 *= p2[i];
34            }
35        }
36        benchmark::DoNotOptimize(a1);
37        benchmark::DoNotOptimize(a2);
38        benchmark::ClobberMemory();
39
40    }
41    state.SetItemsProcessed(state.iterations());
42}
43
44void BM_predict(benchmark::State&amp; state)
45{
46
47    std::srand(1);
48    const unsigned int N = 10000;;
49    std::vector&lt;unsigned long&gt; v1(N), v2(N);
50    std::vector&lt;int&gt; c1(N), c2(N);
51
52    for (int i = 0; i &lt; N; ++i)
53    {
54        v1[i] = rand();
55        v2[i] = rand();
56        c1[i] = rand() &amp; 0x1;
57        c2[i] = !c1[i];
58    }
59
60    unsigned long* p1 = v1.data();
61    unsigned long* p2 = v2.data();
62    int* b1 = c1.data();
63    int* b2 = c2.data();
64
65    for (auto _ : state)
66    {
67        unsigned long a1 = 0, a2 = 0;
68        for (size_t i = 0; i &lt; N; ++i)
69        {
70            if (b1[i] | b2[i]) // Only difference
71            {
72                a1 += p1[i];
73            }
74            else
75            {
76                a2 *= p2[i];
77            }
78        }
79        benchmark::DoNotOptimize(a1);
80        benchmark::DoNotOptimize(a2);
81        benchmark::ClobberMemory();
82
83    }
84    state.SetItemsProcessed(state.iterations());
85}
86

I will not go in all the details explained in the book why the latter is faster, but the idea is that hardware branch predictor is given 2 chances to misspredict in slower version and in the | (bitwise or) version. See the benchmark results below.

enter image description here

So the question is why don't we always use | instead of || in branches?

ANSWER

Answered 2022-Feb-08 at 19:57

Code readability, short-circuiting and it is not guaranteed that Ord will always outperform a || operand. Computer systems are more complicated than expected, even though they are man-made.

There was a case where a for loop with a much more complicated condition ran faster on an IBM. The CPU didn't cool and thus instructions were executed faster, that was a possible reason. What I am trying to say, focus on other areas to improve code than fighting small-cases which will differ depending on the CPU and the boolean evaluation (compiler optimizations).

Source https://stackoverflow.com/questions/71039947

QUESTION

What is the proper evaluation order when assigning a value in a map?

Asked 2022-Feb-02 at 09:25

I know that compiler is usually the last thing to blame for bugs in a code, but I do not see any other explanation for the following behaviour of the following C++ code (distilled down from an actual project):

1#include &lt;iostream&gt;
2#include &lt;map&gt;
3
4int main()
5{
6    auto values = { 1, 3, 5 };
7    std::map&lt;int, int&gt; valMap;
8
9    for (auto const &amp; val : values) {
10        std::cout &lt;&lt; &quot;before assignment: valMap.size() = &quot; &lt;&lt; valMap.size();
11        valMap[val] = valMap.size();
12        std::cout &lt;&lt; &quot; -&gt; set valMap[&quot; &lt;&lt; val &lt;&lt; &quot;] to &quot; &lt;&lt; valMap[val] &lt;&lt; &quot;\n&quot;;
13    }
14}
15

The expected output of this code is:

1#include &lt;iostream&gt;
2#include &lt;map&gt;
3
4int main()
5{
6    auto values = { 1, 3, 5 };
7    std::map&lt;int, int&gt; valMap;
8
9    for (auto const &amp; val : values) {
10        std::cout &lt;&lt; &quot;before assignment: valMap.size() = &quot; &lt;&lt; valMap.size();
11        valMap[val] = valMap.size();
12        std::cout &lt;&lt; &quot; -&gt; set valMap[&quot; &lt;&lt; val &lt;&lt; &quot;] to &quot; &lt;&lt; valMap[val] &lt;&lt; &quot;\n&quot;;
13    }
14}
15before assignment: valMap.size() = 0 -&gt; set valMap[1] to 0
16before assignment: valMap.size() = 1 -&gt; set valMap[3] to 1
17before assignment: valMap.size() = 2 -&gt; set valMap[5] to 2
18

However, when I build a Release version with the (default) C++14 compiler, the output becomes:

1#include &lt;iostream&gt;
2#include &lt;map&gt;
3
4int main()
5{
6    auto values = { 1, 3, 5 };
7    std::map&lt;int, int&gt; valMap;
8
9    for (auto const &amp; val : values) {
10        std::cout &lt;&lt; &quot;before assignment: valMap.size() = &quot; &lt;&lt; valMap.size();
11        valMap[val] = valMap.size();
12        std::cout &lt;&lt; &quot; -&gt; set valMap[&quot; &lt;&lt; val &lt;&lt; &quot;] to &quot; &lt;&lt; valMap[val] &lt;&lt; &quot;\n&quot;;
13    }
14}
15before assignment: valMap.size() = 0 -&gt; set valMap[1] to 0
16before assignment: valMap.size() = 1 -&gt; set valMap[3] to 1
17before assignment: valMap.size() = 2 -&gt; set valMap[5] to 2
18before assignment: valMap.size() = 0 -&gt; set valMap[1] to 1
19before assignment: valMap.size() = 1 -&gt; set valMap[3] to 2
20before assignment: valMap.size() = 2 -&gt; set valMap[5] to 3
21

In other words, all values in valMap are larger by 1 than what they should be - it looks like the map gets appended before the right-hand-side of the assignment is evaluated.

This happens only in a Release build with C++14 language standard (which is the default in VS2019). Debug builds work fine (I hate when this happens - it took me hours to find out what is going on), as do Release builds of C++17 and C++20. This is why it looks like a bug to me.

My question is: is this a compiler bug, or am I doing something wrong/dangerous by using .size() in the assignment?

ANSWER

Answered 2022-Feb-01 at 15:49

The evaluation order of A = B was not specified before c++17, after c++17 B is guaranteed to be evaluated before A, see https://en.cppreference.com/w/cpp/language/eval_order rule 20.

The behaviour of valMap[val] = valMap.size(); is therefore unspecified in c++14, you should use:

1#include &lt;iostream&gt;
2#include &lt;map&gt;
3
4int main()
5{
6    auto values = { 1, 3, 5 };
7    std::map&lt;int, int&gt; valMap;
8
9    for (auto const &amp; val : values) {
10        std::cout &lt;&lt; &quot;before assignment: valMap.size() = &quot; &lt;&lt; valMap.size();
11        valMap[val] = valMap.size();
12        std::cout &lt;&lt; &quot; -&gt; set valMap[&quot; &lt;&lt; val &lt;&lt; &quot;] to &quot; &lt;&lt; valMap[val] &lt;&lt; &quot;\n&quot;;
13    }
14}
15before assignment: valMap.size() = 0 -&gt; set valMap[1] to 0
16before assignment: valMap.size() = 1 -&gt; set valMap[3] to 1
17before assignment: valMap.size() = 2 -&gt; set valMap[5] to 2
18before assignment: valMap.size() = 0 -&gt; set valMap[1] to 1
19before assignment: valMap.size() = 1 -&gt; set valMap[3] to 2
20before assignment: valMap.size() = 2 -&gt; set valMap[5] to 3
21auto size = valMap.size();
22valMap[val] = size;
23

Or avoid the problem by using emplace which is more explicit than relying on [] to automatically insert a value if it doesn't already exist:

1#include &lt;iostream&gt;
2#include &lt;map&gt;
3
4int main()
5{
6    auto values = { 1, 3, 5 };
7    std::map&lt;int, int&gt; valMap;
8
9    for (auto const &amp; val : values) {
10        std::cout &lt;&lt; &quot;before assignment: valMap.size() = &quot; &lt;&lt; valMap.size();
11        valMap[val] = valMap.size();
12        std::cout &lt;&lt; &quot; -&gt; set valMap[&quot; &lt;&lt; val &lt;&lt; &quot;] to &quot; &lt;&lt; valMap[val] &lt;&lt; &quot;\n&quot;;
13    }
14}
15before assignment: valMap.size() = 0 -&gt; set valMap[1] to 0
16before assignment: valMap.size() = 1 -&gt; set valMap[3] to 1
17before assignment: valMap.size() = 2 -&gt; set valMap[5] to 2
18before assignment: valMap.size() = 0 -&gt; set valMap[1] to 1
19before assignment: valMap.size() = 1 -&gt; set valMap[3] to 2
20before assignment: valMap.size() = 2 -&gt; set valMap[5] to 3
21auto size = valMap.size();
22valMap[val] = size;
23valMap.emplace(val, size);
24

Source https://stackoverflow.com/questions/70943170

QUESTION

Unable to specify `edition2021` in order to use unstable packages in Rust

Asked 2022-Feb-02 at 07:05

I want to run an example via Cargo but I am facing an error:

1error: failed to parse manifest at `/Users/aviralsrivastava/dev/subxt/Cargo.toml`
2

The full stacktrace is:

1error: failed to parse manifest at `/Users/aviralsrivastava/dev/subxt/Cargo.toml`
2error: failed to parse manifest at `/Users/aviralsrivastava/dev/subxt/Cargo.toml`
3
4Caused by:
5  feature `edition2021` is required
6
7  The package requires the Cargo feature called `edition2021`, but that feature is not stabilized in this version of Cargo (1.56.0-nightly (b51439fd8 2021-08-09)).
8  Consider adding `cargo-features = [&quot;edition2021&quot;]` to the top of Cargo.toml (above the [package] table) to tell Cargo you are opting in to use this unstable feature.
9  See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#edition-2021 for more information about the status of this feature.
10

Based on the suggestion, I go ahead and modify the Cargo.toml:

1error: failed to parse manifest at `/Users/aviralsrivastava/dev/subxt/Cargo.toml`
2error: failed to parse manifest at `/Users/aviralsrivastava/dev/subxt/Cargo.toml`
3
4Caused by:
5  feature `edition2021` is required
6
7  The package requires the Cargo feature called `edition2021`, but that feature is not stabilized in this version of Cargo (1.56.0-nightly (b51439fd8 2021-08-09)).
8  Consider adding `cargo-features = [&quot;edition2021&quot;]` to the top of Cargo.toml (above the [package] table) to tell Cargo you are opting in to use this unstable feature.
9  See https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#edition-2021 for more information about the status of this feature.
10  Consider adding `cargo-features = [&quot;edition2021&quot;]` to the top of Cargo.toml (above the [package] table) to tell Cargo you are opting in to use this unstable feature.
11diff --git a/Cargo.toml b/Cargo.toml
12index 26a02c7..186d09b 100644
13--- a/Cargo.toml
14+++ b/Cargo.toml
15@@ -1,6 +1,6 @@
16 [workspace]
17 members = [&quot;.&quot;, &quot;cli&quot;, &quot;codegen&quot;, &quot;macro&quot;]
18-
19+cargo-features = [&quot;edition2021&quot;]
20 [package]
21 name = &quot;subxt&quot;
22 version = &quot;0.15.0&quot;
23(END)
24

I still face the same error as if nothing was changed in the toml file.

How do I resolve the above error in order to use unstable packages?

ANSWER

Answered 2021-Dec-14 at 14:09

Update the Rust to satisfy the new edition 2021.

rustup default nightly && rustup update

Thanks to @ken. Yes, you can use the stable channel too!

But I love nightly personally.

Source https://stackoverflow.com/questions/69848319

QUESTION

Android Studio strange code sub-windows after upgrade to Arctic Fox (2020.3.1)

Asked 2022-Jan-14 at 09:18

After Android Studio upgraded itself to version Arctic Fox, I now get these strange sub-windows in my code editor that I can't get rid of. If I click in either of the 2 sub-windows (a one-line window at the top or a 5-line window underneath it (see pic below), it scrolls to the code in question and the sub-windows disappear. But as soon as I navigate away from that code, these sub-windows mysteriously reappear. I can't figure out how to get rid of this.

I restarted Studio and it seemed to go away. Then I refactored a piece of code (Extract to Method Ctrl+Alt+M) and then these windows appeared again. Sometimes these windows appear on a 2nd monitor instead of on top of the code area on the monitor with Android Studio. But eventually they end up back on top of my code editor window.

I have searched hi and low for what this is. Studio help, new features, blog, etc. I am sure that I am just using the wrong terminology to find the answer, so hoping someone else knows.

enter image description here

ANSWER

Answered 2021-Aug-15 at 15:29

Just stumbled upon the same thing (strange windows upon attempting to refactor some code after updating to Arctic Fox). After a lot of searching around the options/menus/internet this fixed it for me:

Navigate to:

File > Settings... > Editor > Code Editing

under

Refactorings > Specify refactoring options:

select

In modal dialogs

Press OK.

Fingers crossed refactoring works.

🤞

Further step: Restart Android Studio

Source https://stackoverflow.com/questions/68682622

QUESTION

ellipsis ... as function in substitute?

Asked 2022-Jan-13 at 06:04

I'm having trouble understanding how/why parentheses work where they otherwise should not work®.

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9

Normally an error is thrown, could not find function "..." or '...' used in an incorrect context, for example when calling (\(...) ...())(5).

What I've tried
I have looked at the source code of substitute to find out why this doesn't happen here. R Internals 1.1.1 and 1.5.2 says ... is of SEXPTYPE DOTSXP, a pairlist of promises. These promises are what is extracted by substitute.

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13

Going line-by-line, I am stuck at substituteList, in which h is the current element of ... being processed. This happens recursively at line 2832 if (TYPEOF(h) == DOTSXP) h = substituteList(h, R_NilValue);. I haven't found exception handling of a ...() case in the source code, so I suspect something before this has happened.

In ?substitute we find substitute works on a purely lexical basis. Does it mean ...() is a parser trick?

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18

The second ellipsis is recognized during lexical analysis as the name of a function call. It doesn't have its own token like |> does. The output is a pairlist ( typeof(f(a, b)) ), which in this case is the same as a regular list (?). I guess it is not a parser trick. But whatever it is, it has been around for a while!

old tricks

Question:
How does ...() work?

ANSWER

Answered 2022-Jan-09 at 16:14

Note: When referring to documentation and source code, I provide links to an unofficial GitHub mirror of R's official Subversion repository. The links are bound to commit 97b6424 in the GitHub repo, which maps to revision 81461 in the Subversion repo (the latest at the time of this edit).


substitute is a "special" whose arguments are not evaluated (doc).

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19
1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20

That means that the return value of substitute may not agree with parser logic, depending on how the unevaluated arguments are processed internally.

In general, substitute receives the call ...(<exprs>) as a LANGSXP of the form (pseudocode) pairlist(R_DotsSymbol, <exprs>) (doc). The context of the substitute call determines how the SYMSXP R_DotsSymbol is processed. Specifically, if substitute was called inside of a function with ... as a formal argument and rho as its execution environment, then the result of

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21

in the body of C utility substituteList (source) is either a DOTSXP or R_MissingArg—the latter if and only if f was called without arguments (doc). In other contexts, the result is R_UnboundValue or (exceptionally) some other SEXP—the latter if and only if a value is bound to the name ... in rho. Each of these cases is handled specially by substituteList.

The multiplicity in the processing of R_DotsSymbol is the reason why these R statements give different results:

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36

Given how ...(n = 1) is parsed, you might have expected f1 to return call("...", n = 1), both g0 and g1 to return call("x", n = 1), and both h0 and h1 to throw an error, but that is not the case for the above, mostly undocumented reasons.

Internals

When called inside of the R function f,

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37

substitute evaluates a call to the C utility do_substitute—you can learn this by looking here—in which argList gets a LISTSXP of the form pairlist(x, R_MissingArg), where x is a LANGSXP of the form pairlist(R_DotsSymbol, <exprs>) (source).

If you follow the body of do_substitute, then you will find that the value of t passed to substituteList from do_substitute is a LISTSXP of the form pairlist(copy_of_x) (source).

It follows that the while loop inside of the substituteList call (source) has exactly one iteration and that the statement CAR(el) == R_DotsSymbol in the body of the loop (source) is false in that iteration.

In the false branch of the conditional (source), h gets the value pairlist(substituteList(copy_of_x, env)). The loop exits and substituteList returns h to do_substitute, which in turn returns CAR(h) to R (source 1, 2, 3).

Hence the return value of substitute is substituteList(copy_of_x, env), and it remains to deduce the identity of this SEXP. Inside of this call to substituteList, the while loop has 1+m iterations, where m is the number of <exprs>. In the first iteration, the statement CAR(el) == R_DotsSymbol in the body of the loop is true.

In the true branch of the conditional (source), h is either a DOTSXP or R_MissingArg, because f has ... as a formal argument (doc). Continuing, you will find that substituteList returns:

  • R_NilValue if h was R_MissingArg in the first while iteration and m = 0,

or, otherwise,

  • a LISTSXP listing the expressions in h (if h was a DOTSXP in the first while iteration) followed by <exprs> (if m > 1), all unevaluated and without substitutions, because the execution environment of f is empty at the time of the substitute call.

Indeed:

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43
Misc

FWIW, it helped me to recompile R after adding some print statements to coerce.c. For example, I added the following before UNPROTECT(3); in the body of do_substitute (source):

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43    Rprintf(&quot;CAR(t) == R_DotsSymbol? %d\n&quot;,
44            CAR(t) == R_DotsSymbol);
45    if (TYPEOF(CAR(t)) == LISTSXP || TYPEOF(CAR(t)) == LANGSXP) {
46        Rprintf(&quot;TYPEOF(CAR(t)) = %s, length(CAR(t)) = %d\n&quot;,
47                type2char(TYPEOF(CAR(t))), length(CAR(t)));
48        Rprintf(&quot;CAR(CAR(t)) = R_DotsSymbol? %d\n&quot;,
49                CAR(CAR(t)) == R_DotsSymbol);
50        Rprintf(&quot;TYPEOF(CDR(CAR(t))) = %s, length(CDR(CAR(t))) = %d\n&quot;,
51                type2char(TYPEOF(CDR(CAR(t)))), length(CDR(CAR(t))));
52    }
53    if (TYPEOF(s) == LISTSXP || TYPEOF(s) == LANGSXP) {
54        Rprintf(&quot;TYPEOF(s) = %s, length(s) = %d\n&quot;,
55                type2char(TYPEOF(s)), length(s));
56        Rprintf(&quot;TYPEOF(CAR(s)) = %s, length(CAR(s)) = %d\n&quot;,
57                type2char(TYPEOF(CAR(s))), length(CAR(s)));
58    }
59

which helped me confirm what was going into and coming out of the substituteList call on the previous line:

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43    Rprintf(&quot;CAR(t) == R_DotsSymbol? %d\n&quot;,
44            CAR(t) == R_DotsSymbol);
45    if (TYPEOF(CAR(t)) == LISTSXP || TYPEOF(CAR(t)) == LANGSXP) {
46        Rprintf(&quot;TYPEOF(CAR(t)) = %s, length(CAR(t)) = %d\n&quot;,
47                type2char(TYPEOF(CAR(t))), length(CAR(t)));
48        Rprintf(&quot;CAR(CAR(t)) = R_DotsSymbol? %d\n&quot;,
49                CAR(CAR(t)) == R_DotsSymbol);
50        Rprintf(&quot;TYPEOF(CDR(CAR(t))) = %s, length(CDR(CAR(t))) = %d\n&quot;,
51                type2char(TYPEOF(CDR(CAR(t)))), length(CDR(CAR(t))));
52    }
53    if (TYPEOF(s) == LISTSXP || TYPEOF(s) == LANGSXP) {
54        Rprintf(&quot;TYPEOF(s) = %s, length(s) = %d\n&quot;,
55                type2char(TYPEOF(s)), length(s));
56        Rprintf(&quot;TYPEOF(CAR(s)) = %s, length(CAR(s)) = %d\n&quot;,
57                type2char(TYPEOF(CAR(s))), length(CAR(s)));
58    }
59f &lt;- function(...) substitute(...(n = 1))
60invisible(f(hello, world, hello(world)))
61
1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43    Rprintf(&quot;CAR(t) == R_DotsSymbol? %d\n&quot;,
44            CAR(t) == R_DotsSymbol);
45    if (TYPEOF(CAR(t)) == LISTSXP || TYPEOF(CAR(t)) == LANGSXP) {
46        Rprintf(&quot;TYPEOF(CAR(t)) = %s, length(CAR(t)) = %d\n&quot;,
47                type2char(TYPEOF(CAR(t))), length(CAR(t)));
48        Rprintf(&quot;CAR(CAR(t)) = R_DotsSymbol? %d\n&quot;,
49                CAR(CAR(t)) == R_DotsSymbol);
50        Rprintf(&quot;TYPEOF(CDR(CAR(t))) = %s, length(CDR(CAR(t))) = %d\n&quot;,
51                type2char(TYPEOF(CDR(CAR(t)))), length(CDR(CAR(t))));
52    }
53    if (TYPEOF(s) == LISTSXP || TYPEOF(s) == LANGSXP) {
54        Rprintf(&quot;TYPEOF(s) = %s, length(s) = %d\n&quot;,
55                type2char(TYPEOF(s)), length(s));
56        Rprintf(&quot;TYPEOF(CAR(s)) = %s, length(CAR(s)) = %d\n&quot;,
57                type2char(TYPEOF(CAR(s))), length(CAR(s)));
58    }
59f &lt;- function(...) substitute(...(n = 1))
60invisible(f(hello, world, hello(world)))
61CAR(t) == R_DotsSymbol? 0
62TYPEOF(CAR(t)) = language, length(CAR(t)) = 2
63CAR(CAR(t)) = R_DotsSymbol? 1
64TYPEOF(CDR(CAR(t))) = pairlist, length(CDR(CAR(t))) = 1
65TYPEOF(s) = pairlist, length(s) = 1
66TYPEOF(CAR(s)) = pairlist, length(CAR(s)) = 4
67
1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43    Rprintf(&quot;CAR(t) == R_DotsSymbol? %d\n&quot;,
44            CAR(t) == R_DotsSymbol);
45    if (TYPEOF(CAR(t)) == LISTSXP || TYPEOF(CAR(t)) == LANGSXP) {
46        Rprintf(&quot;TYPEOF(CAR(t)) = %s, length(CAR(t)) = %d\n&quot;,
47                type2char(TYPEOF(CAR(t))), length(CAR(t)));
48        Rprintf(&quot;CAR(CAR(t)) = R_DotsSymbol? %d\n&quot;,
49                CAR(CAR(t)) == R_DotsSymbol);
50        Rprintf(&quot;TYPEOF(CDR(CAR(t))) = %s, length(CDR(CAR(t))) = %d\n&quot;,
51                type2char(TYPEOF(CDR(CAR(t)))), length(CDR(CAR(t))));
52    }
53    if (TYPEOF(s) == LISTSXP || TYPEOF(s) == LANGSXP) {
54        Rprintf(&quot;TYPEOF(s) = %s, length(s) = %d\n&quot;,
55                type2char(TYPEOF(s)), length(s));
56        Rprintf(&quot;TYPEOF(CAR(s)) = %s, length(CAR(s)) = %d\n&quot;,
57                type2char(TYPEOF(CAR(s))), length(CAR(s)));
58    }
59f &lt;- function(...) substitute(...(n = 1))
60invisible(f(hello, world, hello(world)))
61CAR(t) == R_DotsSymbol? 0
62TYPEOF(CAR(t)) = language, length(CAR(t)) = 2
63CAR(CAR(t)) = R_DotsSymbol? 1
64TYPEOF(CDR(CAR(t))) = pairlist, length(CDR(CAR(t))) = 1
65TYPEOF(s) = pairlist, length(s) = 1
66TYPEOF(CAR(s)) = pairlist, length(CAR(s)) = 4
67invisible(substitute(...()))
68
1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43    Rprintf(&quot;CAR(t) == R_DotsSymbol? %d\n&quot;,
44            CAR(t) == R_DotsSymbol);
45    if (TYPEOF(CAR(t)) == LISTSXP || TYPEOF(CAR(t)) == LANGSXP) {
46        Rprintf(&quot;TYPEOF(CAR(t)) = %s, length(CAR(t)) = %d\n&quot;,
47                type2char(TYPEOF(CAR(t))), length(CAR(t)));
48        Rprintf(&quot;CAR(CAR(t)) = R_DotsSymbol? %d\n&quot;,
49                CAR(CAR(t)) == R_DotsSymbol);
50        Rprintf(&quot;TYPEOF(CDR(CAR(t))) = %s, length(CDR(CAR(t))) = %d\n&quot;,
51                type2char(TYPEOF(CDR(CAR(t)))), length(CDR(CAR(t))));
52    }
53    if (TYPEOF(s) == LISTSXP || TYPEOF(s) == LANGSXP) {
54        Rprintf(&quot;TYPEOF(s) = %s, length(s) = %d\n&quot;,
55                type2char(TYPEOF(s)), length(s));
56        Rprintf(&quot;TYPEOF(CAR(s)) = %s, length(CAR(s)) = %d\n&quot;,
57                type2char(TYPEOF(CAR(s))), length(CAR(s)));
58    }
59f &lt;- function(...) substitute(...(n = 1))
60invisible(f(hello, world, hello(world)))
61CAR(t) == R_DotsSymbol? 0
62TYPEOF(CAR(t)) = language, length(CAR(t)) = 2
63CAR(CAR(t)) = R_DotsSymbol? 1
64TYPEOF(CDR(CAR(t))) = pairlist, length(CDR(CAR(t))) = 1
65TYPEOF(s) = pairlist, length(s) = 1
66TYPEOF(CAR(s)) = pairlist, length(CAR(s)) = 4
67invisible(substitute(...()))
68CAR(t) == R_DotsSymbol? 0
69TYPEOF(CAR(t)) = language, length(CAR(t)) = 1
70CAR(CAR(t)) = R_DotsSymbol? 1
71TYPEOF(CDR(CAR(t))) = NULL, length(CDR(CAR(t))) = 0
72TYPEOF(s) = pairlist, length(s) = 1
73TYPEOF(CAR(s)) = language, length(CAR(s)) = 1
74

Obviously, compiling R with debugging symbols and running R under a debugger helps, too.

Another puzzle

Just noticed this oddity:

1f = function(...) substitute(...()); f(a, b)
2[[1]]
3a
4[[2]]
5b
6# but, substitute returns ..1
7f2 = function(...) substitute(...); f2(a, b)
8a
9#  \-substitute #R
10#    \-do_substitute #C
11#      \-substituteList #C recursive
12#        \-substitute #C
13parse(text = &quot;(\\(...) substitute(...()))(a, b)&quot;) |&gt; getParseData() |&gt; subset(text == &quot;...&quot;, select = c(7, 9))
14
15#&gt;                   token  text
16#&gt; 4        SYMBOL_FORMALS   ...
17#&gt; 10 SYMBOL_FUNCTION_CALL   ...
18typeof(substitute)
19[1] &quot;special&quot;
20findVarInFrame3(rho, R_DotsSymbol, TRUE)
21f0 &lt;- function() substitute(...(n = 1)); f0()
22## ...(n = 1)
23f1 &lt;- function(...) substitute(...(n = 1)); f1()
24## $n
25## [1] 1
26g0 &lt;- function() {... &lt;- quote(x); substitute(...(n = 1))}; g0()
27## Error in g0() : '...' used in an incorrect context
28g1 &lt;- function(...) {... &lt;- quote(x); substitute(...(n = 1))}; g1()
29## Error in g1() : '...' used in an incorrect context
30h0 &lt;- function() {... &lt;- NULL; substitute(...(n = 1))}; h0()
31## $n
32## [1] 1
33h1 &lt;- function(...) {... &lt;- NULL; substitute(...(n = 1))}; h1()
34## $n
35## [1] 1
36f &lt;- function(...) substitute(...(&lt;exprs&gt;))
37f &lt;- function(...) substitute(...())
38is.null(f())
39## [1] TRUE
40f &lt;- function(...) substitute(...(n = 1))
41identical(f(a = sin(x), b = zzz), pairlist(a = quote(sin(x)), b = quote(zzz), n = 1))
42## [1] TRUE
43    Rprintf(&quot;CAR(t) == R_DotsSymbol? %d\n&quot;,
44            CAR(t) == R_DotsSymbol);
45    if (TYPEOF(CAR(t)) == LISTSXP || TYPEOF(CAR(t)) == LANGSXP) {
46        Rprintf(&quot;TYPEOF(CAR(t)) = %s, length(CAR(t)) = %d\n&quot;,
47                type2char(TYPEOF(CAR(t))), length(CAR(t)));
48        Rprintf(&quot;CAR(CAR(t)) = R_DotsSymbol? %d\n&quot;,
49                CAR(CAR(t)) == R_DotsSymbol);
50        Rprintf(&quot;TYPEOF(CDR(CAR(t))) = %s, length(CDR(CAR(t))) = %d\n&quot;,
51                type2char(TYPEOF(CDR(CAR(t)))), length(CDR(CAR(t))));
52    }
53    if (TYPEOF(s) == LISTSXP || TYPEOF(s) == LANGSXP) {
54        Rprintf(&quot;TYPEOF(s) = %s, length(s) = %d\n&quot;,
55                type2char(TYPEOF(s)), length(s));
56        Rprintf(&quot;TYPEOF(CAR(s)) = %s, length(CAR(s)) = %d\n&quot;,
57                type2char(TYPEOF(CAR(s))), length(CAR(s)));
58    }
59f &lt;- function(...) substitute(...(n = 1))
60invisible(f(hello, world, hello(world)))
61CAR(t) == R_DotsSymbol? 0
62TYPEOF(CAR(t)) = language, length(CAR(t)) = 2
63CAR(CAR(t)) = R_DotsSymbol? 1
64TYPEOF(CDR(CAR(t))) = pairlist, length(CDR(CAR(t))) = 1
65TYPEOF(s) = pairlist, length(s) = 1
66TYPEOF(CAR(s)) = pairlist, length(CAR(s)) = 4
67invisible(substitute(...()))
68CAR(t) == R_DotsSymbol? 0
69TYPEOF(CAR(t)) = language, length(CAR(t)) = 1
70CAR(CAR(t)) = R_DotsSymbol? 1
71TYPEOF(CDR(CAR(t))) = NULL, length(CDR(CAR(t))) = 0
72TYPEOF(s) = pairlist, length(s) = 1
73TYPEOF(CAR(s)) = language, length(CAR(s)) = 1
74g &lt;- function(...) substitute(...(n = 1), new.env())
75gab &lt;- g(a = sin(x), b = zzz)
76typeof(gab)
77## [1] &quot;language&quot;
78gab
79## ...(n = 1)
80

Someone here can do another deep dive to find out why the result is a LANGSXP rather than a LISTSXP when you supply env different from environment() (including env = NULL).

Source https://stackoverflow.com/questions/70602963

QUESTION

Python 3.10 pattern matching (PEP 634) - wildcard in string

Asked 2021-Dec-17 at 10:43

I got a large list of JSON objects that I want to parse depending on the start of one of the keys, and just wildcard the rest. A lot of the keys are similar, like "matchme-foo" and "matchme-bar". There is a builtin wildcard, but it is only used for whole values, kinda like an else.

I might be overlooking something but I can't find a solution anywhere in the proposal:

https://docs.python.org/3/whatsnew/3.10.html#pep-634-structural-pattern-matching

Also a bit more about it in PEP-636:

https://www.python.org/dev/peps/pep-0636/#going-to-the-cloud-mappings

My data looks like this:

1data = [{
2          &quot;id&quot;     : &quot;matchme-foo&quot;,
3          &quot;message&quot;: &quot;hallo this is a message&quot;,
4      },{
5          &quot;id&quot;     : &quot;matchme-bar&quot;,
6          &quot;message&quot;: &quot;goodbye&quot;,
7      },{
8          &quot;id&quot;     : &quot;anotherid&quot;,
9          &quot;message&quot;: &quot;completely diffrent event&quot;
10      }, ...]
11

I want to do something that can match the id without having to make a long list of |'s.

Something like this:

1data = [{
2          &quot;id&quot;     : &quot;matchme-foo&quot;,
3          &quot;message&quot;: &quot;hallo this is a message&quot;,
4      },{
5          &quot;id&quot;     : &quot;matchme-bar&quot;,
6          &quot;message&quot;: &quot;goodbye&quot;,
7      },{
8          &quot;id&quot;     : &quot;anotherid&quot;,
9          &quot;message&quot;: &quot;completely diffrent event&quot;
10      }, ...]
11for event in data:
12    match event:
13        case {'id':'matchme-*'}: # Match all 'matchme-' no matter what comes next
14            log.INFO(event['message'])
15        case {'id':'anotherid'}:
16            log.ERROR(event['message'])
17

It's a relatively new addition to Python so there aren't many guides on how to use it yet.

ANSWER

Answered 2021-Dec-17 at 10:43

You can use a guard:

1data = [{
2          &quot;id&quot;     : &quot;matchme-foo&quot;,
3          &quot;message&quot;: &quot;hallo this is a message&quot;,
4      },{
5          &quot;id&quot;     : &quot;matchme-bar&quot;,
6          &quot;message&quot;: &quot;goodbye&quot;,
7      },{
8          &quot;id&quot;     : &quot;anotherid&quot;,
9          &quot;message&quot;: &quot;completely diffrent event&quot;
10      }, ...]
11for event in data:
12    match event:
13        case {'id':'matchme-*'}: # Match all 'matchme-' no matter what comes next
14            log.INFO(event['message'])
15        case {'id':'anotherid'}:
16            log.ERROR(event['message'])
17for event in data:
18    match event:
19        case {'id': x} if x.startswith(&quot;matchme&quot;): # guard
20            print(event[&quot;message&quot;])
21        case {'id':'anotherid'}:
22            print(event[&quot;message&quot;])
23

Quoting from the official documentation,

Guard

We can add an if clause to a pattern, known as a “guard”. If the guard is false, match goes on to try the next case block. Note that value capture happens before the guard is evaluated:

1data = [{
2          &quot;id&quot;     : &quot;matchme-foo&quot;,
3          &quot;message&quot;: &quot;hallo this is a message&quot;,
4      },{
5          &quot;id&quot;     : &quot;matchme-bar&quot;,
6          &quot;message&quot;: &quot;goodbye&quot;,
7      },{
8          &quot;id&quot;     : &quot;anotherid&quot;,
9          &quot;message&quot;: &quot;completely diffrent event&quot;
10      }, ...]
11for event in data:
12    match event:
13        case {'id':'matchme-*'}: # Match all 'matchme-' no matter what comes next
14            log.INFO(event['message'])
15        case {'id':'anotherid'}:
16            log.ERROR(event['message'])
17for event in data:
18    match event:
19        case {'id': x} if x.startswith(&quot;matchme&quot;): # guard
20            print(event[&quot;message&quot;])
21        case {'id':'anotherid'}:
22            print(event[&quot;message&quot;])
23match point:
24     case Point(x, y) if x == y:
25         print(f&quot;The point is located on the diagonal Y=X at {x}.&quot;)
26     case Point(x, y):
27         print(f&quot;Point is not on the diagonal.&quot;)
28

Source https://stackoverflow.com/questions/70111401

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Go

Tutorials and Learning Resources are not available at this moment for Go

Share this Page

share link

Get latest updates on Go