Explore all Generator open source software, libraries, packages, source code, cloud functions and APIs.

Explore Related Topics

Popular New Releases in Generator

generator-jhipster

v7.8.1

openapi-generator

v6.0.0-beta released

auto

AutoValue 1.9

github-profile-readme-generator

GPRG v1.2.0

typedoc

v0.22.15

Popular Libraries in Generator

github-readme-stats

by anuraghazra doticonjavascriptdoticon

star image 38038 doticonMIT

:zap: Dynamically generated stats for your github readmes

generator-jhipster

by jhipster doticonjavascriptdoticon

star image 19466 doticonApache-2.0

JHipster is a development platform to quickly generate, develop, & deploy modern web applications & microservice architectures.

openapi-generator

by OpenAPITools doticonjavadoticon

star image 11886 doticonApache-2.0

OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)

qrcodejs

by davidshimjs doticonjavascriptdoticon

star image 10529 doticonMIT

Cross-browser QRCode generator for javascript

trianglify

by qrohlf doticonjavascriptdoticon

star image 9871 doticonGPL-3.0

Algorithmically generated triangle art

auto

by google doticonjavadoticon

star image 9703 doticonApache-2.0

A collection of source code generators for Java.

github-profile-readme-generator

by rahuldkjain doticonjavascriptdoticon

star image 9217 doticonApache-2.0

🚀 Generate GitHub profile README easily with the latest add-ons like visitors count, GitHub stats, etc using minimal UI.

readme-md-generator

by kefranabg doticonjavascriptdoticon

star image 8929 doticonMIT

📄 CLI that generates beautiful README.md files

generator-angular-fullstack

by angular-fullstack doticonjavascriptdoticon

star image 6129 doticon

Yeoman generator for an Angular app with an Express server

Trending New libraries in Generator

github-readme-stats

by anuraghazra doticonjavascriptdoticon

star image 38038 doticonMIT

:zap: Dynamically generated stats for your github readmes

github-profile-readme-generator

by rahuldkjain doticonjavascriptdoticon

star image 9217 doticonApache-2.0

🚀 Generate GitHub profile README easily with the latest add-ons like visitors count, GitHub stats, etc using minimal UI.

git-cliff

by orhun doticonrustdoticon

star image 3750 doticonGPL-3.0

A highly customizable Changelog Generator that follows Conventional Commit specifications ⛰️

swift-doc

by SwiftDocOrg doticonswiftdoticon

star image 1614 doticonMIT

A documentation generator for Swift projects

OC-Gen-X

by Pavo-IM doticonswiftdoticon

star image 1446 doticon

OpenCore Config Generator

github-readme-streak-stats

by DenverCoder1 doticonphpdoticon

star image 1039 doticonMIT

🔥 Stay motivated and show off your contribution streak! 🌟 Display your total contributions, current streak, and longest streak on your GitHub profile README

CsWin32

by microsoft doticoncsharpdoticon

star image 979 doticonMIT

A source generator to add a user-defined set of Win32 P/Invoke methods and supporting types to a C# project.

vite-plugin-pages

by hannoeru doticontypescriptdoticon

star image 687 doticonMIT

File system based route generator for ⚡️Vite

separator-generator

by wwebdev doticonjavascriptdoticon

star image 631 doticon

A generator for section separators

Top Authors in Generator

1

generate

30 Libraries

star icon337

2

manfredsteyer

29 Libraries

star icon359

3

paritytech

27 Libraries

star icon116

4

rayokota

23 Libraries

star icon848

5

anjali031

18 Libraries

star icon46

6

yeoman

16 Libraries

star icon13918

7

engindemirog

13 Libraries

star icon502

8

jhipster

11 Libraries

star icon20896

9

gitterHQ

10 Libraries

star icon750

10

mgechev

10 Libraries

star icon61

1

30 Libraries

star icon337

2

29 Libraries

star icon359

3

27 Libraries

star icon116

4

23 Libraries

star icon848

5

18 Libraries

star icon46

6

16 Libraries

star icon13918

7

13 Libraries

star icon502

8

11 Libraries

star icon20896

9

10 Libraries

star icon750

10

10 Libraries

star icon61

Trending Kits in Generator

As we are in the digital era, real-time video games are ruling the young generation. Tank games are one of the addictive games of this generation.


The objective of this game is to destroy the enemy's tank with our tank, which will decrease the energy level of the opponent. Similarly, our energy level will be reduced when the opponent attacks us with their tank. The attacking capacity ultimately depends on the energy level. The more is the energy level, the high is the attacking capacity. Following are the steps to be followed for building Tank Fight Game, 1.Graphic design & Sound effects 2.Firing and exploiting the tanks 3.Customize control over keyboard 4.Multi-player 5.3D Tank game

Customize control over keyboard

Key mapper is an open-source library that allows users to use a key or combination of keys to perform a specific action, which can be used for navigating and shooting. The below libraries can help you to create your control.

Graphic design & Sound effects

Listed below libraries help in creating the best graphic design and sound effects for gaming applications using python, C#, JavaScript, which can be used to design tanks, animate the movement of tanks, explosion of tanks, and display energy level bars.

Firing and exploiting the tanks

Random module, an open-source library, generates a random number provided the range, which can be used for firing and exploiting the tanks, decides the playing turn at the start of every game.

3D Tank game

The tank game can be built in 3D by using the below library.

Multi-player

Tank game can be played as multiplayer by using the below libraries. Multiplayers will play on the game field they shoot each other. The more is the energy level. The high is the attacking capacity.

If you are a designer you design logos and mockups, you use the color picker every day when choosing a new project's color scheme. By mixing the three primary colors, we can form any color we like. Red(R), Green(G), Blue(B) are the three primary colors we know. Using this tool, you can experiment with custom colors on the web and adjust them according to your preference. In addition, different color formats can be converted easily supported by JavaScript. These libraries allow building color picker and generator client-side applications in minutes. Following are the steps to be followed for building Color Picker and Code Generator, 1. Color Picker 2. Code Generator

Code Generator

These libraries are used to generate the code based on the color.

Color Picker

These libraries are used to pick the color.

Build custom programs to create character generators for video games and applications by using these open-source character generator libraries.


Character Generator is used to generate characters in video games, smartphone and desktop applications, websites, etc. in various different patterns. You can create 3D and 2D characters for your games and application by using a variety of models, textures and animations. With the help of character generator, you can allow the user to determine aspects of a digital character like attributes, skills, strengths and weaknesses, life events, etc. Also, you can allow them to add animation, text, as well as audio and video effects.


Mentioned below are some of the best and most trending reusable character generator libraries that you can implement in your next project:

A JavaScript documentation generator library helps web developers create comprehensive and organized documentation. It can help developers keep track of their code and its dependencies. It provides helpful examples and explanations of the code within the documentation. Documentation generated by this library can help developers understand their code better. It makes it easier for others to use and modify it. Additionally, the documentation can make the code easier to find and read. It can help make it easier to debug any errors that may arise.  

 

JSDoc is free software in JavaScript documentation generator libraries. It is an open-source project. It supports JSDoc syntax, JSDoc tags, and JSDoc comments. It helps generate documentation output from source code and markdown files. It can document Visual Basic, Code blocks, Type Definitions, and Generated Output. It documents React components, NPM modules, various plugins, and literate programming. JSDoc's default template is configurable. We can configure its parser to process names, static websites, comments, and Webpack. Other free software in JavaScript documentation generator libraries includes Sphinx, GitBook, and CoffeeScript. They also offer various plugins and templates to customize the generated output. We can use it for programming, creating websites, and working with modules and plugins.  

 

JSDoc uses a parser to scan source files for comments written in a special syntax, such as JSDoc comments. CoffeeScript is a programming language. It compiles JavaScript, and it will support some JavaScript documentation generator libraries. Sphinx is a popular documentation generator written that can generate documentation for projects. It can generate type definitions, such as those for GraphQL schemas. GraphQL is a query language we can use for API development in JavaScript. It allows developers to make efficient queries that return only the data they need from an API. GitBook is a free and open-source static site generator powered by Vue. It uses Markdown syntax to create documentation files. It offers a template to create documentation for projects, components, and modules. It supports various plugins and provides a parser to generate the documentation output.  

Different types of documentation that a JavaScript documentation generator library can generate:  

1. API Documentation:  

A JavaScript documentation generator library can generate full API documentation. It provides an overview of a given library's functions, objects, and classes. Also, it can provide information about each function, including arguments and return values.  

2. Tutorials:  

A JavaScript documentation generator library can generate tutorials. It provides step-by-step instructions on using a library. This can be useful for beginners starting with a library or developers needing a refresher.  

3. Examples:  

JavaScript documentation generator libraries can also generate code examples for various use cases. These examples demonstrate how we can use a library. It provides a starting point for developers looking to use the library.  

4. Reference Documentation:  

Reference documentation provides detailed information about a library's objects, functions, and classes. It includes information about arguments, return values, and more. This can be helpful for developers who need an overview of a library's features.  

5. User Guides:  

User guides provide an overview of a library and guidance on how to use it. They can also provide tips and best practices for getting the most out of a library. User guides can be especially helpful for newcomers needing a high-level library introduction.  

Different features available in a JavaScript documentation generator library: 

1. Generating documentation from source code comments and other annotations.  

2. Automatically generated class diagrams and other diagrams from source code.  

3. Automatically generating API reference documentation from source code.  

4. Generating interactive examples from source code.  

5. Generating links to external resources such as tutorials, manuals, and blog posts.  

6. Generating searchable documentation with built-in search capabilities.  

7. Generating documentation in many formats like HTML, PDF, and Markdown.  

8. Generating customization options to align documentation with branding and design guidelines.  

9. Supporting many programming languages such as JavaScript, TypeScript, Java, and Python.  

10. Generating documentation for many versions of source code.  

11. Generating Javadoc for your code.  

Tips for using a JavaScript documentation generator library:  

1. Read the documentation to understand how the library works and its features.  

2. Make sure to use the correct syntax and test the library by writing small snippets of code  

3. Use version controls such as Git to track changes and keep backups of your code.  

4. Take advantage of the library's features, such as markdown support or code generation.  

5. Use a linter to ensure your code meets standards and look for possible errors. 

6. Customize the output of the documentation generator library. It will do so by changing the styling, adding comments, and adjusting formatting.  

7. Generate documentation for your code to create an HTML or Markdown page you have written.  

Key Notes  

1. Become familiar with the library.  

2. Make sure you understand the purpose of the library.  

3. Plan your documentation. 

4. Start generating your documentation:  

Your library will provide commands that you can use to generate your documentation. Ensure that you read the documentation for the library to understand how to use the commands. It will also help them understand what arguments to pass to them.  

5. Test your documentation  

6. Publish your documentation  

 

A block comment in JavaScript is a comment that spans many lines. We write with two forward slashes (//) followed by an asterisk (*) and then the comment. The asterisk is then followed by an asterisk and two forward slashes (*/).  

 

A JavaScript documentation generator library is a valuable tool for web developers. The library can generate a document structure. It includes a list of classes, functions, variables, and a description for each item. It also provides code formatting, syntax highlighting, and comment formatting. The library can generate an HTML page with the generated documentation. It allows developers to share their work with others. This library can help save time, improve code quality, and increase efficiency.  

 

Let's look at each library in detail. The links below are to access package commands, installation notes, and code snippets.  

swagger-ui  

  • Provides a graphical user interface to help developers create and document APIs.  
  • Allows developers to test their APIs in real time.  
  • This open-source project is maintained and updated with new features and bug fixes.  

docsify  

  • It is built on the idea of using Markdown for writing your documentation.  
  • It is designed to be extensible. It allows one to add custom plugins to customize documentation.  
  • It provides an intuitive, user-friendly interface that makes writing and navigating easy.  

jsdoc  

  • Supports extra tags such as @param and @returns to document the code better.  
  • Provides a set of templates for creating custom documentation.  
  • Its large user base makes it easy to find help and tutorials online.  

apidoc  

  • It is designed to create documentation for APIs rather than any JavaScript code.  
  • Automatically generates documentation pages. It includes pages like an overview page, a list of endpoints, and detailed pages for each endpoint.  
  • One does not need to write special comments in their code to generate the documentation.  

plato  

  • It is built with Node.js rather than other frameworks, like Java or .NET.  
  • It has a plugin system, allowing for custom parsers and output formats.  
  • Produces a hierarchical tree of topics, allowing for deeper exploration of topics. 

doctoc  

  • Includes features like the ability to integrate with sources like GitHub and BitBucket.  
  • It offers a simple command-line tool allowing users to generate basic documentation.  
  • Includes an easy-to-use web-based editor that customizes the generated documentation.  

docute  

  • It has a built-in search feature that finds the information they need.  
  • It is an open-source library with no dependencies, making it lightweight and easy to use.  
  • Supports many themes to design their documentation according to their design preferences.  

esdoc  

  • Supports many languages such as JavaScript, TypeScript, Flow, and CoffeeScript.  
  • It can be used in both browser based and Node.js projects.  
  • It is customizable and can meet individual needs.  

kss-node  

  • Allows to use the same syntax to document both CSS and your JavaScript.  
  • It is designed to be used with Sass, Less, and other preprocessors.  
  • It enables one to add notes and annotations to their code. 

styledocco  

  • It uses a templating system to allow developers to customize the look and feel of their style guides.  
  • Allows developers to share and collaborate on style guides. It makes it easier for many teams to work on the same projects.  
  • It has a built-in code search feature to find code snippets related to a particular element. 

FAQ 

What is an API documentation generator, and why is it important for programming?  

An API documentation generator is a computer program. It generates documentation for an application programming interface (API). The documentation is a web page describing functions, classes, parameters, and return values. The generated documentation helps us understand how to use it and what it does. It is important for programming because it provides a way to understand the API and how to save time and effort.  

 

What does a documentation output look like when using a JavaScript library?  

A documentation output includes a description of the library, code examples, and references. The library's description overviews the purpose while it provides code snippets for tasks. The API references provide information about functions, including descriptions, return values, and information.  

 

What advantages does JSDoc have in comparison to other libraries?  

JSDoc has several advantages compared to other libraries.  

  • First, it is open source, meaning anyone can contribute to its development or use it in their projects. This allows for a wide range of customization and extensibility.  
  • Second, JSDoc is designed to be intuitive and easy to use. It provides a simple syntax for documenting code. It generates an HTML page that we can use for reference. This makes it much easier to understand the code and find the necessary information.  
  • Finally, JSDoc is compatible with various other libraries, such as TypeScript and Flow. This makes it easier to integrate it with other tools and projects. Additionally, it supports various programming languages, including JavaScript, Java, and HTML.  

 

How can one use documentation comments to improve the quality of code?  

Documentation comments can improve the quality of code. It does so by describing the code's purpose and its associated components. This helps understand the code by reducing error risk and making it easy to maintain. Additionally, it can improve collaboration between developers. It allows them to understand one another's code. Finally, it can help with debugging. It provides an easy reference point for anyone trying to understand an issue.  

 

Is there a site generator for creating interfaces that use JavaScript libraries?  

There are several site generators for creating interfaces that use JavaScript libraries. Some popular options include React, AngularJS, Vue.js, and Ember.js.  

 

Can we use markdown files with JavaScript document generators?  

Some document generators, such as Docsify and VuePress, can process Markdown files.  

 

Does Visual Studio Code offer integration with any specific JavaScript documentation generators?  

Yes, Visual Studio Code offers integration with JSDoc, a JavaScript documentation generator. JSDoc helps developers create inline documentation for their JavaScript code. It uses a syntax like JavaDoc, compatible with Visual Studio Code.  

 

Are code snippets included in most modern JavaScript library document generators?  

Yes, most modern JavaScript library document generators include code snippets.  

 

Do we need to annotate source files before we include them in generated docs from a library?  

No, we don't need to annotate source files before including them in the generated docs from a library. But adding annotations can help make the documentation easier to read and navigate.  

 

In what way do type annotations enhance the usability of a document generator output?  

Type annotations enhance the usability of a JavaScript document generator library's output. It will use it in its code without guessing what data the output contains. It will do so by providing developers with more information about the data types of the output. This makes it easier for developers to understand the output. Type annotations also make the output more consistent. It can reduce errors and make the code more maintainable. 

DESCRIPTION:

Artistic GAN is a fun and creative project that uses a special kind of AI called Generative Adversarial Networks (GANs) to make unique and artistic images. It's like a creative competition between two AI artists. One AI tries to create beautiful pictures, while the other AI tries to figure out which ones are real and which are made by AI.


As the AI artists keep learning and improving, they create more and more realistic and fascinating images. The project shows how AI can be used to make different styles of art, from abstract and colorful to realistic and detailed.


The cool thing is that these AI-generated images can be used in movies, games, and digital design to make content more interesting and exciting. It's a mix of technology and art that inspires new ideas and possibilities for the future.


The best part is that it's open-source, meaning anyone can access the code and even contribute to make the AI artists even better. It's a collaborative and creative project that brings together AI enthusiasts and artists to explore the fantastic world of Generative AI and its potential in redefining art


DEPENDENT LIBRARIES

SOLUTION SOURCE:

Trending Discussions on Generator

How can I make an object with an interface like a random number generator, but that actually generates a specified sequence?

Why is `np.sum(range(N))` very slow?

How would you implement a lazy "range factory" for C++20 ranges that just calls a generator function?

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in the package.json of a module in node_modules

Can you compress angular image assets on build?

Convert a bytes iterable to an iterable of str, where each value is a line

Problem Updating to .Net 6 - Encrypting String

Ensure that an argument can be iterated twice

Proper way to DI NSwag auto-generated client

How to alias generic types for decorators

QUESTION

How can I make an object with an interface like a random number generator, but that actually generates a specified sequence?

Asked 2022-Mar-31 at 13:47

I'd like to construct an object that works like a random number generator, but generates numbers in a specified sequence.

1# a random number generator
2rng = lambda : np.random.randint(2,20)//2
3
4# a non-random number generator
5def nrng():
6    numbers = np.arange(1,10.5,0.5)
7    for i in range(len(numbers)):
8        yield numbers[i]
9
10for j in range(10):
11    print('random number', rng())
12    print('non-random number', nrng())
13

The issue with the code above that I cannot call nrng in the last line because it is a generator. I know that the most straightforward way to rewrite the code above is to simply loop over the non-random numbers instead of defining the generator. I would prefer getting the example above to work because I am working with a large chunk of code that include a function that accepts a random number generator as an argument, and I would like to add the functionality to pass non-random number sequences without rewriting the entire code.

EDIT: I see some confusion in the comments. I am aware that python's random number generators generate pseudo-random numbers. This post is about replacing a pseudo-random-number generator by a number generator that generates numbers from a non-random, user-specified sequence (e.g., a generator that generates the number sequence 1,1,2,2,1,0,1 if I want it to).

ANSWER

Answered 2022-Mar-29 at 00:47

You can call next() with a generator or iterator as an argument to withdraw exactly one element from it. Saving the generator to a variable beforehand allows you to do this multiple times.

1# a random number generator
2rng = lambda : np.random.randint(2,20)//2
3
4# a non-random number generator
5def nrng():
6    numbers = np.arange(1,10.5,0.5)
7    for i in range(len(numbers)):
8        yield numbers[i]
9
10for j in range(10):
11    print('random number', rng())
12    print('non-random number', nrng())
13# make it a generator
14def _rng():
15    while True:
16        yield np.random.randint(2,20)//2
17
18# a non-random number generator
19def _nrng():
20    numbers = np.arange(1,10.5,0.5)
21    for i in range(len(numbers)):
22        yield numbers[i]
23
24rng = _rng()
25nrng = _nrng()
26for j in range(10):
27    print('random number', next(rng))
28    print('non-random number', next(nrng))
29

Source https://stackoverflow.com/questions/71655179

QUESTION

Why is `np.sum(range(N))` very slow?

Asked 2022-Mar-29 at 14:31

I saw a video about speed of loops in python, where it was explained that doing sum(range(N)) is much faster than manually looping through range and adding the variables together, since the former runs in C due to built-in functions being used, while in the latter the summation is done in (slow) python. I was curious what happens when adding numpy to the mix. As I expected np.sum(np.arange(N)) is the fastest, but sum(np.arange(N)) and np.sum(range(N)) are even slower than doing the naive for loop.

Why is this?

Here's the script I used to test, some comments about the supposed cause of slowing done where I know (taken mostly from the video) and the results I got on my machine (python 3.10.0, numpy 1.21.2):

updated script:

1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8    s = 0
9    i = 0
10    while i < N: # condition is checked in python
11        s += i
12        i += 1 # both additions are done in python
13    return s
14
15def sum1(N = N):
16    s = 0
17    for i in range(N): # increment in C
18        s += i # addition in python
19    return s
20
21def sum2(N = N):
22    return sum(range(N)) # everything in C
23
24def sum3(N = N):
25    return sum(list(range(N)))
26
27def sum4(N = N):
28    return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31    # much faster np.array conversion
32    return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35    # much faster np.array conversion
36    return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39    # possibly slow conversion to Py_long from np.int
40    return sum(np.arange(N))
41
42def sum7(N = N):
43    # list returns a list of np.int-s
44    return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47    # tolist conversion to python int seems faster than the implicit conversion
48    # in sum(list()) (tolist returns a list of python int-s)
49    return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52    return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55    return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58    return np.array(range(N))
59
60def array_dtype(N = N):
61    return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64    # np.sum's source code mentions to use fromiter to convert from generators
65    return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop:         {timeit(sum0, number = repetition)}")
68print(f"for loop:           {timeit(sum1, number = repetition)}")
69print(f"sum_range:          {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist:      {timeit(sum3, number = repetition)}")
71print(f"npsum_range:        {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange:    {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2:  {timeit(sum5, number = repetition)}")
74print(f"sum_arange:         {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange:    {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist:  {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange:       {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum:      {timeit(sum9, number = repetition)}")
79print(f"array_basic:        {timeit(array_basic, number = repetition)}")
80print(f"array_dtype:        {timeit(array_dtype, number = repetition)}")
81print(f"array_iter:         {timeit(array_iter,  number = repetition)}")
82
83print(f"npsumarangeREP:     {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP:     {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop:         11.493371912998555
89# for loop:           7.385945574002108
90# sum_range:          2.4605720699983067
91# sum_rangelist:      4.509678105998319
92# npsum_range:        11.85120212900074
93# npsum_iterrange:    4.464334709002287
94# npsum_iterrangev2:  4.498494338993623
95# sum_arange:         9.537815956995473
96# sum_list_arange:    13.290120724996086
97# sum_arange_tolist:  5.231948580003518
98# npsum_arange:       0.241889145996538
99# nparangenpsum:      0.21876695199898677
100# array_basic:        11.736577274998126
101# array_dtype:        8.71628468400013
102# array_iter:         4.303306431000237
103# npsumarangeREP:     21.240833958996518
104# npsumarangeREP:     16.690092379001726
105
106

ANSWER

Answered 2021-Oct-16 at 17:42

From the cpython source code for sum sum initially seems to attempt a fast path that assumes all inputs are the same type. If that fails it will just iterate:

1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8    s = 0
9    i = 0
10    while i < N: # condition is checked in python
11        s += i
12        i += 1 # both additions are done in python
13    return s
14
15def sum1(N = N):
16    s = 0
17    for i in range(N): # increment in C
18        s += i # addition in python
19    return s
20
21def sum2(N = N):
22    return sum(range(N)) # everything in C
23
24def sum3(N = N):
25    return sum(list(range(N)))
26
27def sum4(N = N):
28    return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31    # much faster np.array conversion
32    return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35    # much faster np.array conversion
36    return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39    # possibly slow conversion to Py_long from np.int
40    return sum(np.arange(N))
41
42def sum7(N = N):
43    # list returns a list of np.int-s
44    return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47    # tolist conversion to python int seems faster than the implicit conversion
48    # in sum(list()) (tolist returns a list of python int-s)
49    return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52    return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55    return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58    return np.array(range(N))
59
60def array_dtype(N = N):
61    return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64    # np.sum's source code mentions to use fromiter to convert from generators
65    return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop:         {timeit(sum0, number = repetition)}")
68print(f"for loop:           {timeit(sum1, number = repetition)}")
69print(f"sum_range:          {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist:      {timeit(sum3, number = repetition)}")
71print(f"npsum_range:        {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange:    {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2:  {timeit(sum5, number = repetition)}")
74print(f"sum_arange:         {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange:    {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist:  {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange:       {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum:      {timeit(sum9, number = repetition)}")
79print(f"array_basic:        {timeit(array_basic, number = repetition)}")
80print(f"array_dtype:        {timeit(array_dtype, number = repetition)}")
81print(f"array_iter:         {timeit(array_iter,  number = repetition)}")
82
83print(f"npsumarangeREP:     {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP:     {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop:         11.493371912998555
89# for loop:           7.385945574002108
90# sum_range:          2.4605720699983067
91# sum_rangelist:      4.509678105998319
92# npsum_range:        11.85120212900074
93# npsum_iterrange:    4.464334709002287
94# npsum_iterrangev2:  4.498494338993623
95# sum_arange:         9.537815956995473
96# sum_list_arange:    13.290120724996086
97# sum_arange_tolist:  5.231948580003518
98# npsum_arange:       0.241889145996538
99# nparangenpsum:      0.21876695199898677
100# array_basic:        11.736577274998126
101# array_dtype:        8.71628468400013
102# array_iter:         4.303306431000237
103# npsumarangeREP:     21.240833958996518
104# npsumarangeREP:     16.690092379001726
105
106/* Fast addition by keeping temporary sums in C instead of new Python objects.
107   Assumes all inputs are the same type.  If the assumption fails, default
108   to the more general routine.
109*/
110

I'm not entirely certain what is happening under the hood, but it is likely the repeated creation/conversion of C types to Python objects that is causing these slow-downs. It's worth noting that both sum and range are implemented in C.


This next bit is not really an answer to the question, but I wondered if we could speed up sum for python ranges as range is quite a smart object.

To do this I've used functools.singledispatch to override the built-in sum function specifically for the range type; then implemented a small function to calculate the sum of an arithmetic progression.

1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8    s = 0
9    i = 0
10    while i < N: # condition is checked in python
11        s += i
12        i += 1 # both additions are done in python
13    return s
14
15def sum1(N = N):
16    s = 0
17    for i in range(N): # increment in C
18        s += i # addition in python
19    return s
20
21def sum2(N = N):
22    return sum(range(N)) # everything in C
23
24def sum3(N = N):
25    return sum(list(range(N)))
26
27def sum4(N = N):
28    return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31    # much faster np.array conversion
32    return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35    # much faster np.array conversion
36    return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39    # possibly slow conversion to Py_long from np.int
40    return sum(np.arange(N))
41
42def sum7(N = N):
43    # list returns a list of np.int-s
44    return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47    # tolist conversion to python int seems faster than the implicit conversion
48    # in sum(list()) (tolist returns a list of python int-s)
49    return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52    return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55    return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58    return np.array(range(N))
59
60def array_dtype(N = N):
61    return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64    # np.sum's source code mentions to use fromiter to convert from generators
65    return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop:         {timeit(sum0, number = repetition)}")
68print(f"for loop:           {timeit(sum1, number = repetition)}")
69print(f"sum_range:          {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist:      {timeit(sum3, number = repetition)}")
71print(f"npsum_range:        {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange:    {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2:  {timeit(sum5, number = repetition)}")
74print(f"sum_arange:         {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange:    {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist:  {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange:       {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum:      {timeit(sum9, number = repetition)}")
79print(f"array_basic:        {timeit(array_basic, number = repetition)}")
80print(f"array_dtype:        {timeit(array_dtype, number = repetition)}")
81print(f"array_iter:         {timeit(array_iter,  number = repetition)}")
82
83print(f"npsumarangeREP:     {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP:     {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop:         11.493371912998555
89# for loop:           7.385945574002108
90# sum_range:          2.4605720699983067
91# sum_rangelist:      4.509678105998319
92# npsum_range:        11.85120212900074
93# npsum_iterrange:    4.464334709002287
94# npsum_iterrangev2:  4.498494338993623
95# sum_arange:         9.537815956995473
96# sum_list_arange:    13.290120724996086
97# sum_arange_tolist:  5.231948580003518
98# npsum_arange:       0.241889145996538
99# nparangenpsum:      0.21876695199898677
100# array_basic:        11.736577274998126
101# array_dtype:        8.71628468400013
102# array_iter:         4.303306431000237
103# npsumarangeREP:     21.240833958996518
104# npsumarangeREP:     16.690092379001726
105
106/* Fast addition by keeping temporary sums in C instead of new Python objects.
107   Assumes all inputs are the same type.  If the assumption fails, default
108   to the more general routine.
109*/
110from functools import singledispatch
111
112def sum_range(range_, /, start=0):
113    """Overloaded `sum` for range, compute arithmetic sum"""
114    n = len(range_)
115    if not n:
116        return start
117    return int(start + (n * (range_[0] + range_[-1]) / 2))
118
119sum = singledispatch(sum)
120sum.register(range, sum_range)
121
122def test():
123    """
124    >>> sum(range(0, 100))
125    4950
126    >>> sum(range(0, 10, 2))
127    20
128    >>> sum(range(0, 9, 2))
129    20
130    >>> sum(range(0, -10, -1))
131    -45
132    >>> sum(range(-10, 10))
133    -10
134    >>> sum(range(-1, -100, -2))
135    -2500
136    >>> sum(range(0, 10, 100))
137    0
138    >>> sum(range(0, 0))
139    0
140    >>> sum(range(0, 100), 50)
141    5000
142    >>> sum(range(0, 0), 10)
143    10
144    """
145
146if __name__ == "__main__":
147    import doctest
148    doctest.testmod()
149

I'm not sure if this is complete, but it's definitely faster than looping.

Source https://stackoverflow.com/questions/69584027

QUESTION

How would you implement a lazy "range factory" for C++20 ranges that just calls a generator function?

Asked 2022-Feb-17 at 17:10

I like the idea of the lazy ranges you can make with std::views::iota but was surprised to see that iota is currently the only thing like it in the standard; it is the only "range factory" besides views::single and views::empty. There is not currently, for example, the equivalent of std::generate as a range factory.

I note however it is trivial to implement the semantics of generate by using a transform view on iota and just ignoring the value iota passes to transform i.e.

1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7    return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15    auto d6 = []() {
16        static std::uniform_int_distribution<> dist(1, 6);
17        return dist(rng);
18    };
19
20    for (int v : generate1(d6) | std::views::take(10)) {
21        std::cout << v << ' ';
22    }
23    std::cout << '\n';
24}
25

My questions is what would be "the real way" to implement something like this? To make a range view object that is pipeable that does not just use iota.

I tried inheriting from ranges::view_interface -- no idea if this is the correct approach -- and just having it return a dummy iterator that calls a generator function but my code doesn't work because of the part where it needs to pipe the range view to std::views::take in order to not cause an infinite loop. The object I define here does not end up being pipeable.

1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7    return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15    auto d6 = []() {
16        static std::uniform_int_distribution<> dist(1, 6);
17        return dist(rng);
18    };
19
20    for (int v : generate1(d6) | std::views::take(10)) {
21        std::cout << v << ' ';
22    }
23    std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32    using value_type = decltype(std::declval<F>()());
33
34    class iterator {
35        const F* gen_func_;
36    public:
37        iterator(const F* f) : gen_func_(f)
38        {}
39
40        value_type operator*() const {
41            return (*gen_func_)();
42        }
43
44        bool operator!=(const iterator&) {
45            return true;
46        }
47
48        iterator& operator++() {
49            return *this;
50        }
51    };
52
53    F generator_func_;
54
55public:
56
57    generate2(const F& f) : generator_func_(f) {
58    }
59
60    iterator begin()  {
61        return iterator(&generator_func_);
62    }
63
64    iterator end()  {
65        return iterator(nullptr);
66    }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74    auto d6 = []() {
75        static std::uniform_int_distribution<> dist(1, 6);
76        return dist(rng);
77    };
78
79    // the following doesnt compile because of the pipe...
80    for (int v : generate2(d6) | std::views::take(10)) { 
81        std::cout << v << ' ';
82    }
83    std::cout << '\n';
84}
85

ANSWER

Answered 2022-Feb-17 at 17:10

The reason why generate2 cannot work is that it does not model the range concept, that is, the type returned by its begin() does not model input_iterator, because input_iterator requires difference_type and value_type to exist and i++ is a valid expression.

In addition, your iterator does not satisfy sentinel_for<iterator>, which means that it cannot serve as its own sentinel, because sentinel_for requires semiregular which requires default_initializable, so you also need to add default constructors for it.

You also need to rewrite bool operator!=(...) to bool operator==(...) const since operator!= does not reverse synthesize operator==. But it's easier to just use default_sentinel_t as sentinel in your case.

if you add them to iterator you will find the code will be well-formed:

1#include &lt;iostream&gt;
2#include &lt;ranges&gt;
3#include &lt;random&gt;
4
5template&lt;typename F&gt;
6auto generate1(const F&amp; func) {
7    return std::views::iota(0) | std::views::transform([&amp;func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15    auto d6 = []() {
16        static std::uniform_int_distribution&lt;&gt; dist(1, 6);
17        return dist(rng);
18    };
19
20    for (int v : generate1(d6) | std::views::take(10)) {
21        std::cout &lt;&lt; v &lt;&lt; ' ';
22    }
23    std::cout &lt;&lt; '\n';
24}
25#include &lt;iostream&gt;
26#include &lt;ranges&gt;
27#include &lt;random&gt;
28
29template&lt;typename F&gt;
30class generate2 : public std::ranges::view_interface&lt;generate2&lt;F&gt;&gt;
31{
32    using value_type = decltype(std::declval&lt;F&gt;()());
33
34    class iterator {
35        const F* gen_func_;
36    public:
37        iterator(const F* f) : gen_func_(f)
38        {}
39
40        value_type operator*() const {
41            return (*gen_func_)();
42        }
43
44        bool operator!=(const iterator&amp;) {
45            return true;
46        }
47
48        iterator&amp; operator++() {
49            return *this;
50        }
51    };
52
53    F generator_func_;
54
55public:
56
57    generate2(const F&amp; f) : generator_func_(f) {
58    }
59
60    iterator begin()  {
61        return iterator(&amp;generator_func_);
62    }
63
64    iterator end()  {
65        return iterator(nullptr);
66    }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74    auto d6 = []() {
75        static std::uniform_int_distribution&lt;&gt; dist(1, 6);
76        return dist(rng);
77    };
78
79    // the following doesnt compile because of the pipe...
80    for (int v : generate2(d6) | std::views::take(10)) { 
81        std::cout &lt;&lt; v &lt;&lt; ' ';
82    }
83    std::cout &lt;&lt; '\n';
84}
85class iterator {
86 public:
87  using value_type = decltype(std::declval&lt;F&gt;()());
88  using difference_type = std::ptrdiff_t;
89  iterator() = default;
90  void operator++(int);
91  bool operator==(const iterator&amp;) const {
92    return false;
93  }
94  // ...
95};
96

However, the operator*() of iterator does not meet the requirements of equality-preserving, that is to say, the results obtained by the two calls before and after are not equal, which means that this will be undefined behavior.

You can refer to the implementation of ranges::istream_view to use a member variable to cache each generated result, then you only need to return the cached value each time iterator::operator*() is called.

1#include &lt;iostream&gt;
2#include &lt;ranges&gt;
3#include &lt;random&gt;
4
5template&lt;typename F&gt;
6auto generate1(const F&amp; func) {
7    return std::views::iota(0) | std::views::transform([&amp;func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15    auto d6 = []() {
16        static std::uniform_int_distribution&lt;&gt; dist(1, 6);
17        return dist(rng);
18    };
19
20    for (int v : generate1(d6) | std::views::take(10)) {
21        std::cout &lt;&lt; v &lt;&lt; ' ';
22    }
23    std::cout &lt;&lt; '\n';
24}
25#include &lt;iostream&gt;
26#include &lt;ranges&gt;
27#include &lt;random&gt;
28
29template&lt;typename F&gt;
30class generate2 : public std::ranges::view_interface&lt;generate2&lt;F&gt;&gt;
31{
32    using value_type = decltype(std::declval&lt;F&gt;()());
33
34    class iterator {
35        const F* gen_func_;
36    public:
37        iterator(const F* f) : gen_func_(f)
38        {}
39
40        value_type operator*() const {
41            return (*gen_func_)();
42        }
43
44        bool operator!=(const iterator&amp;) {
45            return true;
46        }
47
48        iterator&amp; operator++() {
49            return *this;
50        }
51    };
52
53    F generator_func_;
54
55public:
56
57    generate2(const F&amp; f) : generator_func_(f) {
58    }
59
60    iterator begin()  {
61        return iterator(&amp;generator_func_);
62    }
63
64    iterator end()  {
65        return iterator(nullptr);
66    }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74    auto d6 = []() {
75        static std::uniform_int_distribution&lt;&gt; dist(1, 6);
76        return dist(rng);
77    };
78
79    // the following doesnt compile because of the pipe...
80    for (int v : generate2(d6) | std::views::take(10)) { 
81        std::cout &lt;&lt; v &lt;&lt; ' ';
82    }
83    std::cout &lt;&lt; '\n';
84}
85class iterator {
86 public:
87  using value_type = decltype(std::declval&lt;F&gt;()());
88  using difference_type = std::ptrdiff_t;
89  iterator() = default;
90  void operator++(int);
91  bool operator==(const iterator&amp;) const {
92    return false;
93  }
94  // ...
95};
96template&lt;typename F&gt;
97class generate2 : public std::ranges::view_interface&lt;generate2&lt;F&gt;&gt; {
98 public:
99  auto begin() {
100    value_ = generator_func_();
101    return iterator{*this};
102  }
103
104  std::default_sentinel_t end() const noexcept { return std::default_sentinel; }
105
106  class iterator {
107   public:
108   //...
109    value_type operator*() const {
110      return parent_-&gt;value_;
111    }
112   private:
113    generate2* parent_;
114  };
115
116 private:
117   F generator_func_;
118   std::remove_cvref_t&lt;std::invoke_result_t&lt;F&amp;&gt;&gt; value_; 
119};
120

Source https://stackoverflow.com/questions/71148130

QUESTION

Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in the package.json of a module in node_modules

Asked 2022-Jan-31 at 17:22

This is a React web app. When I run

1npm start
2

This error occurred

1npm start
2&gt; dataflow@0.1.0 start
3&gt; react-scripts start
4
5node:internal/modules/cjs/loader:488
6      throw e;
7  ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10    at new NodeError (node:internal/errors:371:5)
11    at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12    at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13    at resolveExports (node:internal/modules/cjs/loader:482:36)
14    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17    at Module.require (node:internal/modules/cjs/loader:999:19)
18    at require (node:internal/modules/cjs/helpers:102:18)
19    at Object.&lt;anonymous&gt; (/Users/juliantc/Desktop/ai-    studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24

This error only occurs when I run this on this specific computer, which I do not have superuser access to. It works on other computers.

For reference, this is ./node_modules/postcss-safe-parser/node_modules/postcss/package.json

1npm start
2&gt; dataflow@0.1.0 start
3&gt; react-scripts start
4
5node:internal/modules/cjs/loader:488
6      throw e;
7  ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10    at new NodeError (node:internal/errors:371:5)
11    at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12    at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13    at resolveExports (node:internal/modules/cjs/loader:482:36)
14    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17    at Module.require (node:internal/modules/cjs/loader:999:19)
18    at require (node:internal/modules/cjs/helpers:102:18)
19    at Object.&lt;anonymous&gt; (/Users/juliantc/Desktop/ai-    studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25  "name": "postcss",
26  "version": "8.2.6",
27  "description": "Tool for transforming styles with JS plugins",
28  "engines": {
29    "node": "^10 || ^12 || &gt;=14"
30  },
31  "exports": {
32    ".": {
33      "require": "./lib/postcss.js",
34      "import": "./lib/postcss.mjs",
35      "types": "./lib/postcss.d.ts"
36    },
37    "./": "./"
38  },
39  "main": "./lib/postcss.js",
40  "types": "./lib/postcss.d.ts",
41  "keywords": [
42    "css",
43    "postcss",
44    "rework",
45    "preprocessor",
46    "parser",
47    "source map",
48    "transform",
49    "manipulation",
50    "transpiler"
51  ],
52  "funding": {
53    "type": "opencollective",
54    "url": "https://opencollective.com/postcss/"
55  },
56  "author": "Andrey Sitnik &lt;andrey@sitnik.ru&gt;",
57  "license": "MIT",
58  "homepage": "https://postcss.org/",
59  "repository": "postcss/postcss",
60  "dependencies": {
61    "colorette": "^1.2.1",
62    "nanoid": "^3.1.20",
63    "source-map": "^0.6.1"
64  },
65  "browser": {
66    "./lib/terminal-highlight": false,
67    "colorette": false,
68    "fs": false
69  }
70}

And this is what I get when I list the files in ./node_modules/postcss-safe-parser/node_modules/postcss/lib/

lgtd-lt-119-mbmt:frontend juliantc$ ls ./node_modules/postcss-safe-parser/node_modules/postcss/lib/

1npm start
2&gt; dataflow@0.1.0 start
3&gt; react-scripts start
4
5node:internal/modules/cjs/loader:488
6      throw e;
7  ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10    at new NodeError (node:internal/errors:371:5)
11    at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12    at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13    at resolveExports (node:internal/modules/cjs/loader:482:36)
14    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17    at Module.require (node:internal/modules/cjs/loader:999:19)
18    at require (node:internal/modules/cjs/helpers:102:18)
19    at Object.&lt;anonymous&gt; (/Users/juliantc/Desktop/ai-    studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25  "name": "postcss",
26  "version": "8.2.6",
27  "description": "Tool for transforming styles with JS plugins",
28  "engines": {
29    "node": "^10 || ^12 || &gt;=14"
30  },
31  "exports": {
32    ".": {
33      "require": "./lib/postcss.js",
34      "import": "./lib/postcss.mjs",
35      "types": "./lib/postcss.d.ts"
36    },
37    "./": "./"
38  },
39  "main": "./lib/postcss.js",
40  "types": "./lib/postcss.d.ts",
41  "keywords": [
42    "css",
43    "postcss",
44    "rework",
45    "preprocessor",
46    "parser",
47    "source map",
48    "transform",
49    "manipulation",
50    "transpiler"
51  ],
52  "funding": {
53    "type": "opencollective",
54    "url": "https://opencollective.com/postcss/"
55  },
56  "author": "Andrey Sitnik &lt;andrey@sitnik.ru&gt;",
57  "license": "MIT",
58  "homepage": "https://postcss.org/",
59  "repository": "postcss/postcss",
60  "dependencies": {
61    "colorette": "^1.2.1",
62    "nanoid": "^3.1.20",
63    "source-map": "^0.6.1"
64  },
65  "browser": {
66    "./lib/terminal-highlight": false,
67    "colorette": false,
68    "fs": false
69  }
70}at-rule.d.ts        css-syntax-error.d.ts   input.d.ts      map-generator.js    postcss.d.ts        processor.js        rule.js         tokenize.js
71at-rule.js      css-syntax-error.js input.js        node.d.ts       postcss.js      result.d.ts     stringifier.js      warn-once.js
72comment.d.ts        declaration.d.ts    lazy-result.d.ts    node.js         postcss.mjs     result.js       stringify.d.ts      warning.d.ts
73comment.js      declaration.js      lazy-result.js      parse.d.ts      previous-map.d.ts   root.d.ts       stringify.js        warning.js
74container.d.ts      fromJSON.d.ts       list.d.ts       parse.js        previous-map.js     root.js         symbols.js
75container.js        fromJSON.js     list.js         parser.js       processor.d.ts      rule.d.ts       terminal-highlight.js
76

ANSWER

Answered 2021-Nov-13 at 18:36

I am also stuck with the same problem because I installed the latest version of Node.js (v17.0.1).

Just go for node.js v14.18.1 and remove the latest version just use the stable version v14.18.1

1npm start
2&gt; dataflow@0.1.0 start
3&gt; react-scripts start
4
5node:internal/modules/cjs/loader:488
6      throw e;
7  ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10    at new NodeError (node:internal/errors:371:5)
11    at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12    at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13    at resolveExports (node:internal/modules/cjs/loader:482:36)
14    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17    at Module.require (node:internal/modules/cjs/loader:999:19)
18    at require (node:internal/modules/cjs/helpers:102:18)
19    at Object.&lt;anonymous&gt; (/Users/juliantc/Desktop/ai-    studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25  "name": "postcss",
26  "version": "8.2.6",
27  "description": "Tool for transforming styles with JS plugins",
28  "engines": {
29    "node": "^10 || ^12 || &gt;=14"
30  },
31  "exports": {
32    ".": {
33      "require": "./lib/postcss.js",
34      "import": "./lib/postcss.mjs",
35      "types": "./lib/postcss.d.ts"
36    },
37    "./": "./"
38  },
39  "main": "./lib/postcss.js",
40  "types": "./lib/postcss.d.ts",
41  "keywords": [
42    "css",
43    "postcss",
44    "rework",
45    "preprocessor",
46    "parser",
47    "source map",
48    "transform",
49    "manipulation",
50    "transpiler"
51  ],
52  "funding": {
53    "type": "opencollective",
54    "url": "https://opencollective.com/postcss/"
55  },
56  "author": "Andrey Sitnik &lt;andrey@sitnik.ru&gt;",
57  "license": "MIT",
58  "homepage": "https://postcss.org/",
59  "repository": "postcss/postcss",
60  "dependencies": {
61    "colorette": "^1.2.1",
62    "nanoid": "^3.1.20",
63    "source-map": "^0.6.1"
64  },
65  "browser": {
66    "./lib/terminal-highlight": false,
67    "colorette": false,
68    "fs": false
69  }
70}at-rule.d.ts        css-syntax-error.d.ts   input.d.ts      map-generator.js    postcss.d.ts        processor.js        rule.js         tokenize.js
71at-rule.js      css-syntax-error.js input.js        node.d.ts       postcss.js      result.d.ts     stringifier.js      warn-once.js
72comment.d.ts        declaration.d.ts    lazy-result.d.ts    node.js         postcss.mjs     result.js       stringify.d.ts      warning.d.ts
73comment.js      declaration.js      lazy-result.js      parse.d.ts      previous-map.d.ts   root.d.ts       stringify.js        warning.js
74container.d.ts      fromJSON.d.ts       list.d.ts       parse.js        previous-map.js     root.js         symbols.js
75container.js        fromJSON.js     list.js         parser.js       processor.d.ts      rule.d.ts       terminal-highlight.js
76nvm uninstall &lt;version&gt;
77

OR

1npm start
2&gt; dataflow@0.1.0 start
3&gt; react-scripts start
4
5node:internal/modules/cjs/loader:488
6      throw e;
7  ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10    at new NodeError (node:internal/errors:371:5)
11    at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12    at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13    at resolveExports (node:internal/modules/cjs/loader:482:36)
14    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17    at Module.require (node:internal/modules/cjs/loader:999:19)
18    at require (node:internal/modules/cjs/helpers:102:18)
19    at Object.&lt;anonymous&gt; (/Users/juliantc/Desktop/ai-    studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25  "name": "postcss",
26  "version": "8.2.6",
27  "description": "Tool for transforming styles with JS plugins",
28  "engines": {
29    "node": "^10 || ^12 || &gt;=14"
30  },
31  "exports": {
32    ".": {
33      "require": "./lib/postcss.js",
34      "import": "./lib/postcss.mjs",
35      "types": "./lib/postcss.d.ts"
36    },
37    "./": "./"
38  },
39  "main": "./lib/postcss.js",
40  "types": "./lib/postcss.d.ts",
41  "keywords": [
42    "css",
43    "postcss",
44    "rework",
45    "preprocessor",
46    "parser",
47    "source map",
48    "transform",
49    "manipulation",
50    "transpiler"
51  ],
52  "funding": {
53    "type": "opencollective",
54    "url": "https://opencollective.com/postcss/"
55  },
56  "author": "Andrey Sitnik &lt;andrey@sitnik.ru&gt;",
57  "license": "MIT",
58  "homepage": "https://postcss.org/",
59  "repository": "postcss/postcss",
60  "dependencies": {
61    "colorette": "^1.2.1",
62    "nanoid": "^3.1.20",
63    "source-map": "^0.6.1"
64  },
65  "browser": {
66    "./lib/terminal-highlight": false,
67    "colorette": false,
68    "fs": false
69  }
70}at-rule.d.ts        css-syntax-error.d.ts   input.d.ts      map-generator.js    postcss.d.ts        processor.js        rule.js         tokenize.js
71at-rule.js      css-syntax-error.js input.js        node.d.ts       postcss.js      result.d.ts     stringifier.js      warn-once.js
72comment.d.ts        declaration.d.ts    lazy-result.d.ts    node.js         postcss.mjs     result.js       stringify.d.ts      warning.d.ts
73comment.js      declaration.js      lazy-result.js      parse.d.ts      previous-map.d.ts   root.d.ts       stringify.js        warning.js
74container.d.ts      fromJSON.d.ts       list.d.ts       parse.js        previous-map.js     root.js         symbols.js
75container.js        fromJSON.js     list.js         parser.js       processor.d.ts      rule.d.ts       terminal-highlight.js
76nvm uninstall &lt;version&gt;
77nvm uninstall v17.0.1
78

then install the LTS one which is v14.18.1

1npm start
2&gt; dataflow@0.1.0 start
3&gt; react-scripts start
4
5node:internal/modules/cjs/loader:488
6      throw e;
7  ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by &quot;exports&quot; in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10    at new NodeError (node:internal/errors:371:5)
11    at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12    at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13    at resolveExports (node:internal/modules/cjs/loader:482:36)
14    at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17    at Module.require (node:internal/modules/cjs/loader:999:19)
18    at require (node:internal/modules/cjs/helpers:102:18)
19    at Object.&lt;anonymous&gt; (/Users/juliantc/Desktop/ai-    studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20  code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25  "name": "postcss",
26  "version": "8.2.6",
27  "description": "Tool for transforming styles with JS plugins",
28  "engines": {
29    "node": "^10 || ^12 || &gt;=14"
30  },
31  "exports": {
32    ".": {
33      "require": "./lib/postcss.js",
34      "import": "./lib/postcss.mjs",
35      "types": "./lib/postcss.d.ts"
36    },
37    "./": "./"
38  },
39  "main": "./lib/postcss.js",
40  "types": "./lib/postcss.d.ts",
41  "keywords": [
42    "css",
43    "postcss",
44    "rework",
45    "preprocessor",
46    "parser",
47    "source map",
48    "transform",
49    "manipulation",
50    "transpiler"
51  ],
52  "funding": {
53    "type": "opencollective",
54    "url": "https://opencollective.com/postcss/"
55  },
56  "author": "Andrey Sitnik &lt;andrey@sitnik.ru&gt;",
57  "license": "MIT",
58  "homepage": "https://postcss.org/",
59  "repository": "postcss/postcss",
60  "dependencies": {
61    "colorette": "^1.2.1",
62    "nanoid": "^3.1.20",
63    "source-map": "^0.6.1"
64  },
65  "browser": {
66    "./lib/terminal-highlight": false,
67    "colorette": false,
68    "fs": false
69  }
70}at-rule.d.ts        css-syntax-error.d.ts   input.d.ts      map-generator.js    postcss.d.ts        processor.js        rule.js         tokenize.js
71at-rule.js      css-syntax-error.js input.js        node.d.ts       postcss.js      result.d.ts     stringifier.js      warn-once.js
72comment.d.ts        declaration.d.ts    lazy-result.d.ts    node.js         postcss.mjs     result.js       stringify.d.ts      warning.d.ts
73comment.js      declaration.js      lazy-result.js      parse.d.ts      previous-map.d.ts   root.d.ts       stringify.js        warning.js
74container.d.ts      fromJSON.d.ts       list.d.ts       parse.js        previous-map.js     root.js         symbols.js
75container.js        fromJSON.js     list.js         parser.js       processor.d.ts      rule.d.ts       terminal-highlight.js
76nvm uninstall &lt;version&gt;
77nvm uninstall v17.0.1
78nvm install --lts
79

This worked for me.

Source https://stackoverflow.com/questions/69693907

QUESTION

Can you compress angular image assets on build?

Asked 2022-Jan-15 at 12:13
What I want

I have very big images in my assets, which slows down the site by a lot for slower networks. (you can read more about the topic on this lighthouse linked page)

  • I would like to compress them at build time (ng build --prod).
  • For local development, it is irrelevant (ng serve).
  • Optimally I would like to generate multiple versions for different screen sizes (example.jpg → should become: example_x265.jpg, example_x128.jpg, ...)

What I have tried

The most promising guide I have found for that is this one here, which describes how to use the imagemin package in combination with the ngx-build-plus package.

Unfortunately, after following the tutorial, I get the following error:

1[error] TypeError: Cannot assign to read only property 'main.977fe6373cfd108d.js' of object '#&lt;Object&gt;'
2    at ImageminPlugin._callee2$ (/.../node_modules/imagemin-webpack-plugin/dist/index.js:264:48)
3    at tryCatch (/.../node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:62:40)
4     // ...
5

Is there any way to compress asset images on build?

1[error] TypeError: Cannot assign to read only property 'main.977fe6373cfd108d.js' of object '#&lt;Object&gt;'
2    at ImageminPlugin._callee2$ (/.../node_modules/imagemin-webpack-plugin/dist/index.js:264:48)
3    at tryCatch (/.../node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:62:40)
4     // ...
5Angular Version: 13.1.0
6

Note: I do not want to know how to upload images to third party storage solutions.
I specifically want to create a compressed version of my static assets on build time.

ANSWER

Answered 2021-Dec-19 at 22:55

I would never do that! because its against the convetions You should try Firebase storage, they give you 1 GB for free, and its easy to implement.

Source https://stackoverflow.com/questions/70415693

QUESTION

Convert a bytes iterable to an iterable of str, where each value is a line

Asked 2022-Jan-10 at 08:29

I have an iterable of bytes, such as

1bytes_iter = (
2    b'col_1,',
3    b'c',
4    b'ol_2\n1',
5    b',&quot;val',
6    b'ue&quot;\n',
7)
8

(but typically this would not be hard coded or available all at once, but supplied from a generator say) and I want to convert this to an iterable of str lines, where line breaks are unknown up front, but could be any of \r, \n or \r\n. So in this case would be:

1bytes_iter = (
2    b'col_1,',
3    b'c',
4    b'ol_2\n1',
5    b',&quot;val',
6    b'ue&quot;\n',
7)
8lines_iter = (
9    'col_1,col_2',
10    '1,&quot;value&quot;',
11)
12

(but again, just as an iterable, not so it's all in memory at once).

How can I do this?

Context: my aim is to then pass the iterable of str lines to csv.reader (that I think needs whole lines?), but I'm interested in this answer just in general.

ANSWER

Answered 2022-Jan-10 at 08:29

I used yield and re.finditer.

The yield expression is used when defining a generator function or an asynchronous generator function and thus can only be used in the body of a function definition. Using a yield expression in a function’s body causes that function to be a generator function

Return an iterator yielding match objects over all non-overlapping matches for the RE pattern in string. The string is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result.
If there are no groups, return a list of strings matching the whole pattern. If there is exactly one group, return a list of strings matching that group. If multiple groups are present, return a list of tuples of strings matching the groups. Non-capturing groups do not affect the form of the result.

The regular expression ([^\r\n]*)(\r\n|\r|\n)? can be divided into two parts to match (that is, two groups). The first group matches the data without \r and \n, and the second group matches \r, \n or \r\n.

1bytes_iter = (
2    b'col_1,',
3    b'c',
4    b'ol_2\n1',
5    b',&quot;val',
6    b'ue&quot;\n',
7)
8lines_iter = (
9    'col_1,col_2',
10    '1,&quot;value&quot;',
11)
12import re
13find_rule = re.compile(&quot;([^\r\n]*)(\r\n|\r|\n)?&quot;)
14
15
16def converter(byte_data):
17    left_d = &quot;&quot;
18    for d in byte_data:
19        # Used to save the previous match result in the `for` loop
20        prev_result = None
21        # Concatenate the last part of the previous data with the current data,
22        # used to deal with the case of `\r\n` being separated.
23        d = left_d + d.decode()
24        left_d = &quot;&quot;
25        # Using `find_rule.finditer` the last value(&quot;&quot;) will be invalid
26        for match_result in find_rule.finditer(d):
27            i = match_result.group()
28            if not i:
29                # The program comes to this point, indicating that i == &quot;&quot;, which is the last matching value
30                left_d, prev_result = prev_result.group(), None
31                continue
32            if prev_result:
33                if prev_result.group(2) is None:
34                    # The program goes here, represented as the last valid value matched
35                    left_d = prev_result.group()
36                else:
37                    # Returns the previous matched value
38                    yield prev_result.group()
39            # Save the current match result
40            prev_result = match_result
41
42    else:
43        yield left_d
44
45
46for i in (converter(iter((
47        b'col_1,\r',
48        b'\nc',
49        b'ol_2\n1',
50        b'\n,&quot;val;\r',
51        b'ue&quot;\n')))
52):
53    print(repr(i))
54
55

Output:

1bytes_iter = (
2    b'col_1,',
3    b'c',
4    b'ol_2\n1',
5    b',&quot;val',
6    b'ue&quot;\n',
7)
8lines_iter = (
9    'col_1,col_2',
10    '1,&quot;value&quot;',
11)
12import re
13find_rule = re.compile(&quot;([^\r\n]*)(\r\n|\r|\n)?&quot;)
14
15
16def converter(byte_data):
17    left_d = &quot;&quot;
18    for d in byte_data:
19        # Used to save the previous match result in the `for` loop
20        prev_result = None
21        # Concatenate the last part of the previous data with the current data,
22        # used to deal with the case of `\r\n` being separated.
23        d = left_d + d.decode()
24        left_d = &quot;&quot;
25        # Using `find_rule.finditer` the last value(&quot;&quot;) will be invalid
26        for match_result in find_rule.finditer(d):
27            i = match_result.group()
28            if not i:
29                # The program comes to this point, indicating that i == &quot;&quot;, which is the last matching value
30                left_d, prev_result = prev_result.group(), None
31                continue
32            if prev_result:
33                if prev_result.group(2) is None:
34                    # The program goes here, represented as the last valid value matched
35                    left_d = prev_result.group()
36                else:
37                    # Returns the previous matched value
38                    yield prev_result.group()
39            # Save the current match result
40            prev_result = match_result
41
42    else:
43        yield left_d
44
45
46for i in (converter(iter((
47        b'col_1,\r',
48        b'\nc',
49        b'ol_2\n1',
50        b'\n,&quot;val;\r',
51        b'ue&quot;\n')))
52):
53    print(repr(i))
54
55'col_1,\r\n'
56'col_2\n'
57'1\n'
58',&quot;val;\r'
59'ue&quot;\n'
60

Source https://stackoverflow.com/questions/70639443

QUESTION

Problem Updating to .Net 6 - Encrypting String

Asked 2021-Dec-20 at 23:09

I'm using a string Encryption/Decryption class similar to the one provided here as a solution.

This worked well for me in .Net 5.
Now I wanted to update my project to .Net 6.

When using .Net 6, the decrypted string does get cut off a certain point depending on the length of the input string.

▶️ To make it easy to debug/reproduce my issue, I created a public repro Repository here.

  • The encryption code is on purpose in a Standard 2.0 Project.
  • Referencing this project are both a .Net 6 as well as a .Net 5 Console project.

Both are calling the encryption methods with the exact same input of "12345678901234567890" with the path phrase of "nzv86ri4H2qYHqc&m6rL".

.Net 5 output: "12345678901234567890"
.Net 6 output: "1234567890123456"

The difference in length is 4.

I also looked at the breaking changes for .Net 6, but could not find something which guided me to a solution.

I'm glad for any suggestions regarding my issue, thanks!

Encryption Class

1public static class StringCipher
2{
3    // This constant is used to determine the keysize of the encryption algorithm in bits.
4    // We divide this by 8 within the code below to get the equivalent number of bytes.
5    private const int Keysize = 128;
6
7    // This constant determines the number of iterations for the password bytes generation function.
8    private const int DerivationIterations = 1000;
9
10    public static string Encrypt(string plainText, string passPhrase)
11    {
12        // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13        // so that the same Salt and IV values can be used when decrypting.  
14        var saltStringBytes = Generate128BitsOfRandomEntropy();
15        var ivStringBytes = Generate128BitsOfRandomEntropy();
16        var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18        {
19            var keyBytes = password.GetBytes(Keysize / 8);
20            using (var symmetricKey = Aes.Create())
21            {
22                symmetricKey.BlockSize = 128;
23                symmetricKey.Mode = CipherMode.CBC;
24                symmetricKey.Padding = PaddingMode.PKCS7;
25                using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26                {
27                    using (var memoryStream = new MemoryStream())
28                    {
29                        using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30                        {
31                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32                            cryptoStream.FlushFinalBlock();
33                            // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34                            var cipherTextBytes = saltStringBytes;
35                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37                            memoryStream.Close();
38                            cryptoStream.Close();
39                            return Convert.ToBase64String(cipherTextBytes);
40                        }
41                    }
42                }
43            }
44        }
45    }
46
47    public static string Decrypt(string cipherText, string passPhrase)
48    {
49        // Get the complete stream of bytes that represent:
50        // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51        var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52        // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53        var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54        // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55        var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56        // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57        var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60        {
61            var keyBytes = password.GetBytes(Keysize / 8);
62            using (var symmetricKey = Aes.Create())
63            {
64                symmetricKey.BlockSize = 128;
65                symmetricKey.Mode = CipherMode.CBC;
66                symmetricKey.Padding = PaddingMode.PKCS7;
67                using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68                {
69                    using (var memoryStream = new MemoryStream(cipherTextBytes))
70                    {
71                        using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72                        {
73                            var plainTextBytes = new byte[cipherTextBytes.Length];
74                            var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75                            memoryStream.Close();
76                            cryptoStream.Close();
77                            return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78                        }
79                    }
80                }
81            }
82        }
83    }
84
85    private static byte[] Generate128BitsOfRandomEntropy()
86    {
87        var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88        using (var rngCsp = RandomNumberGenerator.Create())
89        {
90            // Fill the array with cryptographically secure random bytes.
91            rngCsp.GetBytes(randomBytes);
92        }
93        return randomBytes;
94    }
95}
96

Calling code

1public static class StringCipher
2{
3    // This constant is used to determine the keysize of the encryption algorithm in bits.
4    // We divide this by 8 within the code below to get the equivalent number of bytes.
5    private const int Keysize = 128;
6
7    // This constant determines the number of iterations for the password bytes generation function.
8    private const int DerivationIterations = 1000;
9
10    public static string Encrypt(string plainText, string passPhrase)
11    {
12        // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13        // so that the same Salt and IV values can be used when decrypting.  
14        var saltStringBytes = Generate128BitsOfRandomEntropy();
15        var ivStringBytes = Generate128BitsOfRandomEntropy();
16        var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18        {
19            var keyBytes = password.GetBytes(Keysize / 8);
20            using (var symmetricKey = Aes.Create())
21            {
22                symmetricKey.BlockSize = 128;
23                symmetricKey.Mode = CipherMode.CBC;
24                symmetricKey.Padding = PaddingMode.PKCS7;
25                using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26                {
27                    using (var memoryStream = new MemoryStream())
28                    {
29                        using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30                        {
31                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32                            cryptoStream.FlushFinalBlock();
33                            // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34                            var cipherTextBytes = saltStringBytes;
35                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37                            memoryStream.Close();
38                            cryptoStream.Close();
39                            return Convert.ToBase64String(cipherTextBytes);
40                        }
41                    }
42                }
43            }
44        }
45    }
46
47    public static string Decrypt(string cipherText, string passPhrase)
48    {
49        // Get the complete stream of bytes that represent:
50        // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51        var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52        // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53        var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54        // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55        var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56        // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57        var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60        {
61            var keyBytes = password.GetBytes(Keysize / 8);
62            using (var symmetricKey = Aes.Create())
63            {
64                symmetricKey.BlockSize = 128;
65                symmetricKey.Mode = CipherMode.CBC;
66                symmetricKey.Padding = PaddingMode.PKCS7;
67                using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68                {
69                    using (var memoryStream = new MemoryStream(cipherTextBytes))
70                    {
71                        using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72                        {
73                            var plainTextBytes = new byte[cipherTextBytes.Length];
74                            var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75                            memoryStream.Close();
76                            cryptoStream.Close();
77                            return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78                        }
79                    }
80                }
81            }
82        }
83    }
84
85    private static byte[] Generate128BitsOfRandomEntropy()
86    {
87        var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88        using (var rngCsp = RandomNumberGenerator.Create())
89        {
90            // Fill the array with cryptographically secure random bytes.
91            rngCsp.GetBytes(randomBytes);
92        }
93        return randomBytes;
94    }
95}
96var input = &quot;12345678901234567890&quot;;
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
101
102var output = StringCipher.Decrypt(encrypted, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107

ANSWER

Answered 2021-Nov-10 at 10:25

The reason is this breaking change:

DeflateStream, GZipStream, and CryptoStream diverged from typical Stream.Read and Stream.ReadAsync behavior in two ways:

They didn't complete the read operation until either the buffer passed to the read operation was completely filled or the end of the stream was reached.

And the new behaviour is:

Starting in .NET 6, when Stream.Read or Stream.ReadAsync is called on one of the affected stream types with a buffer of length N, the operation completes when:

At least one byte has been read from the stream, or The underlying stream they wrap returns 0 from a call to its read, indicating no more data is available.

In your case you are affected because of this code in Decrypt method:

1public static class StringCipher
2{
3    // This constant is used to determine the keysize of the encryption algorithm in bits.
4    // We divide this by 8 within the code below to get the equivalent number of bytes.
5    private const int Keysize = 128;
6
7    // This constant determines the number of iterations for the password bytes generation function.
8    private const int DerivationIterations = 1000;
9
10    public static string Encrypt(string plainText, string passPhrase)
11    {
12        // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13        // so that the same Salt and IV values can be used when decrypting.  
14        var saltStringBytes = Generate128BitsOfRandomEntropy();
15        var ivStringBytes = Generate128BitsOfRandomEntropy();
16        var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18        {
19            var keyBytes = password.GetBytes(Keysize / 8);
20            using (var symmetricKey = Aes.Create())
21            {
22                symmetricKey.BlockSize = 128;
23                symmetricKey.Mode = CipherMode.CBC;
24                symmetricKey.Padding = PaddingMode.PKCS7;
25                using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26                {
27                    using (var memoryStream = new MemoryStream())
28                    {
29                        using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30                        {
31                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32                            cryptoStream.FlushFinalBlock();
33                            // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34                            var cipherTextBytes = saltStringBytes;
35                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37                            memoryStream.Close();
38                            cryptoStream.Close();
39                            return Convert.ToBase64String(cipherTextBytes);
40                        }
41                    }
42                }
43            }
44        }
45    }
46
47    public static string Decrypt(string cipherText, string passPhrase)
48    {
49        // Get the complete stream of bytes that represent:
50        // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51        var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52        // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53        var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54        // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55        var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56        // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57        var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60        {
61            var keyBytes = password.GetBytes(Keysize / 8);
62            using (var symmetricKey = Aes.Create())
63            {
64                symmetricKey.BlockSize = 128;
65                symmetricKey.Mode = CipherMode.CBC;
66                symmetricKey.Padding = PaddingMode.PKCS7;
67                using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68                {
69                    using (var memoryStream = new MemoryStream(cipherTextBytes))
70                    {
71                        using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72                        {
73                            var plainTextBytes = new byte[cipherTextBytes.Length];
74                            var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75                            memoryStream.Close();
76                            cryptoStream.Close();
77                            return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78                        }
79                    }
80                }
81            }
82        }
83    }
84
85    private static byte[] Generate128BitsOfRandomEntropy()
86    {
87        var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88        using (var rngCsp = RandomNumberGenerator.Create())
89        {
90            // Fill the array with cryptographically secure random bytes.
91            rngCsp.GetBytes(randomBytes);
92        }
93        return randomBytes;
94    }
95}
96var input = &quot;12345678901234567890&quot;;
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
101
102var output = StringCipher.Decrypt(encrypted, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
108{
109    var plainTextBytes = new byte[cipherTextBytes.Length];
110    var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
111    memoryStream.Close();
112    cryptoStream.Close();
113    return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
114}
115

You do not check how much bytes Read actually read and whether it read them all. You could get away with this in previous versions of .NET because as mentioned CryptoStream behaviour was different from other streams, and because your buffer length is enough to hold all data. However, this is no longer the case and you need to check it as you would do for other streams. Or even better - just use CopyTo:

1public static class StringCipher
2{
3    // This constant is used to determine the keysize of the encryption algorithm in bits.
4    // We divide this by 8 within the code below to get the equivalent number of bytes.
5    private const int Keysize = 128;
6
7    // This constant determines the number of iterations for the password bytes generation function.
8    private const int DerivationIterations = 1000;
9
10    public static string Encrypt(string plainText, string passPhrase)
11    {
12        // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13        // so that the same Salt and IV values can be used when decrypting.  
14        var saltStringBytes = Generate128BitsOfRandomEntropy();
15        var ivStringBytes = Generate128BitsOfRandomEntropy();
16        var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18        {
19            var keyBytes = password.GetBytes(Keysize / 8);
20            using (var symmetricKey = Aes.Create())
21            {
22                symmetricKey.BlockSize = 128;
23                symmetricKey.Mode = CipherMode.CBC;
24                symmetricKey.Padding = PaddingMode.PKCS7;
25                using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26                {
27                    using (var memoryStream = new MemoryStream())
28                    {
29                        using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30                        {
31                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32                            cryptoStream.FlushFinalBlock();
33                            // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34                            var cipherTextBytes = saltStringBytes;
35                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37                            memoryStream.Close();
38                            cryptoStream.Close();
39                            return Convert.ToBase64String(cipherTextBytes);
40                        }
41                    }
42                }
43            }
44        }
45    }
46
47    public static string Decrypt(string cipherText, string passPhrase)
48    {
49        // Get the complete stream of bytes that represent:
50        // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51        var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52        // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53        var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54        // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55        var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56        // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57        var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60        {
61            var keyBytes = password.GetBytes(Keysize / 8);
62            using (var symmetricKey = Aes.Create())
63            {
64                symmetricKey.BlockSize = 128;
65                symmetricKey.Mode = CipherMode.CBC;
66                symmetricKey.Padding = PaddingMode.PKCS7;
67                using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68                {
69                    using (var memoryStream = new MemoryStream(cipherTextBytes))
70                    {
71                        using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72                        {
73                            var plainTextBytes = new byte[cipherTextBytes.Length];
74                            var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75                            memoryStream.Close();
76                            cryptoStream.Close();
77                            return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78                        }
79                    }
80                }
81            }
82        }
83    }
84
85    private static byte[] Generate128BitsOfRandomEntropy()
86    {
87        var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88        using (var rngCsp = RandomNumberGenerator.Create())
89        {
90            // Fill the array with cryptographically secure random bytes.
91            rngCsp.GetBytes(randomBytes);
92        }
93        return randomBytes;
94    }
95}
96var input = &quot;12345678901234567890&quot;;
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
101
102var output = StringCipher.Decrypt(encrypted, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
108{
109    var plainTextBytes = new byte[cipherTextBytes.Length];
110    var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
111    memoryStream.Close();
112    cryptoStream.Close();
113    return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
114}
115using (var plainTextStream = new MemoryStream())
116{
117    cryptoStream.CopyTo(plainTextStream);
118    var plainTextBytes = plainTextStream.ToArray();
119    return Encoding.UTF8.GetString(plainTextBytes, 0, plainTextBytes.Length);
120} 
121

Or even better as another answer suggests, since you decrypt UTF8 text:

1public static class StringCipher
2{
3    // This constant is used to determine the keysize of the encryption algorithm in bits.
4    // We divide this by 8 within the code below to get the equivalent number of bytes.
5    private const int Keysize = 128;
6
7    // This constant determines the number of iterations for the password bytes generation function.
8    private const int DerivationIterations = 1000;
9
10    public static string Encrypt(string plainText, string passPhrase)
11    {
12        // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13        // so that the same Salt and IV values can be used when decrypting.  
14        var saltStringBytes = Generate128BitsOfRandomEntropy();
15        var ivStringBytes = Generate128BitsOfRandomEntropy();
16        var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18        {
19            var keyBytes = password.GetBytes(Keysize / 8);
20            using (var symmetricKey = Aes.Create())
21            {
22                symmetricKey.BlockSize = 128;
23                symmetricKey.Mode = CipherMode.CBC;
24                symmetricKey.Padding = PaddingMode.PKCS7;
25                using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26                {
27                    using (var memoryStream = new MemoryStream())
28                    {
29                        using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30                        {
31                            cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32                            cryptoStream.FlushFinalBlock();
33                            // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34                            var cipherTextBytes = saltStringBytes;
35                            cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36                            cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37                            memoryStream.Close();
38                            cryptoStream.Close();
39                            return Convert.ToBase64String(cipherTextBytes);
40                        }
41                    }
42                }
43            }
44        }
45    }
46
47    public static string Decrypt(string cipherText, string passPhrase)
48    {
49        // Get the complete stream of bytes that represent:
50        // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51        var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52        // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53        var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54        // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55        var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56        // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57        var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59        using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60        {
61            var keyBytes = password.GetBytes(Keysize / 8);
62            using (var symmetricKey = Aes.Create())
63            {
64                symmetricKey.BlockSize = 128;
65                symmetricKey.Mode = CipherMode.CBC;
66                symmetricKey.Padding = PaddingMode.PKCS7;
67                using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68                {
69                    using (var memoryStream = new MemoryStream(cipherTextBytes))
70                    {
71                        using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72                        {
73                            var plainTextBytes = new byte[cipherTextBytes.Length];
74                            var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75                            memoryStream.Close();
76                            cryptoStream.Close();
77                            return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78                        }
79                    }
80                }
81            }
82        }
83    }
84
85    private static byte[] Generate128BitsOfRandomEntropy()
86    {
87        var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88        using (var rngCsp = RandomNumberGenerator.Create())
89        {
90            // Fill the array with cryptographically secure random bytes.
91            rngCsp.GetBytes(randomBytes);
92        }
93        return randomBytes;
94    }
95}
96var input = &quot;12345678901234567890&quot;;
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
101
102var output = StringCipher.Decrypt(encrypted, &quot;nzv86ri4H2qYHqc&amp;m6rL&quot;);
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
108{
109    var plainTextBytes = new byte[cipherTextBytes.Length];
110    var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
111    memoryStream.Close();
112    cryptoStream.Close();
113    return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
114}
115using (var plainTextStream = new MemoryStream())
116{
117    cryptoStream.CopyTo(plainTextStream);
118    var plainTextBytes = plainTextStream.ToArray();
119    return Encoding.UTF8.GetString(plainTextBytes, 0, plainTextBytes.Length);
120} 
121using (var plainTextReader = new StreamReader(cryptoStream))
122{
123    return plainTextReader.ReadToEnd();
124}  
125

Source https://stackoverflow.com/questions/69911084

QUESTION

Ensure that an argument can be iterated twice

Asked 2021-Dec-19 at 12:43

Suppose I have the following function:

1def print_twice(x):
2    for i in x: print(i)
3    for i in x: print(i)
4

When I run:

1def print_twice(x):
2    for i in x: print(i)
3    for i in x: print(i)
4print_twice([1,2,3])
5

or:

1def print_twice(x):
2    for i in x: print(i)
3    for i in x: print(i)
4print_twice([1,2,3])
5print_twice((1,2,3))
6

I get the expected result: the numbers 1,2,3 are printed twice.

But when I run:

1def print_twice(x):
2    for i in x: print(i)
3    for i in x: print(i)
4print_twice([1,2,3])
5print_twice((1,2,3))
6print_twice(zip([1,2,3],[4,5,6]))
7

the pairs (1,4),(2,5),(3,6) are printed only once. Probably, this is because the zip returns a generator that terminates after one pass.

How can I modify the function print_twice such that it will correctly handle all inputs?

I could insert a line at the beginning of the function: x = list(x). But this might be inefficient in case x is already a list, a tuple, a range, or any other iterator that can be iterated more than once. Is there a more efficient solution?

ANSWER

Answered 2021-Dec-16 at 15:48

zip will return an iterator. Once unpacked, it cannot be unpacked again, it gets exhausted.

Maybe if you want to make sure that only zip objects get converted to list as you said it would work but it would not be efficient, you can check for it type:

1def print_twice(x):
2    for i in x: print(i)
3    for i in x: print(i)
4print_twice([1,2,3])
5print_twice((1,2,3))
6print_twice(zip([1,2,3],[4,5,6]))
7if isinstance(x, zip):
8  x = list(x)
9

Source https://stackoverflow.com/questions/70381559

QUESTION

Proper way to DI NSwag auto-generated client

Asked 2021-Dec-04 at 10:51

What is the preferred way to make a VS connected service (NSwag) injected into classes/controllers. I have found a lot of suggestions on the net to use this form:

1services.AddHttpClient&lt;IClient, Client&gt;((provider, client) =&gt;
2    {
3        client.BaseAddress = new System.Uri(&quot;https://some.baseurl/&quot;);
4    });
5

However this results in the error

1services.AddHttpClient&lt;IClient, Client&gt;((provider, client) =&gt;
2    {
3        client.BaseAddress = new System.Uri(&quot;https://some.baseurl/&quot;);
4    });
5{&quot;errorMessage&quot;:&quot;Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'.&quot;}
6

This comes from the auto-generated client class in obj, which seems to force a string BaseUrl in constructor, which of course the DI cannot resolve:

1services.AddHttpClient&lt;IClient, Client&gt;((provider, client) =&gt;
2    {
3        client.BaseAddress = new System.Uri(&quot;https://some.baseurl/&quot;);
4    });
5{&quot;errorMessage&quot;:&quot;Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'.&quot;}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8    BaseUrl = baseUrl;
9    _httpClient = httpClient;
10    _settings = new System.Lazy&lt;Newtonsoft.Json.JsonSerializerSettings&gt;(CreateSerializerSettings);
11}
12

This base URL is later forced into the url builder code, so it cannot really be bypassed. However, even the solutions on the net which use partial extensions to client classes seem to completely ignore baseUrl in auto-gen class (like here). As if it does not exist (which is weird, has the NSwag before generated different constructors?). The class is being generated via csproj:

1services.AddHttpClient&lt;IClient, Client&gt;((provider, client) =&gt;
2    {
3        client.BaseAddress = new System.Uri(&quot;https://some.baseurl/&quot;);
4    });
5{&quot;errorMessage&quot;:&quot;Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'.&quot;}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8    BaseUrl = baseUrl;
9    _httpClient = httpClient;
10    _settings = new System.Lazy&lt;Newtonsoft.Json.JsonSerializerSettings&gt;(CreateSerializerSettings);
11}
12  &lt;ItemGroup&gt;
13    &lt;OpenApiReference Include=&quot;OpenAPIs\swagger.json&quot; CodeGenerator=&quot;NSwagCSharp&quot; Namespace=&quot;xxx&quot; ClassName=&quot;Client&quot;&gt;
14      &lt;SourceUri&gt;https://localhost:44353/swagger/v1/swagger.json&lt;/SourceUri&gt;
15    &lt;/OpenApiReference&gt;
16  &lt;/ItemGroup&gt;
17

And this results in the targeted build call:

1services.AddHttpClient&lt;IClient, Client&gt;((provider, client) =&gt;
2    {
3        client.BaseAddress = new System.Uri(&quot;https://some.baseurl/&quot;);
4    });
5{&quot;errorMessage&quot;:&quot;Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'.&quot;}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8    BaseUrl = baseUrl;
9    _httpClient = httpClient;
10    _settings = new System.Lazy&lt;Newtonsoft.Json.JsonSerializerSettings&gt;(CreateSerializerSettings);
11}
12  &lt;ItemGroup&gt;
13    &lt;OpenApiReference Include=&quot;OpenAPIs\swagger.json&quot; CodeGenerator=&quot;NSwagCSharp&quot; Namespace=&quot;xxx&quot; ClassName=&quot;Client&quot;&gt;
14      &lt;SourceUri&gt;https://localhost:44353/swagger/v1/swagger.json&lt;/SourceUri&gt;
15    &lt;/OpenApiReference&gt;
16  &lt;/ItemGroup&gt;
172&gt;GenerateNSwagCSharp:
182&gt;  &quot;C:\.&lt;path&gt;./tools/Win/NSwag.exe&quot; openapi2csclient /className:Client /namespace:xxx /input:&quot;C:\&lt;projpath&gt;\OpenAPIs\swagger.json&quot; /output:&quot;obj\swaggerClient.cs&quot;
192&gt;NSwag command line tool for .NET 4.6.1+ WinX64, toolchain v13.13.2.0 (NJsonSchema v10.5.2.0 (Newtonsoft.Json v11.0.0.0))
20

So, how is this being done? potentially, without creating another proxy class for a proxy class, I would rather that DI handles my object lifetimes. I would also like to avoid NSwagStudio if possible and would like to keep the tooling supplied by VS.

ANSWER

Answered 2021-Sep-26 at 13:24

Ok, I actually solved this problem by poking around through OpenApiReference, but it requires manual modification of csproj file. Additional Options node has to be added to OpenApiReference item group, to instruct NSwag to NOT expose BaseUrl and to also generate an interface, which eases the work with setting up DI without additional code.

Visual Studio team should really add these two checkboxes to Connected Services screens/configuration for OpenAPI.

1services.AddHttpClient&lt;IClient, Client&gt;((provider, client) =&gt;
2    {
3        client.BaseAddress = new System.Uri(&quot;https://some.baseurl/&quot;);
4    });
5{&quot;errorMessage&quot;:&quot;Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'.&quot;}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8    BaseUrl = baseUrl;
9    _httpClient = httpClient;
10    _settings = new System.Lazy&lt;Newtonsoft.Json.JsonSerializerSettings&gt;(CreateSerializerSettings);
11}
12  &lt;ItemGroup&gt;
13    &lt;OpenApiReference Include=&quot;OpenAPIs\swagger.json&quot; CodeGenerator=&quot;NSwagCSharp&quot; Namespace=&quot;xxx&quot; ClassName=&quot;Client&quot;&gt;
14      &lt;SourceUri&gt;https://localhost:44353/swagger/v1/swagger.json&lt;/SourceUri&gt;
15    &lt;/OpenApiReference&gt;
16  &lt;/ItemGroup&gt;
172&gt;GenerateNSwagCSharp:
182&gt;  &quot;C:\.&lt;path&gt;./tools/Win/NSwag.exe&quot; openapi2csclient /className:Client /namespace:xxx /input:&quot;C:\&lt;projpath&gt;\OpenAPIs\swagger.json&quot; /output:&quot;obj\swaggerClient.cs&quot;
192&gt;NSwag command line tool for .NET 4.6.1+ WinX64, toolchain v13.13.2.0 (NJsonSchema v10.5.2.0 (Newtonsoft.Json v11.0.0.0))
20&lt;ItemGroup&gt;
21  &lt;OpenApiReference Include=&quot;OpenAPIs\swagger.json&quot; CodeGenerator=&quot;NSwagCSharp&quot; Namespace=&quot;xxx&quot; ClassName=&quot;Client&quot;&gt;
22    &lt;SourceUri&gt;https://localhost:44353/swagger/v1/swagger.json&lt;/SourceUri&gt;
23    &lt;Options&gt;/UseBaseUrl:false /GenerateClientInterfaces:true&lt;/Options&gt;
24  &lt;/OpenApiReference&gt;
25&lt;/ItemGroup&gt;
26

Now there is only an HttpClient constructor, andNSwag client proxy uses base address from it, so AddHttpClient works properly through DI.

Source https://stackoverflow.com/questions/69333244

QUESTION

How to alias generic types for decorators

Asked 2021-Nov-23 at 11:23
1import unittest
2from typing import *
3
4T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
7    def decorated_function(self: T) -&gt; None:
8        return func(self)
9    return decorated_function
10

Now I even have a generator that creates these decorators and want to shorthand these decorators. What type do I to the variables variables storing the decorator (simplified example omitting the generator).

1import unittest
2from typing import *
3
4T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
7    def decorated_function(self: T) -&gt; None:
8        return func(self)
9    return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11

This works, but is clunky. So the question is:

How can I alias this type to avoid having to write the the full signature?


Things that don't work:

1import unittest
2from typing import *
3
4T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
7    def decorated_function(self: T) -&gt; None:
8        return func(self)
9    return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13

Gives the error

1import unittest
2from typing import *
3
4T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
7    def decorated_function(self: T) -&gt; None:
8        return func(self)
9    return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13error: Type variable &quot;mypytest.T&quot; is unbound
14note: (Hint: Use &quot;Generic[T]&quot; or &quot;Protocol[T]&quot; base class to bind &quot;T&quot; inside a class)
15note: (Hint: Use &quot;T&quot; in function signature to bind &quot;T&quot; inside a function)
16

In contrast, I can use TD[T] as argument type for a function.

Just using my_decorate: TD = ... yields a --strict error

1import unittest
2from typing import *
3
4T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
7    def decorated_function(self: T) -&gt; None:
8        return func(self)
9    return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13error: Type variable &quot;mypytest.T&quot; is unbound
14note: (Hint: Use &quot;Generic[T]&quot; or &quot;Protocol[T]&quot; base class to bind &quot;T&quot; inside a class)
15note: (Hint: Use &quot;T&quot; in function signature to bind &quot;T&quot; inside a function)
16error: Missing type parameters for generic type &quot;TD&quot;
17

And it no longer detects wrong applications of my_decorate.

ANSWER

Answered 2021-Nov-23 at 10:59

What about this? It is shorter than the full signature:

1import unittest
2from typing import *
3
4T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
7    def decorated_function(self: T) -&gt; None:
8        return func(self)
9    return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13error: Type variable &quot;mypytest.T&quot; is unbound
14note: (Hint: Use &quot;Generic[T]&quot; or &quot;Protocol[T]&quot; base class to bind &quot;T&quot; inside a class)
15note: (Hint: Use &quot;T&quot; in function signature to bind &quot;T&quot; inside a function)
16error: Missing type parameters for generic type &quot;TD&quot;
17import unittest
18from typing import *
19
20T = TypeVar(&quot;T&quot;, bound=unittest.TestCase)
21
22
23def decorate(func: Callable[[T], None]) -&gt; Callable[[T], None]:
24    def decorated_function(self: T) -&gt; None:
25        return func(self)
26
27    return decorated_function
28
29
30decorator_variable: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
31
32U = Callable[[T], None]
33my_decorate: Callable[[U[T]], U[T]] = decorator_variable
34

Source https://stackoverflow.com/questions/69888695

Community Discussions contain sources that include Stack Exchange Network

Tutorials and Learning Resources in Generator

Tutorials and Learning Resources are not available at this moment for Generator

Share this Page

share link

Get latest updates on Generator