By continuing you indicate that you have read and agree to our Terms of service and Privacy policy
By continuing you indicate that you have read and agree to our Terms of service and Privacy policy
Popular Releases
Popular Libraries
New Libraries
Top Authors
Trending Kits
Trending Discussions
Learning
Explore Related Topics
generator-jhipster | v7.8.1 |
openapi-generator | v6.0.0-beta released |
auto | AutoValue 1.9 |
github-profile-readme-generator | GPRG v1.2.0 |
typedoc | v0.22.15 |
generator-jhipster v7.8.1 |
openapi-generator v6.0.0-beta released |
auto AutoValue 1.9 |
github-profile-readme-generator GPRG v1.2.0 |
typedoc v0.22.15 |
by anuraghazra javascript
38038 MIT
:zap: Dynamically generated stats for your github readmes
by jhipster javascript
19466 Apache-2.0
JHipster is a development platform to quickly generate, develop, & deploy modern web applications & microservice architectures.
by OpenAPITools java
11886 Apache-2.0
OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)
by davidshimjs javascript
10529 MIT
Cross-browser QRCode generator for javascript
by qrohlf javascript
9871 GPL-3.0
Algorithmically generated triangle art
by google java
9703 Apache-2.0
A collection of source code generators for Java.
by rahuldkjain javascript
9217 Apache-2.0
🚀 Generate GitHub profile README easily with the latest add-ons like visitors count, GitHub stats, etc using minimal UI.
by kefranabg javascript
8929 MIT
📄 CLI that generates beautiful README.md files
by angular-fullstack javascript
6129
Yeoman generator for an Angular app with an Express server
by anuraghazra javascript
38038 MIT
:zap: Dynamically generated stats for your github readmes
by jhipster javascript
19466 Apache-2.0
JHipster is a development platform to quickly generate, develop, & deploy modern web applications & microservice architectures.
by OpenAPITools java
11886 Apache-2.0
OpenAPI Generator allows generation of API client libraries (SDK generation), server stubs, documentation and configuration automatically given an OpenAPI Spec (v2, v3)
by davidshimjs javascript
10529 MIT
Cross-browser QRCode generator for javascript
by qrohlf javascript
9871 GPL-3.0
Algorithmically generated triangle art
by google java
9703 Apache-2.0
A collection of source code generators for Java.
by rahuldkjain javascript
9217 Apache-2.0
🚀 Generate GitHub profile README easily with the latest add-ons like visitors count, GitHub stats, etc using minimal UI.
by kefranabg javascript
8929 MIT
📄 CLI that generates beautiful README.md files
by angular-fullstack javascript
6129
Yeoman generator for an Angular app with an Express server
by anuraghazra javascript
38038 MIT
:zap: Dynamically generated stats for your github readmes
by rahuldkjain javascript
9217 Apache-2.0
🚀 Generate GitHub profile README easily with the latest add-ons like visitors count, GitHub stats, etc using minimal UI.
by orhun rust
3750 GPL-3.0
A highly customizable Changelog Generator that follows Conventional Commit specifications ⛰️
by SwiftDocOrg swift
1614 MIT
A documentation generator for Swift projects
by Pavo-IM swift
1446
OpenCore Config Generator
by DenverCoder1 php
1039 MIT
🔥 Stay motivated and show off your contribution streak! 🌟 Display your total contributions, current streak, and longest streak on your GitHub profile README
by microsoft csharp
979 MIT
A source generator to add a user-defined set of Win32 P/Invoke methods and supporting types to a C# project.
by hannoeru typescript
687 MIT
File system based route generator for ⚡️Vite
by wwebdev javascript
631
A generator for section separators
by anuraghazra javascript
38038 MIT
:zap: Dynamically generated stats for your github readmes
by rahuldkjain javascript
9217 Apache-2.0
🚀 Generate GitHub profile README easily with the latest add-ons like visitors count, GitHub stats, etc using minimal UI.
by orhun rust
3750 GPL-3.0
A highly customizable Changelog Generator that follows Conventional Commit specifications ⛰️
by SwiftDocOrg swift
1614 MIT
A documentation generator for Swift projects
by Pavo-IM swift
1446
OpenCore Config Generator
by DenverCoder1 php
1039 MIT
🔥 Stay motivated and show off your contribution streak! 🌟 Display your total contributions, current streak, and longest streak on your GitHub profile README
by microsoft csharp
979 MIT
A source generator to add a user-defined set of Win32 P/Invoke methods and supporting types to a C# project.
by hannoeru typescript
687 MIT
File system based route generator for ⚡️Vite
by wwebdev javascript
631
A generator for section separators
1
30 Libraries
337
2
29 Libraries
359
3
27 Libraries
116
4
23 Libraries
848
5
18 Libraries
46
6
16 Libraries
13918
7
13 Libraries
502
8
11 Libraries
20896
9
10 Libraries
750
10
10 Libraries
61
If you are a designer you design logos and mockups, you use the color picker every day when choosing a new project's color scheme. By mixing the three primary colors, we can form any color we like. Red(R), Green(G), Blue(B) are the three primary colors we know. Using this tool, you can experiment with custom colors on the web and adjust them according to your preference. In addition, different color formats can be converted easily supported by JavaScript. These libraries allow building color picker and generator client-side applications in minutes.
Following are the steps to be followed for building Color Picker and Code Generator,
1. Color Picker
2. Code Generator Code Generator These libraries are used to generate the code based on the color.
Color Picker These libraries are used to pick the color.
As we are in the digital era, real-time video games are ruling the young generation. Tank games are one of the addictive games of this generation. The objective of this game is to destroy the enemy's tank with our tank, which will decrease the energy level of the opponent. Similarly, our energy level will be reduced when the opponent attacks us with their tank. The attacking capacity ultimately depends on the energy level. The more is the energy level, the high is the attacking capacity.
Following are the steps to be followed for building Tank Fight Game,
1.Graphic design & Sound effects
2.Firing and exploiting the tanks
3.Customize control over keyboard
4.Multi-player
5.3D Tank game Customize control over keyboard Key mapper is an open-source library that allows users to use a key or combination of keys to perform a specific action, which can be used for navigating and shooting. The below libraries can help you to create your control. Graphic design & Sound effects Listed below libraries help in creating the best graphic design and sound effects for gaming applications using python, C#, JavaScript, which can be used to design tanks, animate the movement of tanks, explosion of tanks, and display energy level bars. Firing and exploiting the tanks Random module, an open-source library, generates a random number provided the range, which can be used for firing and exploiting the tanks, decides the playing turn at the start of every game. 3D Tank game The tank game can be built in 3D by using the below library. Multi-player Tank game can be played as multiplayer by using the below libraries. Multiplayers will play on the game field they shoot each other. The more is the energy level. The high is the attacking capacity.
Build custom programs to create character generators for video games and applications by using these open-source character generator libraries. Character Generator is used to generate characters in video games, smartphone and desktop applications, websites, etc. in various different patterns. You can create 3D and 2D characters for your games and application by using a variety of models, textures and animations. With the help of character generator, you can allow the user to determine aspects of a digital character like attributes, skills, strengths and weaknesses, life events, etc. Also, you can allow them to add animation, text, as well as audio and video effects. Mentioned below are some of the best and most trending reusable character generator libraries that you can implement in your next project:
If you are a designer you design logos and mockups, you use the color picker every day when choosing a new project's color scheme. By mixing the three primary colors, we can form any color we like. Red(R), Green(G), Blue(B) are the three primary colors we know. Using this tool, you can experiment with custom colors on the web and adjust them according to your preference. In addition, different color formats can be converted easily supported by JavaScript. These libraries allow building color picker and generator client-side applications in minutes.
Following are the steps to be followed for building Color Picker and Code Generator,
1. Color Picker
2. Code Generator Code Generator These libraries are used to generate the code based on the color.
Color Picker These libraries are used to pick the color.
As we are in the digital era, real-time video games are ruling the young generation. Tank games are one of the addictive games of this generation. The objective of this game is to destroy the enemy's tank with our tank, which will decrease the energy level of the opponent. Similarly, our energy level will be reduced when the opponent attacks us with their tank. The attacking capacity ultimately depends on the energy level. The more is the energy level, the high is the attacking capacity.
Following are the steps to be followed for building Tank Fight Game,
1.Graphic design & Sound effects
2.Firing and exploiting the tanks
3.Customize control over keyboard
4.Multi-player
5.3D Tank game Customize control over keyboard Key mapper is an open-source library that allows users to use a key or combination of keys to perform a specific action, which can be used for navigating and shooting. The below libraries can help you to create your control. Graphic design & Sound effects Listed below libraries help in creating the best graphic design and sound effects for gaming applications using python, C#, JavaScript, which can be used to design tanks, animate the movement of tanks, explosion of tanks, and display energy level bars. Firing and exploiting the tanks Random module, an open-source library, generates a random number provided the range, which can be used for firing and exploiting the tanks, decides the playing turn at the start of every game. 3D Tank game The tank game can be built in 3D by using the below library. Multi-player Tank game can be played as multiplayer by using the below libraries. Multiplayers will play on the game field they shoot each other. The more is the energy level. The high is the attacking capacity.
Build custom programs to create character generators for video games and applications by using these open-source character generator libraries. Character Generator is used to generate characters in video games, smartphone and desktop applications, websites, etc. in various different patterns. You can create 3D and 2D characters for your games and application by using a variety of models, textures and animations. With the help of character generator, you can allow the user to determine aspects of a digital character like attributes, skills, strengths and weaknesses, life events, etc. Also, you can allow them to add animation, text, as well as audio and video effects. Mentioned below are some of the best and most trending reusable character generator libraries that you can implement in your next project:
QUESTION
How can I make an object with an interface like a random number generator, but that actually generates a specified sequence?
Asked 2022-Mar-31 at 13:47I'd like to construct an object that works like a random number generator, but generates numbers in a specified sequence.
1# a random number generator
2rng = lambda : np.random.randint(2,20)//2
3
4# a non-random number generator
5def nrng():
6 numbers = np.arange(1,10.5,0.5)
7 for i in range(len(numbers)):
8 yield numbers[i]
9
10for j in range(10):
11 print('random number', rng())
12 print('non-random number', nrng())
13
The issue with the code above that I cannot call nrng
in the last line because it is a generator. I know that the most straightforward way to rewrite the code above is to simply loop over the non-random numbers instead of defining the generator. I would prefer getting the example above to work because I am working with a large chunk of code that include a function that accepts a random number generator as an argument, and I would like to add the functionality to pass non-random number sequences without rewriting the entire code.
EDIT: I see some confusion in the comments. I am aware that python's random number generators generate pseudo-random numbers. This post is about replacing a pseudo-random-number generator by a number generator that generates numbers from a non-random, user-specified sequence (e.g., a generator that generates the number sequence 1,1,2,2,1,0,1
if I want it to).
ANSWER
Answered 2022-Mar-29 at 00:47You can call next()
with a generator or iterator as an argument to withdraw exactly one element from it. Saving the generator to a variable beforehand allows you to do this multiple times.
1# a random number generator
2rng = lambda : np.random.randint(2,20)//2
3
4# a non-random number generator
5def nrng():
6 numbers = np.arange(1,10.5,0.5)
7 for i in range(len(numbers)):
8 yield numbers[i]
9
10for j in range(10):
11 print('random number', rng())
12 print('non-random number', nrng())
13# make it a generator
14def _rng():
15 while True:
16 yield np.random.randint(2,20)//2
17
18# a non-random number generator
19def _nrng():
20 numbers = np.arange(1,10.5,0.5)
21 for i in range(len(numbers)):
22 yield numbers[i]
23
24rng = _rng()
25nrng = _nrng()
26for j in range(10):
27 print('random number', next(rng))
28 print('non-random number', next(nrng))
29
QUESTION
Why is `np.sum(range(N))` very slow?
Asked 2022-Mar-29 at 14:31I saw a video about speed of loops in python, where it was explained that doing sum(range(N))
is much faster than manually looping through range
and adding the variables together, since the former runs in C due to built-in functions being used, while in the latter the summation is done in (slow) python. I was curious what happens when adding numpy
to the mix. As I expected np.sum(np.arange(N))
is the fastest, but sum(np.arange(N))
and np.sum(range(N))
are even slower than doing the naive for loop.
Why is this?
Here's the script I used to test, some comments about the supposed cause of slowing done where I know (taken mostly from the video) and the results I got on my machine (python 3.10.0, numpy 1.21.2):
updated script:
1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8 s = 0
9 i = 0
10 while i < N: # condition is checked in python
11 s += i
12 i += 1 # both additions are done in python
13 return s
14
15def sum1(N = N):
16 s = 0
17 for i in range(N): # increment in C
18 s += i # addition in python
19 return s
20
21def sum2(N = N):
22 return sum(range(N)) # everything in C
23
24def sum3(N = N):
25 return sum(list(range(N)))
26
27def sum4(N = N):
28 return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31 # much faster np.array conversion
32 return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35 # much faster np.array conversion
36 return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39 # possibly slow conversion to Py_long from np.int
40 return sum(np.arange(N))
41
42def sum7(N = N):
43 # list returns a list of np.int-s
44 return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47 # tolist conversion to python int seems faster than the implicit conversion
48 # in sum(list()) (tolist returns a list of python int-s)
49 return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52 return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55 return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58 return np.array(range(N))
59
60def array_dtype(N = N):
61 return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64 # np.sum's source code mentions to use fromiter to convert from generators
65 return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop: {timeit(sum0, number = repetition)}")
68print(f"for loop: {timeit(sum1, number = repetition)}")
69print(f"sum_range: {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist: {timeit(sum3, number = repetition)}")
71print(f"npsum_range: {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange: {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2: {timeit(sum5, number = repetition)}")
74print(f"sum_arange: {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange: {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist: {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange: {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum: {timeit(sum9, number = repetition)}")
79print(f"array_basic: {timeit(array_basic, number = repetition)}")
80print(f"array_dtype: {timeit(array_dtype, number = repetition)}")
81print(f"array_iter: {timeit(array_iter, number = repetition)}")
82
83print(f"npsumarangeREP: {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP: {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop: 11.493371912998555
89# for loop: 7.385945574002108
90# sum_range: 2.4605720699983067
91# sum_rangelist: 4.509678105998319
92# npsum_range: 11.85120212900074
93# npsum_iterrange: 4.464334709002287
94# npsum_iterrangev2: 4.498494338993623
95# sum_arange: 9.537815956995473
96# sum_list_arange: 13.290120724996086
97# sum_arange_tolist: 5.231948580003518
98# npsum_arange: 0.241889145996538
99# nparangenpsum: 0.21876695199898677
100# array_basic: 11.736577274998126
101# array_dtype: 8.71628468400013
102# array_iter: 4.303306431000237
103# npsumarangeREP: 21.240833958996518
104# npsumarangeREP: 16.690092379001726
105
106
ANSWER
Answered 2021-Oct-16 at 17:42From the cpython source code for sum
sum initially seems to attempt a fast path that assumes all inputs are the same type. If that fails it will just iterate:
1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8 s = 0
9 i = 0
10 while i < N: # condition is checked in python
11 s += i
12 i += 1 # both additions are done in python
13 return s
14
15def sum1(N = N):
16 s = 0
17 for i in range(N): # increment in C
18 s += i # addition in python
19 return s
20
21def sum2(N = N):
22 return sum(range(N)) # everything in C
23
24def sum3(N = N):
25 return sum(list(range(N)))
26
27def sum4(N = N):
28 return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31 # much faster np.array conversion
32 return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35 # much faster np.array conversion
36 return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39 # possibly slow conversion to Py_long from np.int
40 return sum(np.arange(N))
41
42def sum7(N = N):
43 # list returns a list of np.int-s
44 return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47 # tolist conversion to python int seems faster than the implicit conversion
48 # in sum(list()) (tolist returns a list of python int-s)
49 return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52 return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55 return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58 return np.array(range(N))
59
60def array_dtype(N = N):
61 return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64 # np.sum's source code mentions to use fromiter to convert from generators
65 return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop: {timeit(sum0, number = repetition)}")
68print(f"for loop: {timeit(sum1, number = repetition)}")
69print(f"sum_range: {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist: {timeit(sum3, number = repetition)}")
71print(f"npsum_range: {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange: {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2: {timeit(sum5, number = repetition)}")
74print(f"sum_arange: {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange: {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist: {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange: {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum: {timeit(sum9, number = repetition)}")
79print(f"array_basic: {timeit(array_basic, number = repetition)}")
80print(f"array_dtype: {timeit(array_dtype, number = repetition)}")
81print(f"array_iter: {timeit(array_iter, number = repetition)}")
82
83print(f"npsumarangeREP: {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP: {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop: 11.493371912998555
89# for loop: 7.385945574002108
90# sum_range: 2.4605720699983067
91# sum_rangelist: 4.509678105998319
92# npsum_range: 11.85120212900074
93# npsum_iterrange: 4.464334709002287
94# npsum_iterrangev2: 4.498494338993623
95# sum_arange: 9.537815956995473
96# sum_list_arange: 13.290120724996086
97# sum_arange_tolist: 5.231948580003518
98# npsum_arange: 0.241889145996538
99# nparangenpsum: 0.21876695199898677
100# array_basic: 11.736577274998126
101# array_dtype: 8.71628468400013
102# array_iter: 4.303306431000237
103# npsumarangeREP: 21.240833958996518
104# npsumarangeREP: 16.690092379001726
105
106/* Fast addition by keeping temporary sums in C instead of new Python objects.
107 Assumes all inputs are the same type. If the assumption fails, default
108 to the more general routine.
109*/
110
I'm not entirely certain what is happening under the hood, but it is likely the repeated creation/conversion of C types to Python objects that is causing these slow-downs. It's worth noting that both sum
and range
are implemented in C.
This next bit is not really an answer to the question, but I wondered if we could speed up sum
for python range
s as range
is quite a smart object.
To do this I've used functools.singledispatch
to override the built-in sum
function specifically for the range
type; then implemented a small function to calculate the sum of an arithmetic progression.
1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8 s = 0
9 i = 0
10 while i < N: # condition is checked in python
11 s += i
12 i += 1 # both additions are done in python
13 return s
14
15def sum1(N = N):
16 s = 0
17 for i in range(N): # increment in C
18 s += i # addition in python
19 return s
20
21def sum2(N = N):
22 return sum(range(N)) # everything in C
23
24def sum3(N = N):
25 return sum(list(range(N)))
26
27def sum4(N = N):
28 return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31 # much faster np.array conversion
32 return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35 # much faster np.array conversion
36 return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39 # possibly slow conversion to Py_long from np.int
40 return sum(np.arange(N))
41
42def sum7(N = N):
43 # list returns a list of np.int-s
44 return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47 # tolist conversion to python int seems faster than the implicit conversion
48 # in sum(list()) (tolist returns a list of python int-s)
49 return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52 return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55 return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58 return np.array(range(N))
59
60def array_dtype(N = N):
61 return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64 # np.sum's source code mentions to use fromiter to convert from generators
65 return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop: {timeit(sum0, number = repetition)}")
68print(f"for loop: {timeit(sum1, number = repetition)}")
69print(f"sum_range: {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist: {timeit(sum3, number = repetition)}")
71print(f"npsum_range: {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange: {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2: {timeit(sum5, number = repetition)}")
74print(f"sum_arange: {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange: {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist: {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange: {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum: {timeit(sum9, number = repetition)}")
79print(f"array_basic: {timeit(array_basic, number = repetition)}")
80print(f"array_dtype: {timeit(array_dtype, number = repetition)}")
81print(f"array_iter: {timeit(array_iter, number = repetition)}")
82
83print(f"npsumarangeREP: {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP: {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop: 11.493371912998555
89# for loop: 7.385945574002108
90# sum_range: 2.4605720699983067
91# sum_rangelist: 4.509678105998319
92# npsum_range: 11.85120212900074
93# npsum_iterrange: 4.464334709002287
94# npsum_iterrangev2: 4.498494338993623
95# sum_arange: 9.537815956995473
96# sum_list_arange: 13.290120724996086
97# sum_arange_tolist: 5.231948580003518
98# npsum_arange: 0.241889145996538
99# nparangenpsum: 0.21876695199898677
100# array_basic: 11.736577274998126
101# array_dtype: 8.71628468400013
102# array_iter: 4.303306431000237
103# npsumarangeREP: 21.240833958996518
104# npsumarangeREP: 16.690092379001726
105
106/* Fast addition by keeping temporary sums in C instead of new Python objects.
107 Assumes all inputs are the same type. If the assumption fails, default
108 to the more general routine.
109*/
110from functools import singledispatch
111
112def sum_range(range_, /, start=0):
113 """Overloaded `sum` for range, compute arithmetic sum"""
114 n = len(range_)
115 if not n:
116 return start
117 return int(start + (n * (range_[0] + range_[-1]) / 2))
118
119sum = singledispatch(sum)
120sum.register(range, sum_range)
121
122def test():
123 """
124 >>> sum(range(0, 100))
125 4950
126 >>> sum(range(0, 10, 2))
127 20
128 >>> sum(range(0, 9, 2))
129 20
130 >>> sum(range(0, -10, -1))
131 -45
132 >>> sum(range(-10, 10))
133 -10
134 >>> sum(range(-1, -100, -2))
135 -2500
136 >>> sum(range(0, 10, 100))
137 0
138 >>> sum(range(0, 0))
139 0
140 >>> sum(range(0, 100), 50)
141 5000
142 >>> sum(range(0, 0), 10)
143 10
144 """
145
146if __name__ == "__main__":
147 import doctest
148 doctest.testmod()
149
I'm not sure if this is complete, but it's definitely faster than looping.
QUESTION
How would you implement a lazy "range factory" for C++20 ranges that just calls a generator function?
Asked 2022-Feb-17 at 17:10I like the idea of the lazy ranges you can make with std::views::iota
but was surprised to see that iota
is currently the only thing like it in the standard; it is the only "range factory" besides views::single
and views::empty
. There is not currently, for example, the equivalent of std::generate
as a range factory.
I note however it is trivial to implement the semantics of generate
by using a transform view on iota and just ignoring the value iota passes to transform i.e.
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25
My questions is what would be "the real way" to implement something like this? To make a range view object that is pipeable that does not just use iota
.
I tried inheriting from ranges::view_interface
-- no idea if this is the correct approach -- and just having it return a dummy iterator that calls a generator function but my code doesn't work because of the part where it needs to pipe the range view to std::views::take
in order to not cause an infinite loop. The object I define here does not end up being pipeable.
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32 using value_type = decltype(std::declval<F>()());
33
34 class iterator {
35 const F* gen_func_;
36 public:
37 iterator(const F* f) : gen_func_(f)
38 {}
39
40 value_type operator*() const {
41 return (*gen_func_)();
42 }
43
44 bool operator!=(const iterator&) {
45 return true;
46 }
47
48 iterator& operator++() {
49 return *this;
50 }
51 };
52
53 F generator_func_;
54
55public:
56
57 generate2(const F& f) : generator_func_(f) {
58 }
59
60 iterator begin() {
61 return iterator(&generator_func_);
62 }
63
64 iterator end() {
65 return iterator(nullptr);
66 }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74 auto d6 = []() {
75 static std::uniform_int_distribution<> dist(1, 6);
76 return dist(rng);
77 };
78
79 // the following doesnt compile because of the pipe...
80 for (int v : generate2(d6) | std::views::take(10)) {
81 std::cout << v << ' ';
82 }
83 std::cout << '\n';
84}
85
ANSWER
Answered 2022-Feb-17 at 17:10The reason why generate2
cannot work is that it does not model the range
concept, that is, the type returned by its begin()
does not model input_iterator
, because input_iterator
requires difference_type
and value_type
to exist and i++
is a valid expression.
In addition, your iterator does not satisfy sentinel_for<iterator>
, which means that it cannot serve as its own sentinel, because sentinel_for
requires semiregular
which requires default_initializable
, so you also need to add default constructors for it.
You also need to rewrite bool operator!=(...)
to bool operator==(...) const
since operator!=
does not reverse synthesize operator==
. But it's easier to just use default_sentinel_t
as sentinel in your case.
if you add them to iterator
you will find the code will be well-formed:
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32 using value_type = decltype(std::declval<F>()());
33
34 class iterator {
35 const F* gen_func_;
36 public:
37 iterator(const F* f) : gen_func_(f)
38 {}
39
40 value_type operator*() const {
41 return (*gen_func_)();
42 }
43
44 bool operator!=(const iterator&) {
45 return true;
46 }
47
48 iterator& operator++() {
49 return *this;
50 }
51 };
52
53 F generator_func_;
54
55public:
56
57 generate2(const F& f) : generator_func_(f) {
58 }
59
60 iterator begin() {
61 return iterator(&generator_func_);
62 }
63
64 iterator end() {
65 return iterator(nullptr);
66 }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74 auto d6 = []() {
75 static std::uniform_int_distribution<> dist(1, 6);
76 return dist(rng);
77 };
78
79 // the following doesnt compile because of the pipe...
80 for (int v : generate2(d6) | std::views::take(10)) {
81 std::cout << v << ' ';
82 }
83 std::cout << '\n';
84}
85class iterator {
86 public:
87 using value_type = decltype(std::declval<F>()());
88 using difference_type = std::ptrdiff_t;
89 iterator() = default;
90 void operator++(int);
91 bool operator==(const iterator&) const {
92 return false;
93 }
94 // ...
95};
96
However, the operator*()
of iterator
does not meet the requirements of equality-preserving, that is to say, the results obtained by the two calls before and after are not equal, which means that this will be undefined behavior.
You can refer to the implementation of ranges::istream_view
to use a member variable to cache each generated result, then you only need to return the cached value each time iterator::operator*()
is called.
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32 using value_type = decltype(std::declval<F>()());
33
34 class iterator {
35 const F* gen_func_;
36 public:
37 iterator(const F* f) : gen_func_(f)
38 {}
39
40 value_type operator*() const {
41 return (*gen_func_)();
42 }
43
44 bool operator!=(const iterator&) {
45 return true;
46 }
47
48 iterator& operator++() {
49 return *this;
50 }
51 };
52
53 F generator_func_;
54
55public:
56
57 generate2(const F& f) : generator_func_(f) {
58 }
59
60 iterator begin() {
61 return iterator(&generator_func_);
62 }
63
64 iterator end() {
65 return iterator(nullptr);
66 }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74 auto d6 = []() {
75 static std::uniform_int_distribution<> dist(1, 6);
76 return dist(rng);
77 };
78
79 // the following doesnt compile because of the pipe...
80 for (int v : generate2(d6) | std::views::take(10)) {
81 std::cout << v << ' ';
82 }
83 std::cout << '\n';
84}
85class iterator {
86 public:
87 using value_type = decltype(std::declval<F>()());
88 using difference_type = std::ptrdiff_t;
89 iterator() = default;
90 void operator++(int);
91 bool operator==(const iterator&) const {
92 return false;
93 }
94 // ...
95};
96template<typename F>
97class generate2 : public std::ranges::view_interface<generate2<F>> {
98 public:
99 auto begin() {
100 value_ = generator_func_();
101 return iterator{*this};
102 }
103
104 std::default_sentinel_t end() const noexcept { return std::default_sentinel; }
105
106 class iterator {
107 public:
108 //...
109 value_type operator*() const {
110 return parent_->value_;
111 }
112 private:
113 generate2* parent_;
114 };
115
116 private:
117 F generator_func_;
118 std::remove_cvref_t<std::invoke_result_t<F&>> value_;
119};
120
QUESTION
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in the package.json of a module in node_modules
Asked 2022-Jan-31 at 17:22This is a React web app. When I run
1npm start
2
This error occurred
1npm start
2> dataflow@0.1.0 start
3> react-scripts start
4
5node:internal/modules/cjs/loader:488
6 throw e;
7 ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10 at new NodeError (node:internal/errors:371:5)
11 at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12 at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13 at resolveExports (node:internal/modules/cjs/loader:482:36)
14 at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16 at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17 at Module.require (node:internal/modules/cjs/loader:999:19)
18 at require (node:internal/modules/cjs/helpers:102:18)
19 at Object.<anonymous> (/Users/juliantc/Desktop/ai- studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20 code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24
This error only occurs when I run this on this specific computer, which I do not have superuser access to. It works on other computers.
For reference, this is ./node_modules/postcss-safe-parser/node_modules/postcss/package.json
1npm start
2> dataflow@0.1.0 start
3> react-scripts start
4
5node:internal/modules/cjs/loader:488
6 throw e;
7 ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10 at new NodeError (node:internal/errors:371:5)
11 at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12 at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13 at resolveExports (node:internal/modules/cjs/loader:482:36)
14 at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16 at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17 at Module.require (node:internal/modules/cjs/loader:999:19)
18 at require (node:internal/modules/cjs/helpers:102:18)
19 at Object.<anonymous> (/Users/juliantc/Desktop/ai- studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20 code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25 "name": "postcss",
26 "version": "8.2.6",
27 "description": "Tool for transforming styles with JS plugins",
28 "engines": {
29 "node": "^10 || ^12 || >=14"
30 },
31 "exports": {
32 ".": {
33 "require": "./lib/postcss.js",
34 "import": "./lib/postcss.mjs",
35 "types": "./lib/postcss.d.ts"
36 },
37 "./": "./"
38 },
39 "main": "./lib/postcss.js",
40 "types": "./lib/postcss.d.ts",
41 "keywords": [
42 "css",
43 "postcss",
44 "rework",
45 "preprocessor",
46 "parser",
47 "source map",
48 "transform",
49 "manipulation",
50 "transpiler"
51 ],
52 "funding": {
53 "type": "opencollective",
54 "url": "https://opencollective.com/postcss/"
55 },
56 "author": "Andrey Sitnik <andrey@sitnik.ru>",
57 "license": "MIT",
58 "homepage": "https://postcss.org/",
59 "repository": "postcss/postcss",
60 "dependencies": {
61 "colorette": "^1.2.1",
62 "nanoid": "^3.1.20",
63 "source-map": "^0.6.1"
64 },
65 "browser": {
66 "./lib/terminal-highlight": false,
67 "colorette": false,
68 "fs": false
69 }
70}
And this is what I get when I list the files in ./node_modules/postcss-safe-parser/node_modules/postcss/lib/
lgtd-lt-119-mbmt:frontend juliantc$ ls ./node_modules/postcss-safe-parser/node_modules/postcss/lib/
1npm start
2> dataflow@0.1.0 start
3> react-scripts start
4
5node:internal/modules/cjs/loader:488
6 throw e;
7 ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10 at new NodeError (node:internal/errors:371:5)
11 at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12 at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13 at resolveExports (node:internal/modules/cjs/loader:482:36)
14 at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16 at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17 at Module.require (node:internal/modules/cjs/loader:999:19)
18 at require (node:internal/modules/cjs/helpers:102:18)
19 at Object.<anonymous> (/Users/juliantc/Desktop/ai- studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20 code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25 "name": "postcss",
26 "version": "8.2.6",
27 "description": "Tool for transforming styles with JS plugins",
28 "engines": {
29 "node": "^10 || ^12 || >=14"
30 },
31 "exports": {
32 ".": {
33 "require": "./lib/postcss.js",
34 "import": "./lib/postcss.mjs",
35 "types": "./lib/postcss.d.ts"
36 },
37 "./": "./"
38 },
39 "main": "./lib/postcss.js",
40 "types": "./lib/postcss.d.ts",
41 "keywords": [
42 "css",
43 "postcss",
44 "rework",
45 "preprocessor",
46 "parser",
47 "source map",
48 "transform",
49 "manipulation",
50 "transpiler"
51 ],
52 "funding": {
53 "type": "opencollective",
54 "url": "https://opencollective.com/postcss/"
55 },
56 "author": "Andrey Sitnik <andrey@sitnik.ru>",
57 "license": "MIT",
58 "homepage": "https://postcss.org/",
59 "repository": "postcss/postcss",
60 "dependencies": {
61 "colorette": "^1.2.1",
62 "nanoid": "^3.1.20",
63 "source-map": "^0.6.1"
64 },
65 "browser": {
66 "./lib/terminal-highlight": false,
67 "colorette": false,
68 "fs": false
69 }
70}at-rule.d.ts css-syntax-error.d.ts input.d.ts map-generator.js postcss.d.ts processor.js rule.js tokenize.js
71at-rule.js css-syntax-error.js input.js node.d.ts postcss.js result.d.ts stringifier.js warn-once.js
72comment.d.ts declaration.d.ts lazy-result.d.ts node.js postcss.mjs result.js stringify.d.ts warning.d.ts
73comment.js declaration.js lazy-result.js parse.d.ts previous-map.d.ts root.d.ts stringify.js warning.js
74container.d.ts fromJSON.d.ts list.d.ts parse.js previous-map.js root.js symbols.js
75container.js fromJSON.js list.js parser.js processor.d.ts rule.d.ts terminal-highlight.js
76
ANSWER
Answered 2021-Nov-13 at 18:36I am also stuck with the same problem because I installed the latest version of Node.js (v17.0.1).
Just go for node.js v14.18.1
and remove the latest version just use the stable version v14.18.1
1npm start
2> dataflow@0.1.0 start
3> react-scripts start
4
5node:internal/modules/cjs/loader:488
6 throw e;
7 ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10 at new NodeError (node:internal/errors:371:5)
11 at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12 at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13 at resolveExports (node:internal/modules/cjs/loader:482:36)
14 at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16 at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17 at Module.require (node:internal/modules/cjs/loader:999:19)
18 at require (node:internal/modules/cjs/helpers:102:18)
19 at Object.<anonymous> (/Users/juliantc/Desktop/ai- studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20 code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25 "name": "postcss",
26 "version": "8.2.6",
27 "description": "Tool for transforming styles with JS plugins",
28 "engines": {
29 "node": "^10 || ^12 || >=14"
30 },
31 "exports": {
32 ".": {
33 "require": "./lib/postcss.js",
34 "import": "./lib/postcss.mjs",
35 "types": "./lib/postcss.d.ts"
36 },
37 "./": "./"
38 },
39 "main": "./lib/postcss.js",
40 "types": "./lib/postcss.d.ts",
41 "keywords": [
42 "css",
43 "postcss",
44 "rework",
45 "preprocessor",
46 "parser",
47 "source map",
48 "transform",
49 "manipulation",
50 "transpiler"
51 ],
52 "funding": {
53 "type": "opencollective",
54 "url": "https://opencollective.com/postcss/"
55 },
56 "author": "Andrey Sitnik <andrey@sitnik.ru>",
57 "license": "MIT",
58 "homepage": "https://postcss.org/",
59 "repository": "postcss/postcss",
60 "dependencies": {
61 "colorette": "^1.2.1",
62 "nanoid": "^3.1.20",
63 "source-map": "^0.6.1"
64 },
65 "browser": {
66 "./lib/terminal-highlight": false,
67 "colorette": false,
68 "fs": false
69 }
70}at-rule.d.ts css-syntax-error.d.ts input.d.ts map-generator.js postcss.d.ts processor.js rule.js tokenize.js
71at-rule.js css-syntax-error.js input.js node.d.ts postcss.js result.d.ts stringifier.js warn-once.js
72comment.d.ts declaration.d.ts lazy-result.d.ts node.js postcss.mjs result.js stringify.d.ts warning.d.ts
73comment.js declaration.js lazy-result.js parse.d.ts previous-map.d.ts root.d.ts stringify.js warning.js
74container.d.ts fromJSON.d.ts list.d.ts parse.js previous-map.js root.js symbols.js
75container.js fromJSON.js list.js parser.js processor.d.ts rule.d.ts terminal-highlight.js
76nvm uninstall <version>
77
OR
1npm start
2> dataflow@0.1.0 start
3> react-scripts start
4
5node:internal/modules/cjs/loader:488
6 throw e;
7 ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10 at new NodeError (node:internal/errors:371:5)
11 at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12 at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13 at resolveExports (node:internal/modules/cjs/loader:482:36)
14 at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16 at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17 at Module.require (node:internal/modules/cjs/loader:999:19)
18 at require (node:internal/modules/cjs/helpers:102:18)
19 at Object.<anonymous> (/Users/juliantc/Desktop/ai- studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20 code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25 "name": "postcss",
26 "version": "8.2.6",
27 "description": "Tool for transforming styles with JS plugins",
28 "engines": {
29 "node": "^10 || ^12 || >=14"
30 },
31 "exports": {
32 ".": {
33 "require": "./lib/postcss.js",
34 "import": "./lib/postcss.mjs",
35 "types": "./lib/postcss.d.ts"
36 },
37 "./": "./"
38 },
39 "main": "./lib/postcss.js",
40 "types": "./lib/postcss.d.ts",
41 "keywords": [
42 "css",
43 "postcss",
44 "rework",
45 "preprocessor",
46 "parser",
47 "source map",
48 "transform",
49 "manipulation",
50 "transpiler"
51 ],
52 "funding": {
53 "type": "opencollective",
54 "url": "https://opencollective.com/postcss/"
55 },
56 "author": "Andrey Sitnik <andrey@sitnik.ru>",
57 "license": "MIT",
58 "homepage": "https://postcss.org/",
59 "repository": "postcss/postcss",
60 "dependencies": {
61 "colorette": "^1.2.1",
62 "nanoid": "^3.1.20",
63 "source-map": "^0.6.1"
64 },
65 "browser": {
66 "./lib/terminal-highlight": false,
67 "colorette": false,
68 "fs": false
69 }
70}at-rule.d.ts css-syntax-error.d.ts input.d.ts map-generator.js postcss.d.ts processor.js rule.js tokenize.js
71at-rule.js css-syntax-error.js input.js node.d.ts postcss.js result.d.ts stringifier.js warn-once.js
72comment.d.ts declaration.d.ts lazy-result.d.ts node.js postcss.mjs result.js stringify.d.ts warning.d.ts
73comment.js declaration.js lazy-result.js parse.d.ts previous-map.d.ts root.d.ts stringify.js warning.js
74container.d.ts fromJSON.d.ts list.d.ts parse.js previous-map.js root.js symbols.js
75container.js fromJSON.js list.js parser.js processor.d.ts rule.d.ts terminal-highlight.js
76nvm uninstall <version>
77nvm uninstall v17.0.1
78
then install the LTS
one which is v14.18.1
1npm start
2> dataflow@0.1.0 start
3> react-scripts start
4
5node:internal/modules/cjs/loader:488
6 throw e;
7 ^
8
9Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in /Users/juliantc/Desktop/ai-studio/development/frontend/node_modules/postcss-safe-parser/node_modules/postcss/package.json
10 at new NodeError (node:internal/errors:371:5)
11 at throwExportsNotFound (node:internal/modules/esm/resolve:416:9)
12 at packageExportsResolve (node:internal/modules/esm/resolve:669:3)
13 at resolveExports (node:internal/modules/cjs/loader:482:36)
14 at Function.Module._findPath (node:internal/modules/cjs/loader:522:31)
15 at Function.Module._resolveFilename (node:internal/modules/cjs/loader:919:27)
16 at Function.Module._load (node:internal/modules/cjs/loader:778:27)
17 at Module.require (node:internal/modules/cjs/loader:999:19)
18 at require (node:internal/modules/cjs/helpers:102:18)
19 at Object.<anonymous> (/Users/juliantc/Desktop/ai- studio/development/frontend/node_modules/postcss-safe-parser/lib/safe-parser.js:1:17) {
20 code: 'ERR_PACKAGE_PATH_NOT_EXPORTED'
21}
22
23Node.js v17.0.1
24{
25 "name": "postcss",
26 "version": "8.2.6",
27 "description": "Tool for transforming styles with JS plugins",
28 "engines": {
29 "node": "^10 || ^12 || >=14"
30 },
31 "exports": {
32 ".": {
33 "require": "./lib/postcss.js",
34 "import": "./lib/postcss.mjs",
35 "types": "./lib/postcss.d.ts"
36 },
37 "./": "./"
38 },
39 "main": "./lib/postcss.js",
40 "types": "./lib/postcss.d.ts",
41 "keywords": [
42 "css",
43 "postcss",
44 "rework",
45 "preprocessor",
46 "parser",
47 "source map",
48 "transform",
49 "manipulation",
50 "transpiler"
51 ],
52 "funding": {
53 "type": "opencollective",
54 "url": "https://opencollective.com/postcss/"
55 },
56 "author": "Andrey Sitnik <andrey@sitnik.ru>",
57 "license": "MIT",
58 "homepage": "https://postcss.org/",
59 "repository": "postcss/postcss",
60 "dependencies": {
61 "colorette": "^1.2.1",
62 "nanoid": "^3.1.20",
63 "source-map": "^0.6.1"
64 },
65 "browser": {
66 "./lib/terminal-highlight": false,
67 "colorette": false,
68 "fs": false
69 }
70}at-rule.d.ts css-syntax-error.d.ts input.d.ts map-generator.js postcss.d.ts processor.js rule.js tokenize.js
71at-rule.js css-syntax-error.js input.js node.d.ts postcss.js result.d.ts stringifier.js warn-once.js
72comment.d.ts declaration.d.ts lazy-result.d.ts node.js postcss.mjs result.js stringify.d.ts warning.d.ts
73comment.js declaration.js lazy-result.js parse.d.ts previous-map.d.ts root.d.ts stringify.js warning.js
74container.d.ts fromJSON.d.ts list.d.ts parse.js previous-map.js root.js symbols.js
75container.js fromJSON.js list.js parser.js processor.d.ts rule.d.ts terminal-highlight.js
76nvm uninstall <version>
77nvm uninstall v17.0.1
78nvm install --lts
79
This worked for me.
QUESTION
Can you compress angular image assets on build?
Asked 2022-Jan-15 at 12:13I have very big images in my assets, which slows down the site by a lot for slower networks. (you can read more about the topic on this lighthouse linked page)
ng build --prod
).ng serve
).example.jpg
→ should become: example_x265.jpg
, example_x128.jpg
, ...)The most promising guide I have found for that is this one here, which describes how to use the imagemin package in combination with the ngx-build-plus package.
Unfortunately, after following the tutorial, I get the following error:
1[error] TypeError: Cannot assign to read only property 'main.977fe6373cfd108d.js' of object '#<Object>'
2 at ImageminPlugin._callee2$ (/.../node_modules/imagemin-webpack-plugin/dist/index.js:264:48)
3 at tryCatch (/.../node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:62:40)
4 // ...
5
Is there any way to compress asset images on build?
1[error] TypeError: Cannot assign to read only property 'main.977fe6373cfd108d.js' of object '#<Object>'
2 at ImageminPlugin._callee2$ (/.../node_modules/imagemin-webpack-plugin/dist/index.js:264:48)
3 at tryCatch (/.../node_modules/babel-runtime/node_modules/regenerator-runtime/runtime.js:62:40)
4 // ...
5Angular Version: 13.1.0
6
Note: I do not want to know how to upload images to third party storage solutions.
I specifically want to create a compressed version of my static assets on build time.
ANSWER
Answered 2021-Dec-19 at 22:55I would never do that! because its against the convetions You should try Firebase storage, they give you 1 GB for free, and its easy to implement.
QUESTION
Convert a bytes iterable to an iterable of str, where each value is a line
Asked 2022-Jan-10 at 08:29I have an iterable of bytes
, such as
1bytes_iter = (
2 b'col_1,',
3 b'c',
4 b'ol_2\n1',
5 b',"val',
6 b'ue"\n',
7)
8
(but typically this would not be hard coded or available all at once, but supplied from a generator say) and I want to convert this to an iterable of str
lines, where line breaks are unknown up front, but could be any of \r
, \n
or \r\n
. So in this case would be:
1bytes_iter = (
2 b'col_1,',
3 b'c',
4 b'ol_2\n1',
5 b',"val',
6 b'ue"\n',
7)
8lines_iter = (
9 'col_1,col_2',
10 '1,"value"',
11)
12
(but again, just as an iterable, not so it's all in memory at once).
How can I do this?
Context: my aim is to then pass the iterable of str lines to csv.reader
(that I think needs whole lines?), but I'm interested in this answer just in general.
ANSWER
Answered 2022-Jan-10 at 08:29I used yield
and re.finditer
.
The yield expression is used when defining a generator function or an asynchronous generator function and thus can only be used in the body of a function definition. Using a yield expression in a function’s body causes that function to be a generator function
Return an iterator yielding match objects over all non-overlapping matches for the RE pattern in string. The string is scanned left-to-right, and matches are returned in the order found. Empty matches are included in the result.
If there are no groups, return a list of strings matching the whole pattern. If there is exactly one group, return a list of strings matching that group. If multiple groups are present, return a list of tuples of strings matching the groups. Non-capturing groups do not affect the form of the result.
The regular expression ([^\r\n]*)(\r\n|\r|\n)?
can be divided into two parts to match (that is, two groups). The first group matches the data without \r
and \n
, and the second group matches \r
, \n
or \r\n
.
1bytes_iter = (
2 b'col_1,',
3 b'c',
4 b'ol_2\n1',
5 b',"val',
6 b'ue"\n',
7)
8lines_iter = (
9 'col_1,col_2',
10 '1,"value"',
11)
12import re
13find_rule = re.compile("([^\r\n]*)(\r\n|\r|\n)?")
14
15
16def converter(byte_data):
17 left_d = ""
18 for d in byte_data:
19 # Used to save the previous match result in the `for` loop
20 prev_result = None
21 # Concatenate the last part of the previous data with the current data,
22 # used to deal with the case of `\r\n` being separated.
23 d = left_d + d.decode()
24 left_d = ""
25 # Using `find_rule.finditer` the last value("") will be invalid
26 for match_result in find_rule.finditer(d):
27 i = match_result.group()
28 if not i:
29 # The program comes to this point, indicating that i == "", which is the last matching value
30 left_d, prev_result = prev_result.group(), None
31 continue
32 if prev_result:
33 if prev_result.group(2) is None:
34 # The program goes here, represented as the last valid value matched
35 left_d = prev_result.group()
36 else:
37 # Returns the previous matched value
38 yield prev_result.group()
39 # Save the current match result
40 prev_result = match_result
41
42 else:
43 yield left_d
44
45
46for i in (converter(iter((
47 b'col_1,\r',
48 b'\nc',
49 b'ol_2\n1',
50 b'\n,"val;\r',
51 b'ue"\n')))
52):
53 print(repr(i))
54
55
Output:
1bytes_iter = (
2 b'col_1,',
3 b'c',
4 b'ol_2\n1',
5 b',"val',
6 b'ue"\n',
7)
8lines_iter = (
9 'col_1,col_2',
10 '1,"value"',
11)
12import re
13find_rule = re.compile("([^\r\n]*)(\r\n|\r|\n)?")
14
15
16def converter(byte_data):
17 left_d = ""
18 for d in byte_data:
19 # Used to save the previous match result in the `for` loop
20 prev_result = None
21 # Concatenate the last part of the previous data with the current data,
22 # used to deal with the case of `\r\n` being separated.
23 d = left_d + d.decode()
24 left_d = ""
25 # Using `find_rule.finditer` the last value("") will be invalid
26 for match_result in find_rule.finditer(d):
27 i = match_result.group()
28 if not i:
29 # The program comes to this point, indicating that i == "", which is the last matching value
30 left_d, prev_result = prev_result.group(), None
31 continue
32 if prev_result:
33 if prev_result.group(2) is None:
34 # The program goes here, represented as the last valid value matched
35 left_d = prev_result.group()
36 else:
37 # Returns the previous matched value
38 yield prev_result.group()
39 # Save the current match result
40 prev_result = match_result
41
42 else:
43 yield left_d
44
45
46for i in (converter(iter((
47 b'col_1,\r',
48 b'\nc',
49 b'ol_2\n1',
50 b'\n,"val;\r',
51 b'ue"\n')))
52):
53 print(repr(i))
54
55'col_1,\r\n'
56'col_2\n'
57'1\n'
58',"val;\r'
59'ue"\n'
60
QUESTION
Problem Updating to .Net 6 - Encrypting String
Asked 2021-Dec-20 at 23:09I'm using a string Encryption/Decryption class similar to the one provided here as a solution.
This worked well for me in .Net 5.
Now I wanted to update my project to .Net 6.
When using .Net 6, the decrypted string does get cut off a certain point depending on the length of the input string.
▶️ To make it easy to debug/reproduce my issue, I created a public repro Repository here.
Both are calling the encryption methods with the exact same input of "12345678901234567890"
with the path phrase of "nzv86ri4H2qYHqc&m6rL"
.
.Net 5 output: "12345678901234567890"
.Net 6 output: "1234567890123456"
The difference in length is 4
.
I also looked at the breaking changes for .Net 6, but could not find something which guided me to a solution.
I'm glad for any suggestions regarding my issue, thanks!
Encryption Class
1public static class StringCipher
2{
3 // This constant is used to determine the keysize of the encryption algorithm in bits.
4 // We divide this by 8 within the code below to get the equivalent number of bytes.
5 private const int Keysize = 128;
6
7 // This constant determines the number of iterations for the password bytes generation function.
8 private const int DerivationIterations = 1000;
9
10 public static string Encrypt(string plainText, string passPhrase)
11 {
12 // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13 // so that the same Salt and IV values can be used when decrypting.
14 var saltStringBytes = Generate128BitsOfRandomEntropy();
15 var ivStringBytes = Generate128BitsOfRandomEntropy();
16 var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18 {
19 var keyBytes = password.GetBytes(Keysize / 8);
20 using (var symmetricKey = Aes.Create())
21 {
22 symmetricKey.BlockSize = 128;
23 symmetricKey.Mode = CipherMode.CBC;
24 symmetricKey.Padding = PaddingMode.PKCS7;
25 using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26 {
27 using (var memoryStream = new MemoryStream())
28 {
29 using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30 {
31 cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32 cryptoStream.FlushFinalBlock();
33 // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34 var cipherTextBytes = saltStringBytes;
35 cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36 cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37 memoryStream.Close();
38 cryptoStream.Close();
39 return Convert.ToBase64String(cipherTextBytes);
40 }
41 }
42 }
43 }
44 }
45 }
46
47 public static string Decrypt(string cipherText, string passPhrase)
48 {
49 // Get the complete stream of bytes that represent:
50 // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51 var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52 // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53 var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54 // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55 var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56 // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57 var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60 {
61 var keyBytes = password.GetBytes(Keysize / 8);
62 using (var symmetricKey = Aes.Create())
63 {
64 symmetricKey.BlockSize = 128;
65 symmetricKey.Mode = CipherMode.CBC;
66 symmetricKey.Padding = PaddingMode.PKCS7;
67 using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68 {
69 using (var memoryStream = new MemoryStream(cipherTextBytes))
70 {
71 using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72 {
73 var plainTextBytes = new byte[cipherTextBytes.Length];
74 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75 memoryStream.Close();
76 cryptoStream.Close();
77 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78 }
79 }
80 }
81 }
82 }
83 }
84
85 private static byte[] Generate128BitsOfRandomEntropy()
86 {
87 var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88 using (var rngCsp = RandomNumberGenerator.Create())
89 {
90 // Fill the array with cryptographically secure random bytes.
91 rngCsp.GetBytes(randomBytes);
92 }
93 return randomBytes;
94 }
95}
96
Calling code
1public static class StringCipher
2{
3 // This constant is used to determine the keysize of the encryption algorithm in bits.
4 // We divide this by 8 within the code below to get the equivalent number of bytes.
5 private const int Keysize = 128;
6
7 // This constant determines the number of iterations for the password bytes generation function.
8 private const int DerivationIterations = 1000;
9
10 public static string Encrypt(string plainText, string passPhrase)
11 {
12 // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13 // so that the same Salt and IV values can be used when decrypting.
14 var saltStringBytes = Generate128BitsOfRandomEntropy();
15 var ivStringBytes = Generate128BitsOfRandomEntropy();
16 var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18 {
19 var keyBytes = password.GetBytes(Keysize / 8);
20 using (var symmetricKey = Aes.Create())
21 {
22 symmetricKey.BlockSize = 128;
23 symmetricKey.Mode = CipherMode.CBC;
24 symmetricKey.Padding = PaddingMode.PKCS7;
25 using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26 {
27 using (var memoryStream = new MemoryStream())
28 {
29 using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30 {
31 cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32 cryptoStream.FlushFinalBlock();
33 // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34 var cipherTextBytes = saltStringBytes;
35 cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36 cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37 memoryStream.Close();
38 cryptoStream.Close();
39 return Convert.ToBase64String(cipherTextBytes);
40 }
41 }
42 }
43 }
44 }
45 }
46
47 public static string Decrypt(string cipherText, string passPhrase)
48 {
49 // Get the complete stream of bytes that represent:
50 // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51 var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52 // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53 var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54 // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55 var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56 // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57 var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60 {
61 var keyBytes = password.GetBytes(Keysize / 8);
62 using (var symmetricKey = Aes.Create())
63 {
64 symmetricKey.BlockSize = 128;
65 symmetricKey.Mode = CipherMode.CBC;
66 symmetricKey.Padding = PaddingMode.PKCS7;
67 using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68 {
69 using (var memoryStream = new MemoryStream(cipherTextBytes))
70 {
71 using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72 {
73 var plainTextBytes = new byte[cipherTextBytes.Length];
74 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75 memoryStream.Close();
76 cryptoStream.Close();
77 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78 }
79 }
80 }
81 }
82 }
83 }
84
85 private static byte[] Generate128BitsOfRandomEntropy()
86 {
87 var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88 using (var rngCsp = RandomNumberGenerator.Create())
89 {
90 // Fill the array with cryptographically secure random bytes.
91 rngCsp.GetBytes(randomBytes);
92 }
93 return randomBytes;
94 }
95}
96var input = "12345678901234567890";
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, "nzv86ri4H2qYHqc&m6rL");
101
102var output = StringCipher.Decrypt(encrypted, "nzv86ri4H2qYHqc&m6rL");
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107
ANSWER
Answered 2021-Nov-10 at 10:25The reason is this breaking change:
DeflateStream, GZipStream, and CryptoStream diverged from typical Stream.Read and Stream.ReadAsync behavior in two ways:
They didn't complete the read operation until either the buffer passed to the read operation was completely filled or the end of the stream was reached.
And the new behaviour is:
Starting in .NET 6, when Stream.Read or Stream.ReadAsync is called on one of the affected stream types with a buffer of length N, the operation completes when:
At least one byte has been read from the stream, or The underlying stream they wrap returns 0 from a call to its read, indicating no more data is available.
In your case you are affected because of this code in Decrypt
method:
1public static class StringCipher
2{
3 // This constant is used to determine the keysize of the encryption algorithm in bits.
4 // We divide this by 8 within the code below to get the equivalent number of bytes.
5 private const int Keysize = 128;
6
7 // This constant determines the number of iterations for the password bytes generation function.
8 private const int DerivationIterations = 1000;
9
10 public static string Encrypt(string plainText, string passPhrase)
11 {
12 // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13 // so that the same Salt and IV values can be used when decrypting.
14 var saltStringBytes = Generate128BitsOfRandomEntropy();
15 var ivStringBytes = Generate128BitsOfRandomEntropy();
16 var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18 {
19 var keyBytes = password.GetBytes(Keysize / 8);
20 using (var symmetricKey = Aes.Create())
21 {
22 symmetricKey.BlockSize = 128;
23 symmetricKey.Mode = CipherMode.CBC;
24 symmetricKey.Padding = PaddingMode.PKCS7;
25 using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26 {
27 using (var memoryStream = new MemoryStream())
28 {
29 using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30 {
31 cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32 cryptoStream.FlushFinalBlock();
33 // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34 var cipherTextBytes = saltStringBytes;
35 cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36 cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37 memoryStream.Close();
38 cryptoStream.Close();
39 return Convert.ToBase64String(cipherTextBytes);
40 }
41 }
42 }
43 }
44 }
45 }
46
47 public static string Decrypt(string cipherText, string passPhrase)
48 {
49 // Get the complete stream of bytes that represent:
50 // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51 var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52 // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53 var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54 // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55 var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56 // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57 var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60 {
61 var keyBytes = password.GetBytes(Keysize / 8);
62 using (var symmetricKey = Aes.Create())
63 {
64 symmetricKey.BlockSize = 128;
65 symmetricKey.Mode = CipherMode.CBC;
66 symmetricKey.Padding = PaddingMode.PKCS7;
67 using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68 {
69 using (var memoryStream = new MemoryStream(cipherTextBytes))
70 {
71 using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72 {
73 var plainTextBytes = new byte[cipherTextBytes.Length];
74 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75 memoryStream.Close();
76 cryptoStream.Close();
77 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78 }
79 }
80 }
81 }
82 }
83 }
84
85 private static byte[] Generate128BitsOfRandomEntropy()
86 {
87 var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88 using (var rngCsp = RandomNumberGenerator.Create())
89 {
90 // Fill the array with cryptographically secure random bytes.
91 rngCsp.GetBytes(randomBytes);
92 }
93 return randomBytes;
94 }
95}
96var input = "12345678901234567890";
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, "nzv86ri4H2qYHqc&m6rL");
101
102var output = StringCipher.Decrypt(encrypted, "nzv86ri4H2qYHqc&m6rL");
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
108{
109 var plainTextBytes = new byte[cipherTextBytes.Length];
110 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
111 memoryStream.Close();
112 cryptoStream.Close();
113 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
114}
115
You do not check how much bytes Read
actually read and whether it read them all. You could get away with this in previous versions of .NET because as mentioned CryptoStream
behaviour was different from other streams, and because your buffer length is enough to hold all data. However, this is no longer the case and you need to check it as you would do for other streams. Or even better - just use CopyTo
:
1public static class StringCipher
2{
3 // This constant is used to determine the keysize of the encryption algorithm in bits.
4 // We divide this by 8 within the code below to get the equivalent number of bytes.
5 private const int Keysize = 128;
6
7 // This constant determines the number of iterations for the password bytes generation function.
8 private const int DerivationIterations = 1000;
9
10 public static string Encrypt(string plainText, string passPhrase)
11 {
12 // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13 // so that the same Salt and IV values can be used when decrypting.
14 var saltStringBytes = Generate128BitsOfRandomEntropy();
15 var ivStringBytes = Generate128BitsOfRandomEntropy();
16 var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18 {
19 var keyBytes = password.GetBytes(Keysize / 8);
20 using (var symmetricKey = Aes.Create())
21 {
22 symmetricKey.BlockSize = 128;
23 symmetricKey.Mode = CipherMode.CBC;
24 symmetricKey.Padding = PaddingMode.PKCS7;
25 using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26 {
27 using (var memoryStream = new MemoryStream())
28 {
29 using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30 {
31 cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32 cryptoStream.FlushFinalBlock();
33 // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34 var cipherTextBytes = saltStringBytes;
35 cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36 cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37 memoryStream.Close();
38 cryptoStream.Close();
39 return Convert.ToBase64String(cipherTextBytes);
40 }
41 }
42 }
43 }
44 }
45 }
46
47 public static string Decrypt(string cipherText, string passPhrase)
48 {
49 // Get the complete stream of bytes that represent:
50 // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51 var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52 // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53 var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54 // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55 var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56 // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57 var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60 {
61 var keyBytes = password.GetBytes(Keysize / 8);
62 using (var symmetricKey = Aes.Create())
63 {
64 symmetricKey.BlockSize = 128;
65 symmetricKey.Mode = CipherMode.CBC;
66 symmetricKey.Padding = PaddingMode.PKCS7;
67 using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68 {
69 using (var memoryStream = new MemoryStream(cipherTextBytes))
70 {
71 using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72 {
73 var plainTextBytes = new byte[cipherTextBytes.Length];
74 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75 memoryStream.Close();
76 cryptoStream.Close();
77 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78 }
79 }
80 }
81 }
82 }
83 }
84
85 private static byte[] Generate128BitsOfRandomEntropy()
86 {
87 var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88 using (var rngCsp = RandomNumberGenerator.Create())
89 {
90 // Fill the array with cryptographically secure random bytes.
91 rngCsp.GetBytes(randomBytes);
92 }
93 return randomBytes;
94 }
95}
96var input = "12345678901234567890";
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, "nzv86ri4H2qYHqc&m6rL");
101
102var output = StringCipher.Decrypt(encrypted, "nzv86ri4H2qYHqc&m6rL");
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
108{
109 var plainTextBytes = new byte[cipherTextBytes.Length];
110 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
111 memoryStream.Close();
112 cryptoStream.Close();
113 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
114}
115using (var plainTextStream = new MemoryStream())
116{
117 cryptoStream.CopyTo(plainTextStream);
118 var plainTextBytes = plainTextStream.ToArray();
119 return Encoding.UTF8.GetString(plainTextBytes, 0, plainTextBytes.Length);
120}
121
Or even better as another answer suggests, since you decrypt UTF8 text:
1public static class StringCipher
2{
3 // This constant is used to determine the keysize of the encryption algorithm in bits.
4 // We divide this by 8 within the code below to get the equivalent number of bytes.
5 private const int Keysize = 128;
6
7 // This constant determines the number of iterations for the password bytes generation function.
8 private const int DerivationIterations = 1000;
9
10 public static string Encrypt(string plainText, string passPhrase)
11 {
12 // Salt and IV is randomly generated each time, but is preprended to encrypted cipher text
13 // so that the same Salt and IV values can be used when decrypting.
14 var saltStringBytes = Generate128BitsOfRandomEntropy();
15 var ivStringBytes = Generate128BitsOfRandomEntropy();
16 var plainTextBytes = Encoding.UTF8.GetBytes(plainText);
17 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
18 {
19 var keyBytes = password.GetBytes(Keysize / 8);
20 using (var symmetricKey = Aes.Create())
21 {
22 symmetricKey.BlockSize = 128;
23 symmetricKey.Mode = CipherMode.CBC;
24 symmetricKey.Padding = PaddingMode.PKCS7;
25 using (var encryptor = symmetricKey.CreateEncryptor(keyBytes, ivStringBytes))
26 {
27 using (var memoryStream = new MemoryStream())
28 {
29 using (var cryptoStream = new CryptoStream(memoryStream, encryptor, CryptoStreamMode.Write))
30 {
31 cryptoStream.Write(plainTextBytes, 0, plainTextBytes.Length);
32 cryptoStream.FlushFinalBlock();
33 // Create the final bytes as a concatenation of the random salt bytes, the random iv bytes and the cipher bytes.
34 var cipherTextBytes = saltStringBytes;
35 cipherTextBytes = cipherTextBytes.Concat(ivStringBytes).ToArray();
36 cipherTextBytes = cipherTextBytes.Concat(memoryStream.ToArray()).ToArray();
37 memoryStream.Close();
38 cryptoStream.Close();
39 return Convert.ToBase64String(cipherTextBytes);
40 }
41 }
42 }
43 }
44 }
45 }
46
47 public static string Decrypt(string cipherText, string passPhrase)
48 {
49 // Get the complete stream of bytes that represent:
50 // [32 bytes of Salt] + [16 bytes of IV] + [n bytes of CipherText]
51 var cipherTextBytesWithSaltAndIv = Convert.FromBase64String(cipherText);
52 // Get the saltbytes by extracting the first 16 bytes from the supplied cipherText bytes.
53 var saltStringBytes = cipherTextBytesWithSaltAndIv.Take(Keysize / 8).ToArray();
54 // Get the IV bytes by extracting the next 16 bytes from the supplied cipherText bytes.
55 var ivStringBytes = cipherTextBytesWithSaltAndIv.Skip(Keysize / 8).Take(Keysize / 8).ToArray();
56 // Get the actual cipher text bytes by removing the first 64 bytes from the cipherText string.
57 var cipherTextBytes = cipherTextBytesWithSaltAndIv.Skip((Keysize / 8) * 2).Take(cipherTextBytesWithSaltAndIv.Length - ((Keysize / 8) * 2)).ToArray();
58
59 using (var password = new Rfc2898DeriveBytes(passPhrase, saltStringBytes, DerivationIterations))
60 {
61 var keyBytes = password.GetBytes(Keysize / 8);
62 using (var symmetricKey = Aes.Create())
63 {
64 symmetricKey.BlockSize = 128;
65 symmetricKey.Mode = CipherMode.CBC;
66 symmetricKey.Padding = PaddingMode.PKCS7;
67 using (var decryptor = symmetricKey.CreateDecryptor(keyBytes, ivStringBytes))
68 {
69 using (var memoryStream = new MemoryStream(cipherTextBytes))
70 {
71 using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
72 {
73 var plainTextBytes = new byte[cipherTextBytes.Length];
74 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
75 memoryStream.Close();
76 cryptoStream.Close();
77 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
78 }
79 }
80 }
81 }
82 }
83 }
84
85 private static byte[] Generate128BitsOfRandomEntropy()
86 {
87 var randomBytes = new byte[16]; // 16 Bytes will give us 128 bits.
88 using (var rngCsp = RandomNumberGenerator.Create())
89 {
90 // Fill the array with cryptographically secure random bytes.
91 rngCsp.GetBytes(randomBytes);
92 }
93 return randomBytes;
94 }
95}
96var input = "12345678901234567890";
97var inputLength = input.Length;
98var inputBytes = Encoding.UTF8.GetBytes(input);
99
100var encrypted = StringCipher.Encrypt(input, "nzv86ri4H2qYHqc&m6rL");
101
102var output = StringCipher.Decrypt(encrypted, "nzv86ri4H2qYHqc&m6rL");
103var outputLength = output.Length;
104var outputBytes = Encoding.UTF8.GetBytes(output);
105
106var lengthDiff = inputLength - outputLength;
107using (var cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
108{
109 var plainTextBytes = new byte[cipherTextBytes.Length];
110 var decryptedByteCount = cryptoStream.Read(plainTextBytes, 0, plainTextBytes.Length);
111 memoryStream.Close();
112 cryptoStream.Close();
113 return Encoding.UTF8.GetString(plainTextBytes, 0, decryptedByteCount);
114}
115using (var plainTextStream = new MemoryStream())
116{
117 cryptoStream.CopyTo(plainTextStream);
118 var plainTextBytes = plainTextStream.ToArray();
119 return Encoding.UTF8.GetString(plainTextBytes, 0, plainTextBytes.Length);
120}
121using (var plainTextReader = new StreamReader(cryptoStream))
122{
123 return plainTextReader.ReadToEnd();
124}
125
QUESTION
Ensure that an argument can be iterated twice
Asked 2021-Dec-19 at 12:43Suppose I have the following function:
1def print_twice(x):
2 for i in x: print(i)
3 for i in x: print(i)
4
When I run:
1def print_twice(x):
2 for i in x: print(i)
3 for i in x: print(i)
4print_twice([1,2,3])
5
or:
1def print_twice(x):
2 for i in x: print(i)
3 for i in x: print(i)
4print_twice([1,2,3])
5print_twice((1,2,3))
6
I get the expected result: the numbers 1,2,3 are printed twice.
But when I run:
1def print_twice(x):
2 for i in x: print(i)
3 for i in x: print(i)
4print_twice([1,2,3])
5print_twice((1,2,3))
6print_twice(zip([1,2,3],[4,5,6]))
7
the pairs (1,4),(2,5),(3,6) are printed only once. Probably, this is because the zip
returns a generator that terminates after one pass.
How can I modify the function print_twice
such that it will correctly handle all inputs?
I could insert a line at the beginning of the function: x = list(x)
. But this might be inefficient in case x is already a list, a tuple, a range, or any other iterator that can be iterated more than once. Is there a more efficient solution?
ANSWER
Answered 2021-Dec-16 at 15:48zip
will return an iterator. Once unpacked, it cannot be unpacked again, it gets exhausted.
Maybe if you want to make sure that only zip
objects get converted to list
as you said it would work but it would not be efficient, you can check for it type:
1def print_twice(x):
2 for i in x: print(i)
3 for i in x: print(i)
4print_twice([1,2,3])
5print_twice((1,2,3))
6print_twice(zip([1,2,3],[4,5,6]))
7if isinstance(x, zip):
8 x = list(x)
9
QUESTION
Proper way to DI NSwag auto-generated client
Asked 2021-Dec-04 at 10:51What is the preferred way to make a VS connected service (NSwag) injected into classes/controllers. I have found a lot of suggestions on the net to use this form:
1services.AddHttpClient<IClient, Client>((provider, client) =>
2 {
3 client.BaseAddress = new System.Uri("https://some.baseurl/");
4 });
5
However this results in the error
1services.AddHttpClient<IClient, Client>((provider, client) =>
2 {
3 client.BaseAddress = new System.Uri("https://some.baseurl/");
4 });
5{"errorMessage":"Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'."}
6
This comes from the auto-generated client class in obj
, which seems to force a string BaseUrl in constructor, which of course the DI cannot resolve:
1services.AddHttpClient<IClient, Client>((provider, client) =>
2 {
3 client.BaseAddress = new System.Uri("https://some.baseurl/");
4 });
5{"errorMessage":"Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'."}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8 BaseUrl = baseUrl;
9 _httpClient = httpClient;
10 _settings = new System.Lazy<Newtonsoft.Json.JsonSerializerSettings>(CreateSerializerSettings);
11}
12
This base URL is later forced into the url builder code, so it cannot really be bypassed. However, even the solutions on the net which use partial extensions to client classes seem to completely ignore baseUrl in auto-gen class (like here). As if it does not exist (which is weird, has the NSwag before generated different constructors?). The class is being generated via csproj:
1services.AddHttpClient<IClient, Client>((provider, client) =>
2 {
3 client.BaseAddress = new System.Uri("https://some.baseurl/");
4 });
5{"errorMessage":"Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'."}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8 BaseUrl = baseUrl;
9 _httpClient = httpClient;
10 _settings = new System.Lazy<Newtonsoft.Json.JsonSerializerSettings>(CreateSerializerSettings);
11}
12 <ItemGroup>
13 <OpenApiReference Include="OpenAPIs\swagger.json" CodeGenerator="NSwagCSharp" Namespace="xxx" ClassName="Client">
14 <SourceUri>https://localhost:44353/swagger/v1/swagger.json</SourceUri>
15 </OpenApiReference>
16 </ItemGroup>
17
And this results in the targeted build call:
1services.AddHttpClient<IClient, Client>((provider, client) =>
2 {
3 client.BaseAddress = new System.Uri("https://some.baseurl/");
4 });
5{"errorMessage":"Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'."}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8 BaseUrl = baseUrl;
9 _httpClient = httpClient;
10 _settings = new System.Lazy<Newtonsoft.Json.JsonSerializerSettings>(CreateSerializerSettings);
11}
12 <ItemGroup>
13 <OpenApiReference Include="OpenAPIs\swagger.json" CodeGenerator="NSwagCSharp" Namespace="xxx" ClassName="Client">
14 <SourceUri>https://localhost:44353/swagger/v1/swagger.json</SourceUri>
15 </OpenApiReference>
16 </ItemGroup>
172>GenerateNSwagCSharp:
182> "C:\.<path>./tools/Win/NSwag.exe" openapi2csclient /className:Client /namespace:xxx /input:"C:\<projpath>\OpenAPIs\swagger.json" /output:"obj\swaggerClient.cs"
192>NSwag command line tool for .NET 4.6.1+ WinX64, toolchain v13.13.2.0 (NJsonSchema v10.5.2.0 (Newtonsoft.Json v11.0.0.0))
20
So, how is this being done? potentially, without creating another proxy class for a proxy class, I would rather that DI handles my object lifetimes. I would also like to avoid NSwagStudio if possible and would like to keep the tooling supplied by VS.
ANSWER
Answered 2021-Sep-26 at 13:24Ok, I actually solved this problem by poking around through OpenApiReference
, but it requires manual modification of csproj file. Additional Options
node has to be added to OpenApiReference
item group, to instruct NSwag to NOT expose BaseUrl and to also generate an interface, which eases the work with setting up DI without additional code.
Visual Studio team should really add these two checkboxes to Connected Services screens/configuration for OpenAPI.
1services.AddHttpClient<IClient, Client>((provider, client) =>
2 {
3 client.BaseAddress = new System.Uri("https://some.baseurl/");
4 });
5{"errorMessage":"Unable to resolve service for type 'System.String' while attempting to activate 'xxx.Client'."}
6public Client(string baseUrl, System.Net.Http.HttpClient httpClient)
7{
8 BaseUrl = baseUrl;
9 _httpClient = httpClient;
10 _settings = new System.Lazy<Newtonsoft.Json.JsonSerializerSettings>(CreateSerializerSettings);
11}
12 <ItemGroup>
13 <OpenApiReference Include="OpenAPIs\swagger.json" CodeGenerator="NSwagCSharp" Namespace="xxx" ClassName="Client">
14 <SourceUri>https://localhost:44353/swagger/v1/swagger.json</SourceUri>
15 </OpenApiReference>
16 </ItemGroup>
172>GenerateNSwagCSharp:
182> "C:\.<path>./tools/Win/NSwag.exe" openapi2csclient /className:Client /namespace:xxx /input:"C:\<projpath>\OpenAPIs\swagger.json" /output:"obj\swaggerClient.cs"
192>NSwag command line tool for .NET 4.6.1+ WinX64, toolchain v13.13.2.0 (NJsonSchema v10.5.2.0 (Newtonsoft.Json v11.0.0.0))
20<ItemGroup>
21 <OpenApiReference Include="OpenAPIs\swagger.json" CodeGenerator="NSwagCSharp" Namespace="xxx" ClassName="Client">
22 <SourceUri>https://localhost:44353/swagger/v1/swagger.json</SourceUri>
23 <Options>/UseBaseUrl:false /GenerateClientInterfaces:true</Options>
24 </OpenApiReference>
25</ItemGroup>
26
Now there is only an HttpClient
constructor, andNSwag client proxy uses base address from it, so AddHttpClient
works properly through DI.
QUESTION
How to alias generic types for decorators
Asked 2021-Nov-23 at 11:23Consider the example of a typed decorator bound to certain classes.
1import unittest
2from typing import *
3
4T = TypeVar("T", bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
7 def decorated_function(self: T) -> None:
8 return func(self)
9 return decorated_function
10
Now I even have a generator that creates these decorators and want to shorthand these decorators. What type do I to the variables variables storing the decorator (simplified example omitting the generator).
1import unittest
2from typing import *
3
4T = TypeVar("T", bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
7 def decorated_function(self: T) -> None:
8 return func(self)
9 return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11
This works, but is clunky. So the question is:
How can I alias this type to avoid having to write the the full signature?
Things that don't work:
1import unittest
2from typing import *
3
4T = TypeVar("T", bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
7 def decorated_function(self: T) -> None:
8 return func(self)
9 return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13
Gives the error
1import unittest
2from typing import *
3
4T = TypeVar("T", bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
7 def decorated_function(self: T) -> None:
8 return func(self)
9 return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13error: Type variable "mypytest.T" is unbound
14note: (Hint: Use "Generic[T]" or "Protocol[T]" base class to bind "T" inside a class)
15note: (Hint: Use "T" in function signature to bind "T" inside a function)
16
In contrast, I can use TD[T]
as argument type for a function.
Just using my_decorate: TD = ...
yields a --strict
error
1import unittest
2from typing import *
3
4T = TypeVar("T", bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
7 def decorated_function(self: T) -> None:
8 return func(self)
9 return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13error: Type variable "mypytest.T" is unbound
14note: (Hint: Use "Generic[T]" or "Protocol[T]" base class to bind "T" inside a class)
15note: (Hint: Use "T" in function signature to bind "T" inside a function)
16error: Missing type parameters for generic type "TD"
17
And it no longer detects wrong applications of my_decorate
.
ANSWER
Answered 2021-Nov-23 at 10:59What about this? It is shorter than the full signature:
1import unittest
2from typing import *
3
4T = TypeVar("T", bound=unittest.TestCase)
5
6def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
7 def decorated_function(self: T) -> None:
8 return func(self)
9 return decorated_function
10my_decorate: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
11TD = Callable[[Callable[[T], None]], Callable[[T], None]]
12my_decorate: TD[T] = decorator_variable
13error: Type variable "mypytest.T" is unbound
14note: (Hint: Use "Generic[T]" or "Protocol[T]" base class to bind "T" inside a class)
15note: (Hint: Use "T" in function signature to bind "T" inside a function)
16error: Missing type parameters for generic type "TD"
17import unittest
18from typing import *
19
20T = TypeVar("T", bound=unittest.TestCase)
21
22
23def decorate(func: Callable[[T], None]) -> Callable[[T], None]:
24 def decorated_function(self: T) -> None:
25 return func(self)
26
27 return decorated_function
28
29
30decorator_variable: Callable[[Callable[[T], None]], Callable[[T], None]] = decorate
31
32U = Callable[[T], None]
33my_decorate: Callable[[U[T]], U[T]] = decorator_variable
34
Community Discussions contain sources that include Stack Exchange Network
QUESTION
How can I make an object with an interface like a random number generator, but that actually generates a specified sequence?
Asked 2022-Mar-31 at 13:47I'd like to construct an object that works like a random number generator, but generates numbers in a specified sequence.
1# a random number generator
2rng = lambda : np.random.randint(2,20)//2
3
4# a non-random number generator
5def nrng():
6 numbers = np.arange(1,10.5,0.5)
7 for i in range(len(numbers)):
8 yield numbers[i]
9
10for j in range(10):
11 print('random number', rng())
12 print('non-random number', nrng())
13
The issue with the code above that I cannot call nrng
in the last line because it is a generator. I know that the most straightforward way to rewrite the code above is to simply loop over the non-random numbers instead of defining the generator. I would prefer getting the example above to work because I am working with a large chunk of code that include a function that accepts a random number generator as an argument, and I would like to add the functionality to pass non-random number sequences without rewriting the entire code.
EDIT: I see some confusion in the comments. I am aware that python's random number generators generate pseudo-random numbers. This post is about replacing a pseudo-random-number generator by a number generator that generates numbers from a non-random, user-specified sequence (e.g., a generator that generates the number sequence 1,1,2,2,1,0,1
if I want it to).
ANSWER
Answered 2022-Mar-29 at 00:47You can call next()
with a generator or iterator as an argument to withdraw exactly one element from it. Saving the generator to a variable beforehand allows you to do this multiple times.
1# a random number generator
2rng = lambda : np.random.randint(2,20)//2
3
4# a non-random number generator
5def nrng():
6 numbers = np.arange(1,10.5,0.5)
7 for i in range(len(numbers)):
8 yield numbers[i]
9
10for j in range(10):
11 print('random number', rng())
12 print('non-random number', nrng())
13# make it a generator
14def _rng():
15 while True:
16 yield np.random.randint(2,20)//2
17
18# a non-random number generator
19def _nrng():
20 numbers = np.arange(1,10.5,0.5)
21 for i in range(len(numbers)):
22 yield numbers[i]
23
24rng = _rng()
25nrng = _nrng()
26for j in range(10):
27 print('random number', next(rng))
28 print('non-random number', next(nrng))
29
QUESTION
Why is `np.sum(range(N))` very slow?
Asked 2022-Mar-29 at 14:31I saw a video about speed of loops in python, where it was explained that doing sum(range(N))
is much faster than manually looping through range
and adding the variables together, since the former runs in C due to built-in functions being used, while in the latter the summation is done in (slow) python. I was curious what happens when adding numpy
to the mix. As I expected np.sum(np.arange(N))
is the fastest, but sum(np.arange(N))
and np.sum(range(N))
are even slower than doing the naive for loop.
Why is this?
Here's the script I used to test, some comments about the supposed cause of slowing done where I know (taken mostly from the video) and the results I got on my machine (python 3.10.0, numpy 1.21.2):
updated script:
1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8 s = 0
9 i = 0
10 while i < N: # condition is checked in python
11 s += i
12 i += 1 # both additions are done in python
13 return s
14
15def sum1(N = N):
16 s = 0
17 for i in range(N): # increment in C
18 s += i # addition in python
19 return s
20
21def sum2(N = N):
22 return sum(range(N)) # everything in C
23
24def sum3(N = N):
25 return sum(list(range(N)))
26
27def sum4(N = N):
28 return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31 # much faster np.array conversion
32 return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35 # much faster np.array conversion
36 return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39 # possibly slow conversion to Py_long from np.int
40 return sum(np.arange(N))
41
42def sum7(N = N):
43 # list returns a list of np.int-s
44 return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47 # tolist conversion to python int seems faster than the implicit conversion
48 # in sum(list()) (tolist returns a list of python int-s)
49 return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52 return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55 return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58 return np.array(range(N))
59
60def array_dtype(N = N):
61 return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64 # np.sum's source code mentions to use fromiter to convert from generators
65 return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop: {timeit(sum0, number = repetition)}")
68print(f"for loop: {timeit(sum1, number = repetition)}")
69print(f"sum_range: {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist: {timeit(sum3, number = repetition)}")
71print(f"npsum_range: {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange: {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2: {timeit(sum5, number = repetition)}")
74print(f"sum_arange: {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange: {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist: {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange: {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum: {timeit(sum9, number = repetition)}")
79print(f"array_basic: {timeit(array_basic, number = repetition)}")
80print(f"array_dtype: {timeit(array_dtype, number = repetition)}")
81print(f"array_iter: {timeit(array_iter, number = repetition)}")
82
83print(f"npsumarangeREP: {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP: {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop: 11.493371912998555
89# for loop: 7.385945574002108
90# sum_range: 2.4605720699983067
91# sum_rangelist: 4.509678105998319
92# npsum_range: 11.85120212900074
93# npsum_iterrange: 4.464334709002287
94# npsum_iterrangev2: 4.498494338993623
95# sum_arange: 9.537815956995473
96# sum_list_arange: 13.290120724996086
97# sum_arange_tolist: 5.231948580003518
98# npsum_arange: 0.241889145996538
99# nparangenpsum: 0.21876695199898677
100# array_basic: 11.736577274998126
101# array_dtype: 8.71628468400013
102# array_iter: 4.303306431000237
103# npsumarangeREP: 21.240833958996518
104# npsumarangeREP: 16.690092379001726
105
106
ANSWER
Answered 2021-Oct-16 at 17:42From the cpython source code for sum
sum initially seems to attempt a fast path that assumes all inputs are the same type. If that fails it will just iterate:
1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8 s = 0
9 i = 0
10 while i < N: # condition is checked in python
11 s += i
12 i += 1 # both additions are done in python
13 return s
14
15def sum1(N = N):
16 s = 0
17 for i in range(N): # increment in C
18 s += i # addition in python
19 return s
20
21def sum2(N = N):
22 return sum(range(N)) # everything in C
23
24def sum3(N = N):
25 return sum(list(range(N)))
26
27def sum4(N = N):
28 return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31 # much faster np.array conversion
32 return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35 # much faster np.array conversion
36 return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39 # possibly slow conversion to Py_long from np.int
40 return sum(np.arange(N))
41
42def sum7(N = N):
43 # list returns a list of np.int-s
44 return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47 # tolist conversion to python int seems faster than the implicit conversion
48 # in sum(list()) (tolist returns a list of python int-s)
49 return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52 return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55 return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58 return np.array(range(N))
59
60def array_dtype(N = N):
61 return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64 # np.sum's source code mentions to use fromiter to convert from generators
65 return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop: {timeit(sum0, number = repetition)}")
68print(f"for loop: {timeit(sum1, number = repetition)}")
69print(f"sum_range: {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist: {timeit(sum3, number = repetition)}")
71print(f"npsum_range: {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange: {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2: {timeit(sum5, number = repetition)}")
74print(f"sum_arange: {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange: {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist: {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange: {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum: {timeit(sum9, number = repetition)}")
79print(f"array_basic: {timeit(array_basic, number = repetition)}")
80print(f"array_dtype: {timeit(array_dtype, number = repetition)}")
81print(f"array_iter: {timeit(array_iter, number = repetition)}")
82
83print(f"npsumarangeREP: {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP: {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop: 11.493371912998555
89# for loop: 7.385945574002108
90# sum_range: 2.4605720699983067
91# sum_rangelist: 4.509678105998319
92# npsum_range: 11.85120212900074
93# npsum_iterrange: 4.464334709002287
94# npsum_iterrangev2: 4.498494338993623
95# sum_arange: 9.537815956995473
96# sum_list_arange: 13.290120724996086
97# sum_arange_tolist: 5.231948580003518
98# npsum_arange: 0.241889145996538
99# nparangenpsum: 0.21876695199898677
100# array_basic: 11.736577274998126
101# array_dtype: 8.71628468400013
102# array_iter: 4.303306431000237
103# npsumarangeREP: 21.240833958996518
104# npsumarangeREP: 16.690092379001726
105
106/* Fast addition by keeping temporary sums in C instead of new Python objects.
107 Assumes all inputs are the same type. If the assumption fails, default
108 to the more general routine.
109*/
110
I'm not entirely certain what is happening under the hood, but it is likely the repeated creation/conversion of C types to Python objects that is causing these slow-downs. It's worth noting that both sum
and range
are implemented in C.
This next bit is not really an answer to the question, but I wondered if we could speed up sum
for python range
s as range
is quite a smart object.
To do this I've used functools.singledispatch
to override the built-in sum
function specifically for the range
type; then implemented a small function to calculate the sum of an arithmetic progression.
1import numpy as np
2from timeit import timeit
3
4N = 10_000_000
5repetition = 10
6
7def sum0(N = N):
8 s = 0
9 i = 0
10 while i < N: # condition is checked in python
11 s += i
12 i += 1 # both additions are done in python
13 return s
14
15def sum1(N = N):
16 s = 0
17 for i in range(N): # increment in C
18 s += i # addition in python
19 return s
20
21def sum2(N = N):
22 return sum(range(N)) # everything in C
23
24def sum3(N = N):
25 return sum(list(range(N)))
26
27def sum4(N = N):
28 return np.sum(range(N)) # very slow np.array conversion
29
30def sum5(N = N):
31 # much faster np.array conversion
32 return np.sum(np.fromiter(range(N),dtype = int))
33
34def sum5v2_(N = N):
35 # much faster np.array conversion
36 return np.sum(np.fromiter(range(N),dtype = np.int_))
37
38def sum6(N = N):
39 # possibly slow conversion to Py_long from np.int
40 return sum(np.arange(N))
41
42def sum7(N = N):
43 # list returns a list of np.int-s
44 return sum(list(np.arange(N)))
45
46def sum7v2(N = N):
47 # tolist conversion to python int seems faster than the implicit conversion
48 # in sum(list()) (tolist returns a list of python int-s)
49 return sum(np.arange(N).tolist())
50
51def sum8(N = N):
52 return np.sum(np.arange(N)) # everything in numpy (fortran libblas?)
53
54def sum9(N = N):
55 return np.arange(N).sum() # remove dispatch overhead
56
57def array_basic(N = N):
58 return np.array(range(N))
59
60def array_dtype(N = N):
61 return np.array(range(N),dtype = np.int_)
62
63def array_iter(N = N):
64 # np.sum's source code mentions to use fromiter to convert from generators
65 return np.fromiter(range(N),dtype = np.int_)
66
67print(f"while loop: {timeit(sum0, number = repetition)}")
68print(f"for loop: {timeit(sum1, number = repetition)}")
69print(f"sum_range: {timeit(sum2, number = repetition)}")
70print(f"sum_rangelist: {timeit(sum3, number = repetition)}")
71print(f"npsum_range: {timeit(sum4, number = repetition)}")
72print(f"npsum_iterrange: {timeit(sum5, number = repetition)}")
73print(f"npsum_iterrangev2: {timeit(sum5, number = repetition)}")
74print(f"sum_arange: {timeit(sum6, number = repetition)}")
75print(f"sum_list_arange: {timeit(sum7, number = repetition)}")
76print(f"sum_arange_tolist: {timeit(sum7v2, number = repetition)}")
77print(f"npsum_arange: {timeit(sum8, number = repetition)}")
78print(f"nparangenpsum: {timeit(sum9, number = repetition)}")
79print(f"array_basic: {timeit(array_basic, number = repetition)}")
80print(f"array_dtype: {timeit(array_dtype, number = repetition)}")
81print(f"array_iter: {timeit(array_iter, number = repetition)}")
82
83print(f"npsumarangeREP: {timeit(lambda : sum8(N/1000), number = 100000*repetition)}")
84print(f"npsumarangeREP: {timeit(lambda : sum9(N/1000), number = 100000*repetition)}")
85
86# Example output:
87#
88# while loop: 11.493371912998555
89# for loop: 7.385945574002108
90# sum_range: 2.4605720699983067
91# sum_rangelist: 4.509678105998319
92# npsum_range: 11.85120212900074
93# npsum_iterrange: 4.464334709002287
94# npsum_iterrangev2: 4.498494338993623
95# sum_arange: 9.537815956995473
96# sum_list_arange: 13.290120724996086
97# sum_arange_tolist: 5.231948580003518
98# npsum_arange: 0.241889145996538
99# nparangenpsum: 0.21876695199898677
100# array_basic: 11.736577274998126
101# array_dtype: 8.71628468400013
102# array_iter: 4.303306431000237
103# npsumarangeREP: 21.240833958996518
104# npsumarangeREP: 16.690092379001726
105
106/* Fast addition by keeping temporary sums in C instead of new Python objects.
107 Assumes all inputs are the same type. If the assumption fails, default
108 to the more general routine.
109*/
110from functools import singledispatch
111
112def sum_range(range_, /, start=0):
113 """Overloaded `sum` for range, compute arithmetic sum"""
114 n = len(range_)
115 if not n:
116 return start
117 return int(start + (n * (range_[0] + range_[-1]) / 2))
118
119sum = singledispatch(sum)
120sum.register(range, sum_range)
121
122def test():
123 """
124 >>> sum(range(0, 100))
125 4950
126 >>> sum(range(0, 10, 2))
127 20
128 >>> sum(range(0, 9, 2))
129 20
130 >>> sum(range(0, -10, -1))
131 -45
132 >>> sum(range(-10, 10))
133 -10
134 >>> sum(range(-1, -100, -2))
135 -2500
136 >>> sum(range(0, 10, 100))
137 0
138 >>> sum(range(0, 0))
139 0
140 >>> sum(range(0, 100), 50)
141 5000
142 >>> sum(range(0, 0), 10)
143 10
144 """
145
146if __name__ == "__main__":
147 import doctest
148 doctest.testmod()
149
I'm not sure if this is complete, but it's definitely faster than looping.
QUESTION
How would you implement a lazy "range factory" for C++20 ranges that just calls a generator function?
Asked 2022-Feb-17 at 17:10I like the idea of the lazy ranges you can make with std::views::iota
but was surprised to see that iota
is currently the only thing like it in the standard; it is the only "range factory" besides views::single
and views::empty
. There is not currently, for example, the equivalent of std::generate
as a range factory.
I note however it is trivial to implement the semantics of generate
by using a transform view on iota and just ignoring the value iota passes to transform i.e.
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25
My questions is what would be "the real way" to implement something like this? To make a range view object that is pipeable that does not just use iota
.
I tried inheriting from ranges::view_interface
-- no idea if this is the correct approach -- and just having it return a dummy iterator that calls a generator function but my code doesn't work because of the part where it needs to pipe the range view to std::views::take
in order to not cause an infinite loop. The object I define here does not end up being pipeable.
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32 using value_type = decltype(std::declval<F>()());
33
34 class iterator {
35 const F* gen_func_;
36 public:
37 iterator(const F* f) : gen_func_(f)
38 {}
39
40 value_type operator*() const {
41 return (*gen_func_)();
42 }
43
44 bool operator!=(const iterator&) {
45 return true;
46 }
47
48 iterator& operator++() {
49 return *this;
50 }
51 };
52
53 F generator_func_;
54
55public:
56
57 generate2(const F& f) : generator_func_(f) {
58 }
59
60 iterator begin() {
61 return iterator(&generator_func_);
62 }
63
64 iterator end() {
65 return iterator(nullptr);
66 }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74 auto d6 = []() {
75 static std::uniform_int_distribution<> dist(1, 6);
76 return dist(rng);
77 };
78
79 // the following doesnt compile because of the pipe...
80 for (int v : generate2(d6) | std::views::take(10)) {
81 std::cout << v << ' ';
82 }
83 std::cout << '\n';
84}
85
ANSWER
Answered 2022-Feb-17 at 17:10The reason why generate2
cannot work is that it does not model the range
concept, that is, the type returned by its begin()
does not model input_iterator
, because input_iterator
requires difference_type
and value_type
to exist and i++
is a valid expression.
In addition, your iterator does not satisfy sentinel_for<iterator>
, which means that it cannot serve as its own sentinel, because sentinel_for
requires semiregular
which requires default_initializable
, so you also need to add default constructors for it.
You also need to rewrite bool operator!=(...)
to bool operator==(...) const
since operator!=
does not reverse synthesize operator==
. But it's easier to just use default_sentinel_t
as sentinel in your case.
if you add them to iterator
you will find the code will be well-formed:
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32 using value_type = decltype(std::declval<F>()());
33
34 class iterator {
35 const F* gen_func_;
36 public:
37 iterator(const F* f) : gen_func_(f)
38 {}
39
40 value_type operator*() const {
41 return (*gen_func_)();
42 }
43
44 bool operator!=(const iterator&) {
45 return true;
46 }
47
48 iterator& operator++() {
49 return *this;
50 }
51 };
52
53 F generator_func_;
54
55public:
56
57 generate2(const F& f) : generator_func_(f) {
58 }
59
60 iterator begin() {
61 return iterator(&generator_func_);
62 }
63
64 iterator end() {
65 return iterator(nullptr);
66 }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74 auto d6 = []() {
75 static std::uniform_int_distribution<> dist(1, 6);
76 return dist(rng);
77 };
78
79 // the following doesnt compile because of the pipe...
80 for (int v : generate2(d6) | std::views::take(10)) {
81 std::cout << v << ' ';
82 }
83 std::cout << '\n';
84}
85class iterator {
86 public:
87 using value_type = decltype(std::declval<F>()());
88 using difference_type = std::ptrdiff_t;
89 iterator() = default;
90 void operator++(int);
91 bool operator==(const iterator&) const {
92 return false;
93 }
94 // ...
95};
96
However, the operator*()
of iterator
does not meet the requirements of equality-preserving, that is to say, the results obtained by the two calls before and after are not equal, which means that this will be undefined behavior.
You can refer to the implementation of ranges::istream_view
to use a member variable to cache each generated result, then you only need to return the cached value each time iterator::operator*()
is called.
1#include <iostream>
2#include <ranges>
3#include <random>
4
5template<typename F>
6auto generate1(const F& func) {
7 return std::views::iota(0) | std::views::transform([&func](int) {return func(); });
8}
9
10std::random_device dev;
11std::mt19937 rng(dev());
12
13int main() {
14
15 auto d6 = []() {
16 static std::uniform_int_distribution<> dist(1, 6);
17 return dist(rng);
18 };
19
20 for (int v : generate1(d6) | std::views::take(10)) {
21 std::cout << v << ' ';
22 }
23 std::cout << '\n';
24}
25#include <iostream>
26#include <ranges>
27#include <random>
28
29template<typename F>
30class generate2 : public std::ranges::view_interface<generate2<F>>
31{
32 using value_type = decltype(std::declval<F>()());
33
34 class iterator {
35 const F* gen_func_;
36 public:
37 iterator(const F* f) : gen_func_(f)
38 {}
39
40 value_type operator*() const {
41 return (*gen_func_)();
42 }
43
44 bool operator!=(const iterator&) {
45 return true;
46 }
47
48 iterator& operator++() {
49 return *this;
50 }
51 };
52
53 F generator_func_;
54
55public:
56
57 generate2(const F& f) : generator_func_(f) {
58 }
59
60 iterator begin() {
61 return iterator(&generator_func_);
62 }
63
64 iterator end() {
65 return iterator(nullptr);
66 }
67};
68
69std::random_device dev;
70std::mt19937 rng(dev());
71
72int main() {
73
74 auto d6 = []() {
75 static std::uniform_int_distribution<> dist(1, 6);
76 return dist(rng);
77 };
78
79 // the following doesnt compile because of the pipe...
80 for (int v : generate2(d6) | std::views::take(10)) {
81 std::cout << v << ' ';
82 }
83 std::cout << '\n';
84}
85class iterator {
86 public:
87 using value_type = decltype(std::declval<F>()());
88 using difference_type = std::ptrdiff_t;
89 iterator() = default;
90 void operator++(int);
91 bool operator==(const iterator&) const {
92 return false;
93 }
94 // ...
95};
96template<typename F>
97class generate2 : public std::ranges::view_interface<generate2<F>> {
98 public:
99 auto begin() {
100 value_ = generator_func_();
101 return iterator{*this};
102 }
103
104 std::default_sentinel_t end() const noexcept { return std::default_sentinel; }
105
106 class iterator {
107 public:
108 //...
109 value_type operator*() const {
110 return parent_->value_;
111 }
112 private:
113 generate2* parent_;
114 };
115
116 private:
117 F generator_func_;
118 std::remove_cvref_t<std::invoke_result_t<F&>> value_;
119};
120
QUESTION
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './lib/tokenize' is not defined by "exports" in the package.json of a module in node_modules
Asked 2022-Jan-31 at 17:22This is a React web app. When I run
1npm start
2
This error occurred