Popular New Releases in Batch Processing
dataloader
Release v7.0.0
laravelBatch
Laravel Batch v2.3.3
optbinning
OptBinning 0.14.1
spring-batch-lightmin
2.2.1
async-batch
Include type definitions
Popular Libraries in Batch Processing
by graphql 
javascript
11321 MIT
DataLoader is a generic utility to be used as part of your application's data fetching layer to provide a consistent API over various backends and reduce requests to those backends via batching and caching.
by graph-gophers go
841 MIT
Implementation of Facebook's DataLoader in Golang
by mavinoo php
392 MIT
insert batch and update batch in laravel
by vektah go
370 MIT
go generate based DataLoader
by yc9559 shell
316 LGPL-3.0
optimize the interactive parameters of cpufreq driver
by reidmorrison ruby
264 Apache-2.0
Ruby's missing background and batch processing system
by rocketjob ruby
260 Apache-2.0
Ruby's missing background and batch processing system
by guillermo-navas-palencia python
205 Apache-2.0
Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.
by xpadro java
163
Samples of different Spring Integration modules (jms, batch, integration)
Trending New libraries in Batch Processing
by yuhuixu1993 jupyter notebook
48
Batch Normalization with Enhanced Linear Transformation
by jojoldu java
39
스프링배치와 QuerydslPagingItemReader
by Cybertek-B22 java
21
Java programming repository for batch 22
by DeanReynolds csharp
13 MIT
A set of highly-optimized, flexible and powerful 2D spatial partitions for MonoGame
by volkanozsarac python
12 GPL-3.0
Toolbox for ground motion selection and processing
by mafintosh javascript
9 MIT
Addressable batches of data on top of a Hypercore
by jchristel python
9 LGPL-3.0
Code samples for the Revit Batch Processor
by ZhaoyangLi-nju python
6
dataloader for sunrgbd,citscapes,nyud2,coco,pascalvoc
by anic17 c
5 GPL-3.0
A repository that contains some batch plugins
Top Authors in Batch Processing
1
2 Libraries
5
2
2 Libraries
16
3
2 Libraries
13
4
2 Libraries
6
5
1 Libraries
11321
6
1 Libraries
2
7
1 Libraries
2
8
1 Libraries
2
9
1 Libraries
11
10
1 Libraries
2
1
2 Libraries
5
2
2 Libraries
16
3
2 Libraries
13
4
2 Libraries
6
5
1 Libraries
11321
6
1 Libraries
2
7
1 Libraries
2
8
1 Libraries
2
9
1 Libraries
11
10
1 Libraries
2
Trending Kits in Batch Processing
No Trending Kits are available at this moment for Batch Processing
Trending Discussions on Batch Processing
speeding up 1d convolution in PyTorch
Java Design Pattern (Orchestration/Workflow)
Gluonfx build using docker fails, with error: Cannot run program "ie4uinit": CreateProcess error=2, The system cannot find the file specified
React Native Table with Row Selection
R2DBC vs Spring Jdbc Vs Spring Data JDBC?
Create Pytorch "Stack of Views" to save on GPU memory
Configure depth overflow value - Start-Job
Does Flink's windowing operation process elements at the end of window or does it do a rolling processing?
How to speed up if statements with for loop
Adding a border to an SVG image element with SVG filters
QUESTION
speeding up 1d convolution in PyTorch
Asked 2022-Apr-05 at 12:04For my project I am using pytorch as a linear algebra backend. For the performance part of my code, I need to do 1D convolutions of 2 small (length between 2 and 9) vectors (1D tensors) a very large number of times. My code allows for batch-processing of inputs and thus I can stack a couple of input vectors to create matrices that can then be convolved all at the same time. Since torch.conv1d does not allow for convolving along a single dimension for 2D inputs, I had to write my own convolution function called convolve. This new function however consists of a double for-loop and is therefore very very slow.
Question: how can I make the convolve function perform faster through better code-design and let it be able to deal with batched inputs (=2D tensors)?
Partial answer: somehow avoid the double for-loop
Below are three jupyter notebook cells that recreate a minimal example. Note that the you need line_profiler and the %%writefile magic command to make this work!
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28I write the code to the SO_CONVOLVE_QUESTION.py because that is necessary for line_profiler and to use as a setup for timeit.timeit.
Now we can evaluate the output and performance of the code above on non-batch input (x, y) and batched input (x_batch, y_batch):
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28from SO_CONVOLVE_QUESTION import *
29# Without batch processing
30res1 = conv1d(x, y)
31res = convolve(x, y)
32print(torch.allclose(res1, res)) # True
33
34# With batch processing, NB first dimension!
35x_batch = torch.randn(5, 5)
36y_batch = torch.randn(5, 7)
37
38results = []
39for i in range(5):
40 results.append(conv1d(x_batch[i, :], y_batch[i, :]))
41res1 = torch.stack(results)
42res = convolve(x_batch, y_batch)
43print(torch.allclose(res1, res)) # True
44
45print(timeit.timeit('convolve(x, y)', setup=setup, number=10000)) # 4.83391789999996
46print(timeit.timeit('conv1d(x, y)', setup=setup, number=10000)) # 0.2799923000000035
47In the block above you can see that performing convolution 5 times using the conv1d function produces the same result as convolve on batched inputs. We can also see that convolve (= 4.8s) is much slower than the conv1d (= 0.28s). Below we assess the slow part of the convolve function WITHOUT batch processing using line_profiler:
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28from SO_CONVOLVE_QUESTION import *
29# Without batch processing
30res1 = conv1d(x, y)
31res = convolve(x, y)
32print(torch.allclose(res1, res)) # True
33
34# With batch processing, NB first dimension!
35x_batch = torch.randn(5, 5)
36y_batch = torch.randn(5, 7)
37
38results = []
39for i in range(5):
40 results.append(conv1d(x_batch[i, :], y_batch[i, :]))
41res1 = torch.stack(results)
42res = convolve(x_batch, y_batch)
43print(torch.allclose(res1, res)) # True
44
45print(timeit.timeit('convolve(x, y)', setup=setup, number=10000)) # 4.83391789999996
46print(timeit.timeit('conv1d(x, y)', setup=setup, number=10000)) # 0.2799923000000035
47%load_ext line_profiler
48%lprun -f convolve convolve(x, y) # evaluated without batch-processing!
49Output:
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28from SO_CONVOLVE_QUESTION import *
29# Without batch processing
30res1 = conv1d(x, y)
31res = convolve(x, y)
32print(torch.allclose(res1, res)) # True
33
34# With batch processing, NB first dimension!
35x_batch = torch.randn(5, 5)
36y_batch = torch.randn(5, 7)
37
38results = []
39for i in range(5):
40 results.append(conv1d(x_batch[i, :], y_batch[i, :]))
41res1 = torch.stack(results)
42res = convolve(x_batch, y_batch)
43print(torch.allclose(res1, res)) # True
44
45print(timeit.timeit('convolve(x, y)', setup=setup, number=10000)) # 4.83391789999996
46print(timeit.timeit('conv1d(x, y)', setup=setup, number=10000)) # 0.2799923000000035
47%load_ext line_profiler
48%lprun -f convolve convolve(x, y) # evaluated without batch-processing!
49Timer unit: 1e-07 s
50
51Total time: 0.0010383 s
52File: C:\python_projects\pysumo\SO_CONVOLVE_QUESTION.py
53Function: convolve at line 9
54
55Line # Hits Time Per Hit % Time Line Contents
56==============================================================
57 9 def convolve(a, v):
58 10 1 68.0 68.0 0.7 if a.ndim == 1:
59 11 1 271.0 271.0 2.6 a = a.view(1, -1)
60 12 1 44.0 44.0 0.4 v = v.view(1, -1)
61 13
62 14 1 28.0 28.0 0.3 nrows, vcols = v.shape
63 15 1 12.0 12.0 0.1 acols = a.shape[1]
64 16
65 17 1 4337.0 4337.0 41.8 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
66 18 1 12.0 12.0 0.1 noutdim = vcols + acols - 1
67 19 1 127.0 127.0 1.2 out = torch.zeros((nrows, noutdim))
68 20 6 32.0 5.3 0.3 for i in range(acols):
69 21 40 209.0 5.2 2.0 for j in range(vcols):
70 22 35 5194.0 148.4 50.0 out[:, i+j] += expanded[:, i, j]
71 23 1 49.0 49.0 0.5 return out.squeeze()
72Obviously a double for-loop and the line creating the expanded tensor are the slowest. Are these parts avoidable with better code-design?
ANSWER
Answered 2022-Mar-31 at 17:17Pytorch has a batch analyzing tool called torch.nn.functional and there you have a conv1d function (obviously 2d as well and much much more). we will use conv1d.
Suppose you want to convolve 100 vectors given in v1 with 1 another vector given in v2. v1 has dimension of (minibatch , in channels , weights) and you need 1 channel by default. In addition, v2 has dimensions of * (\text{out_channels} , (out_channels,groups / in_channels,kW)*. You are using 1 channel and therefore 1 group so v1 and v2 will be given by:
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28from SO_CONVOLVE_QUESTION import *
29# Without batch processing
30res1 = conv1d(x, y)
31res = convolve(x, y)
32print(torch.allclose(res1, res)) # True
33
34# With batch processing, NB first dimension!
35x_batch = torch.randn(5, 5)
36y_batch = torch.randn(5, 7)
37
38results = []
39for i in range(5):
40 results.append(conv1d(x_batch[i, :], y_batch[i, :]))
41res1 = torch.stack(results)
42res = convolve(x_batch, y_batch)
43print(torch.allclose(res1, res)) # True
44
45print(timeit.timeit('convolve(x, y)', setup=setup, number=10000)) # 4.83391789999996
46print(timeit.timeit('conv1d(x, y)', setup=setup, number=10000)) # 0.2799923000000035
47%load_ext line_profiler
48%lprun -f convolve convolve(x, y) # evaluated without batch-processing!
49Timer unit: 1e-07 s
50
51Total time: 0.0010383 s
52File: C:\python_projects\pysumo\SO_CONVOLVE_QUESTION.py
53Function: convolve at line 9
54
55Line # Hits Time Per Hit % Time Line Contents
56==============================================================
57 9 def convolve(a, v):
58 10 1 68.0 68.0 0.7 if a.ndim == 1:
59 11 1 271.0 271.0 2.6 a = a.view(1, -1)
60 12 1 44.0 44.0 0.4 v = v.view(1, -1)
61 13
62 14 1 28.0 28.0 0.3 nrows, vcols = v.shape
63 15 1 12.0 12.0 0.1 acols = a.shape[1]
64 16
65 17 1 4337.0 4337.0 41.8 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
66 18 1 12.0 12.0 0.1 noutdim = vcols + acols - 1
67 19 1 127.0 127.0 1.2 out = torch.zeros((nrows, noutdim))
68 20 6 32.0 5.3 0.3 for i in range(acols):
69 21 40 209.0 5.2 2.0 for j in range(vcols):
70 22 35 5194.0 148.4 50.0 out[:, i+j] += expanded[:, i, j]
71 23 1 49.0 49.0 0.5 return out.squeeze()
72import torch
73from torch.nn import functional as F
74
75num_vectors = 100
76len_vectors = 9
77v1 = torch.rand((num_vectors, 1, len_vectors))
78v2 = torch.rand(1, 1, 6)
79now we can simply compute the necessary padding via
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28from SO_CONVOLVE_QUESTION import *
29# Without batch processing
30res1 = conv1d(x, y)
31res = convolve(x, y)
32print(torch.allclose(res1, res)) # True
33
34# With batch processing, NB first dimension!
35x_batch = torch.randn(5, 5)
36y_batch = torch.randn(5, 7)
37
38results = []
39for i in range(5):
40 results.append(conv1d(x_batch[i, :], y_batch[i, :]))
41res1 = torch.stack(results)
42res = convolve(x_batch, y_batch)
43print(torch.allclose(res1, res)) # True
44
45print(timeit.timeit('convolve(x, y)', setup=setup, number=10000)) # 4.83391789999996
46print(timeit.timeit('conv1d(x, y)', setup=setup, number=10000)) # 0.2799923000000035
47%load_ext line_profiler
48%lprun -f convolve convolve(x, y) # evaluated without batch-processing!
49Timer unit: 1e-07 s
50
51Total time: 0.0010383 s
52File: C:\python_projects\pysumo\SO_CONVOLVE_QUESTION.py
53Function: convolve at line 9
54
55Line # Hits Time Per Hit % Time Line Contents
56==============================================================
57 9 def convolve(a, v):
58 10 1 68.0 68.0 0.7 if a.ndim == 1:
59 11 1 271.0 271.0 2.6 a = a.view(1, -1)
60 12 1 44.0 44.0 0.4 v = v.view(1, -1)
61 13
62 14 1 28.0 28.0 0.3 nrows, vcols = v.shape
63 15 1 12.0 12.0 0.1 acols = a.shape[1]
64 16
65 17 1 4337.0 4337.0 41.8 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
66 18 1 12.0 12.0 0.1 noutdim = vcols + acols - 1
67 19 1 127.0 127.0 1.2 out = torch.zeros((nrows, noutdim))
68 20 6 32.0 5.3 0.3 for i in range(acols):
69 21 40 209.0 5.2 2.0 for j in range(vcols):
70 22 35 5194.0 148.4 50.0 out[:, i+j] += expanded[:, i, j]
71 23 1 49.0 49.0 0.5 return out.squeeze()
72import torch
73from torch.nn import functional as F
74
75num_vectors = 100
76len_vectors = 9
77v1 = torch.rand((num_vectors, 1, len_vectors))
78v2 = torch.rand(1, 1, 6)
79padding = torch.min(torch.tensor([v1.shape[-1], v2.shape[-1]])).item() - 1
80and the convolution can be done using
1%%writefile SO_CONVOLVE_QUESTION.py
2import torch
3
4def conv1d(a, v):
5 padding = v.shape[-1] - 1
6 return torch.conv1d(
7 input=a.view(1, 1, -1), weight=v.flip(0).view(1, 1, -1), padding=padding, stride=1
8 ).squeeze()
9
10def convolve(a, v):
11 if a.ndim == 1:
12 a = a.view(1, -1)
13 v = v.view(1, -1)
14
15 nrows, vcols = v.shape
16 acols = a.shape[1]
17
18 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
19 noutdim = vcols + acols - 1
20 out = torch.zeros((nrows, noutdim))
21 for i in range(acols):
22 for j in range(vcols):
23 out[:, i+j] += expanded[:, i, j]
24 return out.squeeze()
25
26x = torch.randn(5)
27y = torch.randn(7)
28from SO_CONVOLVE_QUESTION import *
29# Without batch processing
30res1 = conv1d(x, y)
31res = convolve(x, y)
32print(torch.allclose(res1, res)) # True
33
34# With batch processing, NB first dimension!
35x_batch = torch.randn(5, 5)
36y_batch = torch.randn(5, 7)
37
38results = []
39for i in range(5):
40 results.append(conv1d(x_batch[i, :], y_batch[i, :]))
41res1 = torch.stack(results)
42res = convolve(x_batch, y_batch)
43print(torch.allclose(res1, res)) # True
44
45print(timeit.timeit('convolve(x, y)', setup=setup, number=10000)) # 4.83391789999996
46print(timeit.timeit('conv1d(x, y)', setup=setup, number=10000)) # 0.2799923000000035
47%load_ext line_profiler
48%lprun -f convolve convolve(x, y) # evaluated without batch-processing!
49Timer unit: 1e-07 s
50
51Total time: 0.0010383 s
52File: C:\python_projects\pysumo\SO_CONVOLVE_QUESTION.py
53Function: convolve at line 9
54
55Line # Hits Time Per Hit % Time Line Contents
56==============================================================
57 9 def convolve(a, v):
58 10 1 68.0 68.0 0.7 if a.ndim == 1:
59 11 1 271.0 271.0 2.6 a = a.view(1, -1)
60 12 1 44.0 44.0 0.4 v = v.view(1, -1)
61 13
62 14 1 28.0 28.0 0.3 nrows, vcols = v.shape
63 15 1 12.0 12.0 0.1 acols = a.shape[1]
64 16
65 17 1 4337.0 4337.0 41.8 expanded = a.view((nrows, acols, 1)) * v.view((nrows, 1, vcols))
66 18 1 12.0 12.0 0.1 noutdim = vcols + acols - 1
67 19 1 127.0 127.0 1.2 out = torch.zeros((nrows, noutdim))
68 20 6 32.0 5.3 0.3 for i in range(acols):
69 21 40 209.0 5.2 2.0 for j in range(vcols):
70 22 35 5194.0 148.4 50.0 out[:, i+j] += expanded[:, i, j]
71 23 1 49.0 49.0 0.5 return out.squeeze()
72import torch
73from torch.nn import functional as F
74
75num_vectors = 100
76len_vectors = 9
77v1 = torch.rand((num_vectors, 1, len_vectors))
78v2 = torch.rand(1, 1, 6)
79padding = torch.min(torch.tensor([v1.shape[-1], v2.shape[-1]])).item() - 1
80conv_result = temp = F.conv1d(v1, v2, padding=padding)
81I did not time it but it should be considerably faster than your initial double for loop.
QUESTION
Java Design Pattern (Orchestration/Workflow)
Asked 2022-Apr-01 at 08:19I need to automate the workflow after an event occurred. I do have experience in CRUD applications but not in Workflow/Batch processing. Need help in designing the system.
Requirement
The workflow involves 5 steps. Each step is a REST call and are dependent on previous step. EX of Steps: (VerifyIfUserInSystem, CreateUserIfNeeded, EnrollInOpt1, EnrollInOpt2,..)
My thought process is to maintain 2 DB Tables
WORKFLOW_STATUS Table which contains columns like (foreign key(referring to primary table), Workflow Status: (NEW, INPROGRESS, FINISHED, FAILED), Completed Step: (STEP1, STEP2,..), Processed Time,..)
EVENT_LOG Table to maintain the track of Events/Exceptions for a particular record (foreign key, STEP, ExceptionLog)
Question
#1. Is this a correct approach to orchestrate the system(which is not that complex)?
#2. As the steps involve REST Calls, I might have to stop the process when a service is not available and resume the process in a later point of time. I am not sure for many retry attempts should be made and how to maintain the no of attempts made before marking it as FAILED. (Guessing create another column in the WORKFLOW_STATUS table called RETRY_ATTEMPT and set some limit before marking it Failed)
#3 Is the EVENT_LOG Table a correct design and what datatype(clob or varchar(2048)) should I be using for exceptionlog? Every step/retry attempts will be inserted as a new record to this table.
#4 How to reset/restart a FAILED entry after a dependent service is back up.
Please direct me to an blogs/videos/resources if available. Thanks in advance.
ANSWER
Answered 2022-Mar-07 at 18:32Have you considered using a workflow orchestration engine like Netflix's Conductor? docs, Github.
Conductor comes with a lot of the features you are looking for built in.
Here's an example workflow that uses two sequential HTTP requests (where the 2nd requires a response from the first):
Input supplies an IP address (and a Accuweather API key)
1{
2 "ipaddress": "98.11.11.125"
3}
4HTTP request 1 locates the zipCode of the IP address.
HTTP request 2 uses the ZipCode (and the apikey) to report the weather.
The output from this workflow is:
1{
2 "ipaddress": "98.11.11.125"
3}
4{
5 "zipcode": "04043",
6 "forecast": "rain"
7}
8
9Your questions:
- I'd use an orchestration tool like Conductor.
- Each of these tasks (defined in Conductor) have retry logic built in. How you implement will vary on expected timings etc. Since the 2 APIs I'm calling here are public (and relatively fast), I don't wait very long between retries:
1{
2 "ipaddress": "98.11.11.125"
3}
4{
5 "zipcode": "04043",
6 "forecast": "rain"
7}
8
9 "retryCount": 3,
10 "retryLogic": "FIXED",
11 "retryDelaySeconds": 5,
12
13Inside the connection, there are more parameters you can tweak:
1{
2 "ipaddress": "98.11.11.125"
3}
4{
5 "zipcode": "04043",
6 "forecast": "rain"
7}
8
9 "retryCount": 3,
10 "retryLogic": "FIXED",
11 "retryDelaySeconds": 5,
12
13 "connectionTimeOut": 1600,
14 "readTimeOut": 1600
15There is also exponential retry logic if desired.
- The event log is stored in ElasticSearch.
- You can build error pathways for all your workflows.
I have this workflow up and running in the Conductor Playground called "Stack_overflow_sequential_http". Create a free account. Run the workflow - click "run workflow, select "Stack_overflow_sequential_http" and use the JSON above to see it in action.
The get_weather connection is a very slow API, so it may fail a few times before succeeding. Copy the workflow, and play with the timeout values to improve the success.
QUESTION
Gluonfx build using docker fails, with error: Cannot run program "ie4uinit": CreateProcess error=2, The system cannot find the file specified
Asked 2022-Mar-29 at 12:26I'm building a native image using the Gluonfx plugin. I'm doing this procedure inside a docker container. The image builds the 7 steps, but doing the link throws an error complaining about missing ie4uinit.exe.
These are the error logs:
1[Mon Mar 28 21:14:21 UTC 2022][INFO] Substrate is tested with the Gluon's GraalVM build which you can find at https://github.com/gluonhq/graal/releases.
2While you can still use other GraalVM builds, there is no guarantee that these will work properly with Substrate
3[Mon Mar 28 21:14:22 UTC 2022][INFO] ==================== LINK TASK ====================
4[Mon Mar 28 21:14:22 UTC 2022][INFO] Default icon.ico image generated in C:\temp\target\gluonfx\x86_64-windows\gensrc\windows\assets.
5Consider copying it to C:\temp\src\windows before performing any modification
6[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Microsoft (R) Incremental Linker Version 14.31.31104.0
7[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Copyright (C) Microsoft Corporation. All rights reserved.
8[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB]
9[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Creating library C:\temp\target\gluonfx\x86_64-windows\X XXX.lib and object C:\temp\target\gluonfx\x86_64-windows\X XXX.exp
10java.io.IOException: Cannot run program "ie4uinit": CreateProcess error=2, The system cannot find the file specified
11 at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
12 at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
13 at com.gluonhq.substrate.util.ProcessRunner.setupProcess(ProcessRunner.java:378)
14 at com.gluonhq.substrate.util.ProcessRunner.runProcess(ProcessRunner.java:236)
15 at com.gluonhq.substrate.util.ProcessRunner.runProcess(ProcessRunner.java:222)
16[INFO] ------------------------------------------------------------------------
17 at com.gluonhq.substrate.target.WindowsTargetConfiguration.clearExplorerCache(WindowsTargetConfiguration.java:276)
18 at com.gluonhq.substrate.target.WindowsTargetConfiguration.link(WindowsTargetConfiguration.java:224)
19[INFO] BUILD FAILURE
20[INFO] ------------------------------------------------------------------------
21 at com.gluonhq.substrate.SubstrateDispatcher.nativeLink(SubstrateDispatcher.java:443)
22 at com.gluonhq.NativeLinkMojo.execute(NativeLinkMojo.java:47)
23 at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
24 at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:301)
25 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:211)
26 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:165)
27 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:157)
28[INFO] Total time: 06:39 min
29 at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:121)
30 at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
31 at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
32 at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:127)
33 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294)
34[INFO] Finished at: 2022-03-28T21:14:25Z
35[INFO] ------------------------------------------------------------------------
36 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
37 at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
38 at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
39 at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
40 at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
41I would like to know if it's possible to add this dependency to my docker image. Or, if not possible, a possible way to suppress the use of it in gluonfx through configuration.
Docker image:
1[Mon Mar 28 21:14:21 UTC 2022][INFO] Substrate is tested with the Gluon's GraalVM build which you can find at https://github.com/gluonhq/graal/releases.
2While you can still use other GraalVM builds, there is no guarantee that these will work properly with Substrate
3[Mon Mar 28 21:14:22 UTC 2022][INFO] ==================== LINK TASK ====================
4[Mon Mar 28 21:14:22 UTC 2022][INFO] Default icon.ico image generated in C:\temp\target\gluonfx\x86_64-windows\gensrc\windows\assets.
5Consider copying it to C:\temp\src\windows before performing any modification
6[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Microsoft (R) Incremental Linker Version 14.31.31104.0
7[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Copyright (C) Microsoft Corporation. All rights reserved.
8[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB]
9[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Creating library C:\temp\target\gluonfx\x86_64-windows\X XXX.lib and object C:\temp\target\gluonfx\x86_64-windows\X XXX.exp
10java.io.IOException: Cannot run program "ie4uinit": CreateProcess error=2, The system cannot find the file specified
11 at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
12 at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
13 at com.gluonhq.substrate.util.ProcessRunner.setupProcess(ProcessRunner.java:378)
14 at com.gluonhq.substrate.util.ProcessRunner.runProcess(ProcessRunner.java:236)
15 at com.gluonhq.substrate.util.ProcessRunner.runProcess(ProcessRunner.java:222)
16[INFO] ------------------------------------------------------------------------
17 at com.gluonhq.substrate.target.WindowsTargetConfiguration.clearExplorerCache(WindowsTargetConfiguration.java:276)
18 at com.gluonhq.substrate.target.WindowsTargetConfiguration.link(WindowsTargetConfiguration.java:224)
19[INFO] BUILD FAILURE
20[INFO] ------------------------------------------------------------------------
21 at com.gluonhq.substrate.SubstrateDispatcher.nativeLink(SubstrateDispatcher.java:443)
22 at com.gluonhq.NativeLinkMojo.execute(NativeLinkMojo.java:47)
23 at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
24 at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:301)
25 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:211)
26 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:165)
27 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:157)
28[INFO] Total time: 06:39 min
29 at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:121)
30 at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
31 at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
32 at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:127)
33 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294)
34[INFO] Finished at: 2022-03-28T21:14:25Z
35[INFO] ------------------------------------------------------------------------
36 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
37 at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
38 at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
39 at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
40 at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
41# escape=`
42# Use the latest Windows Server Core image with .NET Framework 4.8.
43FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019
44
45# Restore the default Windows shell for correct batch processing.
46SHELL ["cmd", "/S", "/C"]
47
48RUN `
49 # Download the Build Tools bootstrapper.
50 curl -SL --output vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
51 `
52 # Install Build Tools, excluding workloads and components with known issues.
53 && (start /w vs_buildtools.exe --quiet --wait --norestart --nocache modify `
54 --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" `
55 --add Microsoft.VisualStudio.Workload.AzureBuildTools `
56 --add Microsoft.VisualStudio.Workload.VCTools `
57 --add Microsoft.VisualStudio.Component.VC.140 `
58 --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
59 --add Microsoft.VisualStudio.Component.VC.CMake.Project `
60 --add Microsoft.VisualStudio.Component.Windows10SDK.19041 `
61 --add Microsoft.VisualStudio.Component.VC.Llvm.Clang `
62 --add Microsoft.Net.Component.4.8.SDK `
63 --remove Microsoft.VisualStudio.Component.Windows10SDK.10240 `
64 --remove Microsoft.VisualStudio.Component.Windows10SDK.10586 `
65 --remove Microsoft.VisualStudio.Component.Windows10SDK.14393 `
66 --remove Microsoft.VisualStudio.Component.Windows81SDK `
67 || IF "%ERRORLEVEL%"=="3010" EXIT 0) `
68 `
69 # Cleanup
70 && del /q vs_buildtools.exe
71
72#Installing GraalVM
73ADD https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.0.0.2/graalvm-ce-java11-windows-amd64-22.0.0.2.zip /
74RUN powershell -Command "expand-archive -Path 'c:\graalvm-ce-java11-windows-amd64-22.0.0.2.zip' -DestinationPath 'c:\'"
75
76#ADD https://download2.gluonhq.com/substrate/javafxstaticsdk/openjfx-18-ea+14-windows-x86_64-static.zip / C:\Users\ContainerAdministrator\.gluon\substrate\
77
78#Installing Maven
79ADD https://dlcdn.apache.org/maven/maven-3/3.8.5/binaries/apache-maven-3.8.5-bin.zip /
80RUN powershell -Command "expand-archive -Path 'c:\apache-maven-3.8.5-bin.zip' -DestinationPath 'c:\'"
81
82ENV JAVA_HOME C:\graalvm-ce-java11-22.0.0.2\
83ENV MAVEN_HOME C:\apache-maven-3.8.5\
84ENV GRAALVM_HOME C:\graalvm-ce-java11-22.0.0.2\
85ENV JAVA_OPTS -Xmx1g
86RUN setx PATH "%PATH%;%JAVA_HOME%\bin;%MAVEN_HOME%\bin"
87
88WORKDIR C:/temp/
89
90# Define the entry point for the docker container.
91# This entry point starts the developer command prompt and launches the PowerShell shell.
92ENTRYPOINT ["C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Auxiliary\\Build\\vcvars64.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]
93ANSWER
Answered 2022-Mar-29 at 12:26Finally found the solution:
Changing the base docker image to: mcr.microsoft.com/windows:1809-amd64
This one contains the necessary dependencies to complete the build process. This is finally my Dockerfile (Anyways, if you know a lighter image or docker configuration to perform the build with gluonfx, please post in the comments. I will appreciate):
1[Mon Mar 28 21:14:21 UTC 2022][INFO] Substrate is tested with the Gluon's GraalVM build which you can find at https://github.com/gluonhq/graal/releases.
2While you can still use other GraalVM builds, there is no guarantee that these will work properly with Substrate
3[Mon Mar 28 21:14:22 UTC 2022][INFO] ==================== LINK TASK ====================
4[Mon Mar 28 21:14:22 UTC 2022][INFO] Default icon.ico image generated in C:\temp\target\gluonfx\x86_64-windows\gensrc\windows\assets.
5Consider copying it to C:\temp\src\windows before performing any modification
6[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Microsoft (R) Incremental Linker Version 14.31.31104.0
7[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Copyright (C) Microsoft Corporation. All rights reserved.
8[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB]
9[Mon Mar 28 21:14:22 UTC 2022][INFO] [SUB] Creating library C:\temp\target\gluonfx\x86_64-windows\X XXX.lib and object C:\temp\target\gluonfx\x86_64-windows\X XXX.exp
10java.io.IOException: Cannot run program "ie4uinit": CreateProcess error=2, The system cannot find the file specified
11 at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
12 at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
13 at com.gluonhq.substrate.util.ProcessRunner.setupProcess(ProcessRunner.java:378)
14 at com.gluonhq.substrate.util.ProcessRunner.runProcess(ProcessRunner.java:236)
15 at com.gluonhq.substrate.util.ProcessRunner.runProcess(ProcessRunner.java:222)
16[INFO] ------------------------------------------------------------------------
17 at com.gluonhq.substrate.target.WindowsTargetConfiguration.clearExplorerCache(WindowsTargetConfiguration.java:276)
18 at com.gluonhq.substrate.target.WindowsTargetConfiguration.link(WindowsTargetConfiguration.java:224)
19[INFO] BUILD FAILURE
20[INFO] ------------------------------------------------------------------------
21 at com.gluonhq.substrate.SubstrateDispatcher.nativeLink(SubstrateDispatcher.java:443)
22 at com.gluonhq.NativeLinkMojo.execute(NativeLinkMojo.java:47)
23 at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
24 at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute(MojoExecutor.java:301)
25 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:211)
26 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:165)
27 at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:157)
28[INFO] Total time: 06:39 min
29 at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:121)
30 at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
31 at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:56)
32 at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:127)
33 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:294)
34[INFO] Finished at: 2022-03-28T21:14:25Z
35[INFO] ------------------------------------------------------------------------
36 at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:192)
37 at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:105)
38 at org.apache.maven.cli.MavenCli.execute(MavenCli.java:960)
39 at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:293)
40 at org.apache.maven.cli.MavenCli.main(MavenCli.java:196)
41# escape=`
42# Use the latest Windows Server Core image with .NET Framework 4.8.
43FROM mcr.microsoft.com/dotnet/framework/sdk:4.8-windowsservercore-ltsc2019
44
45# Restore the default Windows shell for correct batch processing.
46SHELL ["cmd", "/S", "/C"]
47
48RUN `
49 # Download the Build Tools bootstrapper.
50 curl -SL --output vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
51 `
52 # Install Build Tools, excluding workloads and components with known issues.
53 && (start /w vs_buildtools.exe --quiet --wait --norestart --nocache modify `
54 --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" `
55 --add Microsoft.VisualStudio.Workload.AzureBuildTools `
56 --add Microsoft.VisualStudio.Workload.VCTools `
57 --add Microsoft.VisualStudio.Component.VC.140 `
58 --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
59 --add Microsoft.VisualStudio.Component.VC.CMake.Project `
60 --add Microsoft.VisualStudio.Component.Windows10SDK.19041 `
61 --add Microsoft.VisualStudio.Component.VC.Llvm.Clang `
62 --add Microsoft.Net.Component.4.8.SDK `
63 --remove Microsoft.VisualStudio.Component.Windows10SDK.10240 `
64 --remove Microsoft.VisualStudio.Component.Windows10SDK.10586 `
65 --remove Microsoft.VisualStudio.Component.Windows10SDK.14393 `
66 --remove Microsoft.VisualStudio.Component.Windows81SDK `
67 || IF "%ERRORLEVEL%"=="3010" EXIT 0) `
68 `
69 # Cleanup
70 && del /q vs_buildtools.exe
71
72#Installing GraalVM
73ADD https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.0.0.2/graalvm-ce-java11-windows-amd64-22.0.0.2.zip /
74RUN powershell -Command "expand-archive -Path 'c:\graalvm-ce-java11-windows-amd64-22.0.0.2.zip' -DestinationPath 'c:\'"
75
76#ADD https://download2.gluonhq.com/substrate/javafxstaticsdk/openjfx-18-ea+14-windows-x86_64-static.zip / C:\Users\ContainerAdministrator\.gluon\substrate\
77
78#Installing Maven
79ADD https://dlcdn.apache.org/maven/maven-3/3.8.5/binaries/apache-maven-3.8.5-bin.zip /
80RUN powershell -Command "expand-archive -Path 'c:\apache-maven-3.8.5-bin.zip' -DestinationPath 'c:\'"
81
82ENV JAVA_HOME C:\graalvm-ce-java11-22.0.0.2\
83ENV MAVEN_HOME C:\apache-maven-3.8.5\
84ENV GRAALVM_HOME C:\graalvm-ce-java11-22.0.0.2\
85ENV JAVA_OPTS -Xmx1g
86RUN setx PATH "%PATH%;%JAVA_HOME%\bin;%MAVEN_HOME%\bin"
87
88WORKDIR C:/temp/
89
90# Define the entry point for the docker container.
91# This entry point starts the developer command prompt and launches the PowerShell shell.
92ENTRYPOINT ["C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Auxiliary\\Build\\vcvars64.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]
93# escape=`
94# Use A windows base image.
95FROM mcr.microsoft.com/windows:1809-amd64
96
97# Restore the default Windows shell for correct batch processing.
98SHELL ["cmd", "/S", "/C"]
99
100RUN `
101 # Download the Build Tools bootstrapper.
102 curl -SL --output vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
103 `
104 # Install Build Tools, excluding workloads and components with known issues.
105 && (start /w vs_buildtools.exe --quiet --wait --norestart --nocache `
106 --installPath "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools" `
107 --add Microsoft.VisualStudio.Workload.AzureBuildTools `
108 --add Microsoft.VisualStudio.Workload.VCTools `
109 --add Microsoft.VisualStudio.Component.VC.140 `
110 --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
111 --add Microsoft.VisualStudio.Component.VC.CMake.Project `
112 --add Microsoft.VisualStudio.Component.Windows10SDK.19041 `
113 --add Microsoft.VisualStudio.Component.VC.Llvm.Clang `
114 --add Microsoft.Net.Component.4.8.SDK `
115 --remove Microsoft.VisualStudio.Component.Windows10SDK.10240 `
116 --remove Microsoft.VisualStudio.Component.Windows10SDK.10586 `
117 --remove Microsoft.VisualStudio.Component.Windows10SDK.14393 `
118 --remove Microsoft.VisualStudio.Component.Windows81SDK `
119 || IF "%ERRORLEVEL%"=="3010" EXIT 0) `
120 `
121 # Cleanup
122 && del /q vs_buildtools.exe
123
124#Installing GraalVM
125ADD https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.0.0.2/graalvm-ce-java11-windows-amd64-22.0.0.2.zip /
126RUN powershell -Command "expand-archive -Path 'c:\graalvm-ce-java11-windows-amd64-22.0.0.2.zip' -DestinationPath 'c:\'"
127
128#Installing Maven
129ADD https://dlcdn.apache.org/maven/maven-3/3.8.5/binaries/apache-maven-3.8.5-bin.zip /
130RUN powershell -Command "expand-archive -Path 'c:\apache-maven-3.8.5-bin.zip' -DestinationPath 'c:\'"
131
132ENV JAVA_HOME C:\graalvm-ce-java11-22.0.0.2\
133ENV MAVEN_HOME C:\apache-maven-3.8.5\
134ENV GRAALVM_HOME C:\graalvm-ce-java11-22.0.0.2\
135ENV JAVA_OPTS -Xmx1g
136RUN setx PATH "%PATH%;%JAVA_HOME%\bin;%MAVEN_HOME%\bin"
137
138WORKDIR C:/temp/
139
140# Define the entry point for the docker container.
141# This entry point starts the developer command prompt and launches the PowerShell shell.
142ENTRYPOINT ["C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Auxiliary\\Build\\vcvars64.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]
143QUESTION
React Native Table with Row Selection
Asked 2022-Mar-26 at 08:29I am trying to create a React Native screen that allows the user to select which items to send to the server for batch processing.
My thought was to have a table, and allow the user to select the rows they want, then click a button to submit to the server.
I need the state to contain a list of the ids from those rows, so that I can use it to allow the user to send a request with that array of ids.
A mock-up of my code is below, but it doesn't work. When the update of state is in place, I get an error of "selected items is not an object". When it is commented out, of course the state update doesn't work, but it also doesn't set the value of the checkbox from the array if I hard code it in the state initialization (meaning is 70 is in the array, the box is still not checked by default), and it does allow the box to get checked but not unchecked. How do I get it working?
1import React, { Component } from 'react';
2import { StyleSheet, View } from 'react-native';
3import CheckBox from '@react-native-community/checkbox';
4import { Table, Row, TableWrapper, Cell } from 'react-native-table-component';
5import moment from 'moment';
6
7
8class FruitGrid extends Component {
9
10 constructor(props) {
11 super(props);
12 }
13 state = {
14 selectedItems : [70],
15 data: []
16 };
17
18 refresh() {
19 let rows = [
20 [69,'David','Apples'],
21 [70,'Teddy','Oranges'],
22 [73,'John','Pears']
23 ];
24 this.setState({data: rows});
25 }
26 componentDidMount() {
27 this.refresh();
28 }
29 setSelection(id) {
30 const { selectedItems } = this.state;
31
32 if (id in selectedItems)
33 {
34 this.setState({selectedItems: selectedItems.filter(i => i != id)});
35 }
36 else
37 {
38 this.setState({selectedItems : selectedItems.push(id)});
39 }
40 }
41 render() {
42 const { selectedItems, data } = this.state;
43 let columns = ['',
44 'Person',
45 'Fruit'];
46
47 return (
48 <View style={{ flex: 1 }}>
49 <Table borderStyle={{borderWidth: 2, borderColor: '#c8e1ff'}}>
50 <Row data = {columns} />
51 {
52 data.map((rowData, index) =>
53 (
54 <TableWrapper key={index} style={styles.row}>
55 <Cell key={0} data = {<CheckBox value={rowData[0] in selectedItems} onValueChange={this.setSelection(rowData[0])} />} />
56 <Cell key={1} data = {rowData[1]} textStyle={styles.text}/>
57 <Cell key={2} data = {rowData[2]} textStyle={styles.text}/>
58 </TableWrapper>
59 )
60 )
61 }
62 </Table>
63 </View>
64 );
65 }
66}
67
68export default FruitGrid;
69
70 const styles = StyleSheet.create({
71 btn: { width: 58, height: 18, backgroundColor: '#8bbaf2', borderRadius: 2 },
72 btnText: { textAlign: 'center', color: '#000000' },
73 text: { margin: 6 },
74 row: { flexDirection: 'row' },
75 });
76
77ANSWER
Answered 2021-Dec-19 at 03:33Thanks to a friend, I found 3 issues in the code that were causing the problem:
When checking to see if the array contains the object, I first need to check that the array is an array and contains items. New check (wrapped in a function for reuse):
checkIfChecked(id, selectedItems) { return selectedItems?.length && selectedItems.includes(id); }
The state update was modifying the state without copying. New state update function:
setSelection(id) { const { selectedItems } = this.state;
1import React, { Component } from 'react';
2import { StyleSheet, View } from 'react-native';
3import CheckBox from '@react-native-community/checkbox';
4import { Table, Row, TableWrapper, Cell } from 'react-native-table-component';
5import moment from 'moment';
6
7
8class FruitGrid extends Component {
9
10 constructor(props) {
11 super(props);
12 }
13 state = {
14 selectedItems : [70],
15 data: []
16 };
17
18 refresh() {
19 let rows = [
20 [69,'David','Apples'],
21 [70,'Teddy','Oranges'],
22 [73,'John','Pears']
23 ];
24 this.setState({data: rows});
25 }
26 componentDidMount() {
27 this.refresh();
28 }
29 setSelection(id) {
30 const { selectedItems } = this.state;
31
32 if (id in selectedItems)
33 {
34 this.setState({selectedItems: selectedItems.filter(i => i != id)});
35 }
36 else
37 {
38 this.setState({selectedItems : selectedItems.push(id)});
39 }
40 }
41 render() {
42 const { selectedItems, data } = this.state;
43 let columns = ['',
44 'Person',
45 'Fruit'];
46
47 return (
48 <View style={{ flex: 1 }}>
49 <Table borderStyle={{borderWidth: 2, borderColor: '#c8e1ff'}}>
50 <Row data = {columns} />
51 {
52 data.map((rowData, index) =>
53 (
54 <TableWrapper key={index} style={styles.row}>
55 <Cell key={0} data = {<CheckBox value={rowData[0] in selectedItems} onValueChange={this.setSelection(rowData[0])} />} />
56 <Cell key={1} data = {rowData[1]} textStyle={styles.text}/>
57 <Cell key={2} data = {rowData[2]} textStyle={styles.text}/>
58 </TableWrapper>
59 )
60 )
61 }
62 </Table>
63 </View>
64 );
65 }
66}
67
68export default FruitGrid;
69
70 const styles = StyleSheet.create({
71 btn: { width: 58, height: 18, backgroundColor: '#8bbaf2', borderRadius: 2 },
72 btnText: { textAlign: 'center', color: '#000000' },
73 text: { margin: 6 },
74 row: { flexDirection: 'row' },
75 });
76
77 if (this.checkIfChecked(id,selectedItems))
78 {
79 this.setState({selectedItems: selectedItems.filter(i => i != id)});
80 }
81 else
82 {
83 let selectedItemsCopy = [...selectedItems]
84 selectedItemsCopy.push(id)
85 this.setState({selectedItems : selectedItemsCopy});
86 }
87}
The onValueChange needed ()=> to prevent immediate triggering, which lead to a "Maximum Depth Reached" error. New version
onValueChange={()=>this.setSelection(rowData[0])} />}
The full working code is here:
1import React, { Component } from 'react';
2import { StyleSheet, View } from 'react-native';
3import CheckBox from '@react-native-community/checkbox';
4import { Table, Row, TableWrapper, Cell } from 'react-native-table-component';
5import moment from 'moment';
6
7
8class FruitGrid extends Component {
9
10 constructor(props) {
11 super(props);
12 }
13 state = {
14 selectedItems : [70],
15 data: []
16 };
17
18 refresh() {
19 let rows = [
20 [69,'David','Apples'],
21 [70,'Teddy','Oranges'],
22 [73,'John','Pears']
23 ];
24 this.setState({data: rows});
25 }
26 componentDidMount() {
27 this.refresh();
28 }
29 setSelection(id) {
30 const { selectedItems } = this.state;
31
32 if (id in selectedItems)
33 {
34 this.setState({selectedItems: selectedItems.filter(i => i != id)});
35 }
36 else
37 {
38 this.setState({selectedItems : selectedItems.push(id)});
39 }
40 }
41 render() {
42 const { selectedItems, data } = this.state;
43 let columns = ['',
44 'Person',
45 'Fruit'];
46
47 return (
48 <View style={{ flex: 1 }}>
49 <Table borderStyle={{borderWidth: 2, borderColor: '#c8e1ff'}}>
50 <Row data = {columns} />
51 {
52 data.map((rowData, index) =>
53 (
54 <TableWrapper key={index} style={styles.row}>
55 <Cell key={0} data = {<CheckBox value={rowData[0] in selectedItems} onValueChange={this.setSelection(rowData[0])} />} />
56 <Cell key={1} data = {rowData[1]} textStyle={styles.text}/>
57 <Cell key={2} data = {rowData[2]} textStyle={styles.text}/>
58 </TableWrapper>
59 )
60 )
61 }
62 </Table>
63 </View>
64 );
65 }
66}
67
68export default FruitGrid;
69
70 const styles = StyleSheet.create({
71 btn: { width: 58, height: 18, backgroundColor: '#8bbaf2', borderRadius: 2 },
72 btnText: { textAlign: 'center', color: '#000000' },
73 text: { margin: 6 },
74 row: { flexDirection: 'row' },
75 });
76
77 if (this.checkIfChecked(id,selectedItems))
78 {
79 this.setState({selectedItems: selectedItems.filter(i => i != id)});
80 }
81 else
82 {
83 let selectedItemsCopy = [...selectedItems]
84 selectedItemsCopy.push(id)
85 this.setState({selectedItems : selectedItemsCopy});
86 }
87import React, { Component } from 'react';
88import { StyleSheet, View } from 'react-native';
89import CheckBox from '@react-native-community/checkbox';
90import { Table, Row, TableWrapper, Cell } from 'react-native-table-component';
91import moment from 'moment';
92
93
94class FruitGrid extends Component {
95
96 constructor(props) {
97 super(props);
98 }
99 state = {
100 selectedItems : [],
101 data: []
102 };
103
104 refresh() {
105 let rows = [
106 [69,'David','Apples'],
107 [70,'Teddy','Oranges'],
108 [73,'John','Pears']
109 ];
110 this.setState({data: rows});
111 }
112 componentDidMount() {
113 this.refresh();
114 }
115 setSelection(id) {
116 const { selectedItems } = this.state;
117
118 if (this.checkIfChecked(id,selectedItems))
119 {
120 this.setState({selectedItems: selectedItems.filter(i => i != id)});
121 }
122 else
123 {
124 let selectedItemsCopy = [...selectedItems]
125 selectedItemsCopy.push(id)
126 this.setState({selectedItems : selectedItemsCopy});
127 }
128 }
129
130 checkIfChecked(id, selectedItems)
131 {
132 return selectedItems?.length && selectedItems.includes(id);
133 }
134 render() {
135 const { selectedItems, data } = this.state;
136 let columns = ['',
137 'Person',
138 'Fruit'];
139
140 return (
141 <View style={{ flex: 1 }}>
142 <Table borderStyle={{borderWidth: 2, borderColor: '#c8e1ff'}}>
143 <Row data = {columns} />
144 {
145 data.map((rowData, index) =>
146 (
147 <TableWrapper key={index} style={styles.row}>
148 <Cell key={0} data = {<CheckBox value={this.checkIfChecked(rowData[0],selectedItems)} onValueChange={()=>this.setSelection(rowData[0])} />} />
149 <Cell key={1} data = {rowData[1]} textStyle={styles.text}/>
150 <Cell key={2} data = {rowData[2]} textStyle={styles.text}/>
151 </TableWrapper>
152 )
153 )
154 }
155 </Table>
156 </View>
157 );
158 }
159}
160
161export default FruitGrid;
162
163 const styles = StyleSheet.create({
164 btn: { width: 58, height: 18, backgroundColor: '#8bbaf2', borderRadius: 2 },
165 btnText: { textAlign: 'center', color: '#000000' },
166 text: { margin: 6 },
167 row: { flexDirection: 'row' },
168 });
169
170QUESTION
R2DBC vs Spring Jdbc Vs Spring Data JDBC?
Asked 2022-Mar-25 at 14:03On Using Spring JDBC which works very well and has some improvements over JPA when using Batch processing.
I would love to learn why to use Spring Data JDBC when you already have Spring JDBC.
I would love to learn why to use R2DBC when you already have Spring JDBC.
ANSWER
Answered 2021-Aug-21 at 15:11When you would use R2DBC is easy, it’s when you are building an application that is non-blocking. Everything in the stack has to be non-blocking, including the database driver. JDBC is inherently blocking, people try schemes to get around it but it is not great. If you aren’t building a non-blocking application you wouldn’t use R2DBC.
For the part about when to use spring data JDBC, it looks like it gives you a simpler way to create repositories and map data, as long as you’re ok with their opinionated framework, you are ok with not having all the complex mappings you get with JPA, and you want the DDD concepts (like aggregate root). Otherwise Spring JDBC requires more code to create your data access objects but may give more control. https://docs.spring.io/spring-data/jdbc/docs/2.2.4/reference/html/#reference describes more about why to use Spring Data JDBC. It is simplicity vs control.
QUESTION
Create Pytorch "Stack of Views" to save on GPU memory
Asked 2022-Mar-10 at 22:29I am trying to expand datasets for analysis in Pytorch such that from one 1D (or 2D) tensor two stacks of views are generated. In the following image A (green) and B (blue) are views of the original tensor that are slid from left to right, which would then be combined into single tensors for batch processing:
The motivation behind using views for this is to save on GPU memory, since for large, multi-dimensional datasets this expansion process can convert a dataset of tens of MB into tens of GB despite tremendous data reuse (if normal tensors are used). Simply returning one view at a time is not desirable since the actual processing of tensors works in large batches.
Is what I'm trying to do possible in Pytorch? Simply using torch.stack(list of views) creates a new tensor with a copy of the original data, as verified by tensor.storage().data_ptr().
Another way to phrase the question: can you create batches of tensor views?
The current steps are:
- Load and pre-process all datasets
- Convert datasets into tensors and expand into stacks of sliding views, as shown above
- Move all stacks to GPU to avoid transfer bottleneck during training
ANSWER
Answered 2022-Mar-10 at 22:28As mentioned in the comments, Tensor.unfold can be used for this task. You provide a tensor, starting index, length value, and step size. This returns a batch of views exactly like I was describing, though you have to unfold tensors one at a time for A and B.
The following code can be used to generate A and B:
1A = source_tensor[:-B_length].unfold(0, A_length, 1)
2B = source_tensor[A_length:].unfold(0, B_length, 1)
3
4A.storage().data_ptr() == source_tensor.storage().data_ptr() returns True
5Since the data pointers are the same it is correctly returning views of the original tensor instead of copies, which saves on memory.
QUESTION
Configure depth overflow value - Start-Job
Asked 2022-Mar-06 at 00:09I have a recursive function that is executed around 750~ times - iterating over XML files and processing. The code is running using Start-Job
Example below:
1$job = Start-Job -ScriptBlock {
2
3 function Test-Function {
4
5 Param
6 (
7 $count
8 )
9 Write-Host "Count is: $count"
10
11 $count++
12 Test-Function -count $count
13 }
14 Test-Function -count 1
15
16}
17Output:
1$job = Start-Job -ScriptBlock {
2
3 function Test-Function {
4
5 Param
6 (
7 $count
8 )
9 Write-Host "Count is: $count"
10
11 $count++
12 Test-Function -count $count
13 }
14 Test-Function -count 1
15
16}
17$job | Receive-Job
18Count is: 224
19Count is: 225
20Count is: 226
21Count is: 227
22The script failed due to call depth overflow.
23The depth overflow occurs at 227 consistently on my machine. If I remove Start-Job, I can reach 750~ (and further). I am using jobs for batch processing.
Is there a way to configure the depth overflow value when using Start-Job?
Is this a limitation of PowerShell Jobs?
ANSWER
Answered 2022-Mar-06 at 00:09I can't answer about the specifics of call depth overflow limitations in PS 5.1 / 7.2 but you could do your recursion based-off a Queue within the job.
So instead of doing the recursion within the function, you do it from outside (still within the job though).
Here's what this look like.
1$job = Start-Job -ScriptBlock {
2
3 function Test-Function {
4
5 Param
6 (
7 $count
8 )
9 Write-Host "Count is: $count"
10
11 $count++
12 Test-Function -count $count
13 }
14 Test-Function -count 1
15
16}
17$job | Receive-Job
18Count is: 224
19Count is: 225
20Count is: 226
21Count is: 227
22The script failed due to call depth overflow.
23$job = Start-Job -ScriptBlock {
24$Queue = [System.Collections.Queue]::new()
25
26 function Test-Function {
27
28 Param
29 (
30 $count
31 )
32 Write-Host "Count is: $count"
33
34 $count++
35 # Next item to process.
36 $Queue.Enqueue($Count)
37 }
38
39 # Call the function once
40 Test-Function -count 1
41 # Process the queue
42 while ($Queue.Count -gt 0) {
43 Test-Function -count $Queue.Dequeue()
44 }
45}
46
47Reference:
QUESTION
Does Flink's windowing operation process elements at the end of window or does it do a rolling processing?
Asked 2022-Jan-20 at 03:35I am having some trouble understanding the way windowing is implemented internally in Flink and could not find any article which explain this in depth. In my mind, there are two ways this can be done. Consider a simple window wordcount code as below
1env.socketTextStream("localhost", 9999)
2 .flatMap(new Splitter())
3 .groupBy(0)
4 .window(Time.of(500, TimeUnit.SECONDS)).sum(1)
5Method 1: Store all events for 500 seconds and at the end of the window, process all of them by applying the sum operation on the stored events.
Method 2: We use a counter to store a rolling sum for every window. As each event in a window comes, we do not store the individual events but keep adding 1 to previously stored counter and output the result at the end of the window.
Could someone kindly help to understand which of the above methods (or maybe a different approach) is used by Flink in reality. The reason is, there are pros and cons to both approach and is important to understand in order configure the resources for the cluster correctly. eg: The Method 1 seems very close to batch processing and might potentially have issues related to spike in processing at every 500 sec interval while sitting idle otherwise etc while Method2 would need to maintain a common counter between all task managers.
ANSWER
Answered 2022-Jan-20 at 03:35sum is a reducing function as mentioned here(https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/#reducefunction). Internally, Flink will apply reduce function to each input element, and simply save the reduced result in ReduceState.
For other windows functions, like windows.apply(WindowFunction). There is no aggregation so all input elements will be saved in the ListState.
This document(https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/operators/windows/#window-functions) about windows stream mentions about how the internal elements are handled in Flink.
QUESTION
How to speed up if statements with for loop
Asked 2022-Jan-17 at 22:39I am trying to create an on-edit function in Google Apps Script that automatically checks/changes formatting for certain checkboxes on Sheet 2 ("interventionSheet") based on which checkboxes are selected on Sheet 1 ("sessionFocus"). Each checkbox on Sheet 1 is associated with its own unique group of checkboxes on Sheet 2. I have figured out how to do this by using repeated if statements, but this is slowing down the processing speed significantly. I am looking for help with how to use batch processing (arrays, for statements, etc.) to make the code more efficient. See current (inefficient) code below.
1function populateInterventions(e)
2{
3 var sessionFocus = SpreadsheetApp.getActive().getSheetByName('Session Focus');
4 var interventionSheet = SpreadsheetApp.getActive().getSheetByName('Therapeutic Intervention');
5
6 if(e.range.getA1Notation()=='B23')
7 {
8 var range3a = interventionSheet.getRangeList(['J78','Q69','Q71','Q72']);
9 var range3b = interventionSheet.getRangeList(['J78','K78','Q69','R69','Q71','R71','Q72','R72']);
10
11 if(sessionFocus.getRange('B23').isChecked()==true)
12 {
13 range3a.check();
14 range3b.setBackground('#cfe2f3');
15 range3a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
16 }
17 else
18 {
19 range3a.uncheck();
20 range3b.setBackground('#ffffff');
21 range3a.setBorder(null,null,null,true,null,null,'#ffffff',null);
22 }
23 }
24
25 if(e.range.getA1Notation()=='B24')
26 {
27 var range4a = interventionSheet.getRangeList(['J18','J20','J32','J84','J85']);
28 var range4b = interventionSheet.getRangeList(['J18','K18','J20','K20','J32','K32','J84','K84','J85','K85']);
29
30 if(sessionFocus.getRange('B24').isChecked()==true)
31 {
32 range4a.check();
33 range4b.setBackground('#cfe2f3');
34 range4a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
35 }
36 else
37 {
38 range4a.uncheck();
39 range4b.setBackground('#ffffff');
40 range4a.setBorder(null,null,null,true,null,null,'#ffffff',null);
41 }
42 }
43
44 if(e.range.getA1Notation()=='B26')
45 {
46 var range5a = interventionSheet.getRangeList(['J86','J87','N61','Q79','Q80']);
47 var range5b = interventionSheet.getRangeList(['J86','K86','J87','K87','N61','O61','Q79','R79','Q80','R80']);
48
49 if(sessionFocus.getRange('B26').isChecked()==true)
50 {
51 range5a.check();
52 range5b.setBackground('#cfe2f3');
53 range5a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
54 }
55 else
56 {
57 range5a.uncheck();
58 range5b.setBackground('#ffffff');
59 range5a.setBorder(null,null,null,true,null,null,'#ffffff',null);
60 }
61 }
62ANSWER
Answered 2022-Jan-17 at 22:39In this specific case it doesn't look to make sense using a loop because the script has code sections that only executes when an specific cell is edited.
In order to make your scripts more efficients in terms of their execution time whenever is possible you have to minimize the calls to the Google Apps Script services.
- Avoid having calls to the Google Apps Script services in loops
- Take advantage of variables
i.e. instead of
1function populateInterventions(e)
2{
3 var sessionFocus = SpreadsheetApp.getActive().getSheetByName('Session Focus');
4 var interventionSheet = SpreadsheetApp.getActive().getSheetByName('Therapeutic Intervention');
5
6 if(e.range.getA1Notation()=='B23')
7 {
8 var range3a = interventionSheet.getRangeList(['J78','Q69','Q71','Q72']);
9 var range3b = interventionSheet.getRangeList(['J78','K78','Q69','R69','Q71','R71','Q72','R72']);
10
11 if(sessionFocus.getRange('B23').isChecked()==true)
12 {
13 range3a.check();
14 range3b.setBackground('#cfe2f3');
15 range3a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
16 }
17 else
18 {
19 range3a.uncheck();
20 range3b.setBackground('#ffffff');
21 range3a.setBorder(null,null,null,true,null,null,'#ffffff',null);
22 }
23 }
24
25 if(e.range.getA1Notation()=='B24')
26 {
27 var range4a = interventionSheet.getRangeList(['J18','J20','J32','J84','J85']);
28 var range4b = interventionSheet.getRangeList(['J18','K18','J20','K20','J32','K32','J84','K84','J85','K85']);
29
30 if(sessionFocus.getRange('B24').isChecked()==true)
31 {
32 range4a.check();
33 range4b.setBackground('#cfe2f3');
34 range4a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
35 }
36 else
37 {
38 range4a.uncheck();
39 range4b.setBackground('#ffffff');
40 range4a.setBorder(null,null,null,true,null,null,'#ffffff',null);
41 }
42 }
43
44 if(e.range.getA1Notation()=='B26')
45 {
46 var range5a = interventionSheet.getRangeList(['J86','J87','N61','Q79','Q80']);
47 var range5b = interventionSheet.getRangeList(['J86','K86','J87','K87','N61','O61','Q79','R79','Q80','R80']);
48
49 if(sessionFocus.getRange('B26').isChecked()==true)
50 {
51 range5a.check();
52 range5b.setBackground('#cfe2f3');
53 range5a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
54 }
55 else
56 {
57 range5a.uncheck();
58 range5b.setBackground('#ffffff');
59 range5a.setBorder(null,null,null,true,null,null,'#ffffff',null);
60 }
61 }
62if(e.range.getA1Notation()=='B23')
63if(e.range.getA1Notation()=='B24')
64if(e.range.getA1Notation()=='B26')
65before do a variable declaration
1function populateInterventions(e)
2{
3 var sessionFocus = SpreadsheetApp.getActive().getSheetByName('Session Focus');
4 var interventionSheet = SpreadsheetApp.getActive().getSheetByName('Therapeutic Intervention');
5
6 if(e.range.getA1Notation()=='B23')
7 {
8 var range3a = interventionSheet.getRangeList(['J78','Q69','Q71','Q72']);
9 var range3b = interventionSheet.getRangeList(['J78','K78','Q69','R69','Q71','R71','Q72','R72']);
10
11 if(sessionFocus.getRange('B23').isChecked()==true)
12 {
13 range3a.check();
14 range3b.setBackground('#cfe2f3');
15 range3a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
16 }
17 else
18 {
19 range3a.uncheck();
20 range3b.setBackground('#ffffff');
21 range3a.setBorder(null,null,null,true,null,null,'#ffffff',null);
22 }
23 }
24
25 if(e.range.getA1Notation()=='B24')
26 {
27 var range4a = interventionSheet.getRangeList(['J18','J20','J32','J84','J85']);
28 var range4b = interventionSheet.getRangeList(['J18','K18','J20','K20','J32','K32','J84','K84','J85','K85']);
29
30 if(sessionFocus.getRange('B24').isChecked()==true)
31 {
32 range4a.check();
33 range4b.setBackground('#cfe2f3');
34 range4a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
35 }
36 else
37 {
38 range4a.uncheck();
39 range4b.setBackground('#ffffff');
40 range4a.setBorder(null,null,null,true,null,null,'#ffffff',null);
41 }
42 }
43
44 if(e.range.getA1Notation()=='B26')
45 {
46 var range5a = interventionSheet.getRangeList(['J86','J87','N61','Q79','Q80']);
47 var range5b = interventionSheet.getRangeList(['J86','K86','J87','K87','N61','O61','Q79','R79','Q80','R80']);
48
49 if(sessionFocus.getRange('B26').isChecked()==true)
50 {
51 range5a.check();
52 range5b.setBackground('#cfe2f3');
53 range5a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
54 }
55 else
56 {
57 range5a.uncheck();
58 range5b.setBackground('#ffffff');
59 range5a.setBorder(null,null,null,true,null,null,'#ffffff',null);
60 }
61 }
62if(e.range.getA1Notation()=='B23')
63if(e.range.getA1Notation()=='B24')
64if(e.range.getA1Notation()=='B26')
65const cellAddress = e.range.getA1Notation()
66then use the variable instead of doing multiple Google Apps Script service calls
1function populateInterventions(e)
2{
3 var sessionFocus = SpreadsheetApp.getActive().getSheetByName('Session Focus');
4 var interventionSheet = SpreadsheetApp.getActive().getSheetByName('Therapeutic Intervention');
5
6 if(e.range.getA1Notation()=='B23')
7 {
8 var range3a = interventionSheet.getRangeList(['J78','Q69','Q71','Q72']);
9 var range3b = interventionSheet.getRangeList(['J78','K78','Q69','R69','Q71','R71','Q72','R72']);
10
11 if(sessionFocus.getRange('B23').isChecked()==true)
12 {
13 range3a.check();
14 range3b.setBackground('#cfe2f3');
15 range3a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
16 }
17 else
18 {
19 range3a.uncheck();
20 range3b.setBackground('#ffffff');
21 range3a.setBorder(null,null,null,true,null,null,'#ffffff',null);
22 }
23 }
24
25 if(e.range.getA1Notation()=='B24')
26 {
27 var range4a = interventionSheet.getRangeList(['J18','J20','J32','J84','J85']);
28 var range4b = interventionSheet.getRangeList(['J18','K18','J20','K20','J32','K32','J84','K84','J85','K85']);
29
30 if(sessionFocus.getRange('B24').isChecked()==true)
31 {
32 range4a.check();
33 range4b.setBackground('#cfe2f3');
34 range4a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
35 }
36 else
37 {
38 range4a.uncheck();
39 range4b.setBackground('#ffffff');
40 range4a.setBorder(null,null,null,true,null,null,'#ffffff',null);
41 }
42 }
43
44 if(e.range.getA1Notation()=='B26')
45 {
46 var range5a = interventionSheet.getRangeList(['J86','J87','N61','Q79','Q80']);
47 var range5b = interventionSheet.getRangeList(['J86','K86','J87','K87','N61','O61','Q79','R79','Q80','R80']);
48
49 if(sessionFocus.getRange('B26').isChecked()==true)
50 {
51 range5a.check();
52 range5b.setBackground('#cfe2f3');
53 range5a.setBorder(null,null,null,true,null,null,'#cfe2f3',null);
54 }
55 else
56 {
57 range5a.uncheck();
58 range5b.setBackground('#ffffff');
59 range5a.setBorder(null,null,null,true,null,null,'#ffffff',null);
60 }
61 }
62if(e.range.getA1Notation()=='B23')
63if(e.range.getA1Notation()=='B24')
64if(e.range.getA1Notation()=='B26')
65const cellAddress = e.range.getA1Notation()
66if(cellAddress=='B23')
67if(cellAddress=='B24')
68if(cellAddress=='B26')
69Regarding using arrays, for loops etc., that might help to write better scripts, making them more readable, maintainable, extensible, etc. but you might find that they will not have a really relevant impact on the script performance when using only the "basic" services (i.e. SpreadsheetApp).
On more complex scripts it might be worthy to use Advanced Sheets Service in order to be able do multiple changes "at once" by using batchUpdate.
Resources
QUESTION
Adding a border to an SVG image element with SVG filters
Asked 2021-Dec-19 at 18:05I'm looking for a way to add a border to an SVG image element using SVG filters not CSS, but I can't figure out how.
The file should be stand-alone, i.e. it is not meant for a webpage. I'm working in Notepad++ intending to develop a script for batch processing were images may be of different sizes, and have different aspect ratios.
Although I could read the image dimension and position a rectangle on top, I was thinking that it may be possible to use SVG filters, as discussed developer.mozilla.org.
Below is a MWE.
1<svg
2 width="210mm"
3 height="297mm"
4 viewBox="0 0 210 297"
5 version="1.1"
6 xmlns:xlink="http://www.w3.org/1999/xlink"
7 xmlns="http://www.w3.org/2000/svg"
8 xmlns:svg="http://www.w3.org/2000/svg">
9
10
11 <defs>
12 <filter id="image">
13 <feImage xlink:href="abstract.jpg"/>
14 </filter>
15 </defs>
16
17
18<rect width="100%" height="100%" fill="DodgerBlue" />
19
20 <rect x="10mm" y="10mm" width="20mm" height="30mm"
21 style="filter:url(#image);"/>
22
23</svg>
24
ANSWER
Answered 2021-Dec-19 at 09:46You could add the image in an <image> tag and apply the filter to that. In this example I put x="1" and y="1" to distance the image from the edges since it can cut off the filter effect where the image is flush with the edge of the SVG.
1<svg
2 width="210mm"
3 height="297mm"
4 viewBox="0 0 210 297"
5 version="1.1"
6 xmlns:xlink="http://www.w3.org/1999/xlink"
7 xmlns="http://www.w3.org/2000/svg"
8 xmlns:svg="http://www.w3.org/2000/svg">
9
10
11 <defs>
12 <filter id="image">
13 <feImage xlink:href="abstract.jpg"/>
14 </filter>
15 </defs>
16
17
18<rect width="100%" height="100%" fill="DodgerBlue" />
19
20 <rect x="10mm" y="10mm" width="20mm" height="30mm"
21 style="filter:url(#image);"/>
22
23</svg>
24<svg viewBox="0 0 210 297" version="1.1" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns="http://www.w3.org/2000/svg" xmlns:svg="http://www.w3.org/2000/svg">
25
26 <filter id="outline">
27 <feMorphology in="SourceAlpha" result="expanded" operator="dilate" radius="1"/>
28 <feFlood flood-color="black"/>
29 <feComposite in2="expanded" operator="in"/>
30 <feComposite in="SourceGraphic"/>
31 </filter>
32 <image filter="url(#outline)" href="https://i.stack.imgur.com/cJwpE.jpg" height="200" width="200" x="1" y="1"/>
33</svg>Technique for adding border taken from this article by CSS Tricks.
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Batch Processing
Tutorials and Learning Resources are not available at this moment for Batch Processing
Share this Page
Get latest updates on Batch Processing