Popular New Releases in Firewall
opensnitch
v1.5.0
fail2ban
0.11.2 (2020/11/23) - heal-the-world-with-security-tools
ModSecurity
v2.9.5
naxsi
1.3
simplewall
Popular Libraries in Firewall
by evilsocket 
python
7096 GPL-3.0
OpenSnitch is a GNU/Linux port of the Little Snitch application firewall
by fail2ban python
6361 GPL-2.0
Daemon to ban hosts that cause multiple authentication errors
by screetsec c
5619 GPL-3.0
Thefatrat a massive exploiting tool : Easy tool to generate backdoor and easy tool to post exploitation attack like browser attack and etc . This tool compiles a malware with popular payload and then the compiled malware can be execute on windows, android, mac . The malware that created with this tool also have an ability to bypass most AV software protection .
by Screetsec c
5320 GPL-3.0
Thefatrat a massive exploiting tool : Easy tool to generate backdoor and easy tool to post exploitation attack like browser attack and etc . This tool compiles a malware with popular payload and then the compiled malware can be execute on windows, android, mac . The malware that created with this tool also have an ability to bypass most AV software protection .
by SpiderLabs c++
5316 Apache-2.0
ModSecurity is an open source, cross platform web application firewall (WAF) engine for Apache, IIS and Nginx that is developed by Trustwave's SpiderLabs. It has a robust event-based programming language which provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring, logging and real-time analysis. With over 10,000 deployments world-wide, ModSecurity is the most widely deployed WAF in existence.
by 0xInfection python
4088 Apache-2.0
๐ฅ Everything about web-application firewalls (WAF).
by nbs-system c
3789 GPL-3.0
NAXSI is an open-source, high performance, low rules maintenance WAF for NGINX
by martin-ger c
3677 MIT
A full functional WiFi Repeater (correctly: a WiFi NAT Router)
by pfsense php
3415 Apache-2.0
Main repository for pfSense
Trending New libraries in Firewall
by ADD-SP c
918 BSD-3-Clause
Handy, High performance, ModSecurity compatible Nginx firewall module & ๆนไพฟใ้ซๆง่ฝใๅ ผๅฎน ModSecurity ็ Nginx ้ฒ็ซๅขๆจกๅ
by hackerschoice c
625 BSD-2-Clause
Connect like there is no firewall. Securely.
by harporoeder c++
588 BSD-3-Clause
Linux Application Level Firewall based on eBPF and NFQUEUE.
by persianhydra python
463 GPL-3.0
Undetectable & Xor encrypting with custom KEY (FUD Metasploit Rat) bypass Top Antivirus like BitDefender,Malwarebytes,Avast,ESET-NOD32,AVG,... & Automatically Add ICON and MANIFEST to excitable
by jptosso go
345 Apache-2.0
Coraza WAF is a golang modsecurity compatible web application firewall library
by arloor rust
313 MIT
nftables nat rule generatorโโnftables nat่งๅ็ๆๅจ
by GeorgePatsias python
277 MIT
Cobalt Strike script for ScareCrow payloads (EDR/AV evasion)
by ollypwn python
243
PoC (DoS + scanner) for CVE-2020-0609 & CVE-2020-0610 - RD Gateway RCE
by fabianishere shell
226 GPL-2.0
Tools for bootstrapping custom kernels on the UniFi Dream Machine
Top Authors in Firewall
1
9 Libraries
492
2
4 Libraries
20
3
4 Libraries
489
4
3 Libraries
179
5
3 Libraries
122
6
3 Libraries
19
7
3 Libraries
126
8
3 Libraries
15
9
3 Libraries
46
10
3 Libraries
99
1
9 Libraries
492
2
4 Libraries
20
3
4 Libraries
489
4
3 Libraries
179
5
3 Libraries
122
6
3 Libraries
19
7
3 Libraries
126
8
3 Libraries
15
9
3 Libraries
46
10
3 Libraries
99
Trending Kits in Firewall
No Trending Kits are available at this moment for Firewall
Trending Discussions on Firewall
Android Studio BumbleBee pair wifi not working
TLS v1.2 Cipher Suites in .NET 6 / GET Request Timeout
Private GKE cluster behind firewall getting calls from external IP
Is there a new solution for downloading package and dependencies for a given R version
Kubernetes NodePort is not available on all nodes - Oracle Cloud Infrastructure (OCI)
How can we mount a file as read-only in Linux through Go?
Terraform: How to modify a public subnet's route table that was created by module 'vpc'?
Why is ArgoCD confusing GitHub.com with my own public IP?
New-CimSession without elevation by providing admin credentials on Windows 10?
Failing to run Tomcat in IntelliJ IDEA
QUESTION
Android Studio BumbleBee pair wifi not working
Asked 2022-Apr-03 at 10:29I am trying to connect my Android 11 device with android studio over adb wifi but it is not working.
I updated to latest stable bumblebee and updated my SDK I tried turning off firewall on my pc but it is same result.
When I use QR code method my android phone just shows "pairing device" and nothing happens If I try the code method, android studio just shows "searching for devices" but nothing happens
and, yes, I enabled wireless debugging on my phone and I am connected to the same wifi network.
I don't know if the problem is with my computer or phone. I do not have any other Android11+ phone to try with
ANSWER
Answered 2022-Jan-30 at 21:44I was having the same problem as you. Neither pairing by QR nor by pairing code worked.
So I tried connecting by typing adb connect [phone_ip]:[port] in the terminal and that worked flawlessly. Didn't even need to plug the phone to the computer with USB. Your phone will tell you the IP and port right above the "pair with QR code" option inside the Wifi-debugging setting. Just connect to that address.
QUESTION
TLS v1.2 Cipher Suites in .NET 6 / GET Request Timeout
Asked 2022-Mar-30 at 12:52I am currently trying to connect to an AWS REST API which requires at least TLS v1.2. The documentation stats that clients must also support cipher suites with perfect forward secrecy (PFS) such as Ephemeral Diffie-Hellman (DHE) or Elliptic Curve Ephemeral Diffie-Hellman (ECDHE).
When sending a GET request using the HttpClient, the connection simply times out. I have set the TLS version explicitely to TLSv1.2 like this:
1httpClientHandler.SslProtocols = SslProtocols.Tls12;
2This works, I can see in the Wireshark trace that the correct TLS version is used. I have also confirmed that there is no firewall issue or similar.
Working Example (CURL)When using cURL, I can see that the cipher suite in the Sever Hello response is TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030), which is also what the server requires.
When using the HttpClient in .NET 6, the above mentioned cipher suite is offered in the Client Hello, but the server response uses all of a sudden TLS_RSA_WITH_AES_256_GCM_SHA384:
I can see that there are additional extensions in the cURL request, for example Extension: psk_key_exchange_modes. Are there any explanations for why the server does not except the first cipher suite? From my understanding, the first offered cipher suite should be the preferred one, is that correct?
Is there a way to force a certain cipher suite in .NET 6?
This is the example I use to reproduce the issue:
1httpClientHandler.SslProtocols = SslProtocols.Tls12;
2public async void PollUrl(string url)
3{
4 HttpResponseMessage msg = new HttpResponseMessage();
5
6 ServicePointManager.Expect100Continue = true;
7 ServicePointManager.SecurityProtocol |= SecurityProtocolType.Tls12 | SecurityProtocolType.Tls13;
8
9 using HttpClientHandler httpClientHandler = new();
10
11 httpClientHandler.ServerCertificateCustomValidationCallback = (message, cert, chain, sslPolicyErrors) => true;
12 httpClientHandler.SslProtocols = SslProtocols.Tls12;
13
14 using HttpClient client = new(httpClientHandler);
15
16 // This content type is required for the API call
17 client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/xml"));
18
19 try
20 {
21 client.Timeout = TimeSpan.FromSeconds(5);
22 msg = await client.GetAsync(url);
23 }
24 catch (Exception e)
25 {
26 Console.WriteLine(e);
27 }
28
29 string stringValue = await msg.Content.ReadAsStringAsync();
30 Console.WriteLine(stringValue);
31}
32The application is running on Server 2016.
ANSWER
Answered 2022-Mar-30 at 12:52We finally found the reason for this. Windows did not have the required cypher suites enabled. We have used IISCrypto to enable the corresponding cypher suites and all is ok now.
It looks like it's possible to force .NET to TLS 1.2, even though it was not enabled on the server itself.
QUESTION
Private GKE cluster behind firewall getting calls from external IP
Asked 2022-Mar-24 at 20:28We are getting logs that calls to k8s are being made, despite our cluster being private, as well as being behind the gcp firewall with a rule that blocks all ingress except IAP IPs (and ICMP). What am I missing?
1"protoPayload":{
2 "@type":"type.googleapis.com/google.cloud.audit.AuditLog"
3 "authenticationInfo":{
4 "principalEmail":"system:anonymous"
5 }
6 "authorizationInfo":["0":{2}]
7 "methodName":"io.k8s.post"
8 "requestMetadata":{
9 "callerIp":"45.*.*.*"
10 "callerSuppliedUserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
11 }
12 "resourceName":"Autodiscover/Autodiscover.xml"
13 "serviceName":"k8s.io"
14 "status":{
15 "code":"7"
16 "message":"Forbidden"
17 }
18}
19ANSWER
Answered 2022-Mar-24 at 20:28The private clusters have a control plane private endpoint and a control plane public endpoint and you can choose to disable the control plane public endpoint, this is the highest level of restricted access. So you can manage the cluster with the private endpoint internal IP address with tools like kubectl and any VM that uses the same subnet that your cluster can also access the private endpoint.However, it is important to say that even if you disable the public endpoint access, Google can use the control plane public endpoint for cluster management purposes, such as scheduled maintenance and automatic control plane upgrades. If you need more information about how to create a private cluster with public endpoint disable, you can consult the following public document.
You can review your public endpoints with the following command:
1"protoPayload":{
2 "@type":"type.googleapis.com/google.cloud.audit.AuditLog"
3 "authenticationInfo":{
4 "principalEmail":"system:anonymous"
5 }
6 "authorizationInfo":["0":{2}]
7 "methodName":"io.k8s.post"
8 "requestMetadata":{
9 "callerIp":"45.*.*.*"
10 "callerSuppliedUserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
11 }
12 "resourceName":"Autodiscover/Autodiscover.xml"
13 "serviceName":"k8s.io"
14 "status":{
15 "code":"7"
16 "message":"Forbidden"
17 }
18}
19gcloud container clusters describe YOUR_CLUSTER_NAME
20Also, you can verify that your cluster's nodes do not have external IP addresses with the following command:
1"protoPayload":{
2 "@type":"type.googleapis.com/google.cloud.audit.AuditLog"
3 "authenticationInfo":{
4 "principalEmail":"system:anonymous"
5 }
6 "authorizationInfo":["0":{2}]
7 "methodName":"io.k8s.post"
8 "requestMetadata":{
9 "callerIp":"45.*.*.*"
10 "callerSuppliedUserAgent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
11 }
12 "resourceName":"Autodiscover/Autodiscover.xml"
13 "serviceName":"k8s.io"
14 "status":{
15 "code":"7"
16 "message":"Forbidden"
17 }
18}
19gcloud container clusters describe YOUR_CLUSTER_NAME
20kubectl get nodes --output wide
21QUESTION
Is there a new solution for downloading package and dependencies for a given R version
Asked 2022-Feb-09 at 03:33I have seen the question asked here from 2018. I'm wondering if there is a better answer today.
Our work computers are bedeviled by an entire IT security department that seems to exist to make them useless. We are allowed to run R 3.6.3 (4.x hasn't been approved yet). We cannot connect to CRAN from behind the corporate firewall. In the past that meant we took our laptops home to install packages. But now we have a download monitor that blocks CRAN downloads even when we're on our own wi-fi.
I was attempting to get around this by downloading the package .zip files on a personal machine, transferring them via CD, and then installing with repos=NULL. I used this code
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9to get the dependencies for the package I wanted and downloaded them all with download.packages(). I was trying to install tidyquant so it turned out there were 113 dependencies.
But of course this downloads the latest version of all 113 packages, several of which it turned out were incompatible with R 3.6.3.
The solution I referenced above suggested finding a compatible version and then using
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12But that would involve manually searching all 113 dependencies...plus I'm looking for a general purpose solution that I can share with the ~10 other people in my office that all have this issue now.
I'm hoping that since 2018 a better solution may have presented itself?
Update: Based on answers below I tried
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24I get
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46And the "no package" warnings go on for all 113 dependencies.
I think the key here is
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47Obviously there wouldn't be a 4.1 repository in April 2020. I assume it's trying that since the machine is running 4.1. So it seems we're tantalizingly close....
If it helps
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47> sessionInfo()
48R version 4.1.2 (2021-11-01)
49Platform: x86_64-w64-mingw32/x64 (64-bit)
50Running under: Windows 10 x64 (build 22000)
51
52Matrix products: default
53
54locale:
55[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
56[4] LC_NUMERIC=C LC_TIME=English_United States.1252
57system code page: 65001
58
59attached base packages:
60[1] stats graphics grDevices utils datasets methods base
61
62loaded via a namespace (and not attached):
63[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
64ANSWER
Answered 2022-Feb-09 at 03:33I'm not sure if it completely addresses your needs, but package checkpoint seems appropriate here. It allows you to download source packages from a snapshot of CRAN taken at a specified date, going back to 2014-09-17. R 4.0.0 was released around 2020-04-24, so the snapshot from 2020-04-01 should work for your purposes.
Here is a reproducible example:
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47> sessionInfo()
48R version 4.1.2 (2021-11-01)
49Platform: x86_64-w64-mingw32/x64 (64-bit)
50Running under: Windows 10 x64 (build 22000)
51
52Matrix products: default
53
54locale:
55[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
56[4] LC_NUMERIC=C LC_TIME=English_United States.1252
57system code page: 65001
58
59attached base packages:
60[1] stats graphics grDevices utils datasets methods base
61
62loaded via a namespace (and not attached):
63[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
64tmp1 <- tempfile()
65dir.create(tmp1)
66cwd <- setwd(tmp1)
67
68getOption("repos")
69## CRAN
70## "https://cloud.r-project.org/"
71
72available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
73## Version Repository
74## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
75## Matrix "1.4-0" "https://cloud.r-project.org/src/contrib"
76## nlme "3.1-155" "https://cloud.r-project.org/src/contrib"
77
78download.packages("Matrix", ".", type = "source")
79## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
80## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
81## ==================================================
82## downloaded 2.7 MB
83##
84## [,1] [,2]
85## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"
86
87tmp2 <- tempfile()
88dir.create(tmp2)
89checkpoint::checkpoint(snapshot_date = "2020-04-01",
90 r_version = "3.6.3",
91 checkpoint_location = tmp2,
92 scan_now = FALSE)
93## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
94## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
95## Warning messages:
96## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
97## Specified R version not the same as current R version
98## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
99## Specified R version not the same as current R version
100
101getOption("repos")
102## CRAN
103## "https://mran.microsoft.com/snapshot/2020-04-01"
104
105available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
106## Version Repository
107## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
108## Matrix "1.2-18" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
109## nlme "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
110
111download.packages("Matrix", ".", type = "source")
112## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
113## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
114## ==================================================
115## downloaded 1.8 MB
116##
117## [,1] [,2]
118## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"
119
120setwd(cwd)
121unlink(c(tmp1, tmp2), recursive = TRUE)
122The warnings about version mismatch occur if you are not actually running R 3.6.3. They can be ignored if you are only downloading source packages, with the intention of installing them on another machine actually running 3.6.3.
You can take a look at the package README and ?checkpoint for more details.
If you are trying to download binary packages (.zip for Windows, .tgz for macOS) rather than source packages (.tar.gz), then checkpoint can get you into trouble. By default, download.packages and friends use contrib.url(repos, type) to construct a URL to search for package binaries.
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47> sessionInfo()
48R version 4.1.2 (2021-11-01)
49Platform: x86_64-w64-mingw32/x64 (64-bit)
50Running under: Windows 10 x64 (build 22000)
51
52Matrix products: default
53
54locale:
55[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
56[4] LC_NUMERIC=C LC_TIME=English_United States.1252
57system code page: 65001
58
59attached base packages:
60[1] stats graphics grDevices utils datasets methods base
61
62loaded via a namespace (and not attached):
63[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
64tmp1 <- tempfile()
65dir.create(tmp1)
66cwd <- setwd(tmp1)
67
68getOption("repos")
69## CRAN
70## "https://cloud.r-project.org/"
71
72available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
73## Version Repository
74## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
75## Matrix "1.4-0" "https://cloud.r-project.org/src/contrib"
76## nlme "3.1-155" "https://cloud.r-project.org/src/contrib"
77
78download.packages("Matrix", ".", type = "source")
79## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
80## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
81## ==================================================
82## downloaded 2.7 MB
83##
84## [,1] [,2]
85## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"
86
87tmp2 <- tempfile()
88dir.create(tmp2)
89checkpoint::checkpoint(snapshot_date = "2020-04-01",
90 r_version = "3.6.3",
91 checkpoint_location = tmp2,
92 scan_now = FALSE)
93## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
94## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
95## Warning messages:
96## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
97## Specified R version not the same as current R version
98## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
99## Specified R version not the same as current R version
100
101getOption("repos")
102## CRAN
103## "https://mran.microsoft.com/snapshot/2020-04-01"
104
105available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
106## Version Repository
107## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
108## Matrix "1.2-18" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
109## nlme "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
110
111download.packages("Matrix", ".", type = "source")
112## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
113## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
114## ==================================================
115## downloaded 1.8 MB
116##
117## [,1] [,2]
118## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"
119
120setwd(cwd)
121unlink(c(tmp1, tmp2), recursive = TRUE)
122contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "win.binary")
123## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1"
124
125contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "mac.binary")
126## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1"
127But there is nothing at either URL. That is (in part) because contrib.url appends the R version that you are currently running, which might not have existed on your snapshot date. Hence:
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47> sessionInfo()
48R version 4.1.2 (2021-11-01)
49Platform: x86_64-w64-mingw32/x64 (64-bit)
50Running under: Windows 10 x64 (build 22000)
51
52Matrix products: default
53
54locale:
55[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
56[4] LC_NUMERIC=C LC_TIME=English_United States.1252
57system code page: 65001
58
59attached base packages:
60[1] stats graphics grDevices utils datasets methods base
61
62loaded via a namespace (and not attached):
63[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
64tmp1 <- tempfile()
65dir.create(tmp1)
66cwd <- setwd(tmp1)
67
68getOption("repos")
69## CRAN
70## "https://cloud.r-project.org/"
71
72available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
73## Version Repository
74## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
75## Matrix "1.4-0" "https://cloud.r-project.org/src/contrib"
76## nlme "3.1-155" "https://cloud.r-project.org/src/contrib"
77
78download.packages("Matrix", ".", type = "source")
79## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
80## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
81## ==================================================
82## downloaded 2.7 MB
83##
84## [,1] [,2]
85## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"
86
87tmp2 <- tempfile()
88dir.create(tmp2)
89checkpoint::checkpoint(snapshot_date = "2020-04-01",
90 r_version = "3.6.3",
91 checkpoint_location = tmp2,
92 scan_now = FALSE)
93## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
94## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
95## Warning messages:
96## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
97## Specified R version not the same as current R version
98## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
99## Specified R version not the same as current R version
100
101getOption("repos")
102## CRAN
103## "https://mran.microsoft.com/snapshot/2020-04-01"
104
105available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
106## Version Repository
107## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
108## Matrix "1.2-18" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
109## nlme "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
110
111download.packages("Matrix", ".", type = "source")
112## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
113## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
114## ==================================================
115## downloaded 1.8 MB
116##
117## [,1] [,2]
118## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"
119
120setwd(cwd)
121unlink(c(tmp1, tmp2), recursive = TRUE)
122contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "win.binary")
123## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1"
124
125contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "mac.binary")
126## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1"
127download.packages("Matrix", ".", type = "win.binary")
128## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
129## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
130## Warning in download.packages("Matrix", ".", type = "win.binary") :
131## no package 'Matrix' at the repositories
132## [,1] [,2]
133
134download.packages("Matrix", ".", type = "mac.binary")
135## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1:
136## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1/PACKAGES'
137## Warning in download.packages("Matrix", ".", type = "mac.binary") :
138## no package 'Matrix' at the repositories
139## [,1] [,2]
140The URLs that you actually need are:
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47> sessionInfo()
48R version 4.1.2 (2021-11-01)
49Platform: x86_64-w64-mingw32/x64 (64-bit)
50Running under: Windows 10 x64 (build 22000)
51
52Matrix products: default
53
54locale:
55[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
56[4] LC_NUMERIC=C LC_TIME=English_United States.1252
57system code page: 65001
58
59attached base packages:
60[1] stats graphics grDevices utils datasets methods base
61
62loaded via a namespace (and not attached):
63[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
64tmp1 <- tempfile()
65dir.create(tmp1)
66cwd <- setwd(tmp1)
67
68getOption("repos")
69## CRAN
70## "https://cloud.r-project.org/"
71
72available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
73## Version Repository
74## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
75## Matrix "1.4-0" "https://cloud.r-project.org/src/contrib"
76## nlme "3.1-155" "https://cloud.r-project.org/src/contrib"
77
78download.packages("Matrix", ".", type = "source")
79## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
80## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
81## ==================================================
82## downloaded 2.7 MB
83##
84## [,1] [,2]
85## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"
86
87tmp2 <- tempfile()
88dir.create(tmp2)
89checkpoint::checkpoint(snapshot_date = "2020-04-01",
90 r_version = "3.6.3",
91 checkpoint_location = tmp2,
92 scan_now = FALSE)
93## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
94## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
95## Warning messages:
96## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
97## Specified R version not the same as current R version
98## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
99## Specified R version not the same as current R version
100
101getOption("repos")
102## CRAN
103## "https://mran.microsoft.com/snapshot/2020-04-01"
104
105available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
106## Version Repository
107## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
108## Matrix "1.2-18" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
109## nlme "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
110
111download.packages("Matrix", ".", type = "source")
112## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
113## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
114## ==================================================
115## downloaded 1.8 MB
116##
117## [,1] [,2]
118## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"
119
120setwd(cwd)
121unlink(c(tmp1, tmp2), recursive = TRUE)
122contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "win.binary")
123## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1"
124
125contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "mac.binary")
126## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1"
127download.packages("Matrix", ".", type = "win.binary")
128## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
129## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
130## Warning in download.packages("Matrix", ".", type = "win.binary") :
131## no package 'Matrix' at the repositories
132## [,1] [,2]
133
134download.packages("Matrix", ".", type = "mac.binary")
135## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1:
136## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1/PACKAGES'
137## Warning in download.packages("Matrix", ".", type = "mac.binary") :
138## no package 'Matrix' at the repositories
139## [,1] [,2]
140## Windows
141"https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"
142
143## macOS, OS X, whatever
144"https://cran.microsoft.com/snapshot/2020-04-01/bin/macosx/el-capitan/contrib/3.6"
145The best way forward, in this case, may be to avoid checkpoint altogether and directly pass the valid URL to available.packages and download.packages:
1getPackages <- function(packs){
2 packages <- unlist(
3 tools::package_dependencies(packs, available.packages(),
4 which=c("Depends", "Imports"), recursive=TRUE)
5 )
6 packages <- union(packs, packages)
7 packages
8}
9install_version("<package name>",
10 version = "<version number>",
11 repos = "http://cran.us.r-project.org")
12tmp <- tempfile()
13dir.create(tmp)
14
15checkpoint::checkpoint(snapshot_date = "2020-04-01",
16 r_version = "3.6.3",
17 checkpoint_location = tmp,
18 scan_now = FALSE)
19
20
21packages <- getPackages(c("tidyquant"))
22
23download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
24Running create_checkpoint in the home directory may result
25in checkpointing very many packages. Continue? (Y/n) Y
26
27Creating checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
28Using checkpoint directory C:/Users/jerem/AppData/Local/Temp/Rtmpa2YEjU/file1efc6daf58e3/.checkpoint/2020-04-01/lib/x86_64-w64-mingw32/3.6.3
29Warning messages:
301: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
31 Specified R version not the same as current R version
322: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
33 Specified R version not the same as current R version
34>
35> packages <- getPackages(c("tidyquant"))
36>
37> download.packages(packages, destdir="C:\\Users\\jerem\\Downloads\\tidyquant", type="win.binary")
38Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
39 cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
40Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
41 no package โtidyquantโ at the repositories
42Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
43 no package โlubridateโ at the repositories
44Warning in download.packages(packages, destdir = "C:\\Users\\jerem\\Downloads\\tidyquant", :
45 no package โPerformanceAnalyticsโ at the repositories
46Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
47> sessionInfo()
48R version 4.1.2 (2021-11-01)
49Platform: x86_64-w64-mingw32/x64 (64-bit)
50Running under: Windows 10 x64 (build 22000)
51
52Matrix products: default
53
54locale:
55[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
56[4] LC_NUMERIC=C LC_TIME=English_United States.1252
57system code page: 65001
58
59attached base packages:
60[1] stats graphics grDevices utils datasets methods base
61
62loaded via a namespace (and not attached):
63[1] compiler_4.1.2 tools_4.1.2 checkpoint_1.0.2
64tmp1 <- tempfile()
65dir.create(tmp1)
66cwd <- setwd(tmp1)
67
68getOption("repos")
69## CRAN
70## "https://cloud.r-project.org/"
71
72available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
73## Version Repository
74## lattice "0.20-45" "https://cloud.r-project.org/src/contrib"
75## Matrix "1.4-0" "https://cloud.r-project.org/src/contrib"
76## nlme "3.1-155" "https://cloud.r-project.org/src/contrib"
77
78download.packages("Matrix", ".", type = "source")
79## trying URL 'https://cloud.r-project.org/src/contrib/Matrix_1.4-0.tar.gz'
80## Content type 'application/x-gzip' length 2849865 bytes (2.7 MB)
81## ==================================================
82## downloaded 2.7 MB
83##
84## [,1] [,2]
85## [1,] "Matrix" "./Matrix_1.4-0.tar.gz"
86
87tmp2 <- tempfile()
88dir.create(tmp2)
89checkpoint::checkpoint(snapshot_date = "2020-04-01",
90 r_version = "3.6.3",
91 checkpoint_location = tmp2,
92 scan_now = FALSE)
93## Creating checkpoint directory /var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T//RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
94## Using checkpoint directory /private/var/folders/n7/v9s56rmd5hn17d3f1qj13l7m0000gn/T/RtmpbrT5Br/filee2045e35c290/.checkpoint/2020-04-01/lib/aarch64-apple-darwin20/3.6.3
95## Warning messages:
96## 1: In create_checkpoint(snapshot_date, r_version, checkpoint_location, :
97## Specified R version not the same as current R version
98## 2: In use_checkpoint(snapshot_date, r_version, checkpoint_location, :
99## Specified R version not the same as current R version
100
101getOption("repos")
102## CRAN
103## "https://mran.microsoft.com/snapshot/2020-04-01"
104
105available.packages()[c("lattice", "Matrix", "nlme"), c("Version", "Repository")]
106## Version Repository
107## lattice "0.20-40" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
108## Matrix "1.2-18" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
109## nlme "3.1-145" "https://mran.microsoft.com/snapshot/2020-04-01/src/contrib"
110
111download.packages("Matrix", ".", type = "source")
112## trying URL 'https://mran.microsoft.com/snapshot/2020-04-01/src/contrib/Matrix_1.2-18.tar.gz'
113## Content type 'application/octet-stream' length 1871705 bytes (1.8 MB)
114## ==================================================
115## downloaded 1.8 MB
116##
117## [,1] [,2]
118## [1,] "Matrix" "./Matrix_1.2-18.tar.gz"
119
120setwd(cwd)
121unlink(c(tmp1, tmp2), recursive = TRUE)
122contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "win.binary")
123## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1"
124
125contrib.url("https://mran.microsoft.com/snapshot/2020-04-01/src/contrib", "mac.binary")
126## [1] "https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1"
127download.packages("Matrix", ".", type = "win.binary")
128## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1:
129## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/4.1/PACKAGES'
130## Warning in download.packages("Matrix", ".", type = "win.binary") :
131## no package 'Matrix' at the repositories
132## [,1] [,2]
133
134download.packages("Matrix", ".", type = "mac.binary")
135## Warning: unable to access index for repository https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1:
136## cannot open URL 'https://mran.microsoft.com/snapshot/2020-04-01/bin/macosx/contrib/4.1/PACKAGES'
137## Warning in download.packages("Matrix", ".", type = "mac.binary") :
138## no package 'Matrix' at the repositories
139## [,1] [,2]
140## Windows
141"https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"
142
143## macOS, OS X, whatever
144"https://cran.microsoft.com/snapshot/2020-04-01/bin/macosx/el-capitan/contrib/3.6"
145pkg <- "tidyquant"
146contriburl <- "https://cran.microsoft.com/snapshot/2020-04-01/bin/windows/contrib/3.6"
147
148db <- available.packages(contriburl)
149deps <- tools::package_dependencies(pkg, db, recursive = TRUE)[[pkg]]
150download.packages(c(pkg, deps), ".", contriburl = contriburl, type = "win.binary")
151QUESTION
Kubernetes NodePort is not available on all nodes - Oracle Cloud Infrastructure (OCI)
Asked 2022-Jan-31 at 14:37I've been trying to get over this but I'm out of ideas for now hence I'm posting the question here.
I'm experimenting with the Oracle Cloud Infrastructure (OCI) and I wanted to create a Kubernetes cluster which exposes some service.
The goal is:
- A running managed Kubernetes cluster (OKE)
- 2 nodes at least
- 1 service that's accessible for external parties
The infra looks the following:
- A VCN for the whole thing
- A private subnet on 10.0.1.0/24
- A public subnet on 10.0.0.0/24
- NAT gateway for the private subnet
- Internet gateway for the public subnet
- Service gateway
- The corresponding security lists for both subnets which I won't share right now unless somebody asks for it
- A containerengine K8S (OKE) cluster in the VCN with public Kubernetes API enabled
- A node pool for the K8S cluster with 2 availability domains and with 2 instances right now. The instances are ARM machines with 1 OCPU and 6GB RAM running Oracle-Linux-7.9-aarch64-2021.12.08-0 images.
- A namespace in the K8S cluster (call it staging for now)
- A deployment which refers to a custom NextJS application serving traffic on port 3000
And now it's the point where I want to expose the service running on port 3000.
I have 2 obvious choices:
- Create a LoadBalancer service in K8S which will spawn a classic Load Balancer in OCI, set up it's listener and set up the backendset referring to the 2 nodes in the cluster, plus it adjusts the subnet security lists to make sure traffic can flow
- Create a Network Load Balancer in OCI and create a NodePort on K8S and manually configure the NLB to the ~same settings as the classic Load Balancer
The first one works perfectly fine but I want to use this cluster with minimal costs so I decided to experiment with option 2, the NLB since it's way cheaper (zero cost).
Long story short, everything works and I can access the NextJS app on the IP of the NLB most of the time but sometimes I couldn't. I decided to look it up what's going on and turned out the NodePort that I exposed in the cluster isn't working how I'd imagine.
The service behind the NodePort is only accessible on the Node that's running the pod in K8S. Assume NodeA is running the service and NodeB is just there chilling. If I try to hit the service on NodeA, everything is fine. But when I try to do the same on NodeB, I don't get a response at all.
That's my problem and I couldn't figure out what could be the issue.
What I've tried so far:
- Switching from ARM machines to AMD ones - no change
- Created a bastion host in the public subnet to test which nodes are responding to requests. Turned out only the node responds that's running the pod.
- Created a regular LoadBalancer in K8S with the same config as the NodePort (in this case OCI will create a classic Load Balancer), that works perfectly
- Tried upgrading to Oracle 8.4 images for the K8S nodes, didn't fix it
- Ran the Node Doctor on the nodes, everything is fine
- Checked the logs of kube-proxy, kube-flannel, core-dns, no error
- Since the cluster consists of 2 nodes, I gave it a try and added one more node and the service was not accessible on the new node either
- Recreated the cluster from scratch
Edit: Some update. I've tried to use a DaemonSet instead of a regular Deployment for the pod to ensure that as a temporary solution, all nodes are running at least one instance of the pod and surprise. The node that was previously not responding to requests on that specific port, it still does not, even though a pod is running on it.
Edit2: Originally I was running the latest K8S version for the cluster (v1.21.5) and I tried downgrading to v1.20.11 and unfortunately the issue is still present.
Edit3: Checked if the NodePort is open on the node that's not responding and it is, at least kube-proxy is listening on it.
1tcp 0 0 0.0.0.0:31600 0.0.0.0:* LISTEN 16671/kube-proxy
2Edit4:: Tried adding whitelisting iptables rules but didn't change anything.
1tcp 0 0 0.0.0.0:31600 0.0.0.0:* LISTEN 16671/kube-proxy
2[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P FORWARD ACCEPT
3[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P INPUT ACCEPT
4[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P OUTPUT ACCEPT
5Edit5: Just as a trial, I created a LoadBalancer once more to verify if I'm gone completely mental and I just didn't notice this error when I tried or it really works. Funny thing, it works perfectly fine through the classic load balancer's IP. But when I try to send a request to the nodes directly on the port that was opened for the load balancer (it's 30679 for now). I get response only from the node that's running the pod. From the other, still nothing yet through the load balancer, I get 100% successful responses.
Bonus, here's the iptables from the Node that's not responding to requests, not too sure what to look for:
1tcp 0 0 0.0.0.0:31600 0.0.0.0:* LISTEN 16671/kube-proxy
2[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P FORWARD ACCEPT
3[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P INPUT ACCEPT
4[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P OUTPUT ACCEPT
5[opc@oke-cn44eyuqdoq-n3ewna4fqra-sx5p5dalkuq-1 ~]$ sudo iptables -L
6Chain INPUT (policy ACCEPT)
7target prot opt source destination
8KUBE-NODEPORTS all -- anywhere anywhere /* kubernetes health check service ports */
9KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
10KUBE-FIREWALL all -- anywhere anywhere
11
12Chain FORWARD (policy ACCEPT)
13target prot opt source destination
14KUBE-FORWARD all -- anywhere anywhere /* kubernetes forwarding rules */
15KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
16KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
17ACCEPT all -- 10.244.0.0/16 anywhere
18ACCEPT all -- anywhere 10.244.0.0/16
19
20Chain OUTPUT (policy ACCEPT)
21target prot opt source destination
22KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
23KUBE-FIREWALL all -- anywhere anywhere
24
25Chain KUBE-EXTERNAL-SERVICES (2 references)
26target prot opt source destination
27
28Chain KUBE-FIREWALL (2 references)
29target prot opt source destination
30DROP all -- anywhere anywhere /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
31DROP all -- !loopback/8 loopback/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
32
33Chain KUBE-FORWARD (1 references)
34target prot opt source destination
35DROP all -- anywhere anywhere ctstate INVALID
36ACCEPT all -- anywhere anywhere /* kubernetes forwarding rules */ mark match 0x4000/0x4000
37ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
38ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
39
40Chain KUBE-KUBELET-CANARY (0 references)
41target prot opt source destination
42
43Chain KUBE-NODEPORTS (1 references)
44target prot opt source destination
45
46Chain KUBE-PROXY-CANARY (0 references)
47target prot opt source destination
48
49Chain KUBE-SERVICES (2 references)
50target prot opt source destination
51Service spec (the running one since it was generated using Terraform):
1tcp 0 0 0.0.0.0:31600 0.0.0.0:* LISTEN 16671/kube-proxy
2[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P FORWARD ACCEPT
3[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P INPUT ACCEPT
4[opc@oke-cdvpd5qrofa-nyx7mjtqw4a-svceq4qaiwq-0 ~]$ sudo iptables -P OUTPUT ACCEPT
5[opc@oke-cn44eyuqdoq-n3ewna4fqra-sx5p5dalkuq-1 ~]$ sudo iptables -L
6Chain INPUT (policy ACCEPT)
7target prot opt source destination
8KUBE-NODEPORTS all -- anywhere anywhere /* kubernetes health check service ports */
9KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
10KUBE-FIREWALL all -- anywhere anywhere
11
12Chain FORWARD (policy ACCEPT)
13target prot opt source destination
14KUBE-FORWARD all -- anywhere anywhere /* kubernetes forwarding rules */
15KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
16KUBE-EXTERNAL-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes externally-visible service portals */
17ACCEPT all -- 10.244.0.0/16 anywhere
18ACCEPT all -- anywhere 10.244.0.0/16
19
20Chain OUTPUT (policy ACCEPT)
21target prot opt source destination
22KUBE-SERVICES all -- anywhere anywhere ctstate NEW /* kubernetes service portals */
23KUBE-FIREWALL all -- anywhere anywhere
24
25Chain KUBE-EXTERNAL-SERVICES (2 references)
26target prot opt source destination
27
28Chain KUBE-FIREWALL (2 references)
29target prot opt source destination
30DROP all -- anywhere anywhere /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
31DROP all -- !loopback/8 loopback/8 /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT
32
33Chain KUBE-FORWARD (1 references)
34target prot opt source destination
35DROP all -- anywhere anywhere ctstate INVALID
36ACCEPT all -- anywhere anywhere /* kubernetes forwarding rules */ mark match 0x4000/0x4000
37ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod source rule */ ctstate RELATED,ESTABLISHED
38ACCEPT all -- anywhere anywhere /* kubernetes forwarding conntrack pod destination rule */ ctstate RELATED,ESTABLISHED
39
40Chain KUBE-KUBELET-CANARY (0 references)
41target prot opt source destination
42
43Chain KUBE-NODEPORTS (1 references)
44target prot opt source destination
45
46Chain KUBE-PROXY-CANARY (0 references)
47target prot opt source destination
48
49Chain KUBE-SERVICES (2 references)
50target prot opt source destination
51{
52 "apiVersion": "v1",
53 "kind": "Service",
54 "metadata": {
55 "creationTimestamp": "2022-01-28T09:13:33Z",
56 "name": "web-staging-service",
57 "namespace": "web-staging",
58 "resourceVersion": "22542",
59 "uid": "c092f99b-7c72-4c32-bf27-ccfa1fe92a79"
60 },
61 "spec": {
62 "clusterIP": "10.96.99.112",
63 "clusterIPs": [
64 "10.96.99.112"
65 ],
66 "externalTrafficPolicy": "Cluster",
67 "ipFamilies": [
68 "IPv4"
69 ],
70 "ipFamilyPolicy": "SingleStack",
71 "ports": [
72 {
73 "nodePort": 31600,
74 "port": 3000,
75 "protocol": "TCP",
76 "targetPort": 3000
77 }
78 ],
79 "selector": {
80 "app": "frontend"
81 },
82 "sessionAffinity": "None",
83 "type": "NodePort"
84 },
85 "status": {
86 "loadBalancer": {}
87 }
88}
89Any ideas are appreciated. Thanks guys.
ANSWER
Answered 2022-Jan-31 at 12:06Might not be the ideal fix, but can you try changing the externalTrafficPolicy to Local. This would prevent the health check on the nodes which don't run the application to fail. This way the traffic will only be forwarded to the node where the application is . Setting externalTrafficPolicy to local is also a requirement to preserve source IP of the connection. Also, can you share the health check config for both NLB and LB that you are using. When you change the externalTrafficPolicy, note that the health check for LB would change and the same needs to be applied to NLB.
Edit: Also note that you need a security list/ network security group added to your node subnet/nodepool, which allows traffic on all protocols from the worker node subnet.
QUESTION
How can we mount a file as read-only in Linux through Go?
Asked 2022-Jan-25 at 16:08I want to know how can we mount an file as read-only in Linux CentOS 7 Server through Golang. I have tried syscall but that doesn't work, syscall mounts the file but as read-write i have tried to give ro argument in the data but still it's mounting as read-write. Here is my go code:
1syscall.Mount(src, dst, "auto", syscall.MS_BIND, "ro")
2You can see I have given ro argument in the data, i have also tried to give only r and readonly and also read-only but none of them works, when i compile the go file and execute it and then when i check /etc/mtab then i am getting this output :
1syscall.Mount(src, dst, "auto", syscall.MS_BIND, "ro")
2cat /etc/mtab | grep "firewall"
3/dev/vda1 /root/firewall.txt ext4 rw,relatime,data=ordered,jqfmt=vfsv1,usrjquota=quota.user 0 0
4Here you can see that firewall.txt is mounted as rw means read-write
I want to mount it as read-only , can anyone help me how can i do that in Golang?
ANSWER
Answered 2022-Jan-25 at 14:23Read-only mode is defined by the syscall flag MS_RDONLY, which is also defined in the syscall package. So the call should be:
1syscall.Mount(src, dst, "auto", syscall.MS_BIND, "ro")
2cat /etc/mtab | grep "firewall"
3/dev/vda1 /root/firewall.txt ext4 rw,relatime,data=ordered,jqfmt=vfsv1,usrjquota=quota.user 0 0
4syscall.Mount(src, dst, "auto", syscall.MS_BIND | syscall.MS_RDONLY, "")
5QUESTION
Terraform: How to modify a public subnet's route table that was created by module 'vpc'?
Asked 2022-Jan-23 at 04:56I used the vpc module to create my VPC via the following code:
1module "vpc" {
2 source = "terraform-aws-modules/vpc/aws"
3 name = "${var.namespace}-vpc"
4 cidr = "10.0.0.0/16"
5 azs = data.aws_availability_zones.available.names
6 private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
7 public_subnets = ["10.0.101.0/24", "10.0.102.0/24"]
8 #assign_generated_ipv6_cidr_block = true
9 create_database_subnet_group = true
10 enable_nat_gateway = true
11 single_nat_gateway = true
12 enable_dns_hostnames = true
13 enable_dns_support = true
14}
15This module automatically creates two public subnets that has a route table that points to an internet gateway. However, I would like to modify one of the two public subnets to have a different route table that points to a firewall that I have created.
What I did was to create a new route table pub_to_firewall, and then create a new aws_route_table_association to associate the public subnet with the new route table.
1module "vpc" {
2 source = "terraform-aws-modules/vpc/aws"
3 name = "${var.namespace}-vpc"
4 cidr = "10.0.0.0/16"
5 azs = data.aws_availability_zones.available.names
6 private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
7 public_subnets = ["10.0.101.0/24", "10.0.102.0/24"]
8 #assign_generated_ipv6_cidr_block = true
9 create_database_subnet_group = true
10 enable_nat_gateway = true
11 single_nat_gateway = true
12 enable_dns_hostnames = true
13 enable_dns_support = true
14}
15resource "aws_route_table_association" "sn_to_fw_rt_association" {
16 subnet_id = module.vpc.public_subnets[0]
17 route_table_id = aws_route_table.pub_to_firewall.id
18 depends_on = [
19 aws_route_table.pub_to_firewall,
20 ]
21}
22I have been able to follow the instructions to import the original association to this new association, and terraform apply to get the public subnet to have this new route table containing the firewall reference.
However, when I run terraform apply again, terraform now wants to go back to the 'default' associations:
1module "vpc" {
2 source = "terraform-aws-modules/vpc/aws"
3 name = "${var.namespace}-vpc"
4 cidr = "10.0.0.0/16"
5 azs = data.aws_availability_zones.available.names
6 private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
7 public_subnets = ["10.0.101.0/24", "10.0.102.0/24"]
8 #assign_generated_ipv6_cidr_block = true
9 create_database_subnet_group = true
10 enable_nat_gateway = true
11 single_nat_gateway = true
12 enable_dns_hostnames = true
13 enable_dns_support = true
14}
15resource "aws_route_table_association" "sn_to_fw_rt_association" {
16 subnet_id = module.vpc.public_subnets[0]
17 route_table_id = aws_route_table.pub_to_firewall.id
18 depends_on = [
19 aws_route_table.pub_to_firewall,
20 ]
21}
22Objects have changed outside of Terraform
23
24Terraform detected the following changes made outside of Terraform since the last "terraform apply":
25
26 # module.networking.module.vpc.aws_route_table_association.public[0] has been deleted
27 - resource "aws_route_table_association" "public" {
28 - id = "rtbassoc-[ ]" -> null
29 - route_table_id = "rtb-0cabc2388adXXXXX" -> null
30 - subnet_id = "subnet-0a2b011cd7aXXXXX" -> null
31 }
32
33
34Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or respond to these
35changes.
36โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
37
38Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
39 + create
40 ~ update in-place
41
42Terraform will perform the following actions:
43
44
45 # module.networking.module.vpc.aws_route_table_association.public[0] will be created
46 + resource "aws_route_table_association" "public" {
47 + id = (known after apply)
48 + route_table_id = "rtb-0cabc2388adXXXXX"
49 + subnet_id = "subnet-0a2b011cd73XXXXX"
50 }
51I do not want this resource to be recreated because it would throw an error that โ Error: error creating Route Table (rtb-0cabc2388adXXXXX) Association: Resource.AlreadyAssociated: the specified association for route table rtb-0cabc2388adXXXXX conflicts with an existing association obviously since I already associated it with the new routing table.
How can I either:
- Force terraform to 'ignore' the default subnet to routing tables setup
- Or update the
vpccreated aws_route_table_association resourcemodule.networking.module.vpc.aws_route_table_association.public[0]to reference the new route table instead?
ANSWER
Answered 2022-Jan-21 at 09:05You can't change that, as this is how the aws vpc module works. You need custom designed VPC for that. So you have to either fork the entire module and made the changes that you want, or create new VPC module from scratch tailored to your needs.
QUESTION
Why is ArgoCD confusing GitHub.com with my own public IP?
Asked 2022-Jan-10 at 17:37I have just set up a kubernetes cluster on bare metal using kubeadm, Flannel and MetalLB. Next step for me is to install ArgoCD.
I installed the ArgoCD yaml from the "Getting Started" page and logged in.
When adding my Git repositories ArgoCD gives me very weird error messages:
The error message seems to suggest that ArgoCD for some reason is resolving github.com to my public IP address (I am not exposing SSH, therefore connection refused).
I can not find any reason why it would do this. When using https:// instead of SSH I get the same result, but on port 443.
I have put a dummy pod in the same namespace as ArgoCD and made some DNS queries. These queries resolved correctly.
What makes ArgoCD think that github.com resolves to my public IP address?
EDIT:
I have also checked for network policies in the argocd namespace and found no policy that was restricting egress.
I have had this working on clusters in the same network previously and have not changed my router firewall since then.
ANSWER
Answered 2022-Jan-08 at 21:04That looks like argoproj/argo-cd issue 1510, where the initial diagnostic was that the cluster is blocking outbound connections to GitHub. And it suggested to check the egress configuration.
Yet, the issue was resolved with an ingress rule configuration:
need to define in
values.yaml.
argo-cddefault provide subdomain but in our case it was/argocd
1ingress:
2 enabled: true
3 annotations:
4 kubernetes.io/ingress.class: nginx
5 nginx.ingress.kubernetes.io/backend-protocol: HTTP
6 nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
7 nginx.ingress.kubernetes.io/rewrite-target: /
8 path: /argocd
9 hosts:
10 - www.example.com
11and this I have defined under
templates >> argocd-server-deployment.yaml
1ingress:
2 enabled: true
3 annotations:
4 kubernetes.io/ingress.class: nginx
5 nginx.ingress.kubernetes.io/backend-protocol: HTTP
6 nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
7 nginx.ingress.kubernetes.io/rewrite-target: /
8 path: /argocd
9 hosts:
10 - www.example.com
11containers:
12 - name: argocd-server
13 image: {{ .Values.server.image.repository }}:{{ .Values.server.image.tag }}
14 imagePullPolicy: {{ .Values.server.image.pullPolicy }}
15 command:
16 - argocd-server
17 - --staticassets - /shared/app - --repo-server - argocd-repo-server:8081 - --insecure - --basehref - /argocd
18The same case includes an instance very similar to yours:
In any case, do check your git configuration (git config -l) as seen in the ArgoCD cluster, to look for any insteadOf which would change automatically github.com into a local URL (as seen here)
QUESTION
New-CimSession without elevation by providing admin credentials on Windows 10?
Asked 2021-Dec-31 at 13:43I need to query some WMI values using PowerShell from Windows 10 devices. The script is executed in the context of a non-admin user by some software distribution tooling.
There is a local admin account, and for the current purpose (retrieving information before wiping the system) it wouldn't be a problem to put the password in the script. As automation is a hard requirement, there is no way to deal with UAC windows or the user to enter some credentials.
Is there any way to get
1$sess = New-CimSession -Credential $admincred
2to work without running into Access is denied, because it isn't run in an elevated context? Can I somehow self-elevate it by just having the admin credentials?
[Edit]
The comments asked to provide more concrete information:
I want to onboard many unmanaged (i.e. no software distribution tool, no domain join) Windows 10 devices to Windows Autopilot.
The devices are not at a specific site.
The device vendor can't provide the information.
The users don't have administrative privileges
The users don't know the local admin password (I do)
Exposing the local admin password is less of a problem than the missing tech knowledge of the users (the password is considered legacy)
The firewall is preventing incoming traffic (no RDP, WinRM)
Code (Source):
$devDetail = (Get-CimInstance -CimSession $session -Namespace root/cimv2/mdm/dmmap -Class MDM_DevDetail_Ext01 -Filter "InstanceID='Ext' AND ParentID='./DevDetail'")
It is too time consuming to get the information using manual remote sessions with a tool like Teamviewer. Getting the users to download a tool from the intranet and running it would be a way to go. So I created a standalone application that builds and runs a customized PowerShell script. What won't work is getting it to run in an elevated session. I always end up with Access denied.
ANSWER
Answered 2021-Dec-31 at 13:43Can I somehow self-elevate it by just having the admin credentials?
No you cannot. UAC is designed to prevent exactly what you are trying to do. Related Q&A:
- elevate without prompt - verb runas start-process
- UAC Getting in the Way of EXE Install Powershell
- Powershell provide credentials for RunAs
There may be many workarounds, but they all will have in common that you have to go to your machines (locally or remotely) at least once, gain administrative privileges and prepare something, e. g.:
- A scheduled task that runs under your local administrator account or under SYSTEM and triggers the execution of your script
- Disabling UAC (temporarily) (not recommended either way)
- Installing any remote management software, services or accounts (with extra run as background job privilege)
QUESTION
Failing to run Tomcat in IntelliJ IDEA
Asked 2021-Nov-09 at 07:09I'm trying to use IntelliJ IDEA to run a Tomcat server but failing to so, I keep getting the following error code:
Application Server was not connected before run configuration stop, reason: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.CommunicationException [Root exception is java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is: java.net.SocketException: Connection reset]
As seen in the following picture: https://i.stack.imgur.com/yqYav.png
Also, I manage to run a Tomcat server in CMD, but it just doesn't seem to work in IntelliJ. This is a pic of me running Tomcat in CMD succesfully and trying to connect to the same port in the IntelliJ (obviously failing to do so) but it does seem to be trying to connect, as in there is nothing really blocking the connection: https://i.stack.imgur.com/OYIvK.png
I tried every solution I could find:
- specify the same port for both JMX and RMI conversation (here and here)
- configure the Tomcat server again (here)
- make the the path to the Tomcat files is correct (here)
- made sure no Firewall/antivirus is blocking the network connection.
- uninstall and install again both IntelliJ and Tomcat
- specify the IP (here)
- using different ports
Thanks!
ANSWER
Answered 2021-Nov-09 at 07:09The problem was solved.
Running the Apache Tomcat Installer instead of just downloading the zip solved it.
Community Discussions contain sources that include Stack Exchange Network
Tutorials and Learning Resources in Firewall
Tutorials and Learning Resources are not available at this moment for Firewall
Share this Page
Get latest updates on Firewall