mechanize | Mechanize is web scraping tool | Scraper library

This is my first time using stack overflow so apologies if I do this wrong.

I'm fairly new to coding in R and I'm trying to make a simple Shiny app using a TidyTuesday dataset. I wanted to make a map with points showing the different types of water systems ("water_tech") and radio buttons to choose which type of water system is plotted on the map. I got the app to load without an error message, however no matter which button is selected, all of the different types of water systems are plotted on the map, not just the one I selected (essentially, the buttons don't work). If anyone has any ideas about what could be causing this to happen I would greatly appreciate it!

Reproducible code:

I am more of a Java programmer and still somewhat new to development (2 years or so, can write Java code & web apps just fine) however the company I work for has 4 Rails applications and was asked to get this application working called CtrlPanel. I have been having to learn Ruby on Rails in order to help get this issue with this app fixed and get it working.

I have been working on this problem for over a week all day long every day and nothing I do is fixing it.

I fixed everything to the point the app comes up, web server runs serves the pages but all views are white screens as long as this application.html.haml file is present. I re-wrote the file with very basic bootstrap and it sort of works but nothing looks right. The problem seems to stem from 1 single like that simply says: = javascript_include_tag "application"

I have been all over the internet and have tried every single fix from changing coffee-script-source to v1.8.0 as I read Windows has an issue with newer rails and that file, I have tried every variation of changing it from application to default, and every type of ending you can think of no matter what I do it gives me this error message which I can not seem to find.

I am not even sure WHAT that line does, I assume it has to do with the new Google Maps API and I verified the key is valid and it was working before.

This is the error is it giving it says the line with "= javascript_include_tag" "application" giving error ExecJS::RuntimeError at / SyntaxError: [stdin]:1:1: unexpected //=

I am running a PC on Windows 10 20H2 x64 UEFI ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x64-mingw32] Rails 6.1.3

(I did also install Ubuntu on another machine and it gives the exact same error, also gives the same error on another Windows machine)

The app is working IF I delete the "application.html.haml" file and put in a skeleton basic version all of the other views start working but of course none of them look right no menus no bootstrap no nothing.
Here is the application.html.haml file.

I'm trying to mechanize calling a docs script routine from a button click (see this question). It seems the only way to do this is to create a sideboard for the document.

Here's my script:

I'm trying to use mechanize in python to login to this site: https://login.haaretz.co.il/ On the surface, it looks like a 2-phase login process, same as google, but following receipts for google login via mechanize gets me nowhere. After submit()-ing the browser seems to remain on the same page, with a single form containing the single userName control. What am I doing wrong?

I'm trying to scrape a particularly troublesome website. Though all the parameters match and the referrer matches, I see different results when perl runs it than when I watch dev tools.

When I do a copy-as-curl from dev tools, the only header I can't confirm as identical is -H 'Cookie: and its contents. Running that curl command gives me the proper results just as I receive in the browser.

So, what syntax do I use with WWW::Mechanize to set the cookie's value explicitly rather than letting Mechanize do it for me based on the past gets/posts?

Also, how can I view what it does want to set the cookie's value to?

I want to use Perl www::mechanize to connect to the webserver and request a resource. E.g. http://www.my.domain/test.html. But I want to specify the IP address independently from the hostname in the URL.

For example: www.my.domain resolves to 1.1.1.1, but I want to connect to 2.2.2.2.

I want to do this to test multiple web servers behind a load balancer.

Im trying to make a basic web scraper with Rails. everytime I hit the scrape button it sends me to the correct location but gives me this error everytime.

here is my restaurants_controller.rb file

I'm working on a project whose aim is to retrieve all the information from a news article (media website), for this I'm using the library newspaper3K which works quite well.

however I have a problem concerning some urls (redirected link), according to my research newspaper3k does not load the redirection url, it only treats the sent url as a parameter.

Here is an example of a link I would like to deal with:

url = "wtm.actualite.20minutes.fr/redirection.html?m=3e2b20a2f1f6dd3c60608f54d7ad4dc5&c=fr&u=https%3A%2F%2Fwww.20minutes.fr%2Fmonde%2F2943823-20210103-bahamas-disparition-bateau-20-personnes-bord%3Fxtor%3DEREC-182-%5Bactualite%5D&dc=yt0U%2FI8COMJyjwQQ1fA2kVEXpoP0nsZydMTZS6jTm2DdKasFuV%2FVA7rEphhqMfGAy%2FlztUlVN4MJt5tg%2FQXfJwmXMRQL8g3Gfwhl%2BsjkkYmd%2BDxDUhb%2BpPRL%2BNsiDETNQeP3MmrQ6ATGJT%2Blf46Zg4DHd%2FzaXy%2B7UAuxatp2UcVd39HKuuMfQHmyDV%2BAxSAJrd4x5CxHqy3uTtZoQEjwGdZ%2FRtoa7YLOWLKhN9tg4TM%3D"

so the goal here with this url is to get the right url (after redirection) and then send it to newspaper3K.

I have tried the following solutions but they don't work on my side;

1 - using the library resquests as follows response = requests.get(url, verify=False, allow_redirects=True)

2- using the mechanize library as follows:

I have a post on my fb page which I need to update several times a day with data elaborated in a python script. I tried using Selenium, but it gets often stuck when saving the post hence the script gets stuck too, so I'm trying to find a way to do the job within python itself without using a web browser.

I wonder is there a way to edit a FB post using a python library such as Facepy or similar?

I'm reading the graph API reference but there are no examples to learn from, but I guess first thing is to set up the login. On the facepy github page is written that

note that Facepy does not do authentication with Facebook; it only consumes its API. To get an access token to consume the API on behalf of a user, use a suitable OAuth library for your platform

I tried logging in with BeautifulSoup