browserless | remote clients to connect , drive , and execute | Continuous Deployment library
kandi X-RAY | browserless Summary
kandi X-RAY | browserless Summary
browserless is a web-service that allows for remote clients to connect, drive, and execute headless work; all inside of docker. It offers first-class integrations for puppeteer, playwright, selenium's webdriver, and a slew of handy REST APIs for doing more common work. On top of all that it takes care of other common issues such as missing system-fonts, missing external libraries, and performance improvements. We even handle edge-cases like downloading files, managing sessions, and have a fully-fledged documentation site. If you've been struggling to get Chrome up and running docker, or scaling out your headless workloads, then browserless was built for you.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of browserless
browserless Key Features
browserless Examples and Code Snippets
Community Discussions
Trending Discussions on browserless
QUESTION
I want to use Playwright.connect() method using Proxy to consume Browserless. According to Browserless doc.
...ANSWER
Answered 2022-Mar-30 at 22:28With the help of my friend Alejandro Loyola at Browserless, I am now able to connect. I will post the snippet:
QUESTION
I am developing a simple prototype with Java + Playwright + Browserless. I was wondering if is possible avoid downloading the browsers in AWS.
...ANSWER
Answered 2022-Mar-17 at 13:31You can skip the browser download by setting an environment variable PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1
See the docs here: https://playwright.dev/java/docs/browsers#skip-browser-downloads
QUESTION
Background:
I am running a browserless application in python using the device code flow to authenticate with Azure Active Directory with token caching using the Python Microsoft Authentication Library (MSAL). I wrote the script based on this blog post. I intend to run this script on a cron job every day, but would like to reduce the time that the msal.acquire_token_by_device_flow()
method waits to receive the token after successful authentication in browser. It appears the default value is about 5 hours. This specific use case is when the 90 day refresh token expires and the user needs to re-authenticate, I'd like the script to not hang for hours waiting for the user to reauthenticate.
What I've Tried:
I tried to control this by using the inherited timeout
argument (in seconds) when instantiating the PublicClientApplication()
object, but it did not affect the timeout. I am not using the http_client
argument (mentioned when using timeout
). Specifically, I have tried both of the following instantiations:
ANSWER
Answered 2021-Feb-26 at 06:24If you want to abort the polling loop at any time, we can change the value of the flow’s “expires_at” key. For more details, please refer to here
For example
QUESTION
I am trying to hit microsoft to do list api using python program.
Api : https://graph.microsoft.com/beta/me/todo/lists , which resulted in error : Access is denied due to invalid credentials. But when I tried the api : https://graph.microsoft.com/v1.0/users, I am getting the results properly. The documentation of the api (https://docs.microsoft.com/en-us/graph/api/resources/todo-overview?view=graph-rest-beta) says "Currently, the API supports only permissions delegated by the signed-in user". Currently I am trying to get auth code and then auth token and then hitting the API. To act as signed in user, what method should I follow in python? Please note I am not using a web frame work. When I tried using flask and then hitting the API , it works. Need to know how a browserless console python application can serve the purpose.
Thank youin advance for the help
...ANSWER
Answered 2020-Oct-07 at 14:46You are getting an App only token using a client credential flow and it doesn't ask for user credentials. Please check these Authentication flows and according to your requirement you can choose one of these and implement it. In other ways you can test your HTTP call in Graph Explorer by adding permissions there itself. Your call works here because you will login as user.
QUESTION
I am developing a browserless app that needs access to two APIs - an Application API (read-only, e.g remote configuration) and a User API (read-write, e.g user preferences).
The app has the Client ID and Secret stored locally and therefore takes advantage of the Client Credentials flow to access the Application API. Now I need users to be able to access a User API and I'm wondering how to configure the backend (which in this case uses IdentityServer4) to do so. The API should be protected so that only authenticated users can access it and am planning on reading the Claims sent to identify the user.
Should I extend the existing Client ID to also allow the Device Flow grant type or create a separate Client just for this "user-only" flow?
I'm new to OAuth and want to make sure I'm doing this right so everything is kept secure.
...ANSWER
Answered 2020-Aug-22 at 12:16Always use different OAuth clients for different client side connections, which will give you more control:
- Rules such as access token lifetime can be set differently when needed
- A compromised client (eg secret stolen) can be disabled without impacting other clients
- Metrics and logging will be more useful also
It is usually easy to easily configure multiple clients in the Authorization Server via a Management UI.
QUESTION
I have two containers, one Puppeteer and one Browserless. I try to connect to Browserless, but it fails. Both containers are in the same network.
Here is my code:
...ANSWER
Answered 2020-Aug-07 at 13:28Websocket secure, short wss was my issue. The following code works fine.
QUESTION
I am currently trying to work with the Dart Puppeteer library. I have been running the following piece of code:
...ANSWER
Answered 2020-Jun-06 at 20:20It turned out to be a problem with Browserless rather than any Dart code on that certain day. I tried it later and it worked flawlessly.
QUESTION
I wanted to know if anyone using Puppeteer-Cluster could elaborate on how the Cluster.Launch({settings}) protects against sharing of cookies and web data between pages in different context.
Do the browser contexts here, actually block cookies and user-data is not shared or tracked? Browserless' now infamous page seems to think no, here and that .launch({}) should be called on the task, not ahead of the queue.
So my question is, how do we know if puppeteer-cluster is sharing cookies / data between queued tasks? And what kind of options are in the library to lower the chances of being labeled a bot?
Setup: I am using page.authenticate with a proxy service, random user agent, and still getting blocked(403) occasionally by the site which I'm performing the test.
...ANSWER
Answered 2020-Jan-16 at 16:01Author of puppeteer-cluster
here. The library does not actively block cookies, but makes use of browser.createIncognitoBrowserContext()
:
Creates a new incognito browser context. This won't share cookies/cache with other browser contexts.
In addition, the docs state that "Incognito browser contexts don't write any browsing data to disk" (source), so that restarting the browser cannot reuse any cookies from disk as there were no data written.
Regarding the library, this means when a job is executed, a new incognito context is created, which does not share any data (cookies, etc.) with other contexts. So as long as Chromium properly implements the incognito browser contexts, there is no data shared between the jobs.
The page you linked only talks about browser.newPage()
(which shares cookies between pages) and not about incognito contexts.
Some websites will still block you, because they use different measures to detect bots. There are headless browser detection tests as well as fingerprinting libraries that might report you as bot if the user agent does not match the browser fingerprint. You might be interested in this answer by me that provides some more detailed explanation how these fingerprints work.
You can try to use a library like puppeteer-extra
that comes with a stealth
plugin to help you solve the problem. However, this basically is a cat-and-mouse game. The fingerprinting tests might be changed or another sites might use a different "detection" mechanism. All-in-all, there is no way to guarantee that a website does not detect you.
In case you want to use puppeteer-extra
, be aware that you can use it in conjunction with puppeteer-cluster
(example code).
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install browserless
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page