datatracker | WIP experimental project to make for a better ML | Machine Learning library
kandi X-RAY | datatracker Summary
kandi X-RAY | datatracker Summary
WIP experimental project to make for a better ML development UX.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
- Train model
- Get k - values from the results
- Get the largest k values in the dataset
- Returns a Pandas DataFrame containing the results
- Install signal handler
- Dump the results to a csv file
- Get the smallest k in descending order
- Calculate the performance drop
- Get the smallest gain
- Compute the gain for a given name
- Get the smallest k k
- Get the highest gain
- Retrieve a group s results
- Add a tracker function
- Get the smallest false negative negatives
- Get the largest false positives
- Dump results to CSV file
- Gets a single individual by id and name
- Evaluate the model
- Gets the most k - th drop in descending order
datatracker Key Features
datatracker Examples and Code Snippets
Community Discussions
Trending Discussions on datatracker
QUESTION
I am trying to write a regexp to extract an URL components. The syntax can be found here: RFC 3986.
Some of the components are optional. So far I have:
...ANSWER
Answered 2022-Apr-10 at 15:22Your regular expression works fine if you just escape the slashes and preferably the colon as well. The result is (.+)\:\/\/(.*@)?(.+?)(:(\d*))?\/((.*)\?)?((.*)#)?(.*)
. Here is a simple script to show how it can be used to filter out invalid URIs:
Update Following the comments I have made the following modification:
- I have added
(\:((\d*)\/))?(\/)*
. Explanation:\:((\d*)
matches a colon and then any string of digits.- the
\/
after this matches a slash which should be after this string of digits. This is because the port must not contain any other characters but digits. So they cannot be found in the port-portion of the uri. - Finally, the entire port-matching expression is optional, hence the
?
. - The last part indicates that many or no slashes can follow the existing/non-existing port
Final regExp:
(.+)\:\/\/(.*\@)?(.+?)(\:((\d*)\/))?(\/)*((.*)\?)?((.*)\#)?(.*)
QUESTION
I am looking into a way to properly implement refresh & access tokens on a simple SPA with Dotnet Core Backend. The more I read about it the more I seem to worry about its impact on server performance especially as the number of logged in users grows.
Take this auth0 post and this specification for example it clearly demonstrates that we need to create a new refresh token every time we create an access token, due to Malicious Client attempting to reuse Refresh Token.
In particular, authorization servers: MUST rotate refresh tokens on each use, in order to be able to detect a stolen refresh token if one is replayed (described in [oauth-security-topics] section 4.12)
Now given that we want to keep the Access token expiry time limited (e.g. 10-20 minutes) and we need to persist every refresh token which we generate in order to recognize malicious activity of old refresh token being reused.
Which means that every 20 minutes n users hit our backend to refresh Access token and create a new refresh token, so for 1k logged in users that`s 1k requests every 20 minutes, also for each of those users our api checks if refresh token they have presented has been already invalidated, if not, we persist the new refresh token.
Hence after a day of user being logged in, we saved: 24 * 60 / 20 = 72 different refresh tokens .. and now we check every user against every single one ??
Am I missing something, how is this scalable?
...ANSWER
Answered 2022-Mar-28 at 21:25You actually don't need to store every refresh token ever made. You only need a list of used tokens to check if the one your user is trying to refresh is being reused, and since your refresh tokens should have an expiry time just like your access tokens, you also don't need to store tokens, even used ones, for longer than their lifespan.
- If a user tries to use a valid refresh token: It's not expired and not in your used tokens list. Good to go!
- If a user tries to use an expired token: It's expired so no need to worry about reuse, no need to check your database for used tokens.
- If a user tries to reuse a valid refresh token: It's in your list.
So while it does scale with the number of users, it's stable over time and won't blow out of proportion (on the condition that you do purge old tokens).
There are other things to consider. As mentioned in this other auth0 post, in case of a reuse, you want to invalidate the whole token family and not just deny access to the one user that reused a token (it might be the legitimate user). Then again, you don't need to store every token from the token family to keep track of what needs to be invalidated: You just need to add a family identifier to your tokens, mark that family identifier itself as invalid in case of a reuse, and deny future refresh attempts if they belong to an invalidated family. The list of invalidated families can be purged as well of all family identifiers invalidated for longer than your refresh tokens lifespan.
On the server request side of things, refresh tokens should be a net performance gain compared to other authorisation means like API keys or HTTP basic authentication, since for every refresh tokens you have to emit, you're also getting 20 minutes worth of requests for which you won't have to query the database to check if the API key is still valid, or if the password provided is the right one (especially since good password hashing functions like bcrypt are slow on purpose).
QUESTION
I have a basic web server that renders blog posts from a database of JSON posts wherein the main paragraphs are built from a JSON string array. I was trying to find a way to easily encode new lines or line breaks and found a lot of difficulty with how the encoding for these values changes from JSON to GoLang and finally to my HTML webpage. When I tried to encode my JSON with newlines I found I had to encode them using \\n
rather than just \n
in order for them to actually appear on my page. One problem however was they simply appeared as text and not line breaks.
I then tried to research ways to replace the \n
portions of the joined string array into
tags, however I could not find any way to do this with go and moved to trying to do so in javascript. This did not work either despite me deferring the calling of my javascript in my link from my HTML. this is that javascript:
ANSWER
Answered 2022-Mar-19 at 06:43You could try to loop over your array inside the template and generate a p tag for every element of the array. This way there is no need to edit your main array in go.
Template:
QUESTION
Yohoho! I am building an application that leverages OAuth to pull user data from provider APIs, and am wondering about the RFC compliance of the flow I have chosen.
Currently, the user signs into the authorization server which sends an auth code to my frontend client. The frontend then passes the auth code to my backend, which exchanges it for an auth token and then makes calls to the provider to pull data.
In this best practices document, it states:
Note: although PKCE so far was recommended as a mechanism to protect native apps, this advice applies to all kinds of OAuth clients, including web applications.
To my understanding, PKCE is designed to ensure the token is granted to the same entity that requested the auth code, in order to prevent attackers from using stolen auth codes to execute unwarranted requests.
Now, it makes sense to me why this is important even if the backend keeps the client secret unexposed, since the attacker can make requests to the backend with the intercepted auth code to receive the token. However in my flow, since I am not creating an authentication scheme and rather trying to authorize premeditated requests, the token stays with the backend.
So why is PKCE recommended here? It seems to me the most a stolen auth code can do is initiate an API request from the backend, with no token or data being returned to the attacker. And assuming a PKCE implementation is the way to go, how exactly would it work? The frontend requesting the auth code and the backend trading it for a token aren't exactly the same, so would it be as simple as passing the code_verifier to the backend to make the call?
Some clarification would be greatly appreciated.
...ANSWER
Answered 2022-Mar-19 at 06:50PKCE ensures that the party who started the login is also completing it, and there are two main variations that I'll summarise below in terms of Single Page Apps (SPA).
PUBLIC CLIENTS
Consider a Single Page App that runs a code flow implemented only in Javascript. This would store a code verifier in session storage during the OpenID Connect redirect. Upon return to the app after login, this would be sent, along with the authorization code and state to the Authorization Server.
This would return tokens to the browser. If there was a Cross Site Scripting vulnerability, the flow could be abused. In particular the malicious code could spin up a hidden iframe and use prompt=none
to get tokens silently.
CONFIDENTIAL CLIENTS
Therefore the current best practice for Single Page Apps is to use a Backend for Frontend (BFF), and never return tokens to the browser. In this model it is more natural for the BFF to operate like a traditional OpenID Connect website, where both the state
and code_verifier
are stored in a login cookie
that lasts for the duration of the sign-in process.
If there was a Cross Site Scripting vulnerability, then session riding
is possible by the malicious code, to send the authorization code to the BFF and complete a login. However, this would just result in writing secure cookies that the browser cannot access. Similarly, the hidden iframe hack would also only rewrite cookies.
The code_verifier
could alternatively be stored in session storage and sent from the browser to the BFF, but it would be easy enough for malicious code to grab it and also send it to the server. This does not really change the risks, and the key point is that no tokens should be returned to the browser. It is OK to use secondary security values in the browser, as long as you can justify them, eg for security reviewers. Generally though it is easier to explain if secure values are in secure cookies and less visible to Javascript.
FURTHER INFO
Best practices often vary depending on the customer use case, and at Curity we provide resources for customers (and general readers) to explain secure design patterns. These are based on security standards and we translate them to customer use cases. You may find our recent SPA Security Whitepaper useful.
QUESTION
I am experimenting with RFC Expect Header in java HttpURLConnection which works perfectly except for one detail.
There is an 5 second wait period between sending the body of an Fixed Length Mode or between each chunk in Chunk Streaming Mode
Here is the client class
...ANSWER
Answered 2022-Mar-16 at 18:413 Chunks. For each chunk respond with 100 Continue
That is not how Expect: 100-Continue
works. Your server code is completely wrong for what you are attempting to do. In fact, your server code is completely wrong for an HTTP server in general. It is not even attempting to parse the HTTP protocol at all. Not the HTTP headers, not the HTTP chunks, nothing. Is there a reason why you are not using an actual HTTP server implementation, such as Java's own HttpServer
?
When using Expect: 100-Continue
, the client is required to send ONLY the request headers, and then STOP AND WAIT a few seconds to see if the server sends a 100
response or not:
If the server responds with
100
, the client can then finish the request by sending the request body, and then receive a final response.If the server responds with anything other than
100
, the client can fail its operation immediately without sending the request body at all.If no response is received, the client can finish the request by sending the request body and receive the final response.
The whole point of Expect: 100-Continue
is for a client to ask for permission before sending a large request body. If the server doesn't want the body (ie, the headers describe unsatisfactory conditions, etc), the client doesn't have to waste effort and bandwidth to send a request body that will just be rejected.
HttpURLConnection
has built-in support for handling 100
responses, but see How to wait for Expect 100-continue response in Java using HttpURLConnection for caveats. Also see JDK-8012625: Incorrect handling of HTTP/1.1 " Expect: 100-continue " in HttpURLConnection.
But, your server code as shown needs a major rewrite to handle HTTP properly, let alone handle chunked requests properly.
QUESTION
Simple question here... According to the SCIM specification (https://datatracker.ietf.org/doc/html/rfc7643) and the seriously limited documentation I could find, there is a "core user schema" provided by SCIM. It is described in section 4 of the RFC but this is just a vague description. Where is the actual schema? It would be significantly easier to design my own schema if I could see how things are defined in the core user schema. My system only requires/accepts one email address for example, whereas the core schema talks about multiple addresses with different types.
...ANSWER
Answered 2022-Mar-15 at 04:55It's located here: https://datatracker.ietf.org/doc/html/rfc7643#section-8.7.1
For the emails scenario you described, the easiest approach would be to only allow one type(work, other..) and reject any request without a type or with a disallowed type.
QUESTION
I want to get a video with start time and end time from RTSP server. From Maxim-zapryanov's answer at the discussion, I know to do it, I need set time range at PLAY packet's "range" header field
To play from a specific time you need to use the RTSP header field "range" for the PLAY command. See section 6.5.1 of https://www.onvif.org/specs/stream/ONVIF-Streaming-Spec.pdf and https://datatracker.ietf.org/doc/html/rfc2326#section-12.29
I used Live555 lib to set the field to PLAY packet success. But I want set the field by FFmpeg lib instead of Live555 lib.
So, does FFmpeg lib support set time range at PLAY packet's "range" header field when get video from RTSP? If yes, how to do it?
If anyone know the answer, please tell me know. Thank you so much!
...ANSWER
Answered 2022-Mar-10 at 03:59You can read answer at ffmpeg libav-user mail list
QUESTION
Options:
- always by timeout
- when there is no messaging
- when there is an error from the diameter-server
It is not clear from the RFC.
...ANSWER
Answered 2022-Mar-08 at 12:20Each peer should send WDR (both client and server) every X seconds. Usually there is a configuration to adjust how much time to wait between WDRs. (It does not have to be the same time in the client and in the server)
QUESTION
In MDN HTTP Strict Transport Security (HSTS), it has an example of HSTS settings as below
...ANSWER
Answered 2022-Mar-07 at 18:45Preload is a big commitment. It will effectively be hardcoded into a browser’s code. Given it takes several months at a minimum to roll out new version, it’s basically irreversible.
Also as it’s down at the domain level, mistakes have been made. For example preloading domain.com but covering that blog.domain.com, or intranet.domain.com have not been upgraded to HTTPS. At this point your options are 1) upgrade side to HTTPS and live with zero users to the site until the or 2) reverse the preload and wait the months for that to roll out to all browsers and deal with zero users until then.
HTTPS is much more common now, so the risks are reduced but when HSTS preload first came out, these were real risks.
Therefore the preload
attribute was a signal that the site owner was ready for that commitment. It also prevent someone else submitting a site that wasn’t using this header (whether maliciously or with good, but misguided, intentions).
You are correct in that it doesn’t “do” anything in the browser.
There was also talk of checking if the preload header was still being sent, and if not removing the preload but not sure if that’s done.
QUESTION
I want to write a Sieve filter like this:
...ANSWER
Answered 2022-Mar-03 at 06:23You can use a string as pseudo-array with separator like :
, that obviously isn't a part of domain name.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install datatracker
You can use datatracker like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page