sort | simple online and realtime tracking algorithm

This function were removed in TensorFlow version 2.6. According to the keras in rstudio reference

update to

I had the same problem and was able to fix it by adding populate=* to the end of the API endpoint.

For example:

I know this does not solve your problem directly, but consider moving to springdoc which most recent release supports Spring Boot 2.6.0. Springfox is so buggy at this point that is a pain to use. I've moved to springdoc 2 years ago because of its Spring WebFlux support and I am very happy about it. Additionally, it also supports Kotlin Coroutines, which I am not sure Springfox does.

If you decide to migrate, springdoc even has a migration guide.

Since you're already using Docker, I'd suggest using a multi-stage build. Using a standard docker image like golang one can build an executable asset which is guaranteed to work with other docker linux images:

Your split splits the list in two ordered halves, so merge consumes its first argument first and then just produces the second half in full. In other words it is equivalent to ++, doing redundant comparisons on the first half which always turn out to be True.

In the true mergesort the merge actually does twice the work on random data because the two parts are not ordered.

The split though spends some work on the partitioning whereas an online bottom-up mergesort would spend no work there at all. But the built-in sort tries to detect ordered runs in the input, and apparently that extra work is not negligible.

It looks like GCC's naïveté about store-forwarding stalls is hurting its auto-vectorization strategy here. See also Store forwarding by example for some practical benchmarks on Intel with hardware performance counters, and What are the costs of failed store-to-load forwarding on x86? Also Agner Fog's x86 optimization guides.

(gcc -O3 enables -ftree-vectorize and a few other options not included by -O2, e.g. if-conversion to branchless cmov, which is another way -O3 can hurt with data patterns GCC didn't expect. By comparison, Clang enables auto-vectorization even at -O2, although some of its optimizations are still only on at -O3.)

It's doing 64-bit loads (and branching to store or not) on pairs of ints. This means, if we swapped the last iteration, this load comes half from that store, half from fresh memory, so we get a store-forwarding stall after every swap. But bubble sort often has long chains of swapping every iteration as an element bubbles far, so this is really bad.

(Bubble sort is bad in general, especially if implemented naively without keeping the previous iteration's second element around in a register. It can be interesting to analyze the asm details of exactly why it sucks, so it is fair enough for wanting to try.)

Anyway, this is pretty clearly an anti-optimization you should report on GCC Bugzilla with the "missed-optimization" keyword. Scalar loads are cheap, and store-forwarding stalls are costly. (Can modern x86 implementations store-forward from more than one prior store? no, nor can microarchitectures other than in-order Atom efficiently load when it partially overlaps with one previous store, and partially from data that has to come from the L1d cache.)

Even better would be to keep buf[x+1] in a register and use it as buf[x] in the next iteration, avoiding a store and load. (Like good hand-written asm bubble sort examples, a few of which exist on Stack Overflow.)

If it wasn't for the store-forwarding stalls (which AFAIK GCC doesn't know about in its cost model), this strategy might be about break-even. SSE 4.1 for a branchless pmind / pmaxd comparator might be interesting, but that would mean always storing and the C source doesn't do that.

If this strategy of double-width load had any merit, it would be better implemented with pure integer on a 64-bit machine like x86-64, where you can operate on just the low 32 bits with garbage (or valuable data) in the upper half. E.g.,

Ok, so to answer my own question, here's how to get the details on a Veracode vulnerability report in less than four weeks and in only fifty-five easy steps.

• Have someone post an issue against your library suggesting that its mapObjIndexed function is subject to the prototype pollution vulnerability.

• Respond to say that you don't think the user has demonstrated that well-known vulnerability, but that you will dig deeper.

• Write a detailed post described what that vulnerability means and demonstrate that the library is not in fact subject to it, or or at least that the example supplied does not demonstrate it.

• Carry on a short conversation with interested parties explaining the point more thoroughly and responding to objections.

• Leave the issue open for a while so the original reporter can argue the point and respond. ¹

• Receive a comment on the issue that says that the user has received

  a VULN ticket to fix this Prototype Pollution vulnerability found in ramda.

• Carry on a discussion regarding this comment to learn that there is a report that claims that

  ramda is vulnerable to prototype pollution. An attacker can inject properties into existing construct prototypes via the _curry2 function and modify attributes such as __proto__, constructor, and prototype.

  and eventually learn that this is due to a report from the software security company Veracode.

• Examine that report to find that it has no details, no explanation of how to trigger the vulnerability, and no suggested fix.

• Examine the report and other parts of the Veracode site to find there is no public mechanism to challenge such a report.

• Report back to the library's issue that the report must be wrong, as the function mentioned could not possibly generate the behavior described.

• Post an actual example of the vulnerability under discussion and a parallel snippet from the library to demonstrate that it doesn't share the problem.

• Find Veracode's online support form, and submit a request for help. Keep your expectations low, as this is probably for the sales department.

• Post a StackOverflow Question² asking how to find details of a Veracode vulnerability report, using enough details that if the community has the knowledge, it should be easy to answer.

• Try to enjoy your Friday and Saturday. Don't obsessively check your email to see if Veracode has responded. Don't visit the StackOverflow question every hour to see if anyone has posted a solution. Really, don't do these things; they don't help.

• Add a 250-reputation point bounty to the StackOverflow question, trying to get additional attention from the smart people who must have dealt with this before.

• Find direct email support addresses on the Veracode site, and send an email asking for details of the supposed vulnerability, a snippet that demonstrates the issue, and procedures to challenge their findings.

• Receive a response from a Veracode Support email addressthat says, in part,

  Are you saying our vuln db is not correct per your github source? If so, I can send it to our research team to ensure it looks good and if not, to update it.

  As for snips of code, we do not provide that.

• Reply, explaining that you find the report missing the details necessary to challenge it, but that yes, you expect it is incorrect.

• Receive a response that this has been "shot up the chain" and that you will be hearing from them soon.

• Again, don't obsessively check your email or the StackOverflow question. But if you do happen to glance at StackOverflow, notice that while there are still no answers to it, there are enough upvotes to cover over half the cost of the bounty. Clearly you're not alone in wanting to know how to do this.

• Receive an email from Veracode:

  Thank you for your interest in Application Security and Veracode.

  Do you have time next week to connect?

  Also, to make sure you are aligned with the right rep, where is your company headquartered?

• Respond that you're not a potential customer and explain again what you're looking for.

• Add a comment to the StackOverflow to explain where the process has gotten to and expressing your frustration.

• Watch another weekend go by without any way to address this concern.

• Get involved in a somewhat interesting discussion about prototype pollution in the comments to the StackOverflow post.

• Receive an actually helpful email from Veracode, sent by someone new, whose signature says he's a sales manager. The email will look like this:

  Hi Scott, I asked my team to help out with your question, here was their response:

  We have based this artifact from the information available in https://github.com/ramda/ramda/pull/3192. In the Pull Request, there is a POC (https://jsfiddle.net/3pomzw5g/2/) clearly demonstrating the prototype pollution vulnerability in the mapObjIndexed function. In the demo, the user object is modified via the __proto__​ property and is
  considered a violation to the Integrity of the CIA triad. This has been reflected in our CVSS scoring for this vulnerability in our vuln db.

  There is also an unmerged fix for the vulnerability which has also been
  included in our artifact (https://github.com/ramda/ramda/pull/3192/commits/774f767a10f37d1f844168cb7e6412ea6660112d )

  Please let me know if there is a dispute against the POC, and we can look further into this.

• Try to avoid banging your head against the wall for too long when you realize that the issue you thought might have been raised by someone who'd seen the Veracode report was instead the source of that report.

• Respond to this helpful person that yes you will have a dispute for this, and ask if you can be put directly in touch with the relevant Veracode people so there doesn't have to be a middleman.

• Receive an email from this helpful person -- who needs a name, let's call him "Kevin" -- receive an email from Kevin adding to the email chain the research team. (I told you he was helpful!)

• Respond to Kevin and the team with a brief note that you will spend some time to write up a response and get back to them soon.

• Look again at the Veracode Report and note that the description has been changed to

  ramda is vulnerable to prototype pollution. An attacker is able to inject and modify attributes of an object through the mapObjIndexed function via the proto property.

  but note also that it still contains no details, no snippets, no dispute process.

• Receive a bounced-email notification because that research team's email is for internal Veracode use only.

• Laugh because the only other option is to cry.

• Tell Kevin what happened and make sure he's willing to remain as an intermediary. Again he's helpful and will agree right away.

• Spend several hours writing up a detailed response, explaining what prototype pollution is and how the examples do not display this behavior. Post it ahead of time on the issue. (Remember the issue? This is a story about the issue.³) Ask those reading for suggestions before you send the email... mostly as a way to ensure you're not sending this in anger.

• Go ahead and email it right away anyway; if you said something too angry you probably don't want to be talked out of it now, anyhow.

• Note that the nonrefundable StackOverflow bounty has expired without a single answer being offered.

• Twiddle your thumbs for a week, but meanwhile...

• Receive a marketing email from Veracode, who has never sent you one before.

• Note that Veracode has again updated the description to say

  ramda allows object prototype manipulation. An attacker is able to inject and modify attributes of an object through the mapObjIndexed function via the proto property. However, due to ramda's design where object immutability is the default, the impact of this vulnerability is limited to the scope of the object instead of the underlying object prototype. Nonetheless, the possibility of object prototype manipulation as demonstrated in the proof-of-concept under References can potentially cause unexpected behaviors in the application. There are currently no known exploits.

  If that's not clear, a translation would be, "Hey, we reported this, and we don't want to back down, so we're going to say that even though the behavior we noted didn't actually happen, the behavior that's there is still, umm, err, somehow wrong."

• Note that a fan of the library whose employer has a Veracode account has been able to glean more information from their reports. It turns out that their details are restricted to logged-in users, leaving it entirely unclear how they thing such vulnerabilities should be fixed.

• Send a follow-up email to Kevin⁴ saying

  I'm wondering if there is any response to this.

  I see that the vulnerability report has been updated but not removed.
  I still dispute the altered version of it. If this behavior is a true vulnerability, could you point me to the equivalent report on JavaScript's Object.assign, which, as demonstrated earlier, has the exact same issue as the function in question.

  My immediate goal is to see this report retracted. But I also want to point out the pain involved in this process, pain that I think Veracode could fix:

  I am not a customer, but your customers are coming to me as Ramda's maintainer to fix a problem you've reported. That report really should have enough information in it to allow me to confirm the vulnerability reported. I've learned that such information is available to a logged- in customer. That doesn't help me or others in my position to find the information. Resorting to email and filtering it through your sales department, is a pretty horrible process. Could you alter your public reports to contain or point to a proof of concept of the vulnerability?
  And could you further offer in the report some hint at a dispute process?

• Receive an email from the still-helpful Kevin, which says

  Thanks for the follow up [ ... ], I will continue to manage the communication with my team, at this time they are looking into the matter and it has been raised up to the highest levels.

  Please reach back out to me if you don’t have a response within 72 hrs.

  Thank you for your patience as we investigate the issue, this is a new process for me as well.

• Laugh out loud at the notion that he thinks you're being patient.

• Respond, apologizing to Kevin that he's caught in the middle, and read his good-natured reply.

• Hear back from Kevin that your main objective has been met:

  Hi Scott, I wanted to provide an update, my engineering team got back
  to me with the following:

  “updating our DB to remove the report is the final outcome”

  I have also asked for them to let me know about your question regarding the ability to contend findings and will relay that back once feedback is received.

  Otherwise, I hope this satisfies your request and please let me know if any further action is needed from us at this time.

• Respond gratefully to Kevin and note that you would still like to hear about how they're changing their processes.

• Reply to your own email to apologize to Kevin for all the misspelling that happened when you try to type anything more than a short text on your mobile device.

• Check with that helpful Ramda user with Veracode log-in abilities whether the site seems to be updated properly.

• Reach out to that same user on Twitter when he hasn't responded in five minutes. It's not that you're anxious and want to put this behind you. Really it's not. You're not that kind of person.

• Read that user's detailed response explaining that all is well.

• Receive a follow-up from the Veracode Support email address telling you that

  After much consideration we have decided to update our db to remove this report.

  and that they're closing the issue.

• Laugh about the fact that they are sending this after what seem likely the close of business for the week (7:00 PM your time on a Friday.)

• Respond politely to say that you're grateful for the result, but that you would still like to see their dispute process modernized.

• Write a 2257-word answer⁵ to your own Stack Overflow question explaining in great detail the process you went through to resolve this issue.

And that's all it takes. So the next time you run into this, you can solve it too!

(because you knew it couldn't be that easy!)

• Receive an email from a new Veracode account executive which says

  Thanks for your interest! Introducing myself as your point of contact at Veracode.

  I'd welcome the chance to answer any questions you may have around Veracode's services and approach to the space.

  Do you have a few minutes free to touch base? Please let me know a convenient time for you and I'll follow up accordingly.

• Politely respond to that email suggesting a talk with Kevin and including a link to this list of steps.

^{1 This is standard behavior with Ramda issues, but it might be the main reason Veracode chose to report this.}

^{2 Be careful not to get into an infinite loop. This recursion does not have a base case.}

^{3 Hey, this was taking place around Thanksgiving. There had to be an Alice's Restaurant reference!}

^{4 If you haven't yet found a Kevin, now would be a good time to insist that Veracode supply you with one.}

^{5 Including footnotes.}

One work around would be to use your map2_dfr code, but then bind the columns that are needed to the map2_dfr output. Though you still have to drop the names column.

Here is a sample code to output what you need. I use the formula below to calculate pct change. percentage_change = 100*(new-old)/old

If old is 0 it is changed to 1 to avoid division by zero error.