predicated | simple predicate model for Ruby | Plugin library

Apologies . . . put in the answer section instead of comment section. But after trying to fix, it didn't matter . . . not enough posts to add a comment.

This is for tables on a Microsoft ERP system. Microsoft has their default indexes on tables that shouldn't be changed or deleted. On any ERP upgrades, the indexes get recreated by Microsoft anyways.

The tables for most of the reporting are order history headers (8 million records) and lines (57 million records). These tables get populated when an order is transferred to invoice or an invoice is posted. For first situation, order goes to history table and an invoice is created in the open table. The 2nd situation, an anvoice is moved to history table when the invoice is posted. For these processes, the ERP system has a thick client (that hasn't changed much since 2010 or earlier). The process is a rather long process with many tables that does not use an explicit SQL transaction. If this process is interrupted, then a manual fix up is required for any tables that were not updated.

The READONLY/READUNCOMMITTED was initially used for large reporting against the live the database. The Views that Vinh is using are used against a replication server that is now in place. The READONLY is normally used against information that is in previous months/days so the current day changes are not a problem. The large reports were slowing down the transfer and posting processes discussed in the previous paragraph. The posting process above currently takes about 1 hr to post 500 transactions, so it is good any time we can keep the process from slowing down.

Why a specific index is specified: The 57 million rows are divided into order types (SOPTYPE 2 (order), 3 (invoice), and 5 (backorder)). Most of the Microsoft indexes use the SOPTYPE as the 1st field in the index. So most of the queries end up using a index scan rather than an index seek. In some cases, just specifying the index reduces the query time from 2 min to 5 sec. When comparing the index scores, both indexes may be at 80% but SQL tends to choose the index with the SOPTYPE as the first index field.

We are probably one of the larger data users of the particular ERP system. I don't believe Microsoft has optimized for this data size.

I hope this information helps.

I think this is due to using a generator expression as was answered in this question Python: any() unexpected performance

This is not an answer to the question you asked but it requires a bit of code so I didn't want to post it as a comment. It is basically a different way of doing things such that your current question should become moot.

In a new WinForms project, I added a new form named MessageForm and I added a Label to it. I did not change any properties of the Label, which means that AutoSize was True and Location was {0,0}. I then added the following code:

You are on the right track with Or predicate, but instead of multiple Or predicates on user.UserDepatments.Any(single_match) you should create single Or based predicate to be used inside the single user.UserDepatments.Any(multi_or_match).

Something like this:

Probably spring-data ignores the default QueryInits declared in the QueryDSL static metamodel. Nothing prevents you from wrapping an EntityPath expression in another EntityPath expression with more initialized paths:

why can't filter just pass the value to whoever is after it in the pipeline as soon as it finds a match?

Because in the iterator model, positioning and accessing are distinct operations. You position an iterator with ++; you access an iterator with *. These are two distinct expressions, which are evaluated at two distinct times, resulting in two distinct function calls that yield two distinct values (++ yields an iterator, * yields a reference).

A filtering iterator, in order to perform its iteration operation, must access the values of its underlying iterator. But that access cannot be communicated to the caller of ++ because that caller only asked to position the iterator, not to get its value. The result of positioning an iterator is a new iterator value, not the value stored in that iterated position.

So there's nobody to return it to.

You can't really delay positioning until after accessing because a user might reposition the iterator multiple times. I mean, you could implement it that way in theory by storing the number of such increments/decrements. But this increases the complexity of the iterator's implementation. Especially since resolving such delayed positioning can happen through something as simple as testing against another iterator or sentinel, which is supposed to be an O(1) operation.

This is simply a limitation of the model of iterators as having both position and value. Alternative models exist where iteration and access are bundled together exist, but they're not how standard library iteration works.

I am wondering what is an operation and what is an instruction here? It seems like a fma is one instruction, but two operations. Whereas add and mul is one instruction and one operation.

Yes, correct. Fused-Multiply-Add instructions count as 2 operations (a multiply, plus an add). A multiply or add instruction counts as one operation.

Are there any instructions that are not counted as operations?

Yes, any instruction that does not use the single-precision (or double-precision for e.g. flop_count_dp) functional units inside the SM will not contribute any operations to these metrics (either inst or op). For example, integer instructions, or load or store instructions, will not affect these metrics. Any instruction that might have some floating point nature (e.g. conversion to/from floating point) to it but does not consist of add or multiply operations would not contribute to the op metric, I don't believe.

Also, when we talk about peak performance in TFLOP/s, the OP here corresponds to Operations i guess?

Yes

If I want to estimate something like compute to global memory access (CGMA), should I use flop_count_sp instead of the inst_fp_32 for the compute part?

I think this might be a matter of opinion. I would use instructions. A fused-multiply-add instruction, as already mentioned, counts as 2 operations, but it does not "double" the pressure on the floating point units. Therefore, when comparing a code to look at the balance between the global memory load/store activity vs. compute "pressure", I would use instructions. Again, possibly a matter of opinion.

Short anser: no, not possible.

Long(er) answer: ANTLR4 can potentially use unlimited lookahead, so it relies on the stream to seek to any position with no delay or parsing speed will drop to nearly a hold. For that reason all runtimes use a normal file stream that reads in the entire file at once.

There were discussions/attempts in the past to create a stream that buffers only part of the input, but I haven't heard of anything that actually works.

When I list nvprof's metrics with

nvprof --query-events

That isn't consistent usage (emphasis added).

Using nvprof (or nvvp), events and metrics are not the same thing.

To query events, you would use: