hashset.c | hash set C implementation | Hashing library

The reason is that your doing it inefficently. If you use indices to iterate over a linked list the list has to count each node reference to find the correct one. And then it acts on that one.

Try it like this and see the difference. Counting even values of a linked list of N items. The second iteration will be much faster.

First, a indexed for loop.

this creates a hashtable

Actually, it creates an array of hashtables, because ConvertFrom-StringData turns each pipeline input object into a separate hashtable.

To create a single hashtable, join the input lines to form a single string first:

The main reason for this big difference in performance is because you are using a culture-sensitive comparison for the case-insensitive hash set, whereas the case-sensitive hash set uses an ordinal one.

Without passing the hash set any equality comparers, the hash set uses the default equality comparer of that type, which calls the Equals method. Notice that String.Equals

performs an ordinal (case-sensitive and culture-insensitive) comparison.

Whereas StringComparer.InvariantCultureIgnoreCase:

performs a case-insensitive string comparison using the word comparison rules of the invariant culture.

It is important to note that "invariant culture" is not the same as "culture-insensitive", or "ordinal":

Console.WriteLine(StringComparer.InvariantCultureIgnoreCase.Equals( "ss", "ß")); // true Console.WriteLine(StringComparer.OrdinalIgnoreCase.Equals( "ss", "ß")); // false

The invariant culture still has collation rules, and looking up and applying those rules takes time.

Changing InvariantCultureIgnoreCase to OrdinalIgnoreCase should make the execution time almost the same. That doesn't mean there aren't other problems with your benchmark, as the comments have pointed out.

The problem is that headersHashset.clone() doesn't clone HashSets in the values.

From the docs:

Returns a shallow copy of this HashMap instance: the keys and values themselves are not cloned.

It means that localHeadersHashset in your tasks, and your headersHashsetConcurrent, and threadSafeItems returned by futures — all of these use the same HashSet objects for the same keys.

Since tasks are executed in parallel threads, it is entirely possible that some task executes HashSet.add() at the same time when the main thread iterates over the elements of the same HashSet inside:

I think this line: Call Function("A", "(A OR TEST) AND B NOT C)") would return false because it does not satisfy "AND B". "A B" or "TEST B" would return true. Perhaps I have it wrong?

A first step I'd consider would be a simple good/bad filter. Parse the expression and determine the good/bad inputs as two hashsets. E.g. (A OR TEST) AND B NOT C could parse out as bad=("C"), good=("A","B","TEST). Easy to check input against the two hashsets [assuming any input not in either hashset is implicitly disallowed].

If you're using a rule string more than once, I'd add a hash-to-parse lookup and re-use the parse rather than re-calculate each time.

It gets harder when you have to deal with required pairs. E.g. (A OR TEST) AND B NOT C means to me that A B and TEST B are valid, not "A", "TEST" or "C" or "D". If you expand your parse process to actually calculate the logic, the "good/bad" sets become valid input strings, e.g. bad=("C"), good=("A B","TEST B").

For "(A AND B) OR (C AND D)": "A B","C D" are valid but nothing else.

Nothing in your code looks like a hash set. As I understand the problem, you receive a collection of numbers in the range 1..9 coming from the database and you want to map that to the range 1..7 for the UI by collapsing the first 3 database values into 1 and shifting the rest down. You can do that e.g. with a new line at the start of your function:

I tried your sample at dotnet fiddle and it seems like the compiler (version) matters. I got the same exception when running your code on the .NET 4.7.2 compiler, but when you change the compiler to Roslyn 3.8 or .NET 5 the sample works.

It is very likely that the data about the frequency of the characters needs to be collected -- so set should be replaced with a map. Then the resulting string may be built with regard to common characters and minimal frequency of the character in both words: