stress-test | server | TCP library

I have a large application, with many dynamic parts/panels/widgets. Upon stress-testing, The GUI goes blank. I doubt that the GUI thread is bombarded with events from other threads.

I have disabled all the events that I doubted, but it still goes blank. So is there like a global handler or logger to log all the events that occur in the wxWidgets main loop ?

N.B: I have around 1000 threads.

In my application, some value can change at any time. A lot of components need to do something when this value changes. The number of components changes during the usage of the application (depending on what the user opens). Currently this is implemented with events: when value changes, an event is raised, on which all interested components are subscribed.

Of course, to avoid memory leaks, these components unsubscribe on dispose.

When we stress-test our application with lots of components (a few million), one of the biggest bottle necks (>90% cpu time during high loads) are these subscriptions: MyApp.ValueChanged -= TheValueChanged; takes very long (multiple seconds).

I assume this is, because there are a lot of subscribers, and this unsubscribe needs to find the correct subscriber in the list of subscribers (worst case: a million time searching in a list of a million items).

Now, my question: What would be a good way to improve this?

I was thinking about weak event handlers, so unsubscribing isn't necessary, but there may be better solutions.

The Contiki OS supports a series of so-called Rime protocols [ http://contiki.sourceforge.net/docs/2.6/a01798.html ] and one of these, "abc", would have been ideal for a lowish-level radio test I have been tasked with writing.

However, I am required to use contiki-ng, and on studying that I can find, to my amazement, no reference to Rime or the "abc" protocol!

Has the Rime protocol been removed from contiki-ng, and if so is there an equivalent low-level protocol for simply transmitting and receiving radio packets over a specified channel without all the higher networky layers?

If the worst comes to the worst, I suppose I can use UDP. But to stress-test the IoT device I am using I would have preferred a lower-level protocol.

consider a cpp function solve(). A string a of numbers (sample test string) is passed to the function solve() as arguments. I wanted stdin to read the numbers in the string a.

It is actually a stress-test. So in this stress test, this function solve() is provided with a string to return a result which will be tested against another result which is obtained from another function solveFast().

NOTE:- Algorithm in the function solve() is already given to us. I wish to stress-test this algorithm against my own algorithm (in solveFast()). It is guaranteed that Algorithm in the function solve() provides the correct output against its test inputs

I recently stress-tested my express server with the following two queries:

My goal is to write a client-server architecture using TCP sockets in C. I've successfully managed to establish a connection between client and server and send and receive messages in both ways. After receiving a message, the message is being processed by other threads. (It will get enqueued in a queue and another thread works on the queue).

I'm now stress-testing my implementation, by send()ing a lot of messages from the client to the server.

So, what it essentially does is:

My client goes in a for loop and sends X messages using send() to the server. After exiting the for loop, the connection to the server is close()d. This happens all immediately with no delay in between.

The server program works this way: it has a thread where it receives messages and processes the messages by enqueuing them into a queue. After enqueuing it will keep waiting for new messages to arrive. What happens though: The server program will just quit, without any error message, besides that the recv() failed (i.e. returning a -1).

I've noticed the following: the server receives maybe 4000 of 5000 messages before the client calls close(). If I wait in the client until the server has received all messages before the client calling close(), the server does not crash and just continues working. If I don't wait before calling close() the server just stops running. No error message and not even the main function of the server will finish (the main function is actually blocked by a sem_wait(), until some thread sem_post()s the semaphore).

So, what am I doing wrong? How can I make the server to be more robust, even if a client for example crashes during its execution? I don't want the server to be dependent on the client.

Or did I maybe overload the server with sending too many messages with no delay in-between and my scenario is unrealistic?

Here are some parts of my code which are crucial for the send/recv mechanism:

The server:

I want to perform some stress-test for the applications that are run inside docker containers. Simple scenarios like turn on/off dropping all the packets for a specified IP address.

I've tried some tools, but none of them seem to satisfy me:

1. Disconnecting containers from the bridge network - sadly it drops all connections, not only to the specified ips.

2. Setting up iptables rules from inside a container - it requires a container to be launched in privileged mode, which is not a great practice (I'd like to have an approach that is suitable even for production environments).

3. Modifying the host's iptables also doesn't look like an option - I'd like to create network issues for a specific container, not for all the containers on the host.

What are some possible ways I should look into?
Are there any chaos engineering tools that are to-go options for such needs?

I am processing a batch of sentences with different lengths, so I am planning to take advantage of the padding + attention_mask functionality in gpt2 for that.

At the same time, for each sentence I need to add a suffix phrase and run N different inferences. For instance, given the sentence "I like to drink coke", I may need to run two different inferences: "I like to drink coke. Coke is good" and "I like to drink coke. Drink is good". Thus, I am trying to improve the inference time for this by using the "past" functionality: https://huggingface.co/transformers/quickstart.html#using-the-past so I just process the original sentence (e.g. "I like to drink coke") once, and then I somehow expand the result to be able to be used with two other sentences: "Coke is good" and "Drink is good".

Below you will find a simple code that is trying to represent how I was trying to do this. For simplicity I'm just adding a single suffix phrase per sentence (...but I still hope my original idea is possible though):

MacOS with mongodb-community@4.2 (installed using brew)

TLDR: MongoDB is only running as one process, seemingly not taking advantage of the 7 other available CPU cores.

I'm running a simple NodeJS application with PM2, making use of all 8 of my CPU cores.

Using Apache Benchmark, I try to stress-test the application for retrieving data. The endpoint I am hitting retrieves data from my MongoDB database. (Only reading, no write operations are performed).

During the stress-test I get these results:

1. There are 8 active NodeJS processes
2. There is only 1 active MongoDB process
3. CPU usage indicates that MongoDB is the bottleneck. How can I ensure that MongoDB takes advantage of more cores?

Screenshot from TOP:

Why is MongoDB only making use of 1 process/core?

Can I increase performance by configuring it to use more than one process/core?

Some additional information, serverStatus() run during the stress-test: