performance-comparison | test goal is to show the difference | Unit Testing library

I have a Spark DataFrame where all fields are integer type. I need to count how many individual cells are greater than 0.

I am running locally and have a DataFrame with 17,000 rows and 450 columns.

I have tried two methods, both yielding slow results:

Version 1:

Usage: In our production we have around 100 thread which can access the cache we are trying to implement. If cache is missed then information will be fetched from the database and cache will be updated via writer thread.

To achieve this we are planning to implement multiple read and single writer We cannot update the g++ version since we are using g++-4.4

Update: Each worker thread can work for both read and write. If cache is missed then information is cached from the DB.

Problem Statement: We need to implement the cache to enhance the performance. For this, cache read are more frequent and write operations to the cache is very much less.

I think we can use boost::shared_mutex boost::shared_lock, boost::upgrade_lock, boost::upgrade_to_unique_lock implementation

But we learnt that boost::shared_mutex has performance issues:

• Performance comparison on reader writer locks
• Lib boost devel

Questions

1. Does boost::shared_mutex impact the performance in case read are much frequent?
2. What are other constructs and design approaches we can take while considering compiler version g++4.4?
3. Is there a way-around on how to design it, such that reads are lock free?

Also, we are intended to use Map to keep the information for cache.

I've always used C# interfaces to achieve inversion of control. But then I found this article which compares a bunch of IoC containers. What is the difference between these containers and a C# interface?

http://www.palmmedia.de/blog/2011/8/30/ioc-container-benchmark-performance-comparison

I have a server talking to a mobile app, and this app potentially does thousands of requests per day. I don't care that much about performance in this particular case, so saving some miliseconds isn't as big as a concern as saving bandwidth - especially since I'm paying for it.

(1) What is the advantage of using JSON over binary here, when bandwidth is a much bigger deal than performance? I mean, I have read some people saying that the size difference between raw data and JSON isn't really that much - and that might as well be partially true, but when you have thousands of daily requests being made by hundreds of thousands of users, merely doubling the amount of bytes will have a huge impact on bandwidth usage - and in the end, on the server bill.

Also, some people said that you can easily alter the JSON output format, while changing the binary serialization might be a little more complicated. Again, I agree, but shouldn't it be a little more complicated than that? Like, what are the odds that we're gonna change our format? Will the ease of change make up for JSON's bandwidth excess?

(2) And finally, I stumbled upon this link while doing some research on this topic, and in the summary table (Ctrl + F, 'summary') it says that the JSON data size is smaller than the actual binary data? How is that even possible?

I would very much appreciate some answers to these questions.

Thank you in advance.

I was using Symfony3 with PHP7 and want to explore Ruby and Ruby on Rails. My first impression was "wow".

I won't asking PHP7 vs Ruby. This is not related to that. What I am asking was stated here:

https://www.linkedin.com/pulse/ruby-vs-php-python-simple-microservice-performance-comparison-per

Not only it's outperforming both Ruby and Python based solution it was only engine that passed 500 concurrent connections test without failing any request.

Is that true? Is Ruby or Python (but in my case only Ruby on Rails) really fails on such amount of requests?

How can I do to prevent? How does GitHub doesn't fail while they are using Rails or GitLab for instance?

Are there any other way to start RoR server for better performance?

I am building an app that will be higher usages and thinking to move from symfony3/PHP to Ruby. Am I doing wrong?

I am stuck at this point. Can't go further without knowing or understanding this subject better.