YCSB | Yahoo! Cloud Serving Benchmark

by brianfrankcooper Java Version: 0.17.0 License: Apache-2.0

Download this library from

kandi X-RAY | YCSB Summary

YCSB is a Java library. YCSB has build file available, it has a Permissive License and it has medium support. However YCSB has 123 bugs and it has 1 vulnerabilities. You can download it from GitHub, Maven.

<!-- Copyright (c) 2010 Yahoo! Inc., 2012 - 2016 YCSB contributors. All rights reserved. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. -→.

Support

Quality

Security

License

Reuse

kandi-support Support

YCSB has a medium active ecosystem.
It has 3792 star(s) with 1882 fork(s). There are 217 watchers for this library.
It had no major release in the last 12 months.
There are 196 open issues and 657 have been closed. On average issues are closed in 53 days. There are 73 open pull requests and 0 closed requests.
It has a neutral sentiment in the developer community.
The latest version of YCSB is 0.17.0

YCSB Support

Best in #Java

Average in #Java

YCSB Support

Best in #Java

Average in #Java

quality kandi Quality

YCSB has 123 bugs (18 blocker, 4 critical, 42 major, 59 minor) and 1499 code smells.

YCSB Quality

Best in #Java

Average in #Java

YCSB Quality

Best in #Java

Average in #Java

security Security

YCSB has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
YCSB code analysis shows 1 unresolved vulnerabilities (1 blocker, 0 critical, 0 major, 0 minor).
There are 117 security hotspots that need review.

YCSB Security

Best in #Java

Average in #Java

YCSB Security

Best in #Java

Average in #Java

license License

YCSB is licensed under the Apache-2.0 License. This license is Permissive.
Permissive licenses have the least restrictions, and you can use them in most projects.

YCSB License

Best in #Java

Average in #Java

YCSB License

Best in #Java

Average in #Java

build Reuse

YCSB releases are available to install and integrate.
Deployable package is available in Maven.
Build file is available. You can build the component from source.
Installation instructions are not available. Examples and code snippets are available.
YCSB saves you 15213 person hours of effort in developing the same functionality from scratch.
It has 30366 lines of code, 1447 functions and 335 files.
It has high code complexity. Code complexity directly impacts maintainability of the code.

YCSB Reuse

Best in #Java

Average in #Java

YCSB Reuse

Best in #Java

Average in #Java

Top functions reviewed by kandi - BETA

kandi has reviewed YCSB and discovered the below as its top functions. This is intended to give you an instant insight into YCSB implemented functionality, and help decide if they suit your requirements.

Initialize AzureCosmos client .
Initialize keys and tags .
Entry point to YCSB .
Sets up table .
This method scans a table .
Update url options .
Write the contents of the object to the S3 object .
Load classes and tables .
Export measurement measurements .
Single item mutation .

YCSB Key Features

Yahoo! Cloud Serving Benchmark

YCSB Examples and Code Snippets

See all related Code Snippets

default

Copy

Download

* To get here, use https://ycsb.site
* [Our project docs](https://github.com/brianfrankcooper/YCSB/wiki)
* [The original announcement from Yahoo!](https://labs.yahoo.com/news/yahoo-cloud-serving-benchmark/)

Getting Started
---------------

1. Download the [latest release of YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest):

    ```sh
    curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz
    tar xfvz ycsb-0.17.0.tar.gz
    cd ycsb-0.17.0
    ```

2. Set up a database to benchmark. There is a README file under each binding
   directory.

3. Run YCSB command.

    On Linux:
    ```sh
    bin/ycsb.sh load basic -P workloads/workloada
    bin/ycsb.sh run basic -P workloads/workloada
    ```

    On Windows:
    ```bat
    bin/ycsb.bat load basic -P workloads\workloada
    bin/ycsb.bat run basic -P workloads\workloada
    ```

  Running the `ycsb` command without any argument will print the usage.

  See https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload
  for a detailed documentation on how to run a workload.

  See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties for
  the list of available workload properties.


Building from source
--------------------

YCSB requires the use of Maven 3; if you use Maven 2, you may see [errors
such as these](https://github.com/brianfrankcooper/YCSB/issues/406).

To build the full distribution, with all database bindings:

    mvn clean package

To build a single database binding:

    mvn -pl site.ycsb:mongodb-binding -am clean package

ArangoDB Request Load balancing

Copy

Download

upstream arangodb {
  server coord-1.my.domain:8529;
  server coord-2.my.domain:8529;
  server coord-3.my.domain:8529;
}

server {
  listen                *:80 default_server;
  server_name           _; # Listens for ALL hostnames
  proxy_next_upstream   error timeout invalid_header;
  
  location / {
    proxy_pass          http://arangodb;
  }
}

Bash Script - Python can't find a path

Copy

Download

> "'logs/tp${throughput}/load-test${test}-workloada.txt'"

> "logs/tp${throughput}/load-test${test}-workloada.txt"

> ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"

> "logs/tp${throughput}/run-test${test}-workload${workload}.txt"

-----------------------

> "'logs/tp${throughput}/load-test${test}-workloada.txt'"

> "logs/tp${throughput}/load-test${test}-workloada.txt"

> ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"

> "logs/tp${throughput}/run-test${test}-workload${workload}.txt"

-----------------------

> "'logs/tp${throughput}/load-test${test}-workloada.txt'"

> "logs/tp${throughput}/load-test${test}-workloada.txt"

> ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"

> "logs/tp${throughput}/run-test${test}-workload${workload}.txt"

-----------------------

> "'logs/tp${throughput}/load-test${test}-workloada.txt'"

> "logs/tp${throughput}/load-test${test}-workloada.txt"

> ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"

> "logs/tp${throughput}/run-test${test}-workload${workload}.txt"

Init static variable in synchronized block for all the threads, read without synchronized

Copy

Download

public class DB {

  private static final Client client = new Client();

  public insert(Object data) {
    client.insert(data);  // Guaranteed to be initialized once class loading is complete.
  }
}

public class DB {
  private static class Holder {
    private static final Client client = new Client();
  }

  public insert(Object data) {
    Holder.client.insert(data);  // Holder.client is initialized on first access.
  }
}

  public insert(Object data) {
    init();
    client.insert(data);
  }

-----------------------

public class DB {

  private static final Client client = new Client();

  public insert(Object data) {
    client.insert(data);  // Guaranteed to be initialized once class loading is complete.
  }
}

public class DB {
  private static class Holder {
    private static final Client client = new Client();
  }

  public insert(Object data) {
    Holder.client.insert(data);  // Holder.client is initialized on first access.
  }
}

  public insert(Object data) {
    init();
    client.insert(data);
  }

-----------------------

public class DB {

  private static final Client client = new Client();

  public insert(Object data) {
    client.insert(data);  // Guaranteed to be initialized once class loading is complete.
  }
}

public class DB {
  private static class Holder {
    private static final Client client = new Client();
  }

  public insert(Object data) {
    Holder.client.insert(data);  // Holder.client is initialized on first access.
  }
}

  public insert(Object data) {
    init();
    client.insert(data);
  }

how to configure spark sql thrift server

Copy

Download

sbin/start-thriftserver.sh --hiveconf hive.server2.thrift.port 8088

YCSB JDBS driver: java.lang.ClassNotFoundException

Copy

Download

db.driver=com.mysql.jdbc.Driver # The JDBC driver class to use.
db.url=jdbc:mysql://master:3306/ycsb # The Database connection URL.
db.user=root # User name for the connection.
db.passwd=

# The JDBC driver class to use.
db.driver=com.mysql.jdbc.Driver 
# The Database connection URL.
db.url=jdbc:mysql://master:3306/ycsb 
# User name for the connection.
db.user=root 
db.passwd=

-----------------------

db.driver=com.mysql.jdbc.Driver # The JDBC driver class to use.
db.url=jdbc:mysql://master:3306/ycsb # The Database connection URL.
db.user=root # User name for the connection.
db.passwd=

# The JDBC driver class to use.
db.driver=com.mysql.jdbc.Driver 
# The Database connection URL.
db.url=jdbc:mysql://master:3306/ycsb 
# User name for the connection.
db.user=root 
db.passwd=

-----------------------

<dependency>
      <groupId>com.mysql.driver</groupId>
      <artifactId>mysqldriver</artifactId>
      <version>5.1.46</version>
      <scope>system</scope>
      <systemPath>/PATH/YCSB/jdbc/lib/mysql-connector-java-5.1.46-bin.jar</systemPath>
</dependency>

How to map MongoDB data in Spark for kmeans?

Copy

Download

df = spark.read.format("com.mongodb.spark.sql.DefaultSource")
               .option("uri","mongodb://127.0.0.1/ycsb.usertable")
               .load()

# Drop _id column and get RDD representation of the DataFrame
rowRDD = df.drop("_id").rdd

# Convert RDD of Row into RDD of numpy.array
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

# Feed into KMeans 
clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")

parsedRdd = rowRDD.map(lambda row: array([x for x in row]))

from numpy import array 
from pyspark.mllib.clustering import KMeans
import org.apache.spark.sql.SparkSession

spark = SparkSession \
.builder \
.appName("myApp") \
.config("spark.mongodb.input.uri", "mongodb://127.0.0.1/ycsb.usertable") \
.config("spark.mongodb.output.uri", "mongodb:/127.0.0.1/ycsb.usertable") \
.getOrCreate()

df = sqlContext.read.format("com.mongodb.spark.sql.DefaultSource").load()

rowRDD = df.drop("_id").rdd
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")
clusters.clusterCenters

-----------------------

df = spark.read.format("com.mongodb.spark.sql.DefaultSource")
               .option("uri","mongodb://127.0.0.1/ycsb.usertable")
               .load()

# Drop _id column and get RDD representation of the DataFrame
rowRDD = df.drop("_id").rdd

# Convert RDD of Row into RDD of numpy.array
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

# Feed into KMeans 
clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")

parsedRdd = rowRDD.map(lambda row: array([x for x in row]))

from numpy import array 
from pyspark.mllib.clustering import KMeans
import org.apache.spark.sql.SparkSession

spark = SparkSession \
.builder \
.appName("myApp") \
.config("spark.mongodb.input.uri", "mongodb://127.0.0.1/ycsb.usertable") \
.config("spark.mongodb.output.uri", "mongodb:/127.0.0.1/ycsb.usertable") \
.getOrCreate()

df = sqlContext.read.format("com.mongodb.spark.sql.DefaultSource").load()

rowRDD = df.drop("_id").rdd
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")
clusters.clusterCenters

-----------------------

df = spark.read.format("com.mongodb.spark.sql.DefaultSource")
               .option("uri","mongodb://127.0.0.1/ycsb.usertable")
               .load()

# Drop _id column and get RDD representation of the DataFrame
rowRDD = df.drop("_id").rdd

# Convert RDD of Row into RDD of numpy.array
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

# Feed into KMeans 
clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")

parsedRdd = rowRDD.map(lambda row: array([x for x in row]))

from numpy import array 
from pyspark.mllib.clustering import KMeans
import org.apache.spark.sql.SparkSession

spark = SparkSession \
.builder \
.appName("myApp") \
.config("spark.mongodb.input.uri", "mongodb://127.0.0.1/ycsb.usertable") \
.config("spark.mongodb.output.uri", "mongodb:/127.0.0.1/ycsb.usertable") \
.getOrCreate()

df = sqlContext.read.format("com.mongodb.spark.sql.DefaultSource").load()

rowRDD = df.drop("_id").rdd
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")
clusters.clusterCenters

-----------------------

df = spark.read.format("com.mongodb.spark.sql.DefaultSource")
               .option("uri","mongodb://127.0.0.1/ycsb.usertable")
               .load()

# Drop _id column and get RDD representation of the DataFrame
rowRDD = df.drop("_id").rdd

# Convert RDD of Row into RDD of numpy.array
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

# Feed into KMeans 
clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")

parsedRdd = rowRDD.map(lambda row: array([x for x in row]))

from numpy import array 
from pyspark.mllib.clustering import KMeans
import org.apache.spark.sql.SparkSession

spark = SparkSession \
.builder \
.appName("myApp") \
.config("spark.mongodb.input.uri", "mongodb://127.0.0.1/ycsb.usertable") \
.config("spark.mongodb.output.uri", "mongodb:/127.0.0.1/ycsb.usertable") \
.getOrCreate()

df = sqlContext.read.format("com.mongodb.spark.sql.DefaultSource").load()

rowRDD = df.drop("_id").rdd
parsedRdd = rowRDD.map(lambda row: array([int(x) for x in row]))

clusters = KMeans.train(parsedRdd, 2, maxIterations=10, initializationMode="random")
clusters.clusterCenters

Huge runtime difference between running YCSB with and without encryption with Workload E

Copy

Download

recordcount=10000000
operationcount=100000
readproportion=0
updateproportion=0
scanproportion=0.95
insertproportion=0.05
requestdistribution=zipfian
maxscanlength=100
scanlengthdistribution=uniform

Cannot find or load main class com.yahoo.ycsb.Client

Copy

Download

"foostore" : "com.yahoo.ycsb.db.FooStoreClient",

cd ycsb-foostore-binding-0.13.0-SNAPSHOT/
/bin/ycsb load foostore -thread 1 -P workload/worloada -s

-----------------------

"foostore" : "com.yahoo.ycsb.db.FooStoreClient",

cd ycsb-foostore-binding-0.13.0-SNAPSHOT/
/bin/ycsb load foostore -thread 1 -P workload/worloada -s

See all related Code Snippets

Community Discussions

Trending Discussions on YCSB

ArangoDB Request Load balancing
Bash Script - Python can't find a path
How to benchmark 2 Redis nodes with YCSB
Cassandra - HDD vs. SSD usage makes no difference in throughput
KUDU for JDBC replication purposes, but not for Off-loaded Analytics
Init static variable in synchronized block for all the threads, read without synchronized
Why the TiDB performance drop for 10 times when the updated field value is random?
Low read throughput in Cloud Spanner
Correlation between throughtput and latency when benchmarking with YCSB
Hadoop with MongoDB storage

Trending Discussions on YCSB

QUESTION

ArangoDB Request Load balancing

Asked 2020-Dec-01 at 21:32

I'm currently doing benchmarks for my studies using YCSB on an ArangoDB cluster (v3.7.3), that I set up using the starter (here).

I'm trying to understand if and how a setup like that ( I'm using 4 VMs e.g.) helps with balancing request load? If I have nodes A,B,C and D and I tell YCSB the IP of node A, all the requests go to node A...

That would mean that a cluster is unnecessary if you want to balance request load, wouldn't it? It would just make sense for data replication.

How would I handle the request load then? I'd normally do that in my application, but I can not do that if I use existing tools like YCSB... (or can I?)

Thanks for the help!

ANSWER

Answered 2020-Dec-01 at 21:32

I had this problem as well and ended up solving it by standing-up nginx in front of my cluster, providing a stable, language-independent way to distribute query load. I found nginx surprisingly simple to configure, but take a look at the upstream module for more details.

upstream arangodb {
  server coord-1.my.domain:8529;
  server coord-2.my.domain:8529;
  server coord-3.my.domain:8529;
}

server {
  listen                *:80 default_server;
  server_name           _; # Listens for ALL hostnames
  proxy_next_upstream   error timeout invalid_header;
  
  location / {
    proxy_pass          http://arangodb;
  }
}

It's not ideal for everyone but works well for those times when you just need to load-balance and don't want to write a bunch of code (which ends up being quite slow) to resolve coordinators.

I've asked ArangoDB for a native proxy server solution, but I would bet it's low on their to-do list as it could be tricky to support, given the huge number of configuration options.

Source https://stackoverflow.com/questions/64959668

QUESTION

Bash Script - Python can't find a path

Asked 2020-Sep-25 at 15:46

Please I need help!

This is my first bash script, and it's calling a python script at some points. But I always get this output at line 28 and 40:

./startTests.sh: line 28: 'logs/tp1/load-test1-workloada.txt': No such file or directory

The directory logs/tp1 do exist! I think it's something with the char '>' that I have to use there, but I don't know how to fix it.

This is my script:

#! /bin/bash

echo -e "\033[01;34m------------------------------"
echo -e "|      Testes YCSB 0.17      |"
echo -e "------------------------------\033[01;37m"

echo -e 'Está rodando o script da raiz do YCSB? (y/n)'
read yesno

if [ $yesno = 'y' ]; then
    echo -e 'Número de throughputs:'
    read numOfThroughputs

    echo -e 'Número de testes:'
    read numOfTests

    echo -e 'Iniciando...'

    for ((throughput = 1; throughput <= $numOfThroughputs; throughput++)); do
        echo -e '\033[01;32m------------------------------'
        echo -e "Throughput  $throughput "
        echo -e '------------------------------\033[01;37m'
        for ((test = 1; test <= $numOfTests; test++)); do
            echo -e '\033[01;32m------------------------------'
            echo -e "Iciando teste $test"
            echo -e '------------------------------\033[01;37m'
            echo -e 'Loading Workload A'
            python  ./bin/ycsb load mongodb-async -P workloads/workloada -p mongodb.url=mongodb://172.18.0.3:27017/ycsb?w=1 -P config/load.dat -s > "'logs/tp${throughput}/load-test${test}-workloada.txt'"

            for workload in {a,b,c,,f,d,e}; do
                if [ $workload = 'e' ]; then
                    echo -e "\033[01;33mPreparação para workload $workload"
                    echo -e "Apague o banco e precione QUALQUER TECLA 2X"
                    read enter
                    echo -e "Precione QUALQUER TECLA\033[01;37m"
                    read enter2
                    python  ./bin/ycsb load mongodb-async -P workloads/workloade -p mongodb.url=mongodb://172.18.0.3:27017/ycsb?w=1 -P config/load.dat -s > "'logs/tp${throughput}/load-test${test}-workloade.txt'"
                fi
                echo -e "Workload $workload"
                python  ./bin/ycsb run mongodb-async -P workloads/workload"${workload}" -p mongodb.url=mongodb://172.18.0.3:27017/ycsb?w=1 -P config/runtp"${throughput}".dat -s > ""'logs/tp${throughput}/run-test${test}-workload${workload}.txt'"
            done
            echo -e "\033[01;32mTeste $test finalizado!"
            echo -e "\033[01;33mApague o banco e precione QUALQUER TECLA 2X"
            read enter
            echo -e "Precione QUALQUER TECLA\033[01;37m"
            read enter2
        done
    done
fi

ANSWER

Answered 2020-Sep-25 at 15:38

ogs/tp1 may be there but this is not same as 'logs/tp1/'. You should remove the single quotes.

Source https://stackoverflow.com/questions/64067147

QUESTION

Cassandra - HDD vs. SSD usage makes no difference in throughput

Asked 2020-Mar-22 at 10:34

The Context
I'm currently running tests with Apache Cassandra on a single node cluster. I've ensured the cluster is up and running using nodetool status, I've done a multitude of reads and writes that suggest as such, and I'm confident my cluster is set up properly. I am now attempting to speed up my throughput by mounting a SSD onto the directory where Cassandra writes its data to.

My Solution
The write location of Cassandra data is generally to /var/lib/cassandra/data, however I've since switched mine using cassandra.yaml to write to another location, where I've mounted my SSD. I've ensured that Cassandra is writing to this location by checking the size of the data directory's contents through watch du -h and other methods. The directory I've mounted the SSD on includes table data, commitlog, hints, a nested data directory, and saved_caches.

The Problem
I've been using YCSB benchmarks (see https://github.com/brianfrankcooper/YCSB) to test the average throughput and ops/sec of Cassandra. I've noticed no difference in the average throughput when mounting HDD vs. SSD on the location where Cassandra writes its data to. I've analyzed disk access through dstat -cd --disk-util --disk-tps and found HDD caps out on CPU usage in multiple instances whereas SSD only spikes to around 80% on several occassions.

The Question
How can I speed up the throughput of Cassandra using a SSD over a HDD? I assume this is the correct place to mount my SSD, but does Cassandra not utilize its extra processing power? Any help would be greatly appreciated!

ANSWER

Answered 2020-Mar-22 at 10:34

SSD should always win over the HDD in terms of latency, etc. It's just a law of physics. I think that your test simply didn't provide enough load on the system. Another problem could be that you mount only data to SSD, but not the commit logs - on HDDs they should be always put onto a separate disk to avoid clashes with data load. On SSDs they could be put on the same disk as data - please point all directories to SSD to see a difference.

I recommend to perform a comparison by using following tools:

perfscripts - it uses fio tool to emulate Cassandra-like workloads, and if you run it on the both HDDs & SSDs, then you will see the difference in latency. You may not even execute it - just look historic folder, where there are results for different disk types;
DSBench - it was recently released by DataStax team, who is specializing in benchmarking Cassandra and DSE. There are built-in workloads described in wiki, that you can use for testing. Only make sure that you run the load long enough to see the effect of compaction, etc.

Source https://stackoverflow.com/questions/60796808

QUESTION

Init static variable in synchronized block for all the threads, read without synchronized

Asked 2019-Oct-04 at 15:11

I often see the following pattern. One thread will initialize the client in init() method in synchronized block. All the other threads, also called init() method before they start to use the other class methods. Client value is not changed after initialization. They dont set the client value as volatile.

My question is that if this is correct to do? Do all of the threads that create client, and call init() method , will after init method finished see the correct initilized value that was initialized byt the first thread that called init() method?

public class DB {

  private static Object lock = new Object();
  private static Client client;

  public init() {

    synchronized (lock) {
      if (client != null) {
        return;
      }
      client = new Client();
    }
  }

  public insert(Object data) {
    client.insert(data); // is this ok to access the client without volatile or synchronized?
  }
}

The rationale behind that pattern is that they think that because they read the client under synchronized block in init() method, the client will be set to the correct initialized value, and because the client is never changed, they can use it without volatile or synchronized after. IS this correct assumption?

You can see this pattern for example here: https://github.com/brianfrankcooper/YCSB/blob/cd1589ce6f5abf96e17aa8ab80c78a4348fdf29a/mongodb/src/main/java/site/ycsb/db/MongoDbClient.java where they initialized the database in init method and used it without synchronization after.

ANSWER

Answered 2019-Oct-04 at 14:59

It looks like the rationale behind this type of pattern is to ensure that you can only have one instance of Client in the application. Multiple invocations (parallel/sequential) of init() method on different/same DB objects will not allow creating a new Client if it is already created and synchronized block is just to ensure that client object will be created only once if multiple threads called init() parallelly.

But it has nothing to do with safe call of insert() method on client object and that totally depends on the implementation of the insert() method that may be thread-safe or may not be.

Source https://stackoverflow.com/questions/58238522

QUESTION

Correlation between throughtput and latency when benchmarking with YCSB

Asked 2018-Oct-15 at 08:04

I'm using YCSB to benchmark a number of different NoSQL databases. However, when playing around with the number of client threads I have a hard time interpreting the throughput vs. latency results.

For example, when benchmarking cassandra running workload a (50/50 reads and updates) with 16 client threads the following command is executed:

bin/ycsb run cassandra-cql -p hosts=xx.xx.xx.xx -p recordcount=525600 -p operationcount=525600 -threads 16 -P workloads/workloada -s > workloada_525600_16_threads_run_res.txt

which gives the following output:

[OVERALL], RunTime(ms), 62751
[OVERALL], Throughput(ops/sec), 8375.962136061577
[TOTAL_GCS_PS_Scavenge], Count, 64
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 289
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.46055042947522745
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 64
[TOTAL_GC_TIME], Time(ms), 289
[TOTAL_GC_TIME_%], Time(%), 0.46055042947522745
[READ], Operations, 262650
[READ], AverageLatency(us), 1844.6075042832667
[READ], MinLatency(us), 290
[READ], MaxLatency(us), 116159
[READ], 95thPercentileLatency(us), 3081
[READ], 99thPercentileLatency(us), 7551
[READ], Return=OK, 262650
[CLEANUP], Operations, 16
[CLEANUP], AverageLatency(us), 139458.5
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 2232319
[CLEANUP], 95thPercentileLatency(us), 19
[CLEANUP], 99thPercentileLatency(us), 2232319
[UPDATE], Operations, 262950
[UPDATE], AverageLatency(us), 1764.8220193953223
[UPDATE], MinLatency(us), 208
[UPDATE], MaxLatency(us), 95807
[UPDATE], 95thPercentileLatency(us), 2901
[UPDATE], 99thPercentileLatency(us), 7031
[UPDATE], Return=OK, 262950

Running the same operation with 32 threads I get:

[OVERALL], RunTime(ms), 51785
[OVERALL], Throughput(ops/sec), 10149.65723665154
[TOTAL_GCS_PS_Scavenge], Count, 124
[TOTAL_GC_TIME_PS_Scavenge], Time(ms), 310
[TOTAL_GC_TIME_%_PS_Scavenge], Time(%), 0.5986289466061601
[TOTAL_GCS_PS_MarkSweep], Count, 0
[TOTAL_GC_TIME_PS_MarkSweep], Time(ms), 0
[TOTAL_GC_TIME_%_PS_MarkSweep], Time(%), 0.0
[TOTAL_GCs], Count, 124
[TOTAL_GC_TIME], Time(ms), 310
[TOTAL_GC_TIME_%], Time(%), 0.5986289466061601
[READ], Operations, 262848
[READ], AverageLatency(us), 2947.844628834916
[READ], MinLatency(us), 363
[READ], MaxLatency(us), 194559
[READ], 95thPercentileLatency(us), 5079
[READ], 99thPercentileLatency(us), 11055
[READ], Return=OK, 262848
[CLEANUP], Operations, 32
[CLEANUP], AverageLatency(us), 69601.5625
[CLEANUP], MinLatency(us), 1
[CLEANUP], MaxLatency(us), 2228223
[CLEANUP], 95thPercentileLatency(us), 3
[CLEANUP], 99thPercentileLatency(us), 2228223
[UPDATE], Operations, 262752
[UPDATE], AverageLatency(us), 2881.930485781269
[UPDATE], MinLatency(us), 316
[UPDATE], MaxLatency(us), 203391
[UPDATE], 95thPercentileLatency(us), 4987
[UPDATE], 99thPercentileLatency(us), 10711
[UPDATE], Return=OK, 262752

The overall runtime is lower and thus, the throughput is higher, but the latencies are higher as well.

I'm not quite sure how to interpret these results, and how would you find the "appropriate" number of client threads to run?

ANSWER

Answered 2018-Oct-15 at 08:04

In order to have a qualified benchmarks you should 1st define the SLA requirements you aim your system to achieve. Say your workload pattern is 50/50 WR/RD and your SLA requirements are 10K ops/sec throughput with 99th percentile latency < 10 millisec. Use YCSB -target flag to generate the needed throughput, and use various thread count to see which one meets your SLA needs.

It makes a lot of sense that when more threads are used, the throughput increased (more ops/sec), but that comes at a latency price. You should look into the relevant database metrics to try and find your bottleneck - it can be the:

Client (need a stronger client, or better parallelism using less threads but more clients)
Network
DB server (Disk / RAM - use a stronger instance).

You can read more about the Do's and Don't of DB benchmarking here

Source https://stackoverflow.com/questions/52759946

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install YCSB

You can download it from GitHub, Maven.
You can use YCSB like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the YCSB component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .