spdk | Storage Performance Development Kit | Performance Testing library

by spdk C Version: v23.05 License: Non-SPDX

X-Ray Key Features Code Snippets Community Discussions(3)Vulnerabilities Install Support

kandi X-RAY | spdk Summary

spdk is a C library typically used in Testing, Performance Testing applications. spdk has no bugs, it has no vulnerabilities and it has medium support. However spdk has a Non-SPDX License. You can download it from GitHub.

Storage Performance Development Kit

Support

Quality

Security

License

Reuse

Support

spdk has a medium active ecosystem.

It has 2446 star(s) with 992 fork(s). There are 199 watchers for this library.

There were 2 major release(s) in the last 12 months.

There are 125 open issues and 2754 have been closed. On average issues are closed in 111 days. There are 1 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of spdk is v23.05

Quality

spdk has 0 bugs and 0 code smells.

Security

spdk has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

spdk code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

spdk has a Non-SPDX License.

Non-SPDX licenses can be open source with a non SPDX compliant license, or non open source licenses, and you need to review them closely before use.

Reuse

spdk releases are available to install and integrate.

Installation instructions, examples and code snippets are available.

Top functions reviewed by kandi - BETA

kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of spdk

Get all kandi verified functions for this library.

spdk Key Features

No Key Features are available at this moment for spdk.

spdk Examples and Code Snippets

No Code Snippets are available at this moment for spdk.

Community Discussions

Trending Discussions on spdk

git branch command works fine as a cli command, but fails when run from loop or script using variables

How to use MySQL database with NVMe SPDK?

What actually happens in asynchronous IO

QUESTION

git branch command works fine as a cli command, but fails when run from loop or script using variables

Asked 2021-Nov-10 at 15:43

In creating setup scripts, I have several git repos that I clone locally. This is done through a temporarily available proxy that may or may not be available later on, so I need to create all the remote branches from the remote repo as local branches that can be switched to. I have a method to extract the names of the remote repos that I want, when get stored as

...

ANSWER

Answered 2021-Nov-10 at 15:43

Figured out a solution:

Source https://stackoverflow.com/questions/69905099

QUESTION

How to use MySQL database with NVMe SPDK?

Asked 2021-Sep-03 at 15:33

I look at the documentation and don't see any practical implementation case in the examples.

Does anybody know how to ensure that incoming requests to my database server are processed via Intel NVMe SPDK?

...

ANSWER

Answered 2021-Sep-03 at 15:33

You would need to develop a custom storage engine.

The existing storage engines in MySQL use POSIX calls for I/O. In other words, they call standard kernel functions. This way they can support a wide range of storage devices and operating systems with portable code.

As I understand SPDK, it is a library of user-space code entry points that bypass the operating system kernel's I/O interface, and read and write NVMe directly.

So I assume it would require a complete rewrite of the I/O code in a MySQL storage engine to utilize SPDK. That would also make the storage engine non-portable. It would only work on hosts with NVMe storage devices.

Then you would need to work on optimizing it for about 25 years, to match the level of maturity of for example the InnoDB code. Just doing the same I/O calls with the same usage pattern may not be the best way to utilize an NVMe drive.

Even if you do all that, you probably would not find that it gives you a great improvement on performance overall for a complex service like MySQL. If your database is I/O bound, you're already losing. The better way to improve performance is to do no I/O, but read and write data in RAM. Even compared to NVMe storage, RAM latency is still at least three orders of magnitude lower.

Source https://stackoverflow.com/questions/69046903

QUESTION

What actually happens in asynchronous IO

Asked 2021-Jan-05 at 13:36

I keep reading about why asynchronous IO is better than synchronous IO, which is because in a-sync IO, your program can keep running, while in sync IO you're blocked until operation is finished.
I do not understand this saying because using sync IO (such as write()) the kernel writes the data to the disk - it doesn't happen by itself. The kernel do need CPU time in order to do it.
So in a-sync IO, it needs it as well, which might result in context switch from my application to the kernel. So it's not really blocking, but there cpu cycles do need to run this operation.

Is that correct?
Is the difference between those two that we assume disk access is slow, so compared to sync IO where you wait for the data to be written to disk, in a-sync IO the time you wait for it to be written to disk can be used to continue doing application processing, and the kernel part of writing it to disk is small?
Let's say I have an application that all it does is get info and write it into files. Is there any benefit for using a-sync IO instead of sync IO?

Examples of sync IO:

write()

Examples of async IO:

io_uring (as I understand has zero copy as well, so it's a benefit)
spdk (should be best, though I don't understand how to use it)
aio

...

ANSWER

Answered 2021-Jan-04 at 22:19

The kernel do need CPU time in order to do it.

Is that correct?.

Pretty much, yes.

Is the difference between those two that we assume disk access is slow ... in a-sync IO the time you wait for it to be written to disk can be used to continue doing application processing, and the kernel part of writing it to disk is small?

Exactly.

Let's say I have an application that all it does is get info and write it into files. Is there any benefit for using a-sync IO instead of sync IO?

Depends on many factors. How does the application "get info"? Is it CPU intensive? Does it use the same IO as the writing? Is it a service that processes multiple requests concurrently? How many simultaneous connections? Is the performance important in the first place? In some cases: Yes, there may be significant benefit in using async IO. In some other cases, you may get most of the benefits by using sync IO in a separate thread. And in other cases single threaded sync IO can be sufficient.

Source https://stackoverflow.com/questions/65570336

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install spdk

FreeBSD: Note: Make sure you have the matching kernel source in /usr/src/ and also note that CONFIG_COVERAGE option is not available right now for FreeBSD builds.
Optional components and other build-time configuration are controlled by settings in the Makefile configuration file in the root of the repository. CONFIG contains the base settings for the configure script. This script generates a new file, mk/config.mk, that contains final build settings. For advanced configuration, there are a number of additional options to configure that may be used, or mk/config.mk can simply be created and edited by hand. A description of all possible options is located in CONFIG.