nDPI | Open Source Deep Packet Inspection Software Toolkit

by ntop C Version: 4.6 License: LGPL-3.0

X-Ray Key Features Code Snippets Community Discussions(10)Vulnerabilities Install Support

kandi X-RAY | nDPI Summary

nDPI is a C library. nDPI has no bugs, it has no vulnerabilities, it has a Weak Copyleft License and it has medium support. You can download it from GitHub.

nDPI is an open source LGPLv3 library for deep-packet inspection. Based on OpenDPI it includes ntop extensions. We have tried to push them into the OpenDPI source tree but nobody answered emails so we have decided to create our own source tree.

Support

Quality

Security

License

Reuse

Support

nDPI has a medium active ecosystem.

It has 3267 star(s) with 833 fork(s). There are 154 watchers for this library.

It had no major release in the last 12 months.

There are 41 open issues and 747 have been closed. On average issues are closed in 197 days. There are 7 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of nDPI is 4.6

Quality

nDPI has no bugs reported.

Security

nDPI has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

License

nDPI is licensed under the LGPL-3.0 License. This license is Weak Copyleft.

Weak Copyleft licenses have some restrictions, but you can use them in commercial projects.

Reuse

nDPI releases are available to install and integrate.

Top functions reviewed by kandi - BETA

kandi's functional review helps you automatically verify the functionalities of the libraries and avoid rework.
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of nDPI

Get all kandi verified functions for this library.

nDPI Key Features

No Key Features are available at this moment for nDPI.

nDPI Examples and Code Snippets

No Code Snippets are available at this moment for nDPI.

Community Discussions

Trending Discussions on nDPI

Custom Malloc() Function :: What Does this Syntax Mean?

Can I Submit Data-In-Memory to pcap_loop()?

C Syntax :: Function passed as parameter, where does function get its arguments?

Am I able to convert a directory path into something that can be fed into a python hdf5 data table?

How can I use VIPS for image normalization?

Where are built in scripts for identifying popular web applications?

Bash script to batch process files in sets of 2; how do I pass on the 2nd filename

Script (python, applescript, bash) to automaticlly convert a set of pyramidal tiled image INTO a set of large tiffs using ndpi2tiff?

libvips extract area of NDPI from Ex40 not from map

C++ code undefined reference to function when using shared library

QUESTION

Custom Malloc() Function :: What Does this Syntax Mean?

Asked 2020-Sep-18 at 19:57

I am writing a C program with the nDPI library, available here. (Coding on a Ubuntu machine, GCC compiler, nDPI version 3.2) nDPI is used to inspect network traffic. The code uses a lot of different structs to represent network stuff, like network flows, network protocols, network hosts, and so on.

So I think that if you want to create these structs, you must use the library's custom malloc() and free() functions, which makes sense. But I'm having a hard time understanding the function prototypes. Here's a few relevant lines of code from the API header file:

...

ANSWER

Answered 2020-Sep-18 at 15:53

The function set_ndpi_flow_malloc doesn't actually do the allocation but allows you to set the function that does. It's argument is a pointer to a function that takes a size_t and returns a void *, and the name of the argument is __ndpi_flow_malloc.

The same goes for set_ndpi_flow_free. It tells the library which function to use as its custom free function.

Most likely, ndpi_flow_malloc is the default custom allocator. So if this is the one you want to use you would do the following to set it as the custom allocation function:

Source https://stackoverflow.com/questions/63958750

QUESTION

Can I Submit Data-In-Memory to pcap_loop()?

Asked 2020-Sep-13 at 18:34

Using this link as a guide, I’ve written a C program which writes a PCAP file to disk. The PCAP contains complete Ethernet/IP/TCP packets, with payload. And after my program is done, I can manually read the generated PCAP files in Wireshark, so I believe the code works great. Here’s the pseudocode:

...

ANSWER

Answered 2020-Sep-13 at 18:33

For anyone following this thread... The solution was to use a function in the library, pcap_fopen_offline(). This was my solution:

Source https://stackoverflow.com/questions/63834561

QUESTION

C Syntax :: Function passed as parameter, where does function get its arguments?

Asked 2020-Jul-13 at 19:58

I’m reading through the C code for nDPI, a network packet inspection program. (available here) Most of the raw code I can follow, but this line threw me (slightly abridged for this forum):

...

ANSWER

Answered 2020-Jul-13 at 19:58

And here’s what flummoxes me: “ndpi_process_packet()” requires three arguments. But as far as I can tell, the “if()” line doesn’t provide any. So how are those arguments being submitted to “ndpi_process_packet()”? Where might I look to see where those arguments are coming from?

You need to look at how pcap_loop uses those arguments. As far as I can tell, pcap_loop is not defined in that project. It's part of an external library, libpcap.

So if you search libpcap, you'll find this function:

Source https://stackoverflow.com/questions/62882858

QUESTION

Am I able to convert a directory path into something that can be fed into a python hdf5 data table?

Asked 2019-Nov-15 at 14:33

I was wondering, how to convert a string or path into something that can be fed into a hdf5 table. For example, I am returning a numpy img array, label, and path to the image, from a Pytorch dataloader, where the path to the image would look like this:

...

ANSWER

Answered 2019-Nov-15 at 14:33

You have a couple of choices when it comes to saving string data:

You can create a standard dataset in h5py or PyTables, and define with an arbitrarily large string size. This is the simplest method, but runs the risk that your arbitrarily large string isn't large enough. :)
Alternately, can create a Variable Length dataset. PyTables calls this dataset type a VLArray and the object it uses is Class VLStringAtom(). h5py uses a standard dataset, but the dtype references special_dtype(vlen=str) (Note if you are using h5py 2.10 you can use string_dtype() instead).

I created an example that shows how to do this for both PyTables and h5py. It is built around the procedures referenced in your comments. I did not copy all of the code -- just what was necessary to retrieve file names and shuffle them. Also, the kaggle dataset I found has a different directory structure, so I modified cat_dog_train_path variable to match.

Source https://stackoverflow.com/questions/58794554

QUESTION

How can I use VIPS for image normalization?

Asked 2019-Nov-05 at 10:51

I want to normalize the exposure and color palettes of a set of images. For context, this is for training a neural net in image classification on medical images. I'm also doing this for hundreds of thousands of images, so efficiency is very important.

So far I've been using VIPS, specifically PyVIPS, and would prefer a solution using that library. After finding this answer and looking through the documentation, I tried

...

ANSWER

Answered 2019-Nov-05 at 10:51

You need hist_equal for histogram equalisation.

The main docs are here:

https://libvips.github.io/libvips/API/current/libvips-histogram.html

However, that will be extremely slow for large slide images. It will need to scan the whole slide once to build the histogram, then scan again to equalise it. It would be much faster to find the histogram of a low-res layer, then use that to equalise the high-res one.

For example:

Source https://stackoverflow.com/questions/58665477

QUESTION

Where are built in scripts for identifying popular web applications?

Asked 2019-May-07 at 17:23

I am investigating bro as a DPI solution to identify popular web applications (something like nDPI). I can identify that conn.log is analogous to netflow.

In the official documentation, it has been said that

In addition to the logs, Bro comes with built-in functionality for a range of analysis and detection tasks,... identifying popular web applicatios...

So I was looking at bro source code and examples, but I could not find any default log which identifies popular web application flows.

I ultimately want, conn.log or similar log to contain "a popular web application service" under service tag.

It would be great if someone points me to the built in script to identify popular webapps and concerned logs.

Thanks in advance!

...

ANSWER

Answered 2019-May-07 at 17:23

This comment in the docs refers to these policies/sigs, Sachin:

https://github.com/zeek/zeek/blob/master/scripts/policy/protocols/http/detect-webapps.zeek https://github.com/zeek/zeek/blob/master/scripts/policy/protocols/http/detect-webapps.sig

These are fairly dated (except for the recent Zeek renaming and compatibility updates).

Source https://stackoverflow.com/questions/55609550

QUESTION

Bash script to batch process files in sets of 2; how do I pass on the 2nd filename

Asked 2019-Feb-14 at 03:21

A colleague wrote a nice script to convert an XML document into a different XML file with formatting compatible with our workflow. Unfortunately, he didn't build in the ability to process thousands of files. This site helped me before to do a similar task, but I couldn't figure out the extrapolation.

There's an image file, with a corresponding XML file with annotation information (filename1.ndpi and filename1.ndpi.ndpa). The syntax is:

...

ANSWER

Answered 2019-Feb-14 at 03:11

$ for f in *.ndpi; do script.py "$f" "$f.ndpa"; done

Source https://stackoverflow.com/questions/54682587

QUESTION

Script (python, applescript, bash) to automaticlly convert a set of pyramidal tiled image INTO a set of large tiffs using ndpi2tiff?

Asked 2018-Nov-27 at 15:06

I feel like this should be a really simple solution, but I still can't seem to get it to work.

I have several hundred (more coming!) large pyramidal tiled images of histopathology cases created a hammamatsu scanner that outputs images in the NDPI format, aka whole slide imaging (WSI). There's already a tool called ndpi2tiff in a package called NdpiTools, that does the hard part. It will analyze a ndpi file (XYZ.ndpi) and create a new tiff file (XYZ.tif). Unfortunately, you can't use the *.ndpi wildcard argument and just batch convert all the files. It would be impractical to convert them one by one. I messed around with applescript (on a Mac, but have python installed) and only got errors where either no input was being directed to the ndpi2tiff OR it would try to act on the directory/folder and not the files within.

Usage is: ndpi2tiff -options [image2convert.ndpi]

Located in /usr/local/bin

https://www.imnc.in2p3.fr/pagesperso/deroulers/software/ndpitools/

Any tips?

...

ANSWER

Answered 2018-Nov-27 at 03:31

You can use find and -exec on the command line (bash):

Source https://stackoverflow.com/questions/53492087

QUESTION

libvips extract area of NDPI from Ex40 not from map

Asked 2018-Nov-22 at 09:36

I am using libvips to get pyramids of .ndpi images.

through this answer and searching the documentation I found this command

...

ANSWER

Answered 2018-Nov-22 at 09:36

I've written this as an answer, though it's not really an answer. It seemed too long as just a comment.

The libvips openslideload operation lets you pick an associated image to load. You can get a list of the associated images from the slide-associated-images metadata tag. For example:

Source https://stackoverflow.com/questions/53420846

QUESTION

C++ code undefined reference to function when using shared library

Asked 2017-Sep-26 at 14:14

I am trying to compile a simple c++ program that calls some functions from the NDPI C library. When compiling using the shared library (.so file) it can't find some of the functions in the library (like set_ndpi_flow_malloc) but seems to be able to find another function called set_ndpi_malloc.

Note: I believe the functions it can't find were newly added to the library when it updated from version 1.8 to 2.0, but the library files I am using were all compiled from source of version 2.0.

Strangely when I compile using the static library (.a) it has no problem finding these functions.

See the image linked above to see the compiler output each time and the code. The two compile attempts were made within a minute of each other and nothing was changed except for one line in the makefile to compile with the static library instead of the shared library.

Update

This has been identified as an issue with the library not exporting all the symbols.

See the issue on github here: https://github.com/ntop/nDPI/issues/459

...

ANSWER

Answered 2017-Sep-26 at 02:23

I made some effort to reproduce your steps.

Clone (download/unpack) the library source from their Github
Enter the source directory of the library
./autogen.sh (as per README.md, also: libpcap-dev package is needed to be installed in the system)
./configure (as per README.md)
make (as per README.md)
Create a file in the home directory called test.cpp and place your code there (however, I think that include statement shall be reworded as simply #include "ndpi_api.h")
g++ -c -O3 -Wall -I ./src/include ~/test.cpp (as per your wish)
g++ -O3 -Wall -o test test.o -L ./lib -lndpi (as per your wish)

So, indeed, there are undefined symbols in this case. I used nm -gC ./lib/libndpi.so to read the symbol list and didn't find set_ndpi_flow_malloc and set_ndpi_flow_free. Not mentioned in either way.

I then inspected ./src/include/ndpi_api.h and ./src/lib/ndpi_main.c and couldn't wrap my head around what might have been wrong there. Everything seemed to be OK since these two functions were implemented and defined similar to their counterparts without _flow_. Nevertheless, the difference certainly exists and could be observed in ./libndpi.sym file. As you might see, set_ndpi_malloc and set_ndpi_free are listed there whilst their _flow_ counterparts are not.

So, I deleted the library source and followed the steps 1 and 2 one more time. Then I added these two lines to ./libndpi.sym just after non-_flow_ function names:

Source https://stackoverflow.com/questions/46393154

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install nDPI

You can download it from GitHub.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: