parquet-metadata | Dump metadata about a Parquet file

by cldellow Python Version: 0.0.1 License: Apache-2.0

X-Ray Key Features Code Snippets(1)Community Discussions(1)Vulnerabilities Install Support

kandi X-RAY | parquet-metadata Summary

parquet-metadata is a Python library typically used in Big Data, Spark, Hadoop applications. parquet-metadata has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can install using 'pip install parquet-metadata' or download it from GitHub, PyPI.

Dump metadata about a Parquet file.

Support

Quality

Security

License

Reuse

Support

parquet-metadata has a low active ecosystem.

It has 9 star(s) with 1 fork(s). There are 2 watchers for this library.

It had no major release in the last 12 months.

parquet-metadata has no issues reported. There are 1 open pull requests and 0 closed requests.

It has a neutral sentiment in the developer community.

The latest version of parquet-metadata is 0.0.1

Quality

parquet-metadata has 0 bugs and 0 code smells.

Security

parquet-metadata has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

parquet-metadata code analysis shows 0 unresolved vulnerabilities.

There are 0 security hotspots that need review.

License

parquet-metadata is licensed under the Apache-2.0 License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

parquet-metadata releases are not available. You will need to build from source code and install.

Deployable package is available in PyPI.

Build file is available. You can build the component from source.

Installation instructions are not available. Examples and code snippets are available.

It has 67 lines of code, 6 functions and 5 files.

It has low code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed parquet-metadata and discovered the below as its top functions. This is intended to give you an instant insight into parquet-metadata implemented functionality, and help decide if they suit your requirements.

Dump data to file
Dump a single column
Dump a row group
Dump metadata to file

Get all kandi verified functions for this library.

parquet-metadata Key Features

No Key Features are available at this moment for parquet-metadata.

parquet-metadata Examples and Code Snippets

parquet-metadata

Python

Lines of Code : 26

License : Permissive (Apache-2.0)

Copy

sudo pip install pyarrow parquet-metadata
parquet-metadata parquet.file

file	created_by	parquet-cpp version 1.4.1-SNAPSHOT
file	columns	9
file	row_groups	1
file	rows	2
row_group	0		size	634
row_group	0		rows	2
row_group	0		columns	9
row_group	0	bool

Community Discussions

Trending Discussions on parquet-metadata

How to write file-wide metadata into parquetfiles with apache parquet in C++

QUESTION

How to write file-wide metadata into parquetfiles with apache parquet in C++

Asked 2021-Dec-03 at 15:05

I use apache parquet to create Parquet tables with process information of a machine and I need to store file wide metadata (Machine ID and Machine Name).

It is stated that parquet files are capable of storing file wide metadata, however i couldn't find anything in the documentation about it.

There is another stackoverflow post that tells how it is done with pyarrow. As far as the post is telling, i need some kind of key value pair (maybe map) and add it to the schema somehow.

I Found a class inside the parquet source code that is called parquet::FileMetaData that may be used for this purpose, however there is nothing in the docs about it.

Is it possible to store file-wide metadata with c++ ?

Currently i am using the stream_reader_writer example for writing parquet files

...

ANSWER

Answered 2021-Dec-03 at 15:05

You can pass the file level metadata when calling parquet::ParquetFileWriter::Open, see the source code here

Source https://stackoverflow.com/questions/70197200

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install parquet-metadata

You can install using 'pip install parquet-metadata' or download it from GitHub, PyPI.
You can use parquet-metadata like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: