pandasticsearch | An Elasticsearch client exposing DataFrame API

by onesuper Python Version: 0.6.0 License: MIT

X-Ray Key Features Code Snippets Community Discussions(2)Vulnerabilities Install Support

kandi X-RAY | pandasticsearch Summary

pandasticsearch is a Python library typically used in Data Science, Pandas applications. pandasticsearch has no bugs, it has build file available, it has a Permissive License and it has low support. However pandasticsearch has 4 vulnerabilities. You can download it from GitHub.

Pandasticsearch is an Elasticsearch client for data-analysis purpose. It provides table-like access to Elasticsearch documents, similar to the Python Pandas library and R DataFrames. Elasticsearch is skilled in real-time indexing, search and data-analysis. Pandasticsearch can convert the analysis results (e.g. multi-level nested aggregation) into Pandas DataFrame objects for subsequent data analysis. Checkout the API doc:

Support

Quality

Security

License

Reuse

Support

pandasticsearch has a low active ecosystem.

It has 263 star(s) with 40 fork(s). There are 12 watchers for this library.

It had no major release in the last 12 months.

There are 8 open issues and 18 have been closed. On average issues are closed in 511 days. There are no pull requests.

It has a neutral sentiment in the developer community.

The latest version of pandasticsearch is 0.6.0

Quality

pandasticsearch has 0 bugs and 17 code smells.

Security

pandasticsearch has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.

pandasticsearch code analysis shows 4 unresolved vulnerabilities (0 blocker, 4 critical, 0 major, 0 minor).

There are 1 security hotspots that need review.

License

pandasticsearch is licensed under the MIT License. This license is Permissive.

Permissive licenses have the least restrictions, and you can use them in most projects.

Reuse

pandasticsearch releases are available to install and integrate.

Build file is available. You can build the component from source.

Installation instructions are not available. Examples and code snippets are available.

pandasticsearch saves you 705 person hours of effort in developing the same functionality from scratch.

It has 1717 lines of code, 191 functions and 19 files.

It has medium code complexity. Code complexity directly impacts maintainability of the code.

Top functions reviewed by kandi - BETA

kandi has reviewed pandasticsearch and discovered the below as its top functions. This is intended to give you an instant insight into pandasticsearch implemented functionality, and help decide if they suit your requirements.

Create a DataFrame from Elasticsearch index
Make a GET request
Prepare the url for a given path
Constructs a startswith field
List of column names
Get the columns from mapping
Resolve mappings from json_map
Resolve mappings in JSON format
Show the results of this query
Execute the query
Make a POST request
Build query
Construct an Agg object from a dictionary
Recursively iterate through all aggregations
Explain the aggregation result
Convert to pandas DataFrame
Recursively resolve nested fields
Convert a hit to a dictionary
Create a new Query from a dictionary
Explain the result
Returns a list of Row objects
Create a new row
Print debug information
Return a pandas DataFrame

Get all kandi verified functions for this library.

pandasticsearch Key Features

No Key Features are available at this moment for pandasticsearch.

pandasticsearch Examples and Code Snippets

No Code Snippets are available at this moment for pandasticsearch.

Community Discussions

Trending Discussions on pandasticsearch

Date format issue when reading data from ElasticSearch

Elasticsearch Aggregation to pandas Dataframe

QUESTION

Date format issue when reading data from ElasticSearch

Asked 2020-Jun-16 at 18:32

i'm reading data from elasticsearch . when i visualize date format on kibana it shows Date Aug 5, 2020 @ 23:00:00.000 => so it's correct but when i'm reading it from elasticsearch to do some machine learning , i noticed that date format is taken wrong Date 1.596665e+12

i'm collecting the content of the index into a dataframe with pyspark and i can do it in scala if there's any solution

`from elasticsearch import Elasticsearch from pandasticsearch import Select

es = Elasticsearch(['http://localhost:9200'],timeout=600) documents = es.search(index='sub1',body={})

pandas_df = Select.from_dict(documents).to_pandas() print(pandas_df) '

it shows wrong date format , so how can i solve it ? any help ? thank u

...

ANSWER

Answered 2020-Jun-11 at 18:17

1.596665e+12 equals 1596665000000 which is a unix millisecond timestamp corresponding to Wednesday, August 5, 2020 10:03:20 PM in GMT.

You've essentially got 3 options:

Use a script_field to parse/convert the ts to a human-readable date. Note that you'll need to extract the script fields from the response b/c they're not part of the _source.
Convert the timestamps after you fetch the docs but before you load them into a df (preferably in a loop/list comprehension/map).
Reindex your data w/ already converted timestamps. This can be done from inside of an _update script, you don't need to drop everything.

UPDATE

Implementation of point 2

Source https://stackoverflow.com/questions/62328452

QUESTION

Elasticsearch Aggregation to pandas Dataframe

Asked 2020-Jan-24 at 08:28

I am working with some ElasticSearch data and i would like to generate the tables from the aggregations like in Kibana. A sample output of the aggregation is below, based on the following code :

...

ANSWER

Answered 2020-Jan-24 at 08:28

Struggling with the same problem, I've come to believe the reason for this being that the response_dict are not normal dicts, but an elasticsearch_dsl.utils.AttrList of elasticsearch_dsl.utils.AttrDict.

If you have an AttrList of AttrDicts, it's possible to do:

Source https://stackoverflow.com/questions/59632290

Community Discussions, Code Snippets contain sources that include Stack Exchange Network

Vulnerabilities

No vulnerabilities reported

Install pandasticsearch

You can download it from GitHub.
You can use pandasticsearch like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

Support

For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .

Find more information at: