pystore | Fast data store for Pandas time-series data | Database library

 by   ranaroussi Python Version: 0.1.23 License: Apache-2.0

kandi X-RAY | pystore Summary

kandi X-RAY | pystore Summary

pystore is a Python library typically used in Database, Pandas applications. pystore has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has low support. You can install using 'pip install pystore' or download it from GitHub, PyPI.

Fast data store for Pandas time-series data
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              pystore has a low active ecosystem.
              It has 494 star(s) with 92 fork(s). There are 37 watchers for this library.
              OutlinedDot
              It had no major release in the last 12 months.
              There are 26 open issues and 29 have been closed. On average issues are closed in 25 days. There are no pull requests.
              It has a neutral sentiment in the developer community.
              The latest version of pystore is 0.1.23

            kandi-Quality Quality

              pystore has 0 bugs and 0 code smells.

            kandi-Security Security

              pystore has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              pystore code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              pystore is licensed under the Apache-2.0 License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              pystore releases are available to install and integrate.
              Deployable package is available in PyPI.
              Build file is available. You can build the component from source.
              It has 430 lines of code, 44 functions and 7 files.
              It has high code complexity. Code complexity directly impacts maintainability of the code.

            Top functions reviewed by kandi - BETA

            kandi has reviewed pystore and discovered the below as its top functions. This is intended to give you an instant insight into pystore implemented functionality, and help decide if they suit your requirements.
            • Wrapper for write
            • Write data to an item
            • List items in the datastore
            • Read metadata
            • Write metadata to path
            • Convert datetime index to int64
            • Convert to pandas DataFrame
            • Append data to item
            • Creates a snapshot
            • List all available snapshots
            • Make a Path from arguments
            • Return list of subdirectories
            • Set the default path
            • Return a Path instance
            • Return the index of the given item
            • Return the path for an item
            • Delete a snapshot
            • Delete all snapshots
            • Delete an item
            • Return a list of all store directories
            Get all kandi verified functions for this library.

            pystore Key Features

            No Key Features are available at this moment for pystore.

            pystore Examples and Code Snippets

            No Code Snippets are available at this moment for pystore.

            Community Discussions

            QUESTION

            Multi-user efficient time-series storing for Django web app
            Asked 2019-Sep-04 at 18:26

            I'm developing a Django app. Use-case scenario is this:

            50 users, each one can store up to 300 time series and each time serie has around 7000 rows.

            Each user can ask at any time to retrieve all of their 300 time series and ask, for each of them, to perform some advanced data analysis on the last N rows. The data analysis cannot be done in SQL but in Pandas, where it doesn't take much time... but retrieving 300,000 rows in separate dataframes does!

            Users can also ask results of some analysis that can be performed in SQL (like aggregation+sum by date) and that is considerably faster (to the point where I wouldn't be writing this post if that was all of it).

            Browsing and thinking around, I've figured storing time series in SQL is not a good solution (read here).

            Ideal deploy architecture looks like this (each bucket is a separate server!):

            Problem: time series in SQL are too slow to retrieve in a multi-user app.

            Researched solutions (from this article):

            Here are some problems:

            1) Although these solutions are massively faster for pulling millions of rows time series into a single dataframe, I might need to pull around 500.000 rows into 300 different dataframes. Would that still be as fast?

            This is the current db structure I'm using:

            ...

            ANSWER

            Answered 2019-Sep-04 at 18:26

            The article you refer to in your post is probably the best answer to your question. Clearly good research and a few good solutions being proposed (don't forget to take a look at InfluxDB).

            Regarding the decoupling of the storage solution from your instances, I don't see the problem:

            • Arctic uses mongoDB as a backing store
            • pyStore uses a file system as a backing store
            • InfluxDB is a database server on its own

            So as long as you decouple the backing store from your instances and make them shared among instances, you'll have the same setup as for your posgreSQL database: mongoDB or InfluxDB can run on a separate centralised instance; the file storage for pyStore can be shared, e.g. using a shared mounted volume. The python libraries that access these stores of course run on your django instances, like psycopg2 does.

            Source https://stackoverflow.com/questions/57789982

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            No vulnerabilities reported

            Install pystore

            You can install using 'pip install pystore' or download it from GitHub, PyPI.
            You can use pystore like any standard Python library. You will need to make sure that you have a development environment consisting of a Python distribution including header files, a compiler, pip, and git installed. Make sure that your pip, setuptools, and wheel are up to date. When using pip it is generally recommended to install packages in a virtual environment to avoid changes to the system.

            Support

            For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
            Install
          • PyPI

            pip install PyStore

          • CLONE
          • HTTPS

            https://github.com/ranaroussi/pystore.git

          • CLI

            gh repo clone ranaroussi/pystore

          • sshUrl

            git@github.com:ranaroussi/pystore.git

          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link