tsfile | columnar file format designed for time-series data | Time Series Database library

 by   thulab Java Version: 0.7.1 License: No License

kandi X-RAY | tsfile Summary

kandi X-RAY | tsfile Summary

tsfile is a Java library typically used in Database, Time Series Database applications. tsfile has no bugs, it has no vulnerabilities, it has build file available and it has low support. You can download it from GitHub, Maven.

THIS REPO HAS MOVED TO TsFile is a columnar file format designed for time-series data, which supports efficient compression and query. It is easy to integrate TsFile with your IOT big data processing frameworks.
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              tsfile has a low active ecosystem.
              It has 103 star(s) with 21 fork(s). There are 21 watchers for this library.
              OutlinedDot
              It had no major release in the last 12 months.
              There are 20 open issues and 48 have been closed. On average issues are closed in 23 days. There are 1 open pull requests and 0 closed requests.
              It has a neutral sentiment in the developer community.
              The latest version of tsfile is 0.7.1

            kandi-Quality Quality

              tsfile has 0 bugs and 0 code smells.

            kandi-Security Security

              tsfile has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              tsfile code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              tsfile does not have a standard license declared.
              Check the repository for any license declaration and review the terms closely.
              OutlinedDot
              Without a license, all rights are reserved, and you cannot use the library in your applications.

            kandi-Reuse Reuse

              tsfile releases are available to install and integrate.
              Deployable package is available in Maven.
              Build file is available. You can build the component from source.
              Installation instructions are not available. Examples and code snippets are available.

            Top functions reviewed by kandi - BETA

            kandi has reviewed tsfile and discovered the below as its top functions. This is intended to give you an instant insight into tsfile implemented functionality, and help decide if they suit your requirements.
            • Test program
            • Converts a global time filter into a global time filter
            • Convert a global time filter into a query filter by using the select series list
            • Execute a QueryExpression
            • Test entry point
            • Flush all row groups to the output stream
            • Ends the TimeSeriesMetaData
            • Add a new StringContainer from the header
            • Adds a string to the container
            • Gets value from timestamp
            • Convert a JSON Schema to a map of Measure Descriptors
            • Finds CompressionType by short name
            • Compares this pair with the specified object
            • Decode int
            • Join a list of byte arrays
            • Returns the next value
            • Read all values from the target page list
            • Read a long value from an input stream
            • Writes test file
            • Clone a string container
            • Construct data point with data type
            • Loads the configuration file
            • Read a double
            • Convert a string to a TSRecord
            • Reads a float from the given stream
            • Encodes all values in the list and writes it to the OutputStream
            Get all kandi verified functions for this library.

            tsfile Key Features

            No Key Features are available at this moment for tsfile.

            tsfile Examples and Code Snippets

            No Code Snippets are available at this moment for tsfile.

            Community Discussions

            QUESTION

            query when disk is full and then the error is reported on Apache IoTDB Database
            Asked 2022-Mar-11 at 02:03

            2022-03-09 10:17:10,158 [pool-12-IoTDB-RPC-Client-1] WARN o.a.i.d.u.ErrorHandlingUtils:61 - Status code: INTERNAL_SERVER_ERROR(500), operation: "select count(tenantId$s) from root.benchmark.type.model_850_7.device_model850_15". executeStatement failed java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: Invalid input: 95 at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.iotdb.db.service.thrift.impl.TSServiceImpl.executeStatement(TSServiceImpl.java:637) at org.apache.iotdb.service.rpc.thrift.TSIService$Processor$executeStatement.getResult(TSIService.java:2853) at org.apache.iotdb.service.rpc.thrift.TSIService$Processor$executeStatement.getResult(TSIService.java:2833) at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:38) at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:38) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:248) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.lang.IllegalArgumentException: Invalid input: 95 at org.apache.iotdb.tsfile.file.metadata.enums.MetadataIndexNodeType.deserialize(MetadataIndexNodeType.java:70) at org.apache.iotdb.tsfile.file.metadata.MetadataIndexNode.deserializeFrom(MetadataIndexNode.java:107) at org.apache.iotdb.tsfile.read.TsFileSequenceReader.getMetadataAndEndOffset(TsFileSequenceReader.java:980) at org.apache.iotdb.tsfile.read.TsFileSequenceReader.getLeafMetadataIndexPair(TsFileSequenceReader.java:471) at org.apache.iotdb.tsfile.read.TsFileSequenceReader.readTimeseriesMetadata(TsFileSequenceReader.java:443) at org.apache.iotdb.db.engine.cache.TimeSeriesMetadataCache.get(TimeSeriesMetadataCache.java:194) at org.apache.iotdb.db.utils.FileLoaderUtils.loadTimeSeriesMetadata(FileLoaderUtils.java:168)

            ...

            ANSWER

            Answered 2022-Mar-11 at 02:03

            You can do this if you have another disk:

            1. Add the path of the second disk to data_dir in the iOTDB configuration file

            2. Move some TS files from the first disk to the second disk

            Source https://stackoverflow.com/questions/71423289

            QUESTION

            the data is not synchronized on Apaceh IoTDB Database
            Asked 2022-Mar-10 at 09:05

            I configured TsFile synchronization. Schema already synchronizes, while the data is not synchronized. Have no idea why it would happen.enter image description here

            ...

            ANSWER

            Answered 2022-Mar-10 at 09:05

            Does the TsFile on the sending end have a TsFile resource? If not, the file is not sealed. Currently, only sealed files can be synchronized.

            Source https://stackoverflow.com/questions/71409779

            QUESTION

            Apache IoTDB compiling error :maven-download-plugin has wget (get-thrift-executable) EOFException
            Asked 2022-Mar-04 at 05:43

            Apache IoTDB compiling error :maven-download-plugin has wget (get-thrift-executable) EOFException

            ran "mvn clean package -pl distribution -am -DskipTests" and then received the error. [INFO] -------------------< org.apache.iotdb:iotdb-thrift >-------------------- [INFO] Building rpc-thrift 0.12.5-SNAPSHOT [4/17] [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ iotdb-thrift --- [INFO] Deleting /Users/qiaojialin/Documents/git_workspace/incubator-iotdb/thrift/target [INFO] [INFO] --- maven-checkstyle-plugin:3.0.0:check (validate) @ iotdb-thrift --- [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ iotdb-thrift --- [INFO] Skipping Rule Enforcement. [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-version-convergence) @ iotdb-thrift --- [INFO] Skipping Rule Enforcement. [INFO] [INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (vulnerability-checks) @ iotdb-thrift --- [INFO] Skipping Rule Enforcement. [INFO] [INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ iotdb-thrift --- [INFO] [INFO] --- download-maven-plugin:1.3.0:wget (get-thrift-executable) @ iotdb-thrift --- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary for Apache IoTDB Project Parent POM 0.12.5-SNAPSHOT: [INFO] [INFO] Apache IoTDB Project Parent POM .................... SUCCESS [ 8.961 s] [INFO] TsFile ............................................. SUCCESS [ 19.869 s] [INFO] sql-antlr .......................................... SUCCESS [ 4.801 s] [INFO] rpc-thrift ......................................... FAILURE [ 0.301 s] [INFO] rpc-thrift-cluster ................................. SKIPPED [INFO] rpc-thrift-sync .................................... SKIPPED [INFO] Service-rpc ........................................ SKIPPED [INFO] IoTDB Jdbc ......................................... SKIPPED [INFO] IoTDB Server ....................................... SKIPPED [INFO] IoTDB Session ...................................... SKIPPED [INFO] IoTDB Cli .......................................... SKIPPED [INFO] IoTDB Grafana ...................................... SKIPPED [INFO] cluster ............................................ SKIPPED [INFO] Compile Tools ...................................... SKIPPED [INFO] Tools: Thrift ...................................... SKIPPED [INFO] Client for cpp ..................................... SKIPPED [INFO] IoTDB Distribution ................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 34.350 s [INFO] Finished at: 2022-03-02T10:31:28+08:00 [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal com.googlecode.maven-download-plugin:download-maven-plugin:1.3.0:wget (get-thrift-executable) on project iotdb-thrift: IO Error: EOFException -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :iotdb-thrift

            ...

            ANSWER

            Answered 2022-Mar-04 at 05:43

            delete directory".m2/repository/.cache/download-maven-plugin". Then compile again. It will work.

            Source https://stackoverflow.com/questions/71347068

            QUESTION

            How often the TsFiles in Apache IoTDB be merged?
            Asked 2022-Feb-22 at 09:09

            I'm currently inserting data into IoTDB and I see some small TsFiles is merging into the large files. I would like to know how often the TsFiles be merged and how the merge process is triggered. Thanks!

            ...

            ANSWER

            Answered 2022-Feb-22 at 09:09

            In the 0.12.x IoTDB iotdb-engine.properties file, there is a configuration merge_fileSelection_time_budget=30000, which means if there are enough amount of TsFiles of one level existed (seq_file_num_in_each_level=6 or unseq_file_num_in_each_level=10), the merge task will start. After 30000ms then, IoTDB will check if there still are enough TsFiles existed and start the next merge task.

            Source https://stackoverflow.com/questions/71217704

            QUESTION

            How to convert TsFile into CSV when paths are unknown in IoTDB
            Asked 2022-Jan-13 at 02:04

            How to convert TsFile into CSV when paths are unknown in IoTDB? The CSV should contain path, time, type, value, etc.

            ...

            ANSWER

            Answered 2022-Jan-13 at 02:04

            Currently, we do not have export CSV tools for TsFile, but you could use export tools for IoTDB.

            Step 1. Load TsFile in a clean IoTDB: https://iotdb.apache.org/UserGuide/Master/System-Tools/Load-External-Tsfile.html

            Step 2. Use export-csv and use https://iotdb.apache.org/zh/UserGuide/Master/System-Tools/Load-External-Tsfile.html or select * from root align by device: https://iotdb.apache.org/UserGuide/Master/System-Tools/CSV-Tool.html

            Source https://stackoverflow.com/questions/70690516

            QUESTION

            How to set the wal_buffer_size when a plan can not fit into the wal buffer in IoTDB
            Asked 2021-Nov-07 at 14:20

            When a plan can not fit into the wal buffer, IoTDB throw an exception. However, I do not know how to set the wal_buffer_size.

            client:

            server:

            ...

            ANSWER

            Answered 2021-Nov-07 at 14:20

            In latest version, IoTDB will print the actual size of plan when it exceeds the wal_buffer_size. For example, Log cannot fit into the buffer, please increase wal_buffer_size to 16777216.

            You can check server log and set wal_buffer_size in iotdb-engine.properties.

            Source https://stackoverflow.com/questions/69871328

            QUESTION

            UnsupportedOperationException occurred in Compaction Merge in IoTDB when TsFile storage in HDFS
            Asked 2021-Oct-27 at 05:55

            I've tried to locale the TSFiles to HDFS, the config as below:

            ...

            ANSWER

            Answered 2021-Oct-27 at 05:55

            the compaction of IoTDB doesn't adapt storage in HDFS yet. Please close the compaction by setting compaction_strategy=NO_COMPACTION and enable_unseq_compaction=false when using HDFS.

            Source https://stackoverflow.com/questions/69726200

            QUESTION

            Too big tsfile / datastore in Apache IoTDB database (version 0.11.2)
            Asked 2021-Oct-06 at 07:13

            We are using Apache IoTDB Server in Version 0.11.2 in a scenario and observe a data directory / tsfiles that are way bigger than they should be (about 130 sensors with 4 Million double values for each sensor but the files are about 200gb).

            Are there known issues or do you have any ideas what could cause this is how to track that down?

            The only thing we could think off could be some merge artefacts as we do write many datapoints out of order so merging has to happen frequently.

            Are there any ideas or tools on how to debug / inspect the tsfiles to get an idea whats happening here?

            Any help or hints is appreciated!

            ...

            ANSWER

            Answered 2021-Oct-06 at 06:55

            this may be due to the compaction strategy.

            You could fix this in two ways (no need at the same time):

            (1) upgrade to 0.12.2 version

            (2) open the configuration in iotdb-engine.properties: force_full_merge=true

            The reason is:

            The unsequenced data compaction in the 0.11.2 version has two strategies.

            E.g.,

            Chunks in a sequence TsFile: [1,3], [4,5]

            Chunks in a unsequence TsFile: [2]

            (I use [1,3] to indicate the timestamp of 2 data points in a Chunk)

            (1) When using full merge(rewrite all data): we get a tidy sequence file: [1,2,3],[4,5]

            (2) However, to speed up the compaction, we use append merge by default, when we get a sequence TsFile: [1,3],[4,5],[1,2,3]. In this TsFile, [1,3] does not have metadata at the end of the File, it is garbage data.

            So, if you have lots of out-of-order data merged frequently, this will happen (get a verrry big TsFile).

            The big TsFile will be tidy after a new compaction.

            You could also use TsFileSketchTool or example/tsfile/TsFileSequenceRead to see the structure of the TsFile.

            Source https://stackoverflow.com/questions/69460998

            QUESTION

            Angular11 - NG0303: Can't bind to 'ngForOf' since it isn't a known property of 'tr'
            Asked 2021-May-21 at 21:35

            the template TR not displayed, and generates an error : NG0303: Can't bind to 'ngForOf' since it isn't a known property of 'tr'. I don't understand why. I am in Angular 11 and navigator Chrone

            Error in the console html :

            ...

            ANSWER

            Answered 2021-May-21 at 21:35

            You should add the component to the declarations array:

            Source https://stackoverflow.com/questions/67643943

            QUESTION

            set mat-select default value if array has only single element else populate dropdown Angular
            Asked 2020-Oct-21 at 09:25

            I have a scenario where I have select dropdownlist(classification array). if my selecList has multiple values then ill display as a dropdown. if it has only one element in the array then by default it should be shown in selectList dropdown with that element selected by default. I need to write a condition where if classification element length is less orequal to 1 then show the element selected by default in matselect or display the list. Could you guys advise

            ...

            ANSWER

            Answered 2020-Oct-20 at 16:00

            I think so you could set has selected value on constructor. I'm assuming you are using reactive forms. (it looks like in your code example).

            Source https://stackoverflow.com/questions/64448729

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            No vulnerabilities reported

            Install tsfile

            You can download it from GitHub, Maven.
            You can use tsfile like any standard Java library. Please include the the jar files in your classpath. You can also use any IDE and you can run and debug the tsfile component as you would do with any other Java program. Best practice is to use a build tool that supports dependency management such as Maven or Gradle. For Maven installation, please refer maven.apache.org. For Gradle installation, please refer gradle.org .

            Support

            For any new features, suggestions and bugs create an issue on GitHub. If you have any questions check and ask questions on community page Stack Overflow .
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
            Install
            Maven
            Gradle
            CLONE
          • HTTPS

            https://github.com/thulab/tsfile.git

          • CLI

            gh repo clone thulab/tsfile

          • sshUrl

            git@github.com:thulab/tsfile.git

          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link