CleverCSV | Python package for handling messy CSV files | CSV Processing library

 by   alan-turing-institute Python Version: 0.8.2 License: MIT

kandi X-RAY | CleverCSV Summary

kandi X-RAY | CleverCSV Summary

CleverCSV is a Python library typically used in Utilities, CSV Processing applications. CleverCSV has no bugs, it has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. You can install using 'pip install CleverCSV' or download it from GitHub, PyPI.

CleverCSV is a Python package that aims to solve some of the pain points of CSV files, while maintaining many of the good things. The package automatically detects (with high accuracy) the format (dialect) of CSV files, thus making it easier to simply point to a CSV file and load it, without the need for human inspection. In the future, we hope to solve some of the other issues of CSV files too. CleverCSV is based on science. We investigated thousands of real-world CSV files to find a robust way to automatically detect the dialect of a file. This may seem like an easy problem, but to a computer a CSV file is simply a long string, and every dialect will give you some table. In CleverCSV we use a technique based on the patterns of row lengths of the parsed file and the data type of the resulting cells. With our method we achieve 97% accuracy for dialect detection, with a 21% improvement on non-standard (messy) CSV files compared to the Python standard library.
Support
    Quality
      Security
        License
          Reuse

            kandi-support Support

              CleverCSV has a medium active ecosystem.
              It has 1083 star(s) with 61 fork(s). There are 18 watchers for this library.
              OutlinedDot
              It had no major release in the last 12 months.
              There are 9 open issues and 26 have been closed. On average issues are closed in 27 days. There are 1 open pull requests and 0 closed requests.
              It has a neutral sentiment in the developer community.
              The latest version of CleverCSV is 0.8.2

            kandi-Quality Quality

              CleverCSV has 0 bugs and 0 code smells.

            kandi-Security Security

              CleverCSV has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              CleverCSV code analysis shows 0 unresolved vulnerabilities.
              There are 0 security hotspots that need review.

            kandi-License License

              CleverCSV is licensed under the MIT License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              CleverCSV releases are available to install and integrate.
              Deployable package is available in PyPI.
              Build file is available. You can build the component from source.
              Installation instructions, examples and code snippets are available.
              CleverCSV saves you 2181 person hours of effort in developing the same functionality from scratch.
              It has 5200 lines of code, 397 functions and 52 files.
              It has medium code complexity. Code complexity directly impacts maintainability of the code.

            Top functions reviewed by kandi - BETA

            kandi has reviewed CleverCSV and discovered the below as its top functions. This is intended to give you an instant insight into CleverCSV implemented functionality, and help decide if they suit your requirements.
            • Implements the command line interface
            • Import an optional optional dependency module
            • Write the csv to a file
            • Detect dialect
            • Parse an integer
            • Import optional dependency
            • Read dicts from file
            • Detect the dialect of the given data
            • Stream data from a file
            • Detect the dialect of the given sample
            • Read data from file
            • Write a table to a file
            • Process csv files
            • Write a list of dictionaries to a file
            • Handles the command line
            • Run dialect detection
            • Create a csv dialect Dialect object
            • Create a simple dialect object
            • Calls pypi
            • Check if cell is a currency
            • Generate known types
            • Build an application
            • Run actions
            • Commit changelog
            • Clears a pre - release
            • Activate command
            • Build the man page
            Get all kandi verified functions for this library.

            CleverCSV Key Features

            No Key Features are available at this moment for CleverCSV.

            CleverCSV Examples and Code Snippets

            Looking for ideal method to "scrub" csv file to be put into Excel
            Pythondot img1Lines of Code : 13dot img1License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            import csv
            
            with open('input.csv', 'r', newline='') as input_file, open('output.csv','w', newline='') as output_file:
                reader = csv.reader(input_file)
                writer = csv.writer(output_file)
            
                for line_number, row in enumerate(reader, s

            Community Discussions

            QUESTION

            Looking for ideal method to "scrub" csv file to be put into Excel
            Asked 2020-Apr-30 at 19:16

            Bit of an involved setup to this question, but bear with me!

            (Copy and pasting the below block into an editor works well)

            I am using clevercsv to load my data from a financial website's csv file. Each row is stored as an item in a list.

            data = clevercsv.wrappers.read_csv(in_file_name)

            After some account info lines, the stock data begins:

            stock_data = data[8:]

            I wish to remove the data: Market, Loan Value - all the way to - Day High (inclusive0

            And Keep Symbol, Description -> % of Positions (inclusive), 52-wk Low, 52-wk High

            Each stock has this data associated with it on the relevant line. Any best practices for removing this data? I have been trying and seem to be having logic errors.

            As of Date,2020-04-29 18:44:29

            Account,TD Direct Investing - HAHAHA

            Cash,123.12

            Investments,1234.12

            Total Value,12345.12

            Margin,123456.12,

            ,

            Symbol,Market,Description,Quantity,Average Cost,Price,Book Cost,Market Value,Unrealized $,Unrealized %,% of Positions,Loan Value,Change Today $,Change Today %,Bid,Bid Lots,Ask,Ask Lots,Volume,Day Low,Day High,52-wk Low,52-wk High

            AFL,US,"AFLAC INC",500,43.79,39.23,21895.79,19615.00,-2280.79,-10.42,7.26,,1.4399986,3.81,39.19,1,40.2,1,3001288,38.31,39.48,23.07,57.18

            AKTS,US,"AKOUSTIS TECHNOLOGIES INC",2500,5.04,8.94,12609.87,22350.00,9740.13,77.24,8.27,,0.35999966,4.20,8.68,1,9.2,10,1161566,8.65,9.25,3.76,9.25

            And here is my code so far:

            ...

            ANSWER

            Answered 2020-Apr-30 at 00:58

            If you already know the column indices and the header length, you can do something like this:

            Source https://stackoverflow.com/questions/61513760

            Community Discussions, Code Snippets contain sources that include Stack Exchange Network

            Vulnerabilities

            No vulnerabilities reported

            Install CleverCSV

            Click here to go to the introduction with more details about CleverCSV. If you're in a hurry, below is a quick overview of how to get started with the CleverCSV Python package and the command line interface.
            CleverCSV is available on PyPI. You can install either the full version, which includes the command line interface and all optional dependencies, using. or you can install a lighter, core version of CleverCSV with.

            Support

            If you want to encourage development of CleverCSV, the best thing to do now is to spread the word!. If you encounter an issue in CleverCSV, please open an issue or submit a pull request. Don't hesitate, you're helping to make this project better for everyone! If GitHub's not your thing but you still want to contact us, you can send an email to gertjanvandenburg at gmail dot com instead. You can also ask questions on Gitter. Note that all contributions to the project must adhere to the Code of Conduct. The CleverCSV package was originally written by Gertjan van den Burg and came out of scientific research on wrangling messy CSV files by Gertjan van den Burg, Alfredo Nazabal, and Charles Sutton.
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
            Install
          • PyPI

            pip install clevercsv

          • CLONE
          • HTTPS

            https://github.com/alan-turing-institute/CleverCSV.git

          • CLI

            gh repo clone alan-turing-institute/CleverCSV

          • sshUrl

            git@github.com:alan-turing-institute/CleverCSV.git

          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link

            Explore Related Topics

            Consider Popular CSV Processing Libraries

            Laravel-Excel

            by Maatwebsite

            PapaParse

            by mholt

            q

            by harelba

            xsv

            by BurntSushi

            countries

            by mledoze

            Try Top Libraries by alan-turing-institute

            sktime

            by alan-turing-institutePython

            the-turing-way

            by alan-turing-instituteHTML

            AIrsenal

            by alan-turing-instituteJupyter Notebook

            rse-course

            by alan-turing-instituteJupyter Notebook

            skpro

            by alan-turing-institutePython