Run SQL queries on Data files with TrdSQL

Trdsql is a command-line python application that execute SQL queries on flat data files like CSV, TSV, LTSV, TBLN and JSON files. It also allows exporting the outputs in several formats.

The application is written in Go language which known for speed and performance.

The reason why did we choose to write about trdSQL is that we used it in some tasks. It is proven to be a useful and reliable tool.

The trdsql supports both MySQL and PostgreSQL queries. It also can execute SQL queries from .sql files.

trdsql -q test.sql

The Trdsql tool is an advanced alternative for "q" which built with Python, and "textql" which allow executing SQL queries against structured CSV and TSV files. It supports more files format, different output options and process compressed and multiple files.

Highlights

1. Supports multiple file formats
2. Supports multiple output formats
3. Multiple file inputs (not possible to mix different formats)
4. Process compressed files: gzip, bz2, zstd, lz4, xz
5. Compress output files
6. Join table
7. Join csv files
8. Switch query driver (MySQL and PostgreSQL)
9. Configuration file config.json
10. It Can be used as Go language library.
11. Process large files

Supported file formats

1. CSV (Comma Separated Values)
2. TSV (Tab Separated Values)
3. LTSV (Labeled Tab-separated Values)
4. JSON
5. JSONL
6. TBLN

Output formats

1. Raw output
2. ASCII Table output
3. Markdown output
4. The vertical format output

Install

The trdsql has different installation options

1. Building the source with Go (requires go 1.13 or higher)
2. Download and install binary releases for Linux, Windows and macOS from here.
3. Use it within Docker container docker pull noborus/trdsql

License

The library is released as an open-source package under MIT license.

Resources

1. https://github.com/noborus/trdsql
2. https://github.com/harelba/q
3. https://github.com/dinedal/textql

More Related Articles in Data science