Run SQL queries against JSON, CSV, Excel, Parquet, and more with dsq

dsq is a free and open-source app that enables you to run SQL queries against JSON, CSV, Excel, Parquet, and more.

It works on Windows, Linux and macOS.

It can also work against log files such as Nginx Access Logs and Apache Access Logs.

Features

• Lightweight
• Developer-friendly
• Easy learning curve
• Pretty results
• REGEXP support
• Support nested objects
• Support multiple files

Supported files and databases

• CSV
• JSON
• TSV
• YAML
• Excel files
• ODS
• Apache Error Logs
• Apache Access Logs
• Nginx Access Logs

Install

macOS

$ brew install dsq

Linux, macOS and WIndows

On macOS, Linux, and WSL you can run the following:

$ VERSION="v0.23.0"
$ FILE="dsq-$(uname -s | awk '{ print tolower($0) }')-x64-$VERSION.zip"
$ curl -LO "https://github.com/multiprocessio/dsq/releases/download/$VERSION/$FILE"
$ unzip $FILE
$ sudo mv ./dsq /usr/local/bin/dsq

Usage

You can either pipe data to dsq or you can pass a file name to it. NOTE: piping data doesn't work on Windows.

If you are passing a file, it must have the usual extension for its content type.

For example:

$ dsq testdata.json "SELECT * FROM {} WHERE x > 10"

Or:

$ dsq testdata.ndjson "SELECT name, AVG(time) FROM {} GROUP BY name ORDER BY AVG(time) DESC"

Pretty print

By default dsq prints ugly JSON. This is the most efficient mode.

$ dsq testdata/userdata.parquet 'select count(*) from {}'
[{"count(*)":1000}
]

If you want prettier JSON you can pipe dsq to jq.

$ dsq testdata/userdata.parquet 'select count(*) from {}' | jq
[
  {
    "count(*)": 1000
  }
]

Or you can enable pretty printing with -p or --pretty in dsq which will display your results in an ASCII table.

$ dsq --pretty testdata/userdata.parquet 'select count(*) from {}'
+----------+
| count(*) |
+----------+
|     1000 |
+-———+

Piping data to dsq

When piping data to dsq you need to set the -s flag and specify the file extension or MIME type.

For example:

$ cat testdata.csv | dsq -s csv "SELECT * FROM {} LIMIT 1"

Or:

$ cat testdata.parquet | dsq -s parquet "SELECT COUNT(1) FROM {}"

Multiple files and joins

License

This software is licensed under an Apache 2.0 license.

Resources & Downloads

• Source-code and download