Announcing shellcheck-gpt

ShellCheck is an open-source tool that performs static analysis on shell scripts, identifying potential bugs and suggesting improvements. These suggestions are based on community-driven rules. While ShellCheck highlights areas for enhancement, implementing these changes can be challenging, especially for those not deeply familiar with shell scripting.

I'd like to introduce shellcheck-gpt, a tool that I built that combines ShellCheck's recommendations with the capabilities of a Large Language Model (LLM). With shellcheck-gpt, the process of applying ShellCheck's suggestions to your script is automated, simplifying the task of refining and improving your shell scripts.

Demo

In this demonstration, we show how shellcheck-gpt can automatically identify and correct issues in shell scripts.

Consider the following bash script which contains some subtle bugs:

#!/bin/sh

for f in $(ls *.m3u)
do
  grep -qi hq.*mp3 $f \
    && echo -e 'Playlist $f contains a HQ file in mp3 format'
done

Running this script through ShellCheck yields several suggestions for improvement:

In example/before.sh line 3:
for f in $(ls *.m3u)
         ^---------^ SC2045 (error): Iterating over ls output is fragile. Use globs.
              ^-- SC2035 (info): Use ./*glob* or -- *glob* so names with dashes won't become options.

In example/before.sh line 5:
  grep -qi hq.*mp3 $f \
           ^-----^ SC2062 (warning): Quote the grep pattern so the shell won't interpret it.
                   ^-- SC2086 (info): Double quote to prevent globbing and word splitting.

Did you mean:
  grep -qi hq.*mp3 "$f" \

... (some suggestions omitted)

Typically, addressing these suggestions would require manual intervention. However, with shellcheck-gpt, the process is automated. After running the script through shellcheck-gpt, the output is a revised script:

#!/bin/sh

for f in ./*.m3u
do
  grep -qi 'hq.*mp3' "$f" \
    && printf 'Playlist %s contains a HQ file in mp3 format\n' "$f"
done

Here, shellcheck-gpt has correctly applied ShellCheck's recommendations. You can verify its effectiveness by rerunning ShellCheck on this updated script and noting the absence of any further suggestions. This example showcases the practical utility of shellcheck-gpt in streamlining script correction.

Getting started

Prerequisites

To utilize shellcheck-gpt, there are a few preliminary steps you need to complete.

Firstly, ShellCheck, the core tool used by shellcheck-gpt, must be installed on your system. It's crucial that ShellCheck is accessible from any directory in your terminal, meaning it should be included in your system's PATH. You can find the specific installation guidelines for your operating system in the ShellCheck official installation instructions.

Secondly, since shellcheck-gpt leverages an OpenAI GPT model for inference, having an active OpenAI API key is required. To obtan an API key, you must first create an account with OpenAI, if you don't have one already. Once your account is set up, you can generate an API key here. Additionally, you should ensure that your billing settings in OpenAI have a positive credit balance, as using GPT models incurs a nominal fee.

Installation

Installing shellcheck-gpt is a straightforward process, though the steps differ slightly depending on your operating system.

MacOS & Linux

For MacOS and Linux users, the process is simplified through the use of Homebrew, a popular package manager. To install shellcheck-gpt using Homebrew, simply run the following command in your terminal:

$ brew install kkyr/tap/shellcheck-gpt

Other platforms

For those that do not or cannot use Homebrew, you have two options:

1. Download a pre-compiled binary tailored for your specific platform from the project's releases page.
2. Build the tool from its source using the Go toolchain. This option requires that Go is installed on your system. To build shellcheck-gpt, execute the following command:

$ go install github.com/kkyr/shellcheck-gpt

Verifying installation

After installing shellcheck-gpt, it's important to confirm that it's properly set up on your system. You can do this easily through your terminal. Just type the following command:

$ shellcheck-gpt -v

This command should display the currently installed version of shellcheck-gpt. If, instead of the version number, you encounter an error stating that the command is not found, it's likely that shellcheck-gpt wasn't installed in a directory included in your system's PATH. The steps to add it to your PATH varies depending on the method of installation you chose.

Usage

Preparing environment

Before using shellcheck-gpt, it's necessary to add your OpenAI API key into your working environment. You can accomplish this with the following command in your terminal:

$ export OPENAI_API_KEY=replace-with-your-key

This step needs to be repeated in each new terminal session, unless you add the command to your terminal's initialization script. If you're using bash, this is ~/.bashrc. This ensures that your OpenAI API key is automatically set every time you open a new terminal session.

Running shellcheck-gpt

You're now ready to use shellcheck-gpt. To start, simply run it in your terminal with your shell script as an argument:

$ shellcheck-gpt script.sh

When you execute this command, shellcheck-gpt will perform a series of actions:

1. Execute ShellCheck on your script to identify and gather improvement suggestions.
2. Supply these suggestions, along with the script's content, to a GPT model. The model is then tasked with rewriting the script, incorporating the suggested corrections.
3. Display the revised script content in your terminal.

With the revised script in your terminal, you now have the option to manually inspect it and modify the original script as needed.

If you'd like the tool to directly apply the recommended changes to your script, you can run the script with the -w flag. When you use this flag, shellcheck-gpt will overwrite the original file with the revised script. Be aware that this action will replace the existing contents of the file.

$ shellcheck-gpt -w script.sh

This streamlines the update process by directly applying the suggested modifications to your original script. However, since this action overwrites the existing file, it's recommended to run this command within a version-controlled repository. This way, if necessary, you can easily revert to previous version of your script.

Configuration

Model selection

To alter the specific GPT model that shellcheck-gpt utilizes, you can employ the -m flag. For instance, to switch to using the gpt-4-turbo model, your command would be:

$ shellcheck-gpt -m gpt-4-turbo script.sh

By default, if no model is specified, shellcheck-gpt uses the gpt-3.5-turbo model. To explore the range of available models, you can refer to the help documentation by executing the following command:

$ shellcheck-gpt --help

This command also provides you with information about other flags and configurations that are available.

Conclusion

In summary, this blog post highlighted the integration of ShellCheck with an LLM, resulting in a tool that can automatically detect and rectify potential problems in shell scripts. We covered the installation process, provided examples of its usage, and discussed how to customize it to your needs.

This tool is entirely open-source, and contributions from the community are highly encouraged. Whether you're interested in contributing or simply want to learn more, you can explore the project further at: https://github.com/kkyr/shellcheck-gpt.

Thanks for reading this far!