Shell History

Inspired by bamos/zsh-history-analysis.

Visualize your usage of Bash/Zsh through a web appthanks to Flask and Highcharts!

Post your charts ideas in this issue!

• Requirements
• Installation
• Setup
• Usage
• Some technical info
  • How it works
  • History file format
  • How we get the values
• License

Requirements

Shell History requires Python 3.6 or above.

# install pyenv
git clone https://github.com/pyenv/pyenv ~/.pyenv

# setup pyenv (you should also put these three lines in .bashrc or similar)
export PATH="${HOME}/.pyenv/bin:${PATH}"
export PYENV_ROOT="${HOME}/.pyenv"
eval "$(pyenv init -)"

# install Python 3.6
pyenv install 3.6.12

# make it available globally
pyenv global system 3.6.12

Installation

With pip:

python3.6 -m pip install shellhistory

With pipx:

python3.6 -m pip install --user pipx

pipx install --python python3.6 shellhistory

Setup

shellhistory needs a lot of info to be able to display various charts.The basic shell history is not enough. In order to generate the necessaryinformation, you have to enable the shell extension.

At shell startup, in .bashrc or .zshrc, put the following:

# only load it for interactive shells
if [[ $- == *i* ]] && command -v shellhistory-location &>/dev/null; then
    . $(shellhistory-location)
    shellhistory enable
fi

... and now use your shell normally!

If you want to stop shellhistory, simply run shellhistory disable.

Note: for performance reasons, you can also use the static,absolute path to the source file.Indeed, calling shellhistory-location spawns a Python processwhich can slow down your shell startup.Get the path once with shellhistory-location, and use . <ABS_PATH>.In my case it's . ~/.local/pipx/venvs/shellhistory/lib/python3.6/site-packages/shellhistory/shellhistory.sh.

Usage

Launch the web app with shellhistory-web.Now go to http://localhost:5000/ and enjoy!

You will need Internet connection since assets are not bundled.

Some technical info

How it works

When you enter a command, shellhistory will compute valuesbefore and after the command execution.In Bash, it uses a trap on DEBUG and the PROMPT_COMMAND variable(man bash for more information).For Zsh, it uses the preexec_functions and precmd_functions arrays(anyone knows where to find the official documentation for these?Some information in man zshmisc).

Before the command is executed, we start a timer, compute the command type,and store the current working directory and the command itself.

After the command has finished, we store the return code, and stop the timer.

History file format

Fields saved along commands are start and stop timestamps, hostname, username,uuid (generated), tty, process' parents, shell, shell level, command type,return code, and working directory (path), in the following format::start:stop:uuid:parents:host:user:tty:path��level:type:code:command.

• multi-line commands are prepended with a semi-colon ; instead of a colon :,starting at second line
• start and stop timestamps are in microseconds since epoch
• process' parents and working directory are encoded in base64 to avoiddelimiter corruption.

Example (multi-line command):

:1510588139930150:1510588139936608:40701d9b-1807-4a3e-994b-dde68692aa14:L2Jpbi9iYXNoCi91c3IvYmluL3B5dGhvbiAvdXNyL2Jpbi94LXRlcm1pbmFsLWVtdWxhdG9yCi91c3IvYmluL29wZW5ib3ggLS1zdGFydHVwIC91c3IvbGliL3g4Nl82NC1saW51eC1nbnUvb3BlbmJveC1hdXRvc3RhcnQgT1BFTkJPWApsaWdodGRtIC0tc2Vzc2lvbi1jaGlsZCAxMiAyMQovdXNyL3NiaW4vbGlnaHRkbQovc2Jpbi9pbml0Cg==:myhost:pawamoy:/dev/pts/1:L21lZGlhL3Bhd2Ftb3kvRGF0YS9naXQvc2hlbGxoaXN0Cg==:/bin/bash:1:builtin:0:echo "a
;b
;c" | wc -c

Note: later we could use CSV formatting, quotingstrings and doubling double-quotes in those if any.It would make the file more readable for humans,and easily importable in other programs.See issue 26.

The previous example would look like this:

1510588139930150,1510588139936608,40701d9b-1807-4a3e-994b-dde68692aa14,"/bin/bash
/usr/bin/python /usr/bin/x-terminal-emulator
/usr/bin/openbox --startup /usr/lib/x86_64-linux-gnu/openbox-autostart OPENBOX
lightdm --session-child 12 21
/usr/sbin/lightdm
/sbin/init",myhost,pawamoy,/dev/pts/1,"/media/pawamoy/Data/git/shellhist",/bin/bash,1,builtin,0,"echo ""a
b
c"" | wc -c"

How we get the values

Start and stop time are obtained with date '+%s%N', return code is passeddirectly with $?, working directory is obtained with $PWD and commandtype with type for Bash and whence for Zsh.

Values for UUID, parents, hostname, and TTY are computed only once, whenshellhistory.sh is sourced. Indeed they do not change during usage of the currentshell process. Hostname and TTY are obtained through commands hostname andtty. UUID is generated with command uuidgen. Also note that UUIDis exported in subshells so we know which shell is a subprocess of another, andso we are able to group shell processes by "sessions", a session being an openedterminal (be it a tab, window, pane or else). Parents are obtained with afunction that iteratively greps ps result with PIDs (see shellhistory.sh).

Values for user, shell, and level are simply obtained through environmentvariables: $USER, $SHELL (though its use here is incorrect:see issue 24),and $SHLVL (also see issue 25).

The last command is obtained with the command fc.Using fc allows shellhistory to have the same behavior as your history:

• if commands starting with spaces are ignored, they will be ignoredin shellhistory as well.
• same for duplicates (entering ls two or more timessaves only the first instance). Note however that if you type the same commandas the previous one in an other terminal, it will still be appended,unless you manage to synchronize your history between terminals,which is another story.

Additionally, if you enter an empty line,or hit Control-C before enter, nothing will be appended either.The trick behind this is to check the command number in the current history(see shellhistory.sh for technical details).