Lose the ZSH history

command historymacosoh-my-zshzsh

Every once in a while I discover my zsh history has been truncated (or maybe lost entirely, difficult to tell), and I have to restore it from backup.

For example, today:

ls -lh ~/.zsh_history
-rw-------  1 stripe  staff    32K 21 Feb 10:20 /Users/stripe/.zsh_history

But in my backup from a few days ago:

-rw-------@  1 stripe  staff  203K 17 Feb 22:36 /Volumes/Time Machine Backups/.../Users/stripe/.zsh_history

I have configured zsh to save lots of history so it shouldn't be a matter of the shell intentionally trimming the file.

unsetopt share_history
setopt inc_append_history
setopt hist_ignore_all_dups
HISTSIZE=500000
SAVEHIST=$HISTSIZE

Has anyone else experienced this and found a way to mitigate it? Is there some zsh spring cleaning feature I'm unaware of?

Best Answer

ZSH

History file can be truncated/lost/cleaned for multiple reasons those can be:

  • Corruption of the zsh history file (because of a power-cut/system-fail while a shell is opened, in this case fsck need to be setup to run when the system fail)
  • Zsh config file is not loaded (for example if $HOME env variable is not defined)
  • Unsupported character on history file can make zsh reset the history
  • Cleaning tools like bleachbit
  • Zsh misconfiguration
  • etc.

Notes

History available setup options

HISTFILE="$HOME/.zsh_history"
HISTSIZE=500000
SAVEHIST=500000
setopt BANG_HIST                 # Treat the '!' character specially during expansion.
setopt EXTENDED_HISTORY          # Write the history file in the ":start:elapsed;command" format.
setopt INC_APPEND_HISTORY        # Write to the history file immediately, not when the shell exits.
setopt SHARE_HISTORY             # Share history between all sessions.
setopt HIST_EXPIRE_DUPS_FIRST    # Expire duplicate entries first when trimming history.
setopt HIST_IGNORE_DUPS          # Don't record an entry that was just recorded again.
setopt HIST_IGNORE_ALL_DUPS      # Delete old recorded entry if new entry is a duplicate.
setopt HIST_FIND_NO_DUPS         # Do not display a line previously found.
setopt HIST_IGNORE_SPACE         # Don't record an entry starting with a space.
setopt HIST_SAVE_NO_DUPS         # Don't write duplicate entries in the history file.
setopt HIST_REDUCE_BLANKS        # Remove superfluous blanks before recording entry.
setopt HIST_VERIFY               # Don't execute immediately upon history expansion.
setopt HIST_BEEP                 # Beep when accessing nonexistent history.

Answer

The following configuration is recommended for this situation (to be setup on ~/.zshrc file)

HISTFILE=/specify/a/fixed/and/different/location/.history
HISTSIZE=500000
SAVEHIST=500000
setopt appendhistory
setopt INC_APPEND_HISTORY  
setopt SHARE_HISTORY

Alternative

You can use a little script that check the history file size and do restore it from backup when necessary (in ~/.zshrc)

if [ /home/my/zsh/hist/file -lt 64000 ]; then
    echo "History file is lower than 64 kbytes, restoring backup..."
    cp -f /mybackup/histfile /home/my/zsh/hist/file
fi

Links

Additional infos are available on this and this questions.

Related Question