How to enable reverse search in zsh

This is the documented behavior:

down-line-or-search
Move down a line in the buffer, or if already at the bottom line, search forward in the history for a line beginning with the first word in the buffer.

There doesn't seem to be an existing widget that does exactly what you want, so you'll have to make your own. Here's how to define a widget that behaves like up-line-or-search, but using the beginning of the line (up to the cursor) rather than the first word as search string. Not really tested, especially not on multi-line input.

up-line-or-search-prefix () {
  local CURSOR_before_search=$CURSOR
  zle up-line-or-search "$LBUFFER"
  CURSOR=$CURSOR_before_search
}
zle -N up-line-or-search-prefix

An alternate approach is to use history-beginning-search-backward, but only call it if the cursor is on the first line. Untested.

up-line-or-history-beginning-search () {
  if [[ -n $PREBUFFER ]]; then
    zle up-line-or-history
  else
    zle history-beginning-search-backward
  fi
}
zle -N up-line-or-history-beginning-search

You can run bindkey with no arguments to get a list of existing bindings, eg:

# Enter vi mode
chopper:~> bindkey -v

# Search for history key bindings
chopper:~> bindkey | fgrep history
"^[OA" up-line-or-history
"^[OB" down-line-or-history
"^[[A" up-line-or-history
"^[[B" down-line-or-history

In emacs mode, the binding you want is history-incremental-search-backward, but that is not bound by default in vi mode. To bind Ctrl-R yourself, you can run this command, or add it to your ~/.zshrc:

bindkey "^R" history-incremental-search-backward

The zshzle manpage (man zshzle) has more information on zsh's line editor, bindkey, and emacs/vi modes.