When in my zsh / oh-my-zsh setup I write something, then press up/down arrow keys, it searches history for only lines starting like that.

It’s a oh-my-zsh plugin but also here https://github.com/zsh-users/zsh-history-substring-search

In Bash

add this to ~/.inputrc:

bind '"\e[A": history-search-backward'
bind '"\e[B": history-search-forward'