On default, yt-dlp downloads a rather low quality version and if you use --list-formats, most of them are either video only or audio only.
By using
yt-dlp -S vcodec:h264,res,acodec:m4ayou’ll get high quality mp4s.
I have it aliased to alias y=yt-dlp -S vcodec:h264,res,acodec:m4a so I don’t have to remember it.