When you don't have a .vimrc file, Vim will load a defaults.vim file from the Vim runtime directory. The defaults.vim file shipped with Vim will enable options to make it more useful by default. Among the settings in defaults.vim, syntax highlighting and filetype indent plug-ins are enabled.
That's why you see such a big change when you create a very minimal .vimrc, as Vim will stop setting all the other settings from defaults.vim.
In order to preserve the options from defaults.vim, but revert its setting enabling filetype indent plug-ins, you might want to update your .vimrc to:
unlet! skip_defaults_vim
source $VIMRUNTIME/defaults.vim
filetype indent off
By explicitly sourcing the defaults.vim file, you'll keep all other settings from it as you introduce your own in your .vimrc.
See also:
:help defaults.vim- Contents of
defaults.vimshipped with Vim.