As this game is on the E drive I know that I can just hit delete to remove those files. But will this remove it completely?
No. Anything outside the main file folder will be left behind, including not only registry entries, but also saved games and/or game settings that were stored in the hidden AppData folder on the C: drive. And if you have "Steam Cloud" active anything the Steam server has collected will remain there.
Is there some reason you can't remove the game using the Programs icon in Control Panel?
Or use Steam Client to remove the game.
Open Steam Client and log in
Library tab
Right-click game and choose Uninstall.
Confirm the uninstall.