The actual password(s) are stored on the site(s) you visit. Remember the registration forms you filled out? Remember you had, in most cases, to enter a password, amongst other things? So deleting Temporary files doesn't delete your passwords, just the information obtained from a site that helps that sites' pages load quicker.
If something gets your goat, it just proves you have a goat to get.