That link is for Windows Live Hotmail, but some of the directions will work...
For the Windows Live Mail email client, I've found you have to set the reminders by clicking on File, New and then calendar event. The next screen has an option to set a reminder and when you want the reminder sent to you. The options are from immediately to 5 minutes to 2 weeks.