The Myst games have been hit and miss on Windows 7 and Windows 8 (both operating systems have separate threads stickied at the top of the Glitches forum).
If you have tried to install Myst III Exile normally and it won't install, try navigating to the setup file on the disk and installing using compatibility mode. If it installs, but won't play, try right clicking on the game icon and click on Troubleshoot Compatibility. You may have to try several different operating systems and sometimes the one you least expect to work, will
And some of these games simply will not play no matter what you try. One other thing to try if it it installs, but won't play, is to right click on the game icon and select "Run as Administrator."