Yes, it's been canceled. A note was posted on the Her Interactive board a few months ago from a HER employee saying that the project has been shelved.
Here's the relevant part of the note that was posted on the HER forums:
Ship of Shadows has been in development for a while, and though you haven’t seen many updates on it, for quite some time there was a good deal of progress pushing for completion. After the feedback we received from fans regarding the first two Dossier games, we wanted to ensure that we added in all the right elements to make the best the series had to offer. This caused an initial delay in the release date. As time pushed ever forward, the Dossier series kept getting held up to make way for new projects and releases. One such release was the latest series, Nancy Drew Mobile Mysteries. Now before you say anything, Mobile Mysteries, was not intended to replace or halt the production of the Dossier series. We strive to provide quality mystery entertainment to fans everywhere, and on every platform (like the Wii or the DS). To be able to do so, we’re constantly re-inventing ourselves and producing the kind of games we know our fans love. The consequence of so many delays to a single project signaled its untimely removal from our launch schedule. Sadly, for the time being, Nancy Drew Dossier: Ship of Shadows is on ‘the bench,’ waiting for an eager team to come finish it. There are no known plans for release or re-initialization of the series, however… the future is always full of possibility.