"Cartoony" is a synonym for "cartoonish." It can refer to graphics, but it can also refer to the the characters. Are they believable to any extent, or are they just an excuse for yet another tiresome joke? I think most people who object to "cartoony" games are objecting to the humor and unrealistic characters. Not that a character is a talking dog, but that a character that is supposedly a human acts like an exaggerated, annoying stereotype.
Since all games that are not FMV are pretty much hand drawn, I'm kind of wondering what is considered "cartoony" and what is not.
Compare Syberia with Randal's Monday or Ceville. Syberia is made to look fairly realistic, the other two are all exaggeration.
Sometimes it's difficult to tell from graphics alone if a game is "cartoony" if it uses pixel art. But there's no doubt that, for example, The Last Door is a horror game and not "cartoony." And I don't consider the Wadjet Eye games to be "cartoony" though they use pixel art.
Then there are games like thishttp://www.mobygames.com/game/dead-synchronicity-tomorrow-comes-today/screenshots
The graphics look like they might have been from a graphic novel. They aren't realistic at all. They are obviously exaggerated. But I would not call them "cartoony" either, since to me "cartoony" implies some form of humor.
If a game that is considered not Cartoony has too many mazes, timed events, and illogical and difficult puzzles, I would pick a cartoony one that is not so difficult.
Who says you have to choose either one?
Also, illogical puzzles are more apt to be found in "cartoony" games than in those with more realistic characters and graphics.