The movie didn't work for me either. It reminded me alot of Ralph Baskshi's Cool World, another film that held tremendous promise yet left me cold. The Idea of a live actor interacting with animated characters in an animated landscape is really fantastic and that both of these films had a dark take on this concept was even more fantastic to me.
Less fantastic was Brendan Fraser and Gabriel Byrne looking beleivable as they interacted with the animated characters in their respective films, neither one of them could completely pull it off. So far for me the best actor with a toon is still Bob Hoskins in Who Framed Rodger Rabbit back in 1988.