<< Odd, I don't remember any reference to the housemate/servant's race in the story, nor anything that would suggest racism.

As I said the racism is latent, a product of the times in which the story was written. Sam is utilized and depicted in the Woolrich story much the same way blacks were used in other literature and movies of the period. But getting back to the original point, I think you'd have to go pretty far to find a suggested homosexual relationship between Sam and his boss, regardless of Woolrich's own sexuality.