'who'd stoop to blame' - the line shows the Duke's male entitled pride : he wont lower himself to talk to his wife and confront her about what he fears, as it would show him as paranoid and weak, further it also suggests there's no point arguing with a women as she is so far below him, and incapable of rational thinking, and he shouldn't have to explain it to her - she should be meek, obedient and submissive