A common canonical rule in horror, or if you’re incredibly spiritual & superstitious, is that a monster must be invited in to cause any real damage to you. In classic literature and occult media, vampires must be invited to cross the threshold of a home and demons must be given permission to enter an object or take possession of body. In much the same way, Victor invited his monster into existence by creating him in the first place. And this is first and foremost how horror begins to befall our rather dim-witted main characters.
Frankenstein cannot admit his sins and thus remains silent in the creature’s reign of terror. He goes back and forth blaming himself and then the creature, but are they not the same? The creature was a monster by sight, but Frankenstein was a monster by soul.