Prevent interactive ssh prompts

I have an ssh script that uses a local key for login to the remote host - nothing too exciting there. The key is passworded and I usually add it to an agent to avoid prompting. Occasionally I run the program before the agent is running and it will hang waiting for the unlock phrase. In such cases, rather than prompt interactively, I want the command to simply fail. Anyone know if there's an option for this?