If you are a reasonable person, you will know if a threat is real and imminent.
Sometimes, yes. At other times, it will be a matter of "reasonable belief."
Either way, that's just step one. Step two is producing evidence to form a basis for judging after the fact whether that belief was reasonable--i.e., whether reasonable people, knowing what you knew at the time, would have believed the same thing. Of course, the testimony of other parties will enter into that determination, and as rough and dangerous as some of those other parties may have appeared to you at the time, you can be assured that they will present an entirely different image in court, should it come to that.
As justified as drawing a gun may have seemed to the actor at the time, and as necessary the use of deadly force may have seemed had it gone that far, others may not see it the same way based on the evidence and testimony presented.
That's one risk. The other is that of sustaining injury or being killed in the confrontation itself. The OP encountered one obnoxious, aggressive person. Later he found that there were two. Could there have been more, coming up from behind?
Someone has posted a paper that I cannot find right now about avoiding trouble, the first strategy being non-attendance at gunfights. Where I live I probably would have looked out before leaving the store, or gone back inside at the first indication of trouble, and called 911. That would work in most places. In the nearby major city, or in the country, it might not, due to longer response times.