To the OP, good self-control and wisdom in the decisions. Everyone here has time to reflect and have their standing heart rates at the normal range when they heard your scenario.
If this individual had targeted you, he knows where you live and could come back. Legally, you met all the considerations and responsibilities of owning a weapon. Being on your own property would help justify a mortality or usage of the weapon.
Police times are similar to ours. Unless you are actively using a weapon, the police probably will not show up under 30 minutes regardless of how threatened you may feel. However, times will be shortened considerably if you say your young child may be harmed or killed. This is a tactic my mother, as a principal of an elementary school, uses when police are needed. The liability of a child actually having harm done to them doesn't outweigh the potential misinformation. Police love saving children from harm, as most of us do.
Giving the individual commands that he can understand and follow show that you are the one in control of the situation. Shows some serious "level-headedness" on your part.
Shaking hands was a good idea. Ending things with a conclusion such as this is best for the safety of your family.
I've carried 11 years. Pulled my weapon twice: one situation involved a man and a minor female having an attempted sexual assault being committed and the second one involved a truck driver running me off the road that had also thrown beer bottles at my vehicle (felony - deadly missiles) and was under the influence of drugs/alcohol.
I disagree with some of the macho statements. There is nothing more terrifying than ending someone's life and existence based upon your judgment call. This is something you must rationalize and live with for the rest of your life, not to mention the many possible negative outcomes that may be associated with it.
From my experience, speed bumps and low speed limits have caused way too many scenarios of road rage. I tend to care about my vehicles, as I assume you do, and slow down for them plus drive the speed limit posted. Many people feel that this is a personal slight being done to them by you because they are impatient, thus resulting in behavior unrelated to their normal personality. I wish this would be taken into account when municipalities install them.