My Aunt had to put her 14 year old Lab mix Sheba, down a year ago and it was not peaceful. Once the drug was injected she started crying and whimpering.
She had to leave the room; she could still hear her in the lobby, so she had to leave the vets office.

In Sheba's case, she was full of tumors and had bad circulation, making it difficult for the drug to take effect instantly.
More times than not it is instant.

I'm so sorry you had to go through that.
Even if his passing seemed not to be peaceful, please take comfort in knowing that he is now at peace.