I had a right miserable old curmudgeon on my local water. The first time we met went badly as I caught him climbing over a fence then he just shouted "ticket" at me when he appeared in my swim. I told him he couldn't be a bailiff as I had seen him climb the fence and asked to see proof. He said he'd left it in the car and I told I wasn't showing it to him until he produced it. He stormed off and turned up 10 minutes later with it and then demanded to do a rig check. He also let slip into the conversion that he had a shotgun, like you do.
I wrote a letter of complaint to the Chairman and did get a call from the head bailiff a month later but not an apology. He did mellow a bit but was still a bit of a Hitler to the general public on the lake who got lost and was often heard shouting across.
Luckily he's gone now and the new bailiff is a really nice old boy who you'd happily make a brew.
I would definitely make a complaint and list all the problems you've had with him in a calm and logical manner. They would have to look into it and confront him and perhaps he is just oblivious to the chaos he causes. If you do nothing, it will continue as no-one will know anything is wrong.