I accidently called my X Boyfriends brother - and he overheard me with guys! ...?

You do have the right to go out with guys since your single, but you two have just broken up. Maybe he feels differently about the break up than you do and he was probably offended and embarrassed. Especially since you two dated for so long.