Credit Card Default and Creditor Harassment?

I took out a $1000 limit credit card over a year ago that I took out with the credit union at my university. I subsequently spent a few months oversees and unemployed, and was not able to make payments for some months. Upon returning, I received a note that my card was in default, and I agreed to terms with a representative that I would make a monthly payment of $50. I have made one payment since our agreement a month ago. However, I have been told twice now that my housemates have answered the door to a man asking to speak with me and saying I have a court date. They asked what for, and for any official documentation, but he said he was in a hurry and left both times. I was not home either of those times. I am worried that this is something that has to do with my debt, but I am making payments and have called my representative and let her know via voicemail that she can expect payment for the end of this month. I am beginning to panic, when I described the house visits to my parents, I was told that it is harassment, and that he has no right to come to my home asking for me. I am spending more time outside of my home and at my parents' house because of this. I am not sure how to proceed, but there is nothing else I have done wrong that would warrant such a visit. Is this legal? And how should I proceed?