I don't spend the weekends with my boyfriend

Check this out!

I have been dating this guy for about 3 and a half months and I have only spent two weekends with him. I only see him during the week. For the weekends he always has an excuse for us not to meet but eager to see me during the week. He comes over to my house from work and spends time but from Friday till Sunday he's no show. Should I confront him about it or just leave him alone? Also, I have never spent the night in his house before, maybe I am the side chick?