Is he just using me????? :(?

Okay so i recently (almost a month ago) started dating a guy i really like. and at first he was really sweet but soon we both started making lots of sex jokes and over the course of the relationship it seems like it recently just started revolving around sex. but it really does seem like he wants us to be together for a long time but in the present it seems like all he wants is sex...we tried it a week ago on Sunday, but we couldn't because he couldn't get hard enough (he is a virgin) so instead i just gave him a B.J and we were both happy until about 20 minutes before he had to go because he didn't take his medication( he is clinically depressed and on meds) and he just wasn't happy. and usually we don't talk much unless it's in person so like whenever we aren't together we don't talk at all...but now its like he wont ever respond to my texts asking if he wants to hang out or anything and i'm worried he only wants me so he can loose his virginity :( please help! what should i do? break it off or stay with him??