Was I Wrong for This?

On Valentine's Day I went out with a guy I'd been talking to for a few weeks, and we'd had two previous dates that went smoothly. The date itself was great. The guy spent about 95$ on dinner ($40 per person plus drinks!) So then we went back to his apartment; I knew I shouldn't have gone in, obviously he was expecting something. We were watching a movie, and he was rubbing my arm, kind of like building up to other things. Then he tried to put his hand in my bra, and that's when I told him I had to go. I felt bad, but at the same time, I was not comfortable just laying it all out on the bed that night. Naturally I haven't heard from him since, nor do I really care, but was I wrong for not putting out? Girls and guys opinions appreciated, please.