I had two pet ferrets (thelma and louise) - they need loads of attention and time (which i willingly gave) and are just so cute that i would have so much fun playing with them in the garden or inside the house when it was raining or cold.
I just found the experience so much more satisfying than sitting in front of the TV. But my husband was rather jealous of my ferrets and like your boyfriend complained because i wasn't sitting beside him on the sofa to watch some boring tv program and my friends also complained that i was always rushing home at a certain hour because the ferrets needed to be fed, cleaned etc....
But eventually they all learned to live with it.
It's just possessiveness..