Do you find your partner attractive? And are you turned on by watching sex? And, in your fantasy world where you have this idea, you get to decide who the person is so you can choose someone who's not threatening to you and/or your partner isn't running off and leaving you for that person? Then it makes a lot of sense that the idea might be arousing.
Of course, these aren't the only reasons why some people enjoy fantasising about their partner being with other people. Some people are just intro group sex and they think about watching their partner do it with other people. Sometimes lots of other people. Sometimes for money. Sometimes in sex clubs. Sometimes with friends. And of course some folks are just swingers. They like sex. They like new partners. And they are happy to share their partner assuming their partner wants to be shared.
Other times people are into humiliation and that "other partner" isn't just any partner, and the scene isn't just any scene, but it's one that makes the fantasiser feel bad about himself or herself in ways that feel sexually exciting. People who go this route may have fantasies about their partner cheating on them fairly openly and making a cuckold out of them.
Other people feel powerful in their fantasies; it's the opposite of humilation. Their partner feels like their property who they can "loan out" to others. And if your partner is a willing participant in what turns you on, some people are happy to consent to being loaned out.
In short, there are lots of reasons why people have less-than-vanilla fantasies that turn them on. Check out the book Insatiable Wives for a bit on the cuckoldry scene and Secret Sex Lives for a peek inside diverse sex lives. Keep in mind that fantasies can stay fantasies. Just because you like to think about something doesn't mean you want to go out and do it. And if you do want to enact it, it goes without saying (I hope) that your partner doesn't have to sign up for it. But if you're both interested and willing? The world is your oyster.
Dr. Debby Herbenick, author of Sex Made Easy and Great In Bed, is the Co-Director of the Center for Sexual Health Promotion in the School of Public Health-Bloomington at Indiana University (IU) where she is a Research Scientist. She is also a sexual health educator at The Kinsey Institute for Research in Sex, Gender and Reproduction where she writes (and hosts audio podcasts of) the Kinsey Confidential column and coordinates educational programming. She has a PhD in Health Behavior from IU, a Master's degree in Public Health Education (also from IU) and a bachelor's degree in psychology from the University of Maryland, College Park. In addition, she is certified as a Sexuality Educator from the American Association of Sex Educators, Counsellors and Therapists.