I know I'm a bit late out the gate, and I hope my suggestion isn't too obvious, but here goes:
I think it might be helpful to try writing him a letter (without the intention of necessarily sending it) in which you can express anything and everything that comes to mind that you need to get out about what he did to you, your relationship before that, and how you feel about him and the abuse now. If I am correctly understanding your question, you haven't seen him since you were a child and a very recent victim of his abuse, both of which limited your agency to express your feelings. Letters are great in that you can say everything you need to without being interrupted or hurt by the other person's response. At the very least, the letter will help you organize your feelings and look at them a bit more objectively than you can when you're just experiencing them- feelings tend to come and go and even conflict, so if you let yourself get everything out in the open it might clarify things and make the decision easier. There is also a possibility that the letter will give you enough closure on its own to resolve your conflicted desire to see him. It may also be difficult and upsetting to the point that you will find that you don't think you can say those things to his face, and that's okay too- if you need to say them but not see him you can send the letter.
Regardless of what you eventually decide, it is important that you remember to put your needs first. He has proven that he is unable or unwilling to prioritize your needs, even when that was his entire job as your parent; it is absolutely correct that your only deciding factor here is what YOU need in order to be as comfortable and healed as is possible for you at this moment. And it is okay to let your family know that they need to support your decision and not make you feel guilty or beholden in any way- that obligation was severed when your father broke his most important responsibility toward you, which is, at minimum, to do no harm. Again, stating the obvious, but if you do decide to see him you needn't feel that that must necessitate you forgiving him. I also absolutely want to endorse Bion Satir's suggestion of a therapist to talk to about your experiences- oftentimes the most upsetting and difficult things in our lives are those which we feel afraid or simply unable to tell our friends and which our families cannot support us through with integrity because of their own involvement.
[Trigger warning: sexual assault] I can't answer the question of what I would do if it was me. Abuse by a parent is a completely different thing than sexual assault by a non-family member. But, if I had the opportunity to safely confront my attacker, I hope that I would be able to say the things that I could not say when he took my safety away. I hope I would be able to say the things I've been biting back every time I see a movie with a rape scene or hear a rape joke in a bar or am cat-called by a stranger on the bus or the street or wake up shaking and sweating at three in the morning or catch a leering look from a man or hear a formerly harmless phrase. He's the one who changed all of those things forever and irrevocably. I hope that, for myself, I would be able to finally break the silence I couldn't break then. Even if it was just to scream and scream at him. But that hope cannot answer your question of what might be right for you. I do hope that you find an answer for yourself that offers some comfort.