how do i handle this after all these years?

many years ago when i was young ,stupid and naive i was in love with a man that was in the armed forces. he switched religions while stationed overseas. he never told me how he felt about me and i never asked him. we exchanged letters all the time. he would come home for about 2 weeks every so often and we would visit for a few days. he came home on one visit and he gave me information on the religion that he had converted to and told me to read it which i did. i chose not to convert. it was never mentioned again. i met a guy at school ( i was still in high school at the time). he did not know about this guy until he came back home on a visit. by this time this guy from school and i were engaged. my fiancee told me after he met the other guy that i had to make a choice. i could either remain friends with him or i could marry my fiance. i called my friend up and said that i couldn't be his friend anymore that i was engaged to be married and my fiance didn't like my friendship with him. of course being the gentleman that he was he has held to that. he got married and had a family of his own. my fiance and i got married and we had a family. all these years i have felt very guilty how i handled this situation and now realize that i had other choices that i could have made at the time. i have my ex-friends address and phone number. should i contact him and let him know that i was stupid to have let our friendship end the way i did and that it has bothered me all these years or should i let bygones be bygones?