Im now with a man that just finally divorced but i new him before then and we got really really close?

we've tried so many times to stay away from eachother but we seem to find our way back to eachother.around this time they were seperated and now he's finally divorced. i am not the cause of it because it was already going bad before i came along.she dislikes me because i knew him almost around the same time. so now we're staying together and lately she been stopping by unannounced. he cant stop talking to her because they have a child together i don't have a problem with that.its just the fact that she souppose to call him before stopping by.she wants to find out who he with now but she doesn't know that its me all she know is that he's with somebody else. but why she stopped by did not have to do with their child.should i just let it go?and not have a problem at all?i know they'll be seeing eachother for the sake of the child but other than that i dont think she should be stopping by whenever she feels like it. am i wrong to think this way because he thinks it is.