so should I say or not say that it bothers/worries me that there's the sudden reappearance of his most recent ex?

I'd say, tell him if it's making you cranky, because he might not know what's bothering you, and then things could spiral.