When a man looks at a woman at the beach or on the street, that’s all he is doing. Looking. He won’t remember her later, he will not act upon his (momentary) interest, and he does not compare his wife or girlfriend to her.
Some things men do are not worth women thinking about. If your man is of good character, this is one of them.