Granted everyone is certainly different and needs a different approach. Some people are coming from one place and some from another. But here I go, nonetheless:
As far as the cleaning lady goes, the question is why (in the world) are you making a big deal out of her? If you had a really tough time, and knew a new cleaning lady is coming, I am gonna be prejudicial here and suspect that you were 'prepared' to check her out - then to resist her should she turn out to be a pretty woman.
This is fear-driven behavior. It is typical of the religious (and failing for some people) approach. It prepares us for the moment of issur: Lo sasuru - gevalt! What if she is hot?! Whatever am I going to do!? Is Hashem strong enough to help? ...I wonder....
Does He love me enough to help me? ...I doubt it...
Then what about the next time?? ....probably then I'll plotz!
So why bother?
No! That's not frum! I am yid and I will fight!!
Squeak!
Plop.
OK. So maybe there is another way. Maybe there are ten other ways - certainly the same old losing way is a dumb choice to make...good intentions ("l'Shem Shomayim") are no excuse for stupidity.
So. In Mesilas Yeshorim it's written that the peasant has no desire for the princess, for he cannot imagine that she would ever be interested in him. It's just not shayach! So the desire itself - which is from the body - never gets started.
I suggest to you that you would not have such a hard time with her if you would not be all worried about her, afraid of her (and her breasts, her face, her shape, her voice, and all her other 'powers') AND if you had no shaychus to getting physical with her. I would guess that you actually do harbor a desire to get sexual with her - a fantasy. But I bet it is in your psyche hiding somewhere, hence she is seen by your body-sense as reachable. Hence, she is 'such a tremendous challenge', as you put it.
Admit it, and then let go of her. Admit it to a safe friend. Don't walk around sounding frum and party-line, if you are not really there - it gets you no where, cuz it's sheker, and Hashem isn't with sheker.
If it is true, then consider admitting clearly and awkwardly that you are muchan and mesugal to use her attributes to fantasize, and that you do lust after her, and that somewhere in you there is a broken wall - an actual consideration of behaviors that would have terribly destructive consequences. That does not mean you are an addict or a pervert at all, but it does mean something is a bit goofy with you.
Admit it so that you can then surrender it - let it go.
If you want to hold onto the fantasies and silly dreams and keep suffering with lusts that you cannot have without paying terrible prices, then gezunderheit. I am not the one who will try to convince you not to - let the frummies do that.
But if you are really ready to let them go, then just make up your mind about it and say to your trusted friend (and to Hashem after that) that you are ready to let her and your lust for her go - to let the 'opportunities' in the fantasies of your heart go. Let them dissolve - whatever happens to them when we ket them go...who knows, who cares? Just drop them, if you want to. Give them up.
That is what we are meaning when we say, don't fight the desires by holding onto them and trying to beat them while you have them - give them up. It is a totally different derech.
Sound OK?
And if you have any fear of this woman and what she might do to you, I suggest that such a feeling is not yir'as Shomayim, nor yir'as cheit. It is yir'as bosor v'dom....literally. By taking the typical hashkofah of the ignorant, you are giving these women tremendous power! You are investing them with god-like strength - to make you shudder and breath shallow and hope and pray you are spared from them...gevalt. This is not the way.
They are irrelevant, and not yours nor mine, at all. Not shayach to us. They are like peasants and we are royalty - there is really no connection. Hashem can help us stay unconnected to them if we stay connected to the truth about ourselves and stay connected properly to others by telling them the truth about ourselves openly!! Choose safe people for this...but do it.
We are only as sick as our secrets.