I'm sorry to hear you are struggling with this. I think it is very good that you are trying to overcome this temptation. I would say that there are a few things I would want to share with you about anger. The first is that I think it can be addictive. There is to my knowledge some dopamine cycle that forms that it is really good to be free from. Being aware of that may assist you, remember it is your freedom that you are fighting for.
Secondly, what has been the most helpful for me to avoid anger is to realize that my knowledge of a situation is likely limited. Or at the very least, that if I wish to communicate with people in a way that gets me what I want, I have to acknowledge that communicating in terms of absolutes may cause the other person to respond in a way that is not what I would like. For instance, I think it is better to say in an argument ``When you do this ... it makes me feel ...'', or ''my opinion about ... is that it is ''. This may not seem important but I've noticed it produces a very different effect than saying things like "you are ..., that is ..." (in the sense of absolutes). Through practicing this, I've come to be more skeptical of thinking that someone really is what I assume they are, and more likely to give them the benefit of the doubt or to assume it is likely they are in fact trying to do the right thing (but that I perhaps cannot understand how). There's a book I've read called Nonviolent Communication: A Language of Life that has assisted me with this - your mileage may vary. Wishing you the best and Shavua Tov. I think it is very good what you are doing and I think Hashem will reward you with many blessings for trying to overcome it
Lastly, I'd recommend remembering Hashem is in control and will safeguard you from the dangerous things you are worried about. I find it interesting you mention viewing this lady as a possible cause, which I take you to be implicitly concerned about as a possible kind of idolatry. I personally struggle with feelings of idolatrous lust towards women. I find it interesting that many men seem to have idolatrous feelings towards females. I think it is great that you are aware of this. What has helped me the most in my struggle is recognizing that Hashem is my only G-d (first commandment). It was not obvious to me how that commandment is so foundational but I think both your example and my example is able to serve as an example of why it is very important to keep Hashem as the only G-d
