Shalom Brother,
I think you are touching on a lot of points, and I'm not sure if I can formulate a response to everything. I'll try to share my thoughts and hopefully some things will fit. (If you are looking for wiser folks, check PY's list of mentors above).
(you wrote 'good' / 'natural' lust and 'bad' lust -) For the sake of clarity, I'm going to use desire as the potentially healthy feeling a man has for their wife. Lust is more of a superficial craving, focused specifically on self-gratification, which even for one's wife, is often negative. As a "normal" human being, I feel both. At this time I don't find myself capable of not feeling any lust at all, and I'm ok with that, as long as I'm aware that it's not the ideal and the goal is to focus on connection and to desire that connection both emotionally and physically. Over time the lust goes down and the real connection gets stronger. When lust is no longer the main driver behind my desire, its effect seems to be much lessened.
Hashem created both lust and desire, as you mentioned. I don't know if this is what you were saying, but the idea isn't to say that these things aren't b'etzem attractive, because Hasem made them so. Like a steak with butter isn't objectively disgusting, it is subjectively not for us Jews because Hashem said 'no' - which makes it "disgusting" for us, but not bad in and of itself. Lust is a bit different, actually a lot different, because it tends to be damaging for everyone, whether technically prohibited or not. But still, I think you can understand that the pull towards these things comes from a normal part of us, and the answer to them is to see them as bad for us, and that's how you get to saying 'no' (fleeing). To say that all women besides my wife aren't attractive won't work to "beat" the lust, because it's not true. They are attractive, but they are not for me to act on that, and I must leave them alone. The easiest (not so easy) way to do that is to start with avoidance and then say 'no' when the steak and butter restaurant smells so tantalizingly good, but I'm not going inside to order one, not because it's awful (though it might be), but because it's awful for me. If anything, I think this is easier to understand by lust than by nonkosher food, because there is logic that can be applied to explain the damages from unbridled lust, though the YH tries to hijack our brain and temporarily block out what we know in order to trip us into taking what we want in the moment.
Desire is not the core issue, but the way we've misused it - mostly unintentionally at first, not understanding fully what we are doing and the real ramifications. Once we have crossed the lines between desire and lust, we have started to take, and we begin to feel like we need or deserve to have it. Once it becomes a "fix" then we start to use it as such. A response to stress, to desire both healthy and not, to pain, to loneliness, to boredom. It starts to fill all the voids and answer all the problems (though it doesn't - it just temporarily attempts to distract and delay them). To borrow a quote from Richtig's signature (miss your deep insights Brother and hope you are well), "Pornography is a bad answer to a good question" - R' Kalish. We have valid questions, and the voids and confusion and pain are too real, but we don't know how to answer them, and insert lust because we've taught ourselves to use it that way (often not deliberately).
The next part really deserves more arichus, but if a person says they cannot do it, either they have unrealistic expectations that are likely hurting more than helping them to make progress where needed, or they takkeh haven't reached yet the relationship with Hashem to accept that some of the pain comes from him (perhaps not the self-inflicted pain that we pile on with guilt and shame), though it is always for our betterment in some way. We also need to believe always that we have the power through Hashem. I wrote about this recently - the importance of recognizing what is outside of our control and comes from Hashem, and what is within our domain to control, of course with help from Hashem, but it starts with our hishtadlus and emunah that we can do it. If we don't believe in ourselves, that there is a way, that makes it a lot harder to keep pushing.
There seems to be many more aspects to your post that I didn't touch on, and that might be worth a nice shmooze with a mentor / Rav.
Hatzlacha Brother, and Kol Tov.