hi n.c.18 hatzlacha in your uncomfortable situation ,are the workers jewish, orthodox,,are you married,at my job there was a non jew who kept on tapping me on my shoulder and i told her after the first tap (didnt want her to think its ok and then after shes used to it ,ill spring it on her hey ,no tapping )anyhow a new worker came and after i heard some giggleing she touched me on my elbow as if to tell me something related to work of course ,and i gave her the cold shoulder "im jewish ,we dont touch women," she knew that i wouldnt appreciate the touch,im sure, but it was a type of starting up.
im sure its not your fault,but i think hakol milimala is mentioning a good check list,which i could see myself slipping up good naturedly ,not thinking it will give any type of message ,hatzlacha raba again.
i would ask what kind of getting close but could be its not important,maybe its just my lust asking, i thought it would give a better picture.