I don't avoid this word entirely, but it can be destructive at times.
Here are some examples that I think are very unhelpful:
To an ADHD child--"If you would just sit still..."; "If you would just pay attention..."
To an obese person--"If you would just eat less..."; "If you would just exercise..."
To a depressed person--"If you would just start thinking positively..."; "If you would just get out and be around people..."
From one spouse to another: "If you would just try to remember what I tell you..."
And so on.
The fact is that oftentimes, if people could do one of these actions, they would.
The fact is that oftentimes, if people could do one of these actions, they would.
What do all of these have in common? They imply that there is a kind of moral failure of willpower which could be remedied by turning on or off a switch. They ignore the complexities of the brain, of past learning history, of the overall body physiology and chemistry, and of reinforcement patterns in the home.
That is not to say that there are not problems that need to be solved. However, using the word "just" does not solve the problems and make them better. It can do the opposite; it can make problems worse by evoking shame in the other person.
When can the word be used? I think it can be used in an encouraging way rather than as a put down. For a child trying to learn fraction division, it might possibly be helpful to say, "All you have to do is just turn the second one upside down and then multiply them." This could encourage the child to see it as easy extension of something they already know how to do rather than having the mindset that it is hard. However, if the child has difficulty with math, then this statement could also become somewhat of a put down.
The word is sometimes helpful in our self talk. For example, if I were anxious about going into a job interview, it might be helpful for me to say to myself, "All I have to do is put on a cheerful face, take a deep breath, and answer his/her questions." In other words, the word "just" can be a way of telling myself that I am making a mountain out of a molehill.
So be careful with the word "just." Use it wisely and carefully; and pay attention to how it affects other people when you use it with them.