How we change what others think, feel, believe and do
A person may retreat from the other person in a number of ways. These can be large movements or small signals where they move only slightly away.
When people are conversing in close proximity, they are also within reach of the other person and thus vulnerable to attack. Whilst this is unlikely, we are programmed to be cautious and a movement away often shows a desire to continue moving away. Politeness, though, often keeps us in place so we just lean back or turn away.
Defensive retreat is often coupled with other defensive acts including pulling in of arms and pulling down of head and body.
Pulling back the head whilst lowering the chin protects the throat. This can be a surprise signal ('No! Well, fancy that!') that uses a mock defensive move to show amazement.
We all have defined personal spaces and when other people enter these we may back away (which is easier and politer than pushing them back). This can happen when one person is attempting a romantic connection with another and steps into their intimate space (and the other person steps back). Cultural spaces are also different -- for example city people usually have smaller spaces and will stand closer to a country person (who will back away).
When a person feels threatened they will probably continue to look at the other person. When they turn away, whilst they may be indicating that they do not want to be there, this also can be a power move, saying 'I am not threatened by you and do not need to monitor your actions.'
There can be other reasons for moving away from a person, for example if they have bad breath.
Another reason to move away is to give space in which to move in. If you are close to a person and want to use a moving-in signal, then you may need to back off first.
People also distance themselves from anything that they do not like, for example a card player with a poor hand may put the cards down and push back from the table.
When people leave a situation, moving away to somewhere else, it might be a simple natural ending, being attracted by something else or getting away from an uncomfortable aspect of the situation.