Your baby sucks his mum's breast, thumb, the corner of a blanket. Sucking is one of the strongest instincts a baby comes to the world with. When sucking on something, your baby calms down, falls asleep and feels happy and safe.
Try to hold a newborn always in a horizontal position. You can try other positions from time to time, but remember that it's the baby's decision. You shouldn't deviate from the horizontal position unless the baby is feeling safe and comfortable. When he grows, you can start using more vertical positions.