Try to hold a newborn always in a horizontal position. You can try other positions from time to time, but remember that it's the baby's decision. You shouldn't deviate from the horizontal position unless the baby is feeling safe and comfortable. When he grows, you can start using more vertical positions.
I recommend this position especially after feeding. Lay the baby on your shoulder. Rest his head on your shoulder and let his hands fall on your back. Hold his bottom, don't let his torso and head lean back too much. Lean your body slightly to the back so the baby can rest comfortably on your chest.
Until the baby is 2-3 months old the most commonly used holding position is the so called kidney-bean, which reminds the baby of the position he took in his mum's womb. In this position the baby lies on one of your forearms. You slide the other hand under the baby's slightly bent legs and support the bottom with your hand. The baby's head rests in the crook of your arm. From one side, his torso is supported by your forearm whereas the other side is cuddled to your belly. The baby's torso is slightly rounded (hence the name of the position), which prevents the baby from leaning back and makes the position healthy for his spine. It's important that the baby has stab support under his head and not just under the neck. This position allows the baby to move his hands and feet freely.
When holding your baby in the kidney-bean position you can turn him on his side. The baby lies on your forearm, with his bottom resting on your belly. This is the position we call "on a tray". Put the hand on which the child is lying under his torso and bottom and slide the other one between his legs. You can also hold him with his face towards you.
The child is sitting on your hand like as if he was sitting on a chair, leaning back. It's important that the child can lean back and rest on your chest in order not to overstrain his joints or spine. In this position you use your other hand to stabilise the baby and stop him from leaning on the sides. If the baby's head wobbles too much or falls to the side, it means the child is still too young for this position. I don't recommend this position for children under 3 months.
When at home we usually carry the baby in our arms and push him in the pram when we go for a walk. However, if you want to have your both hands free and keep your baby close to you at the same time, you can buy a special baby sling or wrap. They allow carrying the baby in a natural, physiological position (his hands and legs are slightly bent). If you keep the proper position, you can even use the wrap/sling to carry a few days old newborn. However, you shouldn't carry a newborn in them for more than 2-3 hours at a time. Remember also that slings/wraps shouldn't be used to carry babies with low muscle tone or hip dysplasia. Don't carry a newborn in baby carrier. At this stage he's not ready yet to hold his body and head erect for such a long time.
Children with high or low muscle tone may react with fear to changes in carrying position. They aren't able to control their body and don't feel safe when they're positioned with their back towards the floor. It's better to carry them in a position where they can see what is happening around them. It's enough to turn them slightly to the side.
You don't have to avoid carrying your baby if he has hip problems (hip dysplasia). You simply must carry him in a proper position. Cuddle him as often as possible, preferably when you're lying next to him. When you want to pick him up, remember to keep his legs protected all the time, and don't try to straighten them. Place him mostly on his tummy (it's good for the hips). But let him also lie in other positions from time to time.