I usually use 2 or 3 risers per side (4 or 6 total, for a height of either 8" or 10" (the latter height is also tall enough to do step-ups). That's the right height to make it easy for me to both lift heavy dumbbells from the floor while seated and get them in place for bench presses, etc., and lift a barbell into place.
You want to make sure the height is one that when you are seated and bent over, you don't have to round your back to pick up dumbbells, but you can place yourself so that the dumbbells aren't touching the floor when doing rear shoulder flyes from a seated position.