It is all about habitat. If you have a barren yard, with no shrubs, there is no place for visiting birds to hide. Sometimes you see a birdfeeder in an open area in the middle of a lawn—many birds will avoid these as they are sitting ducks for hawks. Give them a place to bolt by putting the feeder near a tree.
Personally, I want to be able to see the whole yard and so I have planted trees and shrubs that way. Plant things in clumps to create islands of habitat. Maintain visibility as much as possible and have the islands make them feel safe.