One thing I've learned in purchasing large branching stony colonies is that the coral will never do as well as a frag.
The problem with larger colonies (4-6") is that they did not grow in your aquarium so they are not "custom" tuned to your water flow and other important features. In most cases part of the inner colony may die off from lack of flow, or just turn to a non-intended color (normally orange & green).
Frags, IME, don't have as large of color shift and never have many problems adjusting to the tank. I too find it more fulfilling to watch it grow into a large colony. Frags are also cheaper.
Brains and other larger polyped colonies usually do fine when purchased large. Most are much slower growers than SPS and can take years to get a nice "centerpiece" coral out of them.