I have a similar track record with Yellow Polyps.
Per Alice's quote "...like light and they like food..."
I purchased a rock covered with Yellow Polyps (~30 polyps) about 1 year ago. I placed it on the bottom of the tank (~28" away from my lights) and did not make special efforts to feed them. Within 3-4 months the colony diminished to 5 polyps.

However, those 5 polyps are still very much alive and seemingly healthy. About 1 month ago, I raised them to midway in the tank and I have been target feeding them about 1 time per week. These 5 have held steady for about a 6-8 month period, and I am still hoping that they will eventually spread (but nothing yet!!!)
About a month ago someone gave me another
yellow polyp rock with about 15-20 polyps on it. I also placed it at the bottom of the tank, but have been target feeding them since the gitgo. I have noticed over the past couple of weeks that these are slowly vanishing, so I am going home tonight to move them closer to the light.
I basically feed them the same stuff that I feed the fish--a mixture of
frozen brine shrimp, mysis shrimp, formula 2 flakes, and prime reef flakes. I just take a turkey baster and direct the mixture over the polyps about once per week. I feed the fish about 2-3 times per week, and the polyps will sometimes grab whatever floats their way.
I have a 135 gal. tank with 440 W VHO (2 actinics and 2 daylight bulbs).
Could some folks share their lighting and feeding conditions??