To "Keep" up with the Jones of course.

I never really built one in my Arduin games, when you're a female half demon half Amazon Warrior, you think more about keeping your skin intact, than settling down.
I did however build a couple in the other game and that was a worthwhile endeavor for those characters. My wizard built a tower, hired a staff and used it for performing magical research. My ranger cleared part of a forest built a keep and spends time patrolling it and keeping the forest free of intruders.
To me yes, they are worthwhile and a portion of the game that is more often than not overlooked.

Bel

Dragon Magazine #224 (I happen to have a copy on the PC) has an article dealing with precisely this.

Essentially, the article boils it down to a few basic maxims.

1) Any world in which magic is at all prevalent will have those who are knowledgable enough to defeat those magics, even if through mundane means. (i.e., you can defeat many flying foes by "capping" watch towers to protect the guards manning those towers. You can deter magical transport through walls by mixing lead in with the mortar. Etc.)

2) Anybody with enough with the resources to build a castle probably has the resources to retain a mage of his own that will help to deal with magical assailants.

3) Even in the presence of magic, typical mundane threats (rampaging armies and bandits for instance) can be defeated by a sturdy wall.

Of course, the article goes into much more depth than this, but I can't realistically copy and paste it here. I'm sure that Paizo/WOTC would hunt me down and cause me harm for it.

In the campaign I belong to it took more then 5 years and I think about 8 million gold to build the Lions Den. Our City/Keep became of the Great Cities in Arduin and we now use it for the starting point of many characters and adventures.

We had about 20ish characters between 8-10 people that all pooled their wealth and individual resources to make it possible.

It was worth every coin spent !