I salute you for knowing about the other three verses.
To my knowledge and experience, the other three are virtually never sung. And I've been known to hang out at a fair amount of places where the national anthem is sung or played. Ballparks, boat christenings, harbormaster ceremonies, civic events.
Funny though, I always thought the second verse had more pizazz anyway, it ends with the affirmative "Tis the Star Spangled Banner! O Long may it wave, O'er the Land of the Free and the Home of the Brave." The verse we usually sing (first verse) ends in a question, wondering if it still waves.
But, anyway, to answer your question, I think the 3rd and 4th verses have too much blood, violence, and invocation of God to be really considered appropriate in today's society.
From the third: "Their blood has washed out their foul footstep's pollution." That's kind of graphic, to talk about the blood of the dead British running in sufficiently copious amounts to erode their own footsteps. Plus, they're our friends now.
And from the fourth: "May the Heaven rescued land / Praise the power that hath made and preserved us a nation / Then Conquer we must, for our cause it is just / and this be out motto, in God is our trust." That's basically an invocation and a command to pray, as well as a paen to imperialism. I just can't see that going over real well in a lot of American communities today.
Plus, it takes about 8 to 11 minutes to run through the whole thing. Here's a video of some people who did it in Massachussets (can't embed http://www.youtube.com/watch?v=ICaNsSuxjmo ). As far as I know, the only time you will ever hear it is when a band makes a point of doing all four, to teach the audience that they exist.
I've seen boats launched, stadiums dedicated, races run, ballgames played, politicians sworn in, and a hundred other circumstances where the national anthem is played, and I've never seen the other three verses performed.
Come to think of it, THE only time I've ever heard it sung was as a stupid human trick / bar bet. People don't believe that there are four verses, so if you memorize a few lines you can belt them out when somebody calls bullshit. Set it up right, and you might get them to owe you a beer out of it.
That would, after all, be in the spirit of the song. As you might know, Francis Scott Key just wrote a poem called "The Star Spangled Banner." He didn't write any music. His poem was set to the tune of a British drinking song called "To Anacreon in Heaven" and later became our national anthem.