I'd go with the surgery. The low blood supply makes the cast only option that much riskier. It simply may not heal and then you're faced with surgery all over again.
I had the same break almost twenty years ago. I had it initially misdiagnosed so didn't have an option other than the surgery (plus bone plug from my hip).
I've spent most of my professional life swinging a hammer and a lot of my recreational time rock climbing. I have some slight decrease in range of motion but it has held up remarkably well.
You don't want risk the long-term stability of your wrist by not dealing with this right the first time.