Kelli, that is a gorgeous barrel!!! I collect those...especially the big giant specimens like yours. It's worth a lot of money here as they are uncommon in that size. It looks VERY healthy and wonderful, I think it's fine!
If the pot isn't precious to you, you could pick up the pot, place it in another bigger pot, then smash the clay pot with a hammer while it's in there, pick up the big chunks of pot and leave the smaller ones in there for drainage, then add soil and gravel at the top. That way you wouldn't have to handle the cactus itself. You can cover it with a blanket while you're working with it.
If picking up pot with the cactus in it is out of the question, then don't worry about it! That cactus looks so healthy, doesn't seem like there's any immediate issue anyway.