Work in progress!
The reason I didn't have the 4x4 block go all the way up is because I didn't want to lose any growing room. But using a 2x4 instead of a 4x4 would have been perfect and I should have done that. The good news is that I bet these 4x4 blocks can be easily removed and the 2x4's put in. The soil just needs to be pulled away and the work done.