For this purpose I keep a large plant pot (or two) full of stones that we dig from the garden when we're tilling a new bed. And from the pots at the end of the season. They're varied sizes and I use either several large or many smaller ... it depends what I've got planted. The smallest I've used would be about the size of a hen's egg.
Our gardens used to have a lot of larva rock around juniper shrubs (previous owners choice ... all long gone by us now) and I think you could even use something like that but I dislike larva rock in the garden, so don't use it for that reason. I do bring home large smooth beach pebbles in my pockets after rambling walks with our grandson on the beach, so they go into the pots too.