I am sorry for this delayed reply to your gardening question. The spring rush has brought a deluge of questions and we are working hard to catch up!
It is not aphid poop. If it is larger than a BB you are probably seeing poop from a large caterpillar. I can't picture what you are describing so I suggest you take a sample to your County Extension Office and see if they can identify the problem and if necessary prescribe an appropriate remedy.
Thanks for the question. Best wishes for a wonderful gardening season. Please stop in again soon!
Q&A Library Searching Tips