Are your rocks clean? Gluing rocks are no different than gluing any two surfaces together. The surfaces must be clean. Dirt, dust, oils, etc. will always cause problems. Pea rock, for instance, used for mulching is very dusty. You would need to wash it.
