I did it!
How should I explain this.
1. I duplicated the original footage, and then I pre-composed the duplicated footage, I call the composition "grass layer". I duplicate the footage so I can see what I am cloning, just as a reference.
2. Then I applied a clone stamp on that pre-composed footage, 'grass layer', and starting cloning the grass.
3. When I was done, I opened the clone properties on the 'grass layer comp', and changed the "Clone source" to be the original footage from the original composition.
4. And whabam! Now I can just delete the duplicated footage inside the "grass layer" comp, and have the grass only thing visible, and make the 'grass layer' a 3D layer, so it sticks on the footage
0.o You deserve a cake if you managed to understand what I wrote above. Maybe I can just show it in pictures if needed