It can depend on what you're doing and the likelihood of issues. If you're shooting for youtube or for a film festival without a hope of being picked up by a distributor and you're not blatantly showing them off you might be ok. But my first response still stands, the safe option is to pay for the rights.
Production designers/set decorators usually make sure those things won't be on the set to start with so you won't get into post with these issues.
You can motion track the covers in question and replace them with your home made generic record covers, you can attempt to blur them out so long as it doesn't ruin the composition of your shot(s) or you could add some element to your shot(s) in front of them, essentially blocking the problem out. Depending on the footage, there may be many other ways to solve this.
Good luck.