I know this sounds a bit controversial, but
Just cause it’s an opinion doesn’t mean it’s ok to say
Using “oh but it’s my opinion” to excuse something you said that can be harmful or rude is not a good excuse.
If you have to use “it’s my opinion” as an excuse for something you said, it most likely wasn’t ok to say it in the first place.
It's not really your fault then. Sometimes, trying to avoid offending anyone is just impossible.