I don't like peas on their own.
I don't like carrots on their own.
Therefore, the phrase, "[Two things that go really well together] go together like peas and carrots," makes absolutely no sense to me. Why would I want to put two things together that I didn't like individually ? I wouldn't like that. I would hate that.