We know at the Harrenhal tourney Ashara Dayne hooked up with someone according to Barristan Selmy. We know from Selmy's own statements that he was deeply in love with the beauty of Ashara Dayne, but supposedly stood aside for Brandon Stark?????

The Kingsguard is sworn to celibacy and Barristan reminds himself of it when he mentions the Tourney. He regrets never telling Ashara of his love,\ but he knows that nothing would have come from it either way. He's a man of impeccable honour (moreorless), he stood aside because he swore a vow.

She died never knowing that Ser Barristan had loved her. How could she? He was a knight of the Kingsguard, sworn to celibacy. No good could have come from telling her his feelings.

The Kingbreaker (ADWD)

Additionally we know Rhaegar's closest friend was Arthur Dayne and his skills could have impressed Rhaegar enough to begin to have eyes for his sister if it meant their child would inherent the blood of Arthur Dayne.

I don't get this.. Are you saying that he looked to Ashara in the hopes that their child would be as skilled as Ser Arthur? A little clarification would help

We know at this time Rhaegar's actions were deeply influenced by prophecy & creating the three heads of the dragon, and was looking for exceptional women from the realm to mother them.

Rhaegar wasn't out scouting for anyone and everyone, he wanted the right one. He found that in Lyanna, hoping their child would be the Song of Ice and Fire as he had believed his son Aegon was. We have no information that Rhaegar ever looked to someone else. Additionally his wife, Elia Martell, wasn't the perfect mother to a child who would one day lead the Kingdom. She was frail and unable to have more than two children, not exactly prime parent material when you hope to have three prophesied children.