The only concern I would have is disease transmission.
Doves are housed in a coop and are allowed to fly, seen them doing wide flying laps around their coop over here. Birds that are allowed out and then come back in to mingle with our fids have the possibility of bringing something with them IMO.
This would be my concern also. If the doves were kept as an indoor/outdoor pet (exercised outside) there would be a greater potential for disease transmission.
I remember someone here on the forum that had a pet crow. I don't remember if he had parrots as well.
I have a friend at work that has a cockatiel and found a dove at her backyard door. She kept it inside overnight and let it loose the next morning.
The next evening she had 2 doves at her backyard door
She keeps them in seperate cages but has had no problems.