YTA. From what I understand, this is not the first time your daughter feels neglected, and it is not the first time that your brother feels that she is being neglected - because of how fast he came to her, because of the conditions he had for you to take her, because he wanted to make sure she was safe in your house. Something has clearly happened before.

Now, by the description you gave of your house, and because capitalism sucks and a lot of people can not afford a house with 4 bedrooms, I will assume that you have some money. Did you not think about spending a day with Harper and have a nanny to help your wife with Mark? Or ask a family member to help her? And there are baby monitors, you know? You can check on your child with those.

Finally, I do not like your wife's attitude. I don't know why she doesn't like your family, but I also do not think from what you have written that you have a bad relationship with them, so she made you move away from your family without caring for you, for Harper or even for Mark. She is a problem and a big, big one. Her saying that your brother's conditions are 'not reasonable' is completely unjustified and selfish.