I made a video-essay about the film not too long ago (If you're interested: https://vimeo.com/117335917) so I can hopefully sort of claim to know a little more about the film and its background. While I didn't touch on the relationship between Brandon and Sissy, I did look into it and researched many interviews with McQueen in which this question was asked.

Now, McQueen is an artist who doesn't like to explain himself too much, he says, because his works should ultimately be a mirror to the audience. He does confirm that there's a deliberate hinting at a troubled past that both characters share, possibly abuse or incest. That's the most he's willing to elaborate on it and that's great because it allows these discussions which show what a complex film it is and how everyone can interpret it differently.

I myself think it's not really important what it ultimately was because I think the function of these hints in the present narrative are more important. Sissy and Brandon are two polar opposites, bound by them being brother and sister. Sissy is this very fragile, open and longing thing who easily attaches herself to people and pours her heart out. Brandon is stark and closed and has difficulties truly connecting with people. It's two people dealing with the same problem, a shared trauma, in totally opposing and escalating ways. They recognise themselves in eachother and that's what pains them and what sets the narrative in motion. Notice how Brandon is fully clothed while Sissy is naked when we first see them together, that's a very clear opposition. She invades his private space, his safety, by quite litteraly breaking in, and turns his space into hers. He's very clean, she makes a mess of the whole apartment etc. So to me the function of these hints at a past trauma is to make both characters confront with eachother and through it with themselves. But there could be multiple explanations ofcourse. Cheers!