There is a spiraling downside to his experiments in sexuality, which he admits and regrets. It's nowhere near as horrible Chogyam Trungpa or others, but that doesn't make it OK. Lots of people left the order, and the continued negative press in England give it the air of a cult.
Of course when you become devoted to your guru, that devotion can be dangerous, and you should never suspend your critical faculties, and never do anything you don't wish to do. Never override common sense, even for the spiritual life.
Sex and the Spiritual Teacher should be required reading for everyone. Hannah Nydahl has created more sangha in the west without scandal. Not that I'm aware of, but I thought Ray was pure, and someone corrected me so with all the breaking scandals it's hard to keep track. Shambhala is still reverberating from sexual scandals and even Rignald Ray is being buffeted by accusations and scandal. I couldn't be bother to watch Pema Chodron talk to Oprah. I sure she is deeply hurt by these scandals. Shoes At The Door is also a good book. Also The Buddha of Brooklyn. It's almost a main theme and not some sort of leitmotif. Watch out.
Having said that, I am truly truly grateful and feel lucky to have lived when Sangharakshita lived, and that I had the available resources to study in his Triratna Buddhist Community sangha for a time. The one year anniversary of his death is a good time to remember all that I am grateful for.