What turned me off was when everyone started turning on each other mid-season 2. I felt like shaking them all and yelling "Look, you dumb fucks! You're marooned on an island with another group of people kidnapping you one by one and you're worrying about petty squabbles! Get your shit together and organize!" They had a room full of guns & ammo, but because Charlie turned emo over Locke "moving in on his territory" he helps the grifter hide them, allowing the "bad guys" to find them. Meanwhile, the grifter guy is mocking the doctor because girl #1 can't decide which cock to suck first, but is leaning towards the grifter's.
I mean, they don't all have to get along, but fuck - realize that together you stand a better chance of surviving than apart. It just got really aggravating seeing the plot start to go somewhere and then fall apart because the writers needed to add more soap opera to the show. I don't watch tv to get upset, thanks. I have enough of it in my actual life.
The flashbacks got old after a while as well, but that's more of a minor annoyance.