"We have some major issues at this house. All of these problems are highly likely a result of conditioning, by us the parents regrettably. Perhaps you could provide some helpful insights / tips on how we can make it better before it gets any worse, if that's even possible (probably possible but I don't dare to envision it).
Baby Boy is 31.5 months old.
Let me start with the sleep problem. For background information, Baby Boy has never slept in a bed / room for an entire night alone, ever. We were clueless first-time parents. As an infant he had reflux and lots of spit ups, we held him upright a lot. He slept in our arms / swing / car seat a lot. I was nursing and working night shift and it was easiest to have him in our bed, so we co-slept. He went to an in-home care where the woman carried him on her back to fall asleep for naps and such -- which I did at home sometimes as well. Other times it was rocking. He was then moved to a bed / crib once he's fallen asleep. He was still waking up and having milk once during the night until he was 14 months old. The co-sleeping went as far as 20 months at which point my husband has had enough of rib-kicking that he said The Boy HAS to move out of our bed. At that point I first tried to have Baby Boy nap in a crib by himself, without assistance from someone else to help him fall asleep. And no surprise, it caused havoc and huge resistance. He's the type who's cries escalates into hysterical screaming, red in the face, even vomited after only 10 or 15 minutes of crying. So we gave in, I mean I gave in after a few tries.
So we made some small changes. I'd lie in a full-sized bed (in another room) with him, once he's fallen asleep I'd leave the room to do chores and whatnots. When he wakes in the wee hours, he always comes looking for me or dad. He woke one to three times at night. One of us (usually me) goes back to the room and sleep with him, usually until morning because I'm too darn tired to get up again. This arrangement went on for about 6 months until Baby Girl was born (Baby Boy was 26 months old). Then hubby slept with Baby Boy in one room and mommy slept with Baby Girl in another room. This arrangement went on for another 6 months. And meanwhile, Baby Boy still night wakes, and goes back to sleep easily by himself if there is someone in the bed with him. If not, he gets upset, sits on the bed and cries until one of us go to him, on very few nights he'll come out looking for us. (You can see that the horrible parenting choices just keeps getting worse and worse...)
Last week we made a bigger change. Eight nights ago to be exact.
We moved him to his own room. By himself. He has (very) reluctantly accepted it. One of us stays in the room (sitting on a chair or standing, making no eye contact with him) until he falls asleep. Most of the time it goes quickly and he's out within 5-15 minutes. For naps, he sometimes will wake up crying, obviously quite upset (note: he does this sometimes even if we're out and he wakes up in the stroller). Other times he will just come out and gets his vitamin gummy and all is well. At night, he still night wakes at least once or twice. Every single night without fail. Sometimes he will sit and cry for us, other times he will come out of the room, and one of us (usually hubby) bring him back to his room (sometimes he wants to bring a certain toy or book back to bed) and stay with him until he's asleep. About half of the time hubby's too tired and left the room before Baby Boy fell asleep, and miraculously he didn't complain too much and soon fell asleep by himself until morning (anywhere between 5-6).
However the last two nights have been nightmares. He woke up at least 3 times per night and does not not go back to sleep and yet he's clearly dead tired. Last night he was awake from 2:30 a.m. to 5:50 a.m., crying every 30-45 minutes for us. Hubby was up late, so I'd go and tuck him back in bed, then leave. He didn't sleep. I finally gave in and stayed in the room at 5:50 a.m. and that's when he passed out within a minute. He woke up at 8:10 a.m. I hope these last 2 days has been a fluke and not the start of another trend. We got him Toy Story and he watched it part of it on Friday night. He decided it was too scary and turned it off about halfway or less. Could this be the cause? I sure hope it is as simple as this -- but that still doesn't solve explain his once or twice night wakings.
Overall, I think this is good progress (minus the last two nights!) considering the history. At least he's out of our bed & room! But it's not what we would like to do for another 6 months or year(s). We need him to sleep all night. STAY in his room until the sun comes up. We need him to stop having night wakings (impossible to ask because everyone night wakes..) well, we need him to go BACK to sleep by himself when he night wakes, without crying or coming out of the room at all. We've tried to tell him that he cannot come out of the room until the sun comes up. Or Seven O'clock is waking time, not 2 a.m. Etc. Nothing. Works.
So the question is, how can we get him to sleep all night, by himself, STAYING in his own room??"
First of all, I fail to see where there are any bad parenting choices being made. In fact, I think you guys have done an excellent job with a kid who sounds like he has a hard time sleeping by himself.
So many people think if you just put a kid down s/he'll sleep. Those people have never had a child who has a harder time. You're looking at it as if the things you did to respond and reassure your son caused him to continue to wake up. I think it's the opposite--he was going to wake up anyway and have problems sleeping, so you did what he needed by being there for him and reassuring him. He's obviously not the kind of kid for whom crying releases tension and helps wind down. Think how much worse things would be if you had ignored him and persisted with CIO! Then you'd have an anxious kid who might stay quiet at night (because the CIO would have "worked" to let him know not to bother crying for you, which is what happens to kids who don't release tension by crying) but who wouldn't be well-rested and who wouldn't trust that you'd be there for him. Sleep would be even more of a struggle for him, and he'd probably end up an insomniac.
Let me tell the story one more time of my brother (who's now a happy, well-adjusted 30-year-old, with his own apartment, a job he loves, a college degree, tons of friends, and a car--if he could find a girlfriend who lives in his actual same city he'd be all set). He needed to be nursed to sleep, for nighttime and naps--every single time until he was 3 1/2 years old. He never took a bottle, and wouldn't let anyone else put him to sleep. He needed our mom, and the nursing. He even had a one-word request: "nursenadtakeanap." He woke up every night at least once, and would call for my mom, and she'd have to go in and nurse him back to sleep.
(Let me interrupt the story to answer the obvious questions: Didn't this drive my mother insane? And why did she keep doing it? I think it did wear on my mom a lot, although at least she wasn't working outside the home at that time, so at least she could catch a nap later on. She kept on doing it because I had been such an easy sleeper that she knew he slept the way he did because of personality differences, not because of anything she'd done. She figured he'd grow out of it eventually, and it would go faster if she gave him all the emotional support he needed. Smart lady, my mom.)
So he finally stopped waking in the night around age 3 1/2. Or so we thought. When I had my first son and was with my family and was complaining about my 9-month-old still waking at night to nurse, my mom told the story of my brother. He heard her and said, "Mom, I still wake up at least once in the middle of the night, every night. I'm awake for anywhere from 10 seconds to 20 minutes. I just stopped calling you because I figured out I didn't need you to get me back to sleep." (Our jaws dropped.)
My mom thought she was laying the foundation for my brother to sleep through the night, but what she was actually doing was laying the foundation for my brother to feel secure and self-sufficient enough not to need her when he woke up. Did it take a lot longer with him than with other kids? Hell yeah. But that doesn't mean there's anything wrong with him. And it certainly doesn't mean she did anything that caused him to be that way. (She treated us both the same--actually she probably coddled me more, since I was the first--and I went down from being awake at only a few months, while he needed her for years.)
So I want to say you're doing an amazing job of responding to a kid who is not an easy sleeper at all.
Now for the practical suggestions (which may not be miracle cures, since he may need to grow out of this on his own).
Definitely stop watching movies at night before he goes to bed. This could absolutely be the cause of his night wakings. Anything scary, or even not scary but exciting, could get his body all geared up and make the adrenaline flow, and that could cause multiple night wakings.
Keep his schedule as boring and predictable as possible. Again, some kids (easy sleeping kids) could sleep after any stimulus and not have any problems, but you know he's not that way. He needs the schedule to stay the same and be nice and calm and boring to be able to relax into happy sleep. So no exciting late-night parties, movies, late-afternoon playdates. When he starts school you should try to get him into a morning program instead of an afternoon program, if possible.
Definitely give him a nice, sleepy snack before bed (involving milk, definitely) that will stay in his stomach and make him drowsy. You mgiht want to try giving him a backrub to sleep with something like Badger Sleep Balm (it has lavender and fir and other scents that really help people relax and fall asleep).
In short, you really want bedtime to be as boring and comforting and stress-free as possible, so he has as easy a time as possible drfting off, and then drifting off again when he wakes in the night.
The other thing I'm going to suggest is to try to help him not be afraid in the night. I'm assuming he already has a night-light in his room. If not, make a big deal about getting one, and let him go along to buy it.
Also consider getting a monster-scarer (ideas in the post and also in the comments) so that when he wakes up he'll be able to trust that the monster-scarer will work, and maybe he won't need you to come in.
Another idea is a musical bear. We have a bear that has an old-fashioned twist music box inside that a 3-year-old could definitely operate himself. When he wakes up he could wind up his bear and have some music to go to sleep with.
You could do a different version of the music theme and get a small CD player to sit by his bed, and he could press "Play" when he woke up. The CD would stop playing when it was done.
I think it's going to be easier to stop his night-wakings than it will be to stop the early waking. And I think there's a 0% chance of fixing them both at the same time. I'm assuming the night-wakings bother you more, so those are the things I've focused on. I also think morning-waking is a problem that's easier to solve in an older child (my son is 4 and we're just now having success with getting him to stay in bed until 7). So try the night-waking suggestions and see how they work.
In the meantime, continue to provide him with as much emotional support as he needs to get to sleep. You've done an amazing job already of responding with care to a kid who, for whatever reason, isn't an easy sleeper. I know it's hard to believe, but you're in the home stretch. Someday he'll be 30 and living on his own, and you won't know or care how many times he wakes up.