lightsoul, have you found some answers to your questions?
Also Daniel I want to ask, what sort of conditions cause a person who has experienced an unnatural death, for his ghost to become vengeful/malevolent? Do these ghosts ever move on to the reincarnation cycle after a certain period? Would a 'voluntary death' (ie suicide) still be classified as an 'unnatural death'? Is the consequence the same or more severe if one chooses to end their life prematurely? Sorry for these intense questions Daniel. Appreciate your time in answering them if you get around to it.
WHAT KEEPS PEOPLE AROUND
Regarding questions like these, a lot does depend on the psychic make-up of the person and on the choices they make when they shed their physical body (we all understand pre-death karma plays a role but I assume u already considered that in your question). My short answer is that it depends on how much the person learns before death, and after death in hindsight. One doesn't need to be "punished" in order to learn a lesson - only realize and understand all facets of it completely up to the point that can be understood on this evolutionary level (there are higher ones where more understanding can be gained, but that won't matter when reincarnation is at stake). Forgiveness is definitely one aspect, as one would need to be able and willing to completely let go of any emotional anchors keeping one here.
I also do believe in the existence of pre-incarnation contractual agreements. I also believe that some are revocable, while others are not. For instance, if we had an agreement that I would give u hell on earth so u can learn forgiveness, and I haven't done that, that's unfinished business. It might play a role.
ENERGETIC REQUIREMENTS TO MOVE ON
I understand daniel to have explained some energy mechanics that plays a role in reincarnation (and the ability to properly pass from space-time on-wards). If enough of a certain superior type of energy is generated, then energetically speaking, it is no longer necessary to reincarnate. But personally I think there's more to it than simply not having sufficient high-level energy to move on from this space-time incarnation into a time-space incarnation, or to escape reincarnation entirely.
REFERENCES FOR OWN STUDIES
Since none of us are experts on the topic, but can only put together the research we've done (that might even contradict each other's conclusions), it will be up to you what you make of it. I've found some nuggets in the website http://www.near-death.com/
which includes stuff covering all belief systems, bad experiences, good experiences, etc. from people having Out of Body Experiences triggered by physical death. There's a LOT of stuff to go through there. My favorite sections are the recount of people's personal experiences. It is mentioned here that those spirits that somehow lost their connection to "Source" (some would call it connection to God) completely, and can no longer "generate" energy on their own - they then need to prey on the energies of other beings to sustain themselves, much like parasites. However, this has a lot to do with choice. If a person chooses to "hate" another to such an extent that they would rather "hide from the light exposing their motives and actions" to keep justifying their hate - then by cutting themselves off from that light (the light would impose an unpleasant experience on their horrible state of being, and th)
When evaluating the content there, keep in mind, that those are subjective experiences. Just because a person says "They saw God" etc. doesn't mean the same thing as we would understand it in this physical realm. Also consider reading up on soul traps so you can get a balanced view on the "unfinished business" type of near-death experience. Some would argue that a person will never need to return for unfinished business, and such experiences are soul traps. I don't think that is the case, because of pre-incarnation contractual agreements also playing a role. But I do believe soul traps do exist (some are in forms we have some understanding of, others are in forms that we don't understand at all yet). I am personally aware of a good personal friend, of a very personal good friend (i.e. 2nd-level connection) that have done some work in Germany regarding trapped souls from jewish bloodlines in areas where there were concentration camps. Forms of soul traps WERE used and they had a very real effect. Probably more along the lines of what you were thinking of "ghosts" not moving on though, more so than people actually dying "normally" but reincarnating when they shouldn't (which is a soul trap research topic currently going on). Some soul traps work by using trauma, others don't work that way at all. Near-death OBE's must be considered under the latter type as a possibility.
I would also, while evaluating the content there, consider how memory can be altered through suggestion. I don't recall the links and topics now, but TED Talks had some interesting scary stuff on there where social scientists studied how people can walk into psychiatric therapy with one set of issues, and walk out with MORE issues and very different ones. They found that through merely systematic suggestion by the therapist, they could cause patients to "remember" things like satanic ritual abuse where there never were any (pretty extreme traumatic stuff to "make up"). They could also cause patients to develop multiple personality disorders where they never had those before they entered therapy. The point of it is, that when people share experiences years after having them, its not that easy to tell how much of it was "altered memory" where the person decided to remember things differently to make more sense of it, and how much of it "really happened". Our brains DO rewrite our memories if we let them. Some people WANT to remember things differently than they happened, and so they do.
This is part of the reason its very important to rather go for first-hand experiences in finding answers, and to be carefully aware of one's own motivations before doing so - as to their influences on one's interpretation of one's findings.
Other people can give you some reference points on what to look out for - and those are SUPER USEFUL, but they can never give you the "real full answer".