Nowadays, in order to prevent criminal behaviors or traffic accidents, video surveillance systems have become more and more popular in both outdoor and indoor places such as offices, departmental stores, public places, railway stations, and airports, etc. So, there is a great demand for an intelligent system to detect abnormal events in videos. In the surveillance tasks, people are generally the main objects of interest. Even though, recognition of human action is an emerging topic in the field of computer vision, detection of abnormal event is recently attracting more research attention. Abnormal behaviors can be identified as irregular behavior from the normal ones. Certainly, various techniques and approaches are proposed in order to ensure human safety. This paper presents a survey on different human activity prediction techniques in video surveillance system. Initially, different techniques developed by previous researchers are studied in detail. Then, the limitations in those techniques are also addressed to suggest further improvement on human activity prediction in videos using advanced techniques. The efficiency of the different human activity prediction techniques is proved by comparing their parameters. The comparison results show the best human activity prediction technique among them.