Malware is suspicious software that nobody wants. It poses threats to the security of the networked system and is also designed to harm services or devices. They are designed to damage any data or to extract data so that cybercriminals can leverage over victims, which is advancing rapidly, states a big challenge for researchers. In this literature review, we did the manual research on the publications. The motive is to look at the available literary works on malware detection and determine how the investigation evolved and progressed based on quantity, content, and publication potential. Described the problems and challenges faced in analyzing malware and identifying system requirements. The goal of this study is to review and examine the literature on malware analysis and analyze how the study of malware has evolved from time to time. In terms of quality and publication outlets, malware has progressed as time goes on and so it becomes complex and later time, it can be difficult to understand its every detail and composition. Enlightening people about what the term malware is about and the variety of anti-evaluation techniques that can be used to thwart forensic evaluations and lessons through the post-investigation results on how to avoid malware analysis.