There are generally two types of technologies used for detecting malware (which is short for malicious software, for example: virus, spyware, worms, etc.) these days. The first method is called 'signature-based protection', the second is called 'behavioral analysis' (sometimes known as 'heuristic' or 'non-signature' based analysis).
Signature-based protection is the most traditional and popular form of malware protection around. It detects malicious programs by matching the code in programs against a known database of malware. This is a very accurate way of detecting malware, but it can only find malicious programs it knows about. Also if a new malicious program comes out it can only be detected when a signature has been created for it. Many anti-malware vendors are struggling to keep up with amount of malicious code that is constantly being released.
Behavioral analysis (or non-signature) based malware protection tries to detect malicious programs by watching the applications on your computer for certain patterns of behavior. For example, if some program tries to infect another file, or perform another type of malicious behavior then it will be flagged. The advantage of behavioral analysis is that it doesn't require signatures to detect malware, so it can detect new malicious programs before traditional signature-based programs can detect it. The problem with this technology is that it can produce a lot of false positives if its tuned wrong, which can be annoying.
Ultimately, the best type of anti-malware protection program is one that combines both types of technologies. Signature based software is only as good as its last update, and behavioral analysis based protection is good but still has a ways to go to over come the problems with false positives.
It seems like most modern anti-malware protection programs are gradually moving in this hybrid direction, but it might take a while to perfect the technology.