Running suspicious software in a virtual machine seems like a basic precaution to determine if the software contains malicious code.
However, it is generally easy to detect whether software runs inside a VM. A list of ways to detect a VirtualBox VM from inside the guest OS is available.
There are obvious detection methods, such as the occurrence of the word 'VirtualBox', as well as more subtle ways.
A proof-of-concept 'malware' application called Al-Khaser demonstrates this, and can be used to verify anti-malware systems.
Al-Khaser can be used to verify one's anti-malware systems.
Author's summary: Virtual machines can be detected by malware.