Next generation sequencing is extensively applied to catalogue somatic mutations 
in cancer, in research settings and increasingly in clinical settings for 
molecular diagnostics, guiding therapy decisions. Somatic variant callers 
perform paired comparisons of sequencing data from cancer tissue and matched 
normal tissue in order to detect somatic mutations. The advent of many new 
somatic variant callers creates a need for comparison and validation of the 
tools, as no de facto standard for detection of somatic mutations exists and 
only limited comparisons have been reported. We have performed a comprehensive 
evaluation using exome sequencing and targeted deep sequencing data of paired 
tumor-normal samples from five breast cancer patients to evaluate the 
performance of nine publicly available somatic variant callers: EBCall, Mutect, 
Seurat, Shimmer, Indelocator, Somatic Sniper, Strelka, VarScan 2 and Virmid for 
the detection of single nucleotide mutations and small deletions and insertions. 
We report a large variation in the number of calls from the nine somatic variant 
callers on the same sequencing data and highly variable agreement. Sequencing 
depth had markedly diverse impact on individual callers, as for some callers, 
increased sequencing depth highly improved sensitivity. For SNV calling, we 
report EBCall, Mutect, Virmid and Strelka to be the most reliable somatic 
variant callers for both exome sequencing and targeted deep sequencing. For 
indel calling, EBCall is superior due to high sensitivity and robustness to 
changes in sequencing depths.