Generally, for a program that support multi-threading, the elapsed time will reduce with the increasing number of used CPUs. However, I found a strang case that picard MarkDuplicates will run slower with more CPUs. When I run picard MarkDuplicates in a node with 160 CPUs, it will cost 1 to 2 hours, but it only cost about half an hour for a node with 32 CPUs.

picard MarkDuplicates is a java-based program without the option to set number of threads. I found it will automatically detect the available CPUs and try to use them all. For the reason picard MarkDuplicatesrunning slower with more CPUs, I assume it cost too much time on looping spliting jobs and retracting data.

Another case that running time has non-linear relationship with number of CPUs is RAxML (unlike picard MarkDuplicates, RAxML allows you to set number of threads). A few years ago, I tested the elapsed time with number of CPUs from 2 to 20 and found that RAxML working faster when you set number of threads as 8 to 10.