Time Command – Understanding System Time in Command Line

Why execution times of the same binary differ on multiple executions

The key problem here is the non-deterministic behavior due to the way the CPU works. Modern superscalar CPUs can execute multiple instructions at a time or change the order of commands to be executed (out-of-order execution). Regarding your example, it's also possible that optimization in cache use applies. RAM is several magnitudes slower than the CPU itself, that's why intensive caching is used. The second run of your binary might run in the cache, thus consuming less CPU cycles (cycles the CPU would have waited for the data to be retrieved from memory).

On the difference of real, user and sys process time statistics

One of these things is not like the other. Real refers to actual elapsed time; User and Sys refer to CPU time used only by the process.

• Real is wall clock time - time from start to finish of the call. This is all elapsed time including time slices used by other processes and time the process spends blocked (for example if it is waiting for I/O to complete).

• User is the amount of CPU time spent in user-mode code (outside the kernel) within the process. This is only actual CPU time used in executing the process. Other processes and time the process spends blocked do not count towards this figure.

• Sys is the amount of CPU time spent in the kernel within the process. This means executing CPU time spent in system calls within the kernel, as opposed to library code, which is still running in user-space. Like 'user', this is only CPU time used by the process. See below for a brief description of kernel mode (also known as 'supervisor' mode) and the system call mechanism.

User+Sys will tell you how much actual CPU time your process used. Note that this is across all CPUs, so if the process has multiple threads it could potentially exceed the wall clock time reported by Real. Note that in the output these figures include the User and Sys time of all child processes (and their descendants) as well when they could have been collected, e.g. by wait(2) or waitpid(2), although the underlying system calls return the statistics for the process and its children separately.

Source

The answer is: You can not! Linux is no real time system. The idea of UNIX and therefore Linux, too, is to provide minimum answer times, while the system is shared between multiple users and system processes. Depending on when you start the command, you might have to wait for a important system process to give you your share of processor time. Further the file system might buffer your file you read from the disk, but eventually these file system buffer fail to load the data from cache, when another process on your system occupies that. Generally the time a process needs in a Linux system depends on the surrounding entropy of the machine, where ever it is located in time and space in the universe.

You will need a real time system and specially real time tuned commands and a specific amount of resources reserved for you only. You might get somewhere near that with new CGROUP features from newer kernels, where you can reserve a processor, a part of the memory and an own reflection of the underlying file system for your environment.

One of the major timing problems in your example is the way grep and wc reads the inputs. You might be able to make the timing more stable when you copy your file into a ramfs and work there.