The run times of scripts vary quite much.
I would like to find similar time -function as Matlab's timeit, described here.
For instance, I am evaluating these commands here about counting quickly matches, running time LC_ALL=C grep -ao CDA r328_0002.raw | wc -l
in loop
---------------------------------------------
Events real user sys
----------- ----------- ----------- ---------
40 0m0.044s 0m0.042s 0m0.005s
40 0m0.064s 0m0.062s 0m0.005s
40 0m0.046s 0m0.044s 0m0.005s
40 0m0.043s 0m0.042s 0m0.005s
40 0m0.047s 0m0.044s 0m0.005s
---------------------------------------------
Table: Events when Macbook Air 2013-Mid in Power Supply.
---------------------------------------------
Events real user sys
----------- ----------- ----------- ---------
40 0m0.056s 0m0.041s 0m0.011s
40 0m0.060s 0m0.047s 0m0.008s
40 0m0.041s 0m0.039s 0m0.006s
40 0m0.046s 0m0.044s 0m0.006s
40 0m0.047s 0m0.045s 0m0.006s
---------------------------------------------
Table: Events when Macbook Air in Battery Supply, 6h later.
where you see the real-time fluctuates from 0.044s to 0.064s, the user-time from 0.042s to 0.062s, while the sys-time keeps rather stable at 0.005s.
My idea in timing
- iterate the command first 1k before timing
- do timing 10 times and take the average and standard deviation
Stout to /dev/nul
This idea is in lcd047's comment, running time LC_ALL=C ggrep -ao CDA r328_0002.raw >/dev/null
in loop
--------------------------------------------
real user sys
-------------- -------------- ------------
0m0.006s 0m0.003s 0m0.002s
0m0.006s 0m0.003s 0m0.002s
0m0.006s 0m0.003s 0m0.002s
0m0.008s 0m0.003s 0m0.003s
0m0.006s 0m0.003s 0m0.002s
0m0.005s 0m0.002s 0m0.002s
0m0.006s 0m0.002s 0m0.002s
0m0.009s 0m0.003s 0m0.003s
0m0.007s 0m0.003s 0m0.003s
0m0.006s 0m0.003s 0m0.002s
0m0.006s 0m0.003s 0m0.002s
0m0.008s 0m0.003s 0m0.003s
--------------------------------------------
Table: Events when Macbook Air 2013-Mid in Battery Supply.
I think these times can be even more improved when keeping the laptop in power supply and keeping less programs on.
How can you time stably shell scripts?
Best Answer
The answer is: You can not! Linux is no real time system. The idea of UNIX and therefore Linux, too, is to provide minimum answer times, while the system is shared between multiple users and system processes. Depending on when you start the command, you might have to wait for a important system process to give you your share of processor time. Further the file system might buffer your file you read from the disk, but eventually these file system buffer fail to load the data from cache, when another process on your system occupies that. Generally the time a process needs in a Linux system depends on the surrounding entropy of the machine, where ever it is located in time and space in the universe.
You will need a real time system and specially real time tuned commands and a specific amount of resources reserved for you only. You might get somewhere near that with new CGROUP features from newer kernels, where you can reserve a processor, a part of the memory and an own reflection of the underlying file system for your environment.
One of the major timing problems in your example is the way grep and wc reads the inputs. You might be able to make the timing more stable when you copy your file into a ramfs and work there.