Awk inplace option with print statement

awkreplace

In recent version of awk there is a inplace option that can be used to do in place operations similar to sed's -i option. However I can't make it work with print statement. Let's see my example.
The content of the file test is:

11 aa
22 bb

root@localhost:~# cat test
11 aa
22 bb

If I don't use -i inplace, I can get my desired result in the console.

root@localhost:~# awk 'BEGIN{print "begin"} $1=="11"{print "111" $2; next} 1; END{print "end"}' test
begin
111aa
22 bb
end

When I add -i inplace, this is what I got.

root@localhost:~# awk -i inplace 'BEGIN{print "begin"} $1=="11"{print "111" $2; next} 1; END{print "end"}' test
begin
end
root@localhost:~# cat test
111aa
22 bb
root@localhost:~#

How to improve my code and get what I want?

Update

My awk version is 4.1.1.

root@localhost:~# awk --version
GNU Awk 4.1.1, API: 1.1 (GNU MPFR 3.1.2-p3, GNU MP 6.0.0)
Copyright (C) 1989, 1991-2014 Free Software Foundation.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
root@localhost:~#

Update2

root@localhost:~# awk --version
GNU Awk 4.1.2, API: 1.1
Copyright (C) 1989, 1991-2015 Free Software Foundation.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program. If not, see http://www.gnu.org/licenses/.
root@localhost:~# cat file
11 aa
22 bb
root@localhost:~# awk 'BEGIN{print "begin"} $1=="11"{print "111" $2; next} 1; END{print "end"}' file
begin
111aa
22 bb
end
root@localhost:~# awk -i inplace 'BEGIN{print "begin"} $1=="11"{print "111" $2; next} 1; END{print "end"}' file
begin
end
root@localhost:~# cat file
111aa
22 bb
root@localhost:~#

Best Answer

I think you need to use BEGINFILE instead of BEGIN and ENDFILE instead of END for the begin and end to print into the file.

At least that works on cygwin.

Rob@Rob-PC /cygdrive/c/tmp
$ awk --version
GNU Awk 4.1.1, API: 1.1 (GNU MPFR 3.1.2, GNU MP 6.0.0)
Copyright (C) 1989, 1991-2014 Free Software Foundation.


Rob@Rob-PC /cygdrive/c/tmp
$ echo "11 aa
22 bb" > test

Rob@Rob-PC /cygdrive/c/tmp
$ awk -i inplace 'BEGINFILE{print "begin"} $1=="11"{print "111" $2; next} 1; ENDFILE{print "end" >> "test"}' test

Rob@Rob-PC /cygdrive/c/tmp
$ cat test
begin
111aa
22 bb
end

Related Solutions

Pass Colors from ls Through Pipe to awk Print Statement

By using ls --color=always the colors should be preserved. I'm unsure if it is gnu specific though. [Edit from OP: yes, it is.]

The case is that ls detect whether output is to tty or not.

If it is not it normally do not print colors and do not translate "unprintable characters" to question marks. This can be added by the -q option.

More on the subject: Number of lines outputted by ls.

gnu coreutils source ls; ref. on some of the colors data:

Edit:

For gnu coreutils ls: --color=always
For OS X's ls: set environment variable CLICOLOR_FORCE and G means color. The core is conforming to ls IEEE Std 1003.1-2001. Other options are extensions that wary between implementations.

OK. Had a deeper look at this. Though from what I read around people also install gnu coreutils on OS X, I guess you perhaps use the original.

The OS X man pages for ls state:

CLICOLOR_FORCE  
    Color sequences are normally disabled if the output isn't directed to a
    terminal.  This can be overridden by setting this flag.  The TERM 
    variable still needs to reference a color capable terminal however
    otherwise it is not possible to determine which color sequences to use.

From the source:

Mac OS X 10.8.2 Source -> file_cmds-220.7 -> ls/

From that code it is also quickly clear that one can set

CLICOLOR_FORCE

to force color.

As it first parses arguments and setenv color on G:

    case 'G':
        setenv("CLICOLOR", "", 1);
        break;

And after argv parsing:

/* CLICOLOR is set    AND    is a tty            OR */
if (getenv("CLICOLOR") && (isatty(STDOUT_FILENO) || 
etenv("CLICOLOR_FORCE")))
/*  force cli color is set */

By next section if TERM is set, color is used:

if (tgetent(termcapbuf, getenv("TERM")) == 1) {

echo $TERM should yield a color capable terminal. But if you get colors by normal this should be the case.

---

Edit to last update:

Not sure then. It is strange. Not only the extra line but also that you do not have any total ... at the start. You could start out by checking the output of something like this:

function dir() {
/bin/ls -FaGl "${@}" | awk '
    function chr2hex(chr) {
        return sprintf("%02x", index(i2x, chr));
    }
    function str2hex(str) {
        r = ""
        for (i = 1; i <= length(str); ++i)
            r = r chr2hex(substr(str, i, 1));
        return r;
    }
    BEGIN {
        i2x = ""
        for (i = 0; i < 256; ++i)
            i2x = sprintf("%s%c", i2x, i);
        printf("FNR        : %d\n", FNR);
        printf("FIELDWIDTHS: \"%s\"\n", FIELDWIDTHS);
        printf("FS         : \"%s\"\n", str2hex(FS));
        printf("RS         : \"%s\"\n", str2hex(RS));
        printf("OFS        : \"%s\"\n", str2hex(OFS));
        printf("ORS        : \"%s\"\n", str2hex(OFS));
    }
    {
        printf("%2d:%2d:%2d \"%s\"\n", FNR, NR, NF, $0);
        total += $4;
    }
    END {
        printf("%21s: %d %s\n", "total", total / 1024, "KiB");
        printf("FNR        : %d\n", FNR);
        printf("From       : \"%s\"\n", FILENAME);
    }
'
}

Should give you something like – (From could be "-" by e.g. gawk):

$ ./dir testdir
FNR        : 0
FIELDWIDTHS: ""
FS         : "20"
RS         : "0a"
OFS        : "20"
ORS        : "20"
 1: 1: 2 "total 24"
 2: 2: 8 "drwxrwxr-x 6 mattdmo 4096 Apr 17 21:46 ./"
 3: 3: 8 "drwxrwxr-x 8 mattdmo 4096 Apr 18 10:47 ../"
 4: 4: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab1/"
 5: 5: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab2/"
 6: 6: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab3/"
 7: 7: 8 "drwxrwxr-x 2 mattdmo 4096 Apr 17 21:46 ab4/"
                total: 24 KB
FNR        : 7
From       : ""

Unix awk begin statement

That's because you did not put your print in the BEGIN. All you have in the BEGIN block is OFS="\t". Which, by the way, means you don't need to add "\t" to your print call. So, what you're after is (I changed the formatting a bit for clarity):

awk 'BEGIN { OFS="\t"; 
            print "MARKER", "CHR", "BP", "EA", "NEA","EAF", "P","OR","SE", 
             "OR_95L","OR95U", "N","N_CASES", "N_CONTROLS", "STRAND","INFO", 
             "HWE_P","IMPUTED"
        }
FNR>16 && $45!="NA" && $9>=0.4 { 
    if ($1=="---") print $2,"'"$chr"'",$4,$6, $5,$45,$42,$48, 
                     $43,$44,$18,$23, $28,"+",$9,$33,"0" ; 
    else print $2,"'"$chr"'",$4,$6,$5,$45, $42,$48,$43,$44,$18, 
               $23,$28,"+",$9,$33,"1" 
}' ./out/expected_dcct_1kg_only${chr}_${chunk}.res > \
  ./temp/expected_dcct_1kg_chr${chr}_${chunk}.tmp