Shell – Split zsh array from subshell by linebreak

arrayshell-scriptzsh

I would like to instantiate a zsh array from a subshell

myarray=($(somecommand)

and I check if I received what I wanted with

for element in $myarray ; do echo "===" ; echo $element ; echo "---" ; done

By default I see that whitespaces (spaces and line breaks) are used to separate elements.

I also found that I can use ${(s:-:)"$(somecommand)"} to separate by -, in which case spaces and line breaks do not break elements (i.e. array elements can contain line breaks).

So far I fail to split at line breaks only. I.e. if somecommand returns

Alice
Bob
Arthur C Clarke
Neo
Thomas Anderson

I want my above for loop to print:

===
Alice
---
===
Bob
---
===
Arthur C Clarke
---
===
Neo
---
===
Thomas Anderson
---

How do I achieve that? (And possibly pointers where to look that up in the manual.)

Best Answer

The native splitting operator (beside the Bourne-like $IFS word splitting which is done on unquoted command substitutions) is with the s parameter expansion flag:

array=(${(ps:\n:)"$(cmd)"})

would split the output of cmd on newline, discarding the empty elements (the empty lines).

p is to enable those \x expansions. As ps:\n: is a common one, it's got a shorter alias: f (to split on line feeds):

array=(${(f)"$(cmd)"})

To preserve the empty lines, you'd do:

array=("${(f@)$(cmd)"})

Now, beware that like in most other shells, command substitution strips all the trailing newline characters, so all trailing empty lines. To preserve those, you can do:

array=("${(f@)$(cmd; echo .)}")
array[-1]=()  # remove that last line added by echo .

With $IFS word splitting:

IFS=$'\n'
array=($(cmd)) # removes empty lines. Note that contrary to other
               # Bourne-like shells, zsh doesn't do globbing there
               # so you don't need the "set -o noglob"

IFS=$'\n\n' # like in ksh93, doubling an IFS-whitespace character
            # removes its special treatment as a whitespace character
array=($(cmd)) # preserves empty lines except the trailing ones

IFS=$'\n\n'
array=($(cmd; echo .)); array[-1]=() # preserves all empty lines.

To avoid modifying $IFS globally, you can do the above in an anonymous function:

(){
  local IFS=$'\n\n'
  array=($(cmd; echo .)); array[-1]=()
}

Also beware that the $array expansion skips the empty elements. So if you wanted to loop over all the elements including the empty ones, you'd need:

for i ("$array[@]") ...

for i ("${(@)array}") ...

not

for i ($array) ...
for i in $array

Related Solutions

Zsh – How to Collect an Array of Lines Properly

TL, DR:

array_of_lines=("${(@f)$(my_command)}")

First mistake (→ Q2): IFS='\n' sets IFS to the two characters \ and n. To set IFS to a newline, use IFS=$'\n'.

Second mistake: to set a variable to an array value, you need parentheses around the elements: array_of_lines=(foo bar).

This would work, except that it strips empty lines, because consecutive whitespace counts as a single separator:

IFS=$'\n' array_of_lines=($(my_command))

You can retain the empty lines except at the very end by doubling the whitespace character in IFS:

IFS=$'\n\n' array_of_lines=($(my_command))

To keep trailing empty lines as well, you'd have to add something to the command's output, because this happens in the command substitution itself, not from parsing it.

IFS=$'\n\n' array_of_lines=($(my_command; echo .)); unset 'array_of_lines[-1]'

(assuming the output of my_command doesn't end in a non-delimited line; also note that you lose the exit status of my_command)

Note that all the snippets above leave IFS with its non-default value, so they may mess up subsequent code. To keep the setting of IFS local, put the whole thing into a function where you declare IFS local (here also taking care of preserving the command's exit status):

collect_lines() {
  local IFS=$'\n\n' ret
  array_of_lines=($("$@"; ret=$?; echo .; exit $ret))
  ret=$?
  unset 'array_of_lines[-1]'
  return $ret
}
collect_lines my_command

But I recommend not to mess with IFS; instead, use the f expansion flag to split on newlines (→ Q1):

array_of_lines=("${(@f)$(my_command)}")

Or to preserve trailing empty lines:

array_of_lines=("${(@f)$(my_command; echo .)}")
unset 'array_of_lines[-1]'

The value of IFS doesn't matter there. I suspect that you used a command that splits on IFS to print $array_of_lines in your tests (→ Q3).

Shell – echo result from subshell in zsh

In zsh, unquoted variables are not automatically split and glob, while unquoted command substitution will be split into words using values in IFS.

In your case, the first command saved content of file /etc/hosts to variable a, echo $a prints value of a variable without split and glob, you get the contents of /etc/hosts (split and glob are not performed in RHS of variable assignment).

echo $(</etc/hosts) used unquoted command substitution, the result will be split into words, so you get content of /etc/hosts as all words delimited by space.

Best Answer

Related Solutions

Zsh – How to Collect an Array of Lines Properly

Shell – echo result from subshell in zsh

Related Question