How to return an array in bash without using globa

Question

I have a function that creates an array and I want to return the array to the caller:

create_array() {
  local my_list=("a", "b", "c")
  echo "${my_list[@]}"
}

my_algorithm() {
  local result=$(create_array)
}

With this, I only get an expanded string. How can I "return" my_list without using anything global?

Answer 1

I tried various implementations, and none preserved arrays that had elements with spaces ... because they all had to use echo.

# These implementations only work if no array items contain spaces.
use_array() {  eval echo  '(' \"\${${1}\[\@\]}\" ')';  }
use_array() {  local _array="${1}[@]"; echo '(' "${!_array}" ')';  }

Solution

Then I came across Dennis Williamson's answer. I incorporated his method into the following functions so they can a) accept an arbitrary array and b) be used to pass, duplicate and append arrays.

# Print array definition to use with assignments, for loops, etc.
#   varname: the name of an array variable.
use_array() {
    local r=$( declare -p $1 )
    r=${r#declare\ -a\ *=}
    # Strip keys so printed definition will be a simple list (like when using
    # "${array[@]}").  One side effect of having keys in the definition is 
    # that when appending arrays (i.e. `a1+=$( use_array a2 )`), values at
    # matching indices merge instead of pushing all items onto array.
    echo ${r//\[[0-9]\]=}
}
# Same as use_array() but preserves keys.
use_array_assoc() {
    local r=$( declare -p $1 )
    echo ${r#declare\ -a\ *=}
}  

Then, other functions can return an array using catchable output or indirect arguments.

# catchable output
return_array_by_printing() {
    local returnme=( "one" "two" "two and a half" )
    use_array returnme
}
eval test1=$( return_array_by_printing )

# indirect argument
return_array_to_referenced_variable() {
    local returnme=( "one" "two" "two and a half" )
    eval $1=$( use_array returnme )
}
return_array_to_referenced_variable test2

# Now both test1 and test2 are arrays with three elements

Answer 2

Bash can't pass around data structures as return values. A return value must be a numeric exit status between 0-255. However, you can certainly use command or process substitution to pass commands to an eval statement if you're so inclined.

This is rarely worth the trouble, IMHO. If you must pass data structures around in Bash, use a global variable--that's what they're for. If you don't want to do that for some reason, though, think in terms of positional parameters.

Your example could easily be rewritten to use positional parameters instead of global variables:

use_array () {
    for idx in "$@"; do
        echo "$idx"
    done
}

create_array () {
    local array=("a" "b" "c")
    use_array "${array[@]}"
}

This all creates a certain amount of unnecessary complexity, though. Bash functions generally work best when you treat them more like procedures with side effects, and call them in sequence.

# Gather values and store them in FOO.
get_values_for_array () { :; }

# Do something with the values in FOO.
process_global_array_variable () { :; }

# Call your functions.
get_values_for_array
process_global_array_variable

If all you're worried about is polluting your global namespace, you can also use the unset builtin to remove a global variable after you're done with it. Using your original example, let my_list be global (by removing the local keyword) and add unset my_list to the end of my_algorithm to clean up after yourself.

Answer 3

With Bash version 4.3 and above, you can make use of a nameref so that the caller can pass in the array name and the callee can use a nameref to populate the named array, indirectly.

#!/usr/bin/env bash

create_array() {
    local -n arr=$1              # use nameref for indirection
    arr=(one "two three" four)
}

use_array() {
    local my_array
    create_array my_array       # call function to populate the array
    echo "inside use_array"
    declare -p my_array         # test the array
}

use_array                       # call the main function

Produces the output:

inside use_array
declare -a my_array=([0]="one" [1]="two three" [2]="four")

You could make the function update an existing array as well:

update_array() {
    local -n arr=$1              # use nameref for indirection
    arr+=("two three" four)      # update the array
}

use_array() {
    local my_array=(one)
    update_array my_array       # call function to update the array
}

This is a more elegant and efficient approach since we don't need command substitution $() to grab the standard output of the function being called. It also helps if the function were to return more than one output - we can simply use as many namerefs as the number of outputs.

---

Here is what the Bash Manual says about nameref:

A variable can be assigned the nameref attribute using the -n option to the declare or local builtin commands (see Bash Builtins) to create a nameref, or a reference to another variable. This allows variables to be manipulated indirectly. Whenever the nameref variable is referenced, assigned to, unset, or has its attributes modified (other than using or changing the nameref attribute itself), the operation is actually performed on the variable specified by the nameref variable’s value. A nameref is commonly used within shell functions to refer to a variable whose name is passed as an argument to the function. For instance, if a variable name is passed to a shell function as its first argument, running

declare -n ref=$1 inside the function creates a nameref variable ref whose value is the variable name passed as the first argument. References and assignments to ref, and changes to its attributes, are treated as references, assignments, and attribute modifications to the variable whose name was passed as $1.

Answer 4

I needed a similar functionality recently, so the following is a mix of the suggestions made by RashaMatt and Steve Zobell.

1. echo each array/list element as separate line from within a function
2. use mapfile to read all array/list elements echoed by a function.

As far as I can see, strings are kept intact and whitespaces are preserved.

#!bin/bash

function create-array() {
  local somearray=("aaa" "bbb ccc" "d" "e f g h")
  for elem in "${somearray[@]}"
  do
    echo "${elem}"
  done
}

mapfile -t resa <<< "$(create-array)"

# quick output check
declare -p resa

Some more variations…

#!/bin/bash

function create-array-from-ls() {
  local somearray=("$(ls -1)")
  for elem in "${somearray[@]}"
  do
    echo "${elem}"
  done
}

function create-array-from-args() {
  local somearray=("$@")
  for elem in "${somearray[@]}"
  do
    echo "${elem}"
  done
}


mapfile -t resb <<< "$(create-array-from-ls)"
mapfile -t resc <<< "$(create-array-from-args 'xxx' 'yy zz' 't s u' )"

sentenceA="create array from this sentence"
sentenceB="keep this sentence"

mapfile -t resd <<< "$(create-array-from-args ${sentenceA} )"
mapfile -t rese <<< "$(create-array-from-args "$sentenceB" )"
mapfile -t resf <<< "$(create-array-from-args "$sentenceB" "and" "this words" )"

# quick output check
declare -p resb
declare -p resc
declare -p resd
declare -p rese
declare -p resf

Answer 5

If your source data is formatted with each list element on a separate line, then the mapfile builtin is a simple and elegant way to read a list into an array:

$ list=$(ls -1 /usr/local)           # one item per line

$ mapfile -t arrayVar <<<"$list"     # -t trims trailing newlines

$ declare -p arrayVar | sed 's#\[#\n[#g'
declare -a arrayVar='(
[0]="bin"
[1]="etc"
[2]="games"
[3]="include"
[4]="lib"
[5]="man"
[6]="sbin"
[7]="share"
[8]="src")'

Note that, as with the read builtin, you would not ordinarily* use mapfile in a pipeline (or subshell) because the assigned array variable would be unavailable to subsequent statements (* unless bash job control is disabled and shopt -s lastpipe is set).

$ help mapfile
mapfile: mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] [array]
    Read lines from the standard input into an indexed array variable.

    Read lines from the standard input into the indexed array variable ARRAY, or
    from file descriptor FD if the -u option is supplied.  The variable MAPFILE
    is the default ARRAY.

    Options:
      -n count  Copy at most COUNT lines.  If COUNT is 0, all lines are copied.
      -O origin Begin assigning to ARRAY at index ORIGIN.  The default index is 0.
      -s count  Discard the first COUNT lines read.
      -t                Remove a trailing newline from each line read.
      -u fd             Read lines from file descriptor FD instead of the standard input.
      -C callback       Evaluate CALLBACK each time QUANTUM lines are read.
      -c quantum        Specify the number of lines read between each call to CALLBACK.

    Arguments:
      ARRAY             Array variable name to use for file data.

    If -C is supplied without -c, the default quantum is 5000.  When
    CALLBACK is evaluated, it is supplied the index of the next array
    element to be assigned and the line to be assigned to that element
    as additional arguments.

    If not supplied with an explicit origin, mapfile will clear ARRAY before
    assigning to it.

    Exit Status:
    Returns success unless an invalid option is given or ARRAY is readonly or
    not an indexed array.

Answer 6

A pure bash, minimal and robust solution based on the 'declare -p' builtin — without insane global variables

This approach involves the following three steps:

1. Convert the array with 'declare -p' and save the output in a variable.
   myVar="$( declare -p myArray )"
   The output of the declare -p statement can be used to recreate the array. For instance the output of declare -p myVar might look like this:
   declare -a myVar='([0]="1st field" [1]="2nd field" [2]="3rd field")'
2. Use the echo builtin to pass the variable to a function or to pass it back from there.
   • In order to preserve whitspaces in array fields when echoing the variable, IFS is temporarly set to a control character (e.g. a vertical tab).
   • Only the right-hand-side of the declare statement in the variable is to be echoed - this can be achieved by parameter expansion of the form ${parameter#word}. As for the example above: ${myVar#*=}
3. Finally, recreate the array where it is passed to using the eval and the 'declare -a' builtins.

Example 1 - return an array from a function

#!/bin/bash

# Example 1 - return an array from a function

function my-fun () {
 # set up a new array with 3 fields - note the whitespaces in the
 # 2nd (2 spaces) and 3rd (2 tabs) field
 local myFunArray=( "1st field" "2nd  field" "3rd       field" )

 # show its contents on stderr (must not be output to stdout!)
 echo "now in $FUNCNAME () - showing contents of myFunArray" >&2
 echo "by the help of the 'declare -p' builtin:" >&2
 declare -p myFunArray >&2

 # return the array
 local myVar="$( declare -p myFunArray )"
 local IFS=$'\v';
 echo "${myVar#*=}"

 # if the function would continue at this point, then IFS should be
 # restored to its default value: <space><tab><newline>
 IFS=' '$'\t'$'\n';
}

# main

# call the function and recreate the array that was originally
# set up in the function
eval declare -a myMainArray="$( my-fun )"

# show the array contents
echo ""
echo "now in main part of the script - showing contents of myMainArray"
echo "by the help of the 'declare -p' builtin:"
declare -p myMainArray

# end-of-file

Output of Example 1:

now in my-fun () - showing contents of myFunArray
by the help of the 'declare -p' builtin:
declare -a myFunArray='([0]="1st field" [1]="2nd  field" [2]="3rd       field")'

now in main part of the script - showing contents of myMainArray
by the help of the 'declare -p' builtin:
declare -a myMainArray='([0]="1st field" [1]="2nd  field" [2]="3rd      field")'

---

Example 2 - pass an array to a function

#!/bin/bash

# Example 2 - pass an array to a function

function my-fun () {
 # recreate the array that was originally set up in the main part of
 # the script
 eval declare -a myFunArray="$( echo "$1" )"

 # note that myFunArray is local - from the bash(1) man page: when used
 # in a function, declare makes each name local, as with the local
 # command, unless the ‘-g’ option is used.

 # IFS has been changed in the main part of this script - now that we
 # have recreated the array it's better to restore it to the its (local)
 # default value: <space><tab><newline>
 local IFS=' '$'\t'$'\n';

 # show contents of the array
 echo ""
 echo "now in $FUNCNAME () - showing contents of myFunArray"
 echo "by the help of the 'declare -p' builtin:"
 declare -p myFunArray
}

# main

# set up a new array with 3 fields - note the whitespaces in the
# 2nd (2 spaces) and 3rd (2 tabs) field
myMainArray=( "1st field" "2nd  field" "3rd     field" )

# show the array contents
echo "now in the main part of the script - showing contents of myMainArray"
echo "by the help of the 'declare -p' builtin:"
declare -p myMainArray

# call the function and pass the array to it
myVar="$( declare -p myMainArray )"
IFS=$'\v';
my-fun $( echo "${myVar#*=}" )

# if the script would continue at this point, then IFS should be restored
# to its default value: <space><tab><newline>
IFS=' '$'\t'$'\n';

# end-of-file

Output of Example 2:

now in the main part of the script - showing contents of myMainArray
by the help of the 'declare -p' builtin:
declare -a myMainArray='([0]="1st field" [1]="2nd  field" [2]="3rd      field")'

now in my-fun () - showing contents of myFunArray
by the help of the 'declare -p' builtin:
declare -a myFunArray='([0]="1st field" [1]="2nd  field" [2]="3rd       field")'