Lists

In the first tutorial we created a simple "Hello World"-like task in cuneiform, using Bash. In this tutorial, we want to introduce lists, allowing you to execute the same tasks with different input very easily. Let's have a look at the code.

deftask greet( out : person ) in bash *{
	out="Hello $person";
}*

greet(person: 'World' 'Wide' 'Web');

Comparing this to the previous tutorial, we don't apply a single value to the input parameter "person", but three different values. Cuneiform will interpret this as a list, which is fine for this task. How to specify the number of values in the task? Don't worry about that, Cuneiform will handle the input type itself. In this case the task "greet" is executed three times, once for each value in the list. It's the same as calling the task three times with one of each values (though the output is different, but more about that later on). Instead of directly writing the input for the greet task into the call, we could define a variable containing the values first and use this variable as input

deftask greet( out : person ) in bash *{
	out="Hello $person";
}*

list = 'World' 'Wide' 'Web';
greet(person: list);

As noted before, the output is different from a task call with a single input value. Instead of receiving a string with the value "Hello World", a string "Hello Wide" and a string "Hello Web", the output is a list of strings as well. You might have guessed by now, that this list contains the three string values mentioned before:

Query [...] finished: 'Hello World' 'Hello Wide' 'Hello Web'
We have seen a task with a single input parameter, what if more parameters receive a list as input value?

deftask table(out : row col) in bash *{
    out="$row$col";
 }*
 
 list1 = "a" "b" "c" "d";
 list2 = "1" "2" "3";
 table(row:list1 col:list2);

The workflow has the output:

'a1' 'a2' 'a3' 'b1' 'b2' 'b3' 'c1' 'c2' 'c3' 'd1' 'd2' 'd3'
As you see, each input value from list1 is combined with each value from list2. But what if you have lists as input and don't want to match each of the values with each other? This calls for compound lists. Compound lists are used to match each element of a list only with elements from other lists, that have the same position in the list:

deftask table(out : [a b]) in bash *{
    out="$a$b";
 }*
 
 list1 = "a" "b" "c";
 list2 = "1" "2" "3";
 table(a:list1 b:list2);

Instead of

'a1' 'a2' 'a3' 'b1' 'b2' 'b3' 'c1' 'c2' 'c3'
the output is
'a1' 'b2' 'c3'
All list elements are paired up nicely. To make this work, you have to ensure that the lists have an equal number of elements, otherwise an IndexOutOfBoundsException will be raised. Of course you can mix this concept with the cross product concept:

deftask table(out : [a b] c) in bash *{
    out="$a$b$c";
}*
 
 list1 = "a" "b" "c";
 list2 = "1" "2" "3";
 list3 = "Q" "W";
 table(a:list1 b:list2 c:list3);

This results in the following output:

Query [...] finished: 'a1Q' 'b2Q' 'c3Q' 'a1W' 'b2W' 'c3W'
You could also combine compound list structures:

deftask table(out : [a b] [c d]) in bash *{
    out="$a$b _$c$d";
}*
 
 list1 = "a" "b" "c";
 list2 = "1" "2" "3";
 list3 = "Q" "W";
 list4 = "8" "9";
 table(a:list1 b:list2 c:list3 d:list4);

Query [...] finished: 'a1 _Q8' 'b2 _Q8' 'c3 _Q8' 'a1 _W9' 'b2 _W9' 'c3 _W9'
My personal impression is that handling many lists can be confusing, so whenever you are in doubt whether all elements are matched with each other as you planned, create a dummy task with a simple body like the ones we use in this tutorial, to see if everythings works out nicely. As an example, this is one of the dummy tasks I had to use in a recent workflow:

deftask dummy1(out : [a b]) in bash *{
    out="$a$b";
}*

deftask dummy2(out : [c d] e) in bash *{
    out="$c$d $e";
}*
 
 l1 = "a" "b";
 l2 = "1" "2";
 l3 = "Q" "W";
 l4 = "8" "9";
 l5 = "x" "y" "z";
 c1 = dummy1(a: l1 b:l2);
 c2 = dummy1(a:l3 b:l4);
 dummy2(c:c1 d:c2 e:l5);

If you are using the CuneiformIDE (CFIDE), you can switch to the DAG panel to see the graph of your workflow. For example, the dummy workflow presented above generates the following graph: Directed acyclic graph for the dummy workflow Analysing the graph can give you further information if you happen to have problems with your workflow.

If you followed along, you might have guessed the following output:

Query [...] finished: 'a1Q8 x' 'a1Q8 y' 'a1Q8 z' 'b2W9 x' 'b2W9 y' 'b2W9 z'
You might tend to say "I could have done this with just one task". I won't say you cannot, maybe you see a way I cannot see at the moment, but the most obvious answer, a task with the signatur
deftask dummy2(out : [[a b] [c d]] e)
will result in a parsing error (which will tell you that the deftask is incomplete "Incomplete Task definition. Task name expected.") as you cannot use nested compound lists. As simple way around this problem is the presented example, just use another task to build you lists and use the output as input for the next task. As we have seen in the beginning of this tutorial, the output of tasks that receive lists as input is a list as well. We are not quiet finished yet, we have seen several options to use lists with tasks, but what if you want to use a complete list as input and not just the elements? Imagine you have task in native cuneiform that performs whatever work you want it to do with an element and a list as input. The task always uses the same list in this workflow (but it may be another list in another workflow) and combines this list with the single element that is the other input. How could you accomplish that? With the following construct:

deftask combine(out : <first> <second>) in bash *{
    out="${first[@]} ${second[@]}"
 }*
 
 a = "1" "2";
 l = "a" "b" "c";
 combine(first:a second:l);

Instead of combining each element of "first" with each element of "second" as before, which would result in a list of 2*3=6 elements in total, we receive one single element that contains all five elements from the input lists:

Query [...] finished: '1 2 a b c'
The task receives the whole list as a single element, which means by combining both lists to form the output, we actually chained the lists. Note that you have to use a different call for the variables inside the task body, if you are handling complete lists. The regular "$varname" is allowed though and maybe you want to do this, as it results in a different output:

deftask head(out : <element> <list>) in bash *{
    out=$element$list;
 }*
 
 a = "1" "2";
 l = "a" "b" "c";
 head(element:a list:l);

This time only the first element of each list is used. Think of it like the "head" methode of a queue.