use tasksref

Showcase gradually the lengthy configuration evolve to simpler skeleton by using ref element to externalized task/flow

Externalized the task definition - c0062-tasks-main.yml

    flow:
      -
        func: shell
        do:
          - echo "task step 1"
          - echo "task step 2"
    
      -
        func: call
        do:
          - task_a
          - task_c
          - task_a
        if: '{{eq .student_name "tom"}}'
    
      -
        func: call
        do:
          - task_b
        loop: "classes"
        if: '{{eq .student_name "tom"}}'
    
      -
        func: shell
        do:
          - echo "task step 3"
          - echo "task step 4"
    
      -
        func: call
        do:
          - a_very_complicated_task

Externalized the task definition - c0062-tasks-simple.yml

    
    tasks:
    
      -
        name: task_a
        task:
          -
            func: shell
            do:
              - echo "task_a_step1"
      -
        name: task_c
        task:
          -
            func: shell
            do:
              - echo "task_c_step1"
    

Externalized the task definition - c0062-tasks-complicated.yml

    
    tasks:
    
      -
        name: a_very_complicated_task
        desc: "a_very_complicated_task"
        ref: c0061-task-complicate.yml
    
      -
        name: task_b
        task:
          -
            func: shell
            do:
              - echo "task_b_step1"
              - echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    

tasksref element

Introduce tasksref to manage a list of tasks, in such a way it offloads the less concerned tasks to different locations. You could manage your tasks in flexible way with your own preference, such as classify them into different names of folders

Demo

source

Main task yaml file
    vars:
      student_name: tom
      classes: [1k, 2b, 3j, 4s]
    tasksref:
    - c0062-tasks-simple.yml
    - c0062-tasks-complicated.yml
    tasks:
    - name: task
      ref: c0062-task-main.yml
    
Main log file
    loading [Config]:  ./tests/functests/upconfig.yml
    Main config:
                 Version -> 1.0.0
                  RefDir -> ./tests/functests
                 WorkDir -> cwd
              AbsWorkDir -> /up_project/up
                TaskFile -> c0062
                 Verbose -> vvv
              ModuleName -> self
               ShellType -> /bin/sh
           MaxCallLayers -> 8
                 Timeout -> 3600000
     MaxModuelCallLayers -> 256
               EntryTask -> task
      ModRepoUsernameRef -> 
      ModRepoPasswordRef -> 
    work dir: /up_project/up
    -exec task: task
    loading [Task]:  ./tests/functests/c0062
    module: [self], instance id: [dev], exec profile: []
    profile -  envVars:
    
    (*core.Cache)({
    })
    
    loading [c0062-tasks-simple.yml]:  ./tests/functests/c0062-tasks-simple.yml
    loading [c0062-tasks-complicated.yml]:  ./tests/functests/c0062-tasks-complicated.yml
    loading [flow ref]:  ./tests/functests/c0062-task-main.yml
    loading [flow ref]:  ./tests/functests/c0061-task-complicate.yml
    Task1: [task ==> task:  ]
    -Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "student_name": "tom",
      "up_runtime_task_layer_number": 0
    })
    
    cmd( 1):
    echo "task step 1"
    
    -
    task step 1
    
    -
     .. ok
    cmd( 2):
    echo "task step 2"
    
    -
    task step 2
    
    -
     .. ok
    . ok
    -Step2:
    self: final context exec vars:
    
    (*core.Cache)({
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "up_runtime_task_layer_number": 0
    })
    
    =Task2: [task ==> task_a:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "up_runtime_task_layer_number": 1,
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "student_name": "tom"
    })
    
    cmd( 1):
    echo "task_a_step1"
    
    -
    task_a_step1
    
    -
     .. ok
    . ok
    =Task3: [task ==> task_c:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task_a_step1\"",
        Code: 0,
        Output: "task_a_step1",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "up_runtime_task_layer_number": 1,
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      }
    })
    
    cmd( 1):
    echo "task_c_step1"
    
    -
    task_c_step1
    
    -
     .. ok
    . ok
    =Task2: [task ==> task_a:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "up_runtime_task_layer_number": 1,
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task_c_step1\"",
        Code: 0,
        Output: "task_c_step1",
        ErrMsg: ""
      })
    })
    
    cmd( 1):
    echo "task_a_step1"
    
    -
    task_a_step1
    
    -
     .. ok
    . ok
    -Step3:
    self: final context exec vars:
    
    (*core.Cache)({
      "up_runtime_task_layer_number": 1,
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      }
    })
    
    =Task5: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "up_runtime_task_layer_number": 1,
      "loopitem": "1k",
      "loopindex": 0,
      "loopindex1": 1,
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      }
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    0 -> student nameed tom has been in class [1k]
    
    -
     .. ok
    . ok
    =Task5: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "loopitem": "2b",
      "loopindex": 1,
      "loopindex1": 2,
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"0 -> student nameed tom has been in class [1k]\"",
        Code: 0,
        Output: "0 -> student nameed tom has been in class [1k]",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "up_runtime_task_layer_number": 1
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    1 -> student nameed tom has been in class [2b]
    
    -
     .. ok
    . ok
    =Task5: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"1 -> student nameed tom has been in class [2b]\"",
        Code: 0,
        Output: "1 -> student nameed tom has been in class [2b]",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "up_runtime_task_layer_number": 1,
      "loopitem": "3j",
      "loopindex": 2,
      "loopindex1": 3
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    2 -> student nameed tom has been in class [3j]
    
    -
     .. ok
    . ok
    =Task5: [task ==> task_b:  ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "loopindex1": 4,
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"2 -> student nameed tom has been in class [3j]\"",
        Code: 0,
        Output: "2 -> student nameed tom has been in class [3j]",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "up_runtime_task_layer_number": 1,
      "loopitem": "4s",
      "loopindex": 3
    })
    
    cmd( 1):
    echo "task_b_step1"
    
    -
    task_b_step1
    
    -
     .. ok
    cmd( 2):
    echo "{{.loopindex}} -> student nameed {{.student_name}} has been in class [{{.loopitem}}]"
    
    -
    3 -> student nameed tom has been in class [4s]
    
    -
     .. ok
    . ok
    -Step4:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 2\"",
        Code: 0,
        Output: "task step 2",
        ErrMsg: ""
      }),
      "up_runtime_task_layer_number": 1
    })
    
    cmd( 1):
    echo "task step 3"
    
    -
    task step 3
    
    -
     .. ok
    cmd( 2):
    echo "task step 4"
    
    -
    task step 4
    
    -
     .. ok
    . ok
    -Step5:
    self: final context exec vars:
    
    (*core.Cache)({
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 4\"",
        Code: 0,
        Output: "task step 4",
        ErrMsg: ""
      }),
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "up_runtime_task_layer_number": 1
    })
    
    =Task4: [task ==> a_very_complicated_task: a_very_complicated_task ]
    --Step1:
    self: final context exec vars:
    
    (*core.Cache)({
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "up_runtime_task_layer_number": 1,
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"task step 4\"",
        Code: 0,
        Output: "task step 4",
        ErrMsg: ""
      }),
      "student_name": "tom"
    })
    
    cmd( 1):
    echo "i am a very complicated flow of step1"
    
    -
    i am a very complicated flow of step1
    
    -
     .. ok
    cmd( 2):
    echo "i am a very complicated flow of step2"
    
    -
    i am a very complicated flow of step2
    
    -
     .. ok
    cmd( 3):
    echo "i am a very complicated flow of step3"
    
    -
    i am a very complicated flow of step3
    
    -
     .. ok
    . ok
    --Step2:
    self: final context exec vars:
    
    (*core.Cache)({
      "student_name": "tom",
      "classes": {
        "1k",
        "2b",
        "3j",
        "4s"
      },
      "up_runtime_task_layer_number": 1,
      "last_result": (*utils.ExecResult)({
        Cmd: "echo \"i am a very complicated flow of step3\"",
        Code: 0,
        Output: "i am a very complicated flow of step3",
        ErrMsg: ""
      })
    })
    
    cmd( 1):
    echo "i am a very complicated flow of step4"
    
    -
    i am a very complicated flow of step4
    
    -
     .. ok
    cmd( 2):
    echo "{{.student_name}}"
    
    -
    tom
    
    -
     .. ok
    . ok
    
Logs with different verbose level
Raw logs with different verbose level