Is it possible to build a dynamic task list in nodejs Async (waterfall, series, etc...)

Question

I am pulling information from some collections in mongo that contain node and edge data. First i must get the node so i can grab its edges. Once i have a list of edges i then go back out and grab more nodes (etc.. based on a depth value). The following code is an loose example of how i am attempting to use async.waterfall and the task list. Initially i have only a single task but once i make my first query i add to the task array. Unfortunately this does not seem to register with async and it does not continue to process the tasks i am adding. Is there a better way to do this? ``` var async = require('async') var mongoose = require('mongoose') var _ = requrie('underscore') var Client = this.Mongo.connect(/*path to mongo*/) var Node = mongoose.Schema({ id : String, graph_id : String }) var Edge = mongoose.Schema({ id : String, source_id : String, destination_id : String }) var Nodes = Client.model('myNode', Node) var Edges = Client.model('myEdge', Edge) var funcs = [] var round = 1 var…

Accepted Answer

A task in your function_array can only add a new task to the array provided it is NOT the last task in the array.

In your case, your function_array contained only 1 task. That task itself cannot add additional tasks since it's the last task.

The solution is to have 2 tasks in the array. A startTask to bootstrap the process, and a finalTask that is more of a dummy task. In that case,

`function_array = [startTask, finalTask];`

Then startTask would add taskA, taskB will add task C and eventually

`function_array = [startTask, taskA, taskB, taskC, finalTask];`

The sample code below that illustrates the concepts.

```
    var async = require('async');
    var max = 6;

var nodeTask = function(taskId, value, callback){
        var r = Math.floor(Math.random() * 20) + 1;
        console.log("From Node Task %d: %d", taskId, r);

// add an edge task
        if (taskId < max) {
            function_array.splice(function_array.length-1, 0, edgeTask);
        }

callback(null, taskId + 1, value + r);
    };

var edgeTask = function(taskId, value, callback){
        var r = Math.floor(Math.random() * 20) + 1;
        console.log("From Edge Task %d: %d", taskId, r);

// add a node task
        if (taskId < max) {
            function_array.splice(function_array.length-1, 0, nodeTask);
        }

callback(null, taskId + 1, value + r);
    };

var startTask = function(callback) {
        function_array.splice(function_array.length-1, 0, nodeTask);
        callback(null, 1, 0);
    };

var finalTask = function(taskId, value, callback) {
        callback(null, value);
    };

var function_array = [startTask, finalTask];

async.waterfall(function_array, function (err, result) {
        console.log("Sum is ", result);
    });
```

Is it possible to build a dynamic task list in nodejs Async (waterfall, series, etc...)

Solution

Problem