DARPA ULTRALOG Final Report - Industrial and Manufacturing ...

Ultra*Log 

PSU/IAI Final Report for Ultra*Log 

Document Revision Number: 2.0 

Date: 09/01/2005 

Prepared for: 

Defense Advanced Research Projects Agency 

Information Systems Office 

3701 North Fairfax Drive 

Arlington, VA 22203-1714 

Prepared by: 

The Pennsylvania State University 

Intelligent Automation, Inc. 

Contact Persons: 

Soundar Kumara (PSU) 

skumara@psu.edu (814) 863-2359 

Vikram Manikonda (IAI) 

vikram@i-a-i.com (301) 294-5045 

Wilbur Peng (IAI) 

wpeng@i-a-i.com (301) 294-50455 

i

Document History 

Revision 

Date 

Revised By Comments Date Reviewed 

with Team 

08/24/05 Soundar Kumara Initial creation 

* Document will be approved as part of CCB process. 

Approved * 

Software Release History 

Date 

Comments 

LEGEND (example; legend/copyright statement optional) 

Use, duplication, or disclosure by the Government is as set forth in the Rights in technical data 

noncommercial items clause DFAR 252.227-7013 and Rights in noncommercial computer software and 

noncommercial computer software documentation clause DFAR 252.227-7014. 

© Copyright 2001 

ii

Contents 

Contents .............................................................................................................. iii 

Executive Summary..............................................................................................v 

1 Introduction .....................................................................................................7 

2 Design and survivability of distributed multi-agent systems ............................7 

2.1 Designing a Network Infrastructure for Survivability of Multi-Agent 

Systems .................................................................................................7 

2.2 Survivability of Multi-agent based Supply Networks: A Topological 

perspective.............................................................................................7 

2.3 Survivability of a distributed multi-agent application – A performance 

control perspective .................................................................................8 

2.4 Survivability through Implementation Alternatives in Large-scale 

Information Networks with Finite Load ...................................................8 

3 Monitoring, situation identification and pattern extraction ...............................8 

3.1 Situation Identification Using Dynamic Parameters in Complex Agent- 

Based Planning Systems .......................................................................8 

3.2 Estimating global stress environment through local behavior in a 

multiagent-based planning system.........................................................8 

3.3 Using Predictors to Improve the Robustness of Multi-Agent Systems: 

Design and Implementation in Cougaar .................................................8 

3.4 Survivability of Complex System – Support Vector Machine Based 

Approach................................................................................................8 

4 Control ............................................................................................................8 

4.1 A Framework For Performance Control of Distributed Autonomous 

Agents....................................................................................................8 

4.2 An Autonomous Performance Control Framework for Distributed Multi- 

Agent Systems: A Queueing Theory Based Approach...........................9 

4.3 Adaptive control for large-scale information networks through alternative 

algorithms to support survivability ..........................................................9 

4.4 Self-organizing resource allocation for minimizing completion time in 

large-scale distributed information networks ..........................................9 

4.5 Efficient method of quantifying minimal completion time for componentbased 

service networks: Network topology and resource allocation ......9 

4.6 Market-based model predictive control for large-scale information 

networks: Completion time and value of solution ...................................9 

4.7 Coordinating control decisions of software agents for adaptation to 

dynamic environments. ..........................................................................9 

iii

5 CPE society modeling and performance analysis...........................................9 

5.1 Understanding agent societies using distributed monitoring and profiling 

...............................................................................................................9 

5.2 Reliable MAS Performance Prediction Using Queueing Models............9 

6 Characterization and analysis of supply chains from complex systems 

perspective .........................................................................................................10 

6.1 Supply Chain Network: A Complex Adaptive Systems Perspective .....10 

6.2 Decision Making in Logistics: A Chaos Theory Based Approach.........10 

iv

Executive Summary 

Ultra*Log is a Defense Advanced Research Projects Agency (DARPA) sponsored 

research project focused on creating a distributed agent-based architecture that is 

inherently survivable and capable of operating effectively in very chaotic environments. 

The project is pursuing the development of technologies to enhance the security, 

robustness, and scalability of large-scale, distributed agent-based systems operating in 

chaotic wartime environments. Ultra*Log's goal is to operate with up to 45% information 

infrastructure loss in a very chaotic environment with not more than 20% capabilities 

degradation and not more than 30% performance degradation for a period representing 

180 days of sustained military operations in a major regional contingency. 

In order to achieve the above goals, we are concentrating on the complexity studies for 

analysis, estimation and control. The efforts are geared towards realizing a robust theory 

for analyzing and controlling the complexity in distributed multi-agent systems. This 

would in turn help to define the theoretical and application grounds for adaptivity in 

distributed systems. The application area is military logistics where the studies 

concentrate on sensing to logistics in a network centric warfare environment. The 

research under Ultra*Log is expected to lay the foundation for the next generation 

logistics. 

In this document we discuss significant accomplishments of PSU/IAI as a part of 

Ultra*Log project. We present the results in the form of papers we published/submitted to 

different refereed conferences and journals. During the course of the Ultra*Log project, 

we have proposed three different research and development areas, namely: 

1. Research in design and survivability of distributed multi-agent systems 

2. Research in Monitoring, Situation Identification and Pattern Extraction 

3. Research in characterization, analysis and control of complex adaptive systems. 

We discuss the details of several results and findings related to the above research areas 

in this document. 

The total period of the project is four and half years from the start of the project. The 

team members include The Pennsylvania State University (PSU), and Intelligent 

Automation Incorporated, Rockville, MD., who is a sub-contractor to PSU. The regular 

team members from PSU include graduate students (Y. Hong, S. Lee, H. P. 

Thadakamalla, A. Surana, V. Narayanan, H. Gupta, N. Gnanasambandam, K. Tang, X. 

Ding, E. Pinto and U. N. Raghavan). These students worked under the direct supervision 

of Professor Kumara. Most of them are Ph.D., students. In addition, Professors G. 

Natarajan and C.R. Rao * participated in the project. From IAI, V. Manikonda, W. Peng 

and H. Gupta were the participants. 

* (Winner of National medal of Science for Mathematics from President Bush in 2002) 

v

1 Introduction 

The main focus of the proposed research is breakthrough technology development based on 

theory of chaos, knowledge mining, queueing theory and market based control theoretic 

principles combined with chaos to improve scalability, robustness and survivability of Cougaar 

Architecture. In specific, our focus was on adaptive logistics. Such a development will introduce 

new-operation capabilities in Cougaar in terms of: 

• Dynamic fault isolation and recovery, 

• Dynamic adaptation to environment, and 

• Variable fidelity to adaptive processes. 

We have published/submitted many papers in different refereed conferences and journals that 

address these principles. We have classified these papers into five different sections as given 

below: 

• Design and survivability of distributed multi-agent systems 

• Monitoring, situation identification and pattern extraction 

• Control 

• CPE society modeling and performance analysis 

• Characterization and analysis of supply chains from complex systems perspective 

Please note that the papers are hyper-linked. 

2 Design and survivability of distributed multi-agent systems 

It is extremely important to design multi-agent systems architecture that is survivable even 

in war-time or critical situations. Survivability can be improved both from functionality and 

topological perspective. We published the following papers, where we discuss different ways of 

improving survivability for distributed multi-agent systems. 

2.1 Surana, A., Gautam, N., Kumara, S. R. T., and Greaves, M., “Designing a 

Network Infrastructure for Survivability of Multi-Agent Systems”, IASTED 

Conference on Parallel and Distributed Computing and Networks, 2005. 

2.2 Thadakamalla, H. P., Raghavan, U. N., Kumara, S. R. T. and Albert, R., 

“Survivability of Multi-agent based Supply Networks: A Topological 

perspective,” IEEE Intelligent Systems, Vol. 19, No. 5, 2004. 

7

2.3 Gnanasambandam, N., Lee, S., Kumara, S. R. T., Gautam, N., Peng, W., 

Manikonda, V., Brinn, M. and Greaves, M., “Survivability of a distributed multiagent 

application – A performance control perspective”, IEEE Symposium on 

Multi-agent Security and Survivability (MAS&S 2005), Philadelphia, 2005. 

2.4 Lee, S., and Kumara, S. R. T., “Survivability through Implementation 

Alternatives in Large-scale Information Networks with Finite Load,” Proceedings 

of Open Cougaar Conference, July 2004. 

3 Monitoring, situation identification and pattern extraction 

An essential task for control is sensing. We have developed different tools to monitor and 

sense distributed systems. With the help of these tools, we devised many situation identification 

and pattern extraction algorithms based on theory of chaos, knowledge mining and Kalman filter 

principles. The following are the papers published in this field of research. 

3.1 Lee, S., Gautam, N., Kumara, S. R. T., Surana, A., Gupta, H., Hong, Y., 

Narayanan, V., and Thadakamalla, H. P., “Situation Identification Using Dynamic 

Parameters in Complex Agent-Based Planning Systems,” Intelligent Engineering 

Systems Through Artificial Neural Networks, v 12, 2002. 

3.2 Lee, S., and Kumara, S. R. T., “Estimating global stress environment through 

local behavior in a multiagent-based planning system,” IEEE Conference on 

Automation Science and Engineering (CASE 05), Edmonton, Canada, August 

2005. 

3.3 Gupta, H., Hong, Y., Thadakamalla, H. P., Manikonda, V., Kumara, S. R. T. and 

Peng, W., “Using Predictors to Improve the Robustness of Multi-Agent Systems: 

Design and Implementation in Cougaar”, Proceedings of Open Cougaar 

Conference, July 2004. 

3.4 Hong, Y., Gautam, N., Kumara, S. R. T., Surana, A., Gupta, H., Lee, S., 

Narayanan, V., and Thadakamalla, H. P., “Survivability of Complex System – 

Support Vector Machine Based Approach,” Intelligent Engineering Systems 

Through Artificial Neural Networks, v 12, 2002. 

4 Control 

The heart of adaptivity is control. In our work we therefore, build different control 

frameworks and methods for distributed systems. The following are the papers published in this 

research area. 

4.1 Gnanasambandam, N., Lee, S., Kumara, S. R. T. and Gautam, N., “A Framework 

For Performance Control of Distributed Autonomous Agents,” Industrial 

Engineering Research Conference (IERC), Atlanta, August 2005.

4.2 Gnanasambandam, N., Lee, S., Gautam, N., Kumara, S. R. T., Peng, W., 

Manikonda, V., Brinn, M. and Greaves, M., “An Autonomous Performance 

Control Framework for Distributed Multi-Agent Systems: A Queueing Theory 

Based Approach,” Autonomous Agents and Multi-Agent Systems (AAMAS), 

Utrecht, Netherlands, July 2005. 

4.3 Lee, S. and Kumara, S. R. T. “Adaptive control for large-scale information 

networks through alternative algorithms to support survivability”, submitted to 

IEEE Transactions on Automatic Control. 

4.4 Lee, S., Kumara, S. R. T. and Gautam, N., “Self-organizing resource allocation 

for minimizing completion time in large-scale distributed information networks”, 

submitted to IEEE Transactions on Systems, Man, and Cybernetics. 

4.5 Lee, S., Kumara, S. R. T. and Gautam, N., “Efficient method of quantifying 

minimal completion time for component-based service networks: Network 

topology and resource allocation”, submitted to IEEE Transactions on 

Computers. 

4.6 Lee, S., Kumara, S. R. T. and Gautam, N., “Market-based model predictive 

control for large-scale information networks: Completion time and value of 

solution”, submitted to IEEE Transactions on Parallel and Distributed Systems. 

4.7 Hong, Y. and Kumara, S. R. T., “Coordinating control decisions of software 

agents for adaptation to dynamic environments” 37 th CIRP International Seminar 

on Manufacturing Systems (ISMS-2004), Budapest, Hungary, May 2004. 

5 CPE society modeling and performance analysis 

We have built a demo society, “CPEDemo”, for identifying the key aspects within a 

continuous planning and execution scenario. This will help us to identify and demonstrate key 

concepts in the argument for and concept of “design for survivability”. The following are the 

papers published related to the CPE society. 

5.1 Peng, W., Manikonda, V. and Kumara S. R. T., “Understanding agent societies 

using distributed monitoring and profiling” Proceedings of Open Cougaar 

Conference, July 2004. 

5.2 Gnanasambandam, N., Lee, S., Gautam, N., Kumara, S. R. T., Peng, W., 

Manikonda, V., Brinn, M. and Greaves, M., “Reliable MAS Performance 

Prediction Using Queueing Models,” IEEE Symposium on Multi-agent Security 

and Survivability (MAS&S 2004), Philadelphia, PA, 2004

6 Characterization and analysis of supply chains from complex 

systems perspective 

With the advent of information technology, supply chains have acquired complexity that is 

almost equivalent to biological systems. In the following papers, we argue why supply chains 

should be treated as complex systems and propose how various concepts, tools and techniques 

used in complex adaptive systems literature can be exploited to characterize and analyze supply 

chain networks. 

6.1 Surana, A., Kumara, S. R. T., Greaves, M. and Raghavan, U. N. “Supply Chain 

Network: A Complex Adaptive Systems Perspective”, International Journal of 

Production Research (to be published), 2005. 

6.2 Kumara, S. R. T., Ranjan, P., Surana, A. and Narayanan, V., “Decision Making in 

Logistics: A Chaos Theory Based Approach”, CIRP Annals, p.381, 2003.

DESIGNING A NETWORK INFRASTRUCTURE FOR SURVIVABILITY OF 

MULTI-AGENT SYSTEMS 

A. Surana 

MIT, 

Cambridge, MA 02139 

email: surana@mit.edu 

N. Gautam, S.R.T. Kumara 

Penn. State University, 

University Park, PA 16802 

email: {ngautam, skumara}@psu.edu 

M. Greaves 

DARPA, 3701 Fairfax Drive 

Arlington, VA 22203-1714 

email: mgreaves@darpa.mil 

ABSTRACT 

In this paper we consider a society of agents whose interactions 

are known. Our objective is to solve a strategic 

network infrastructure design problem to determine: (i) 

number of nodes (usually computers or servers) and their 

processing speeds, (ii) set of links between nodes and their 

bandwidths, and (iii) assignment of agents to nodes. From 

a performance standpoint, on one hand all the agents can 

reside in a single node thereby stressing the processor, on 

the other hand the agents can be distributed so that there 

is a maximum of one agent per node thereby increasing 

communication cost. From a robustness standpoint since 

links and arcs can fail (possibly due to attacks) we would 

like to build a network that is least disruptive to the multiagent 

system functionality. Although we do not explicitly 

consider tactical issues such as moving agents to different 

nodes upon failure, we would like to design an infrastructure 

that facilitates such agent migrations. We formulate 

and solve a mathematical program for the network infrastructure 

design problem by minimizing a cost function subject 

to satisfying quality of service (QoS) as well as robustness 

requirements. We test our methodology on Cougaar 

multi-agent societies. 

KEY WORDS 

network design, QoS, robustness, optimization. 


As the number of applications requiring distributed multiagent 

systems (MAS) is continuously growing, it becomes 

extremely important to build a network infrastructure that 

can guarantee a survivable MAS architecture. By survivable 

we mean a system that is robust, secure as well as able 

to provide excellent quality of service (QoS) even when 

stressed. For example Brinn and Greaves [6] state that the 

Cougaar MAS in Ultra*Log [14], would be considered survivable 

if it would maintain at least x% of system capabilities 

and y% of system performance in the face of z% 

infrastructure loss and wartime loads (where x, y and z are 

provided by the system users). 

In order to build such a survivable system, there are 

several decisions that need to be made at different time 

granularities. These can be broken down as strategic (once 

a year or just one time), tactical (once a week to once a day 

depending on how often the configuration changes), and 

operational (typically milliseconds to seconds, depending 

on the granularity of information exchange) decisions. The 

strategic decisions typically involve designing the network 

infrastructure (in terms of both number and capacity) for 

the MAS such as computers, servers, cables, etc. The tactical 

decisions include where to migrate the agents if a node 

fails or is cut off from the other nodes. Operational decisions 

include adaptive control methods for deciding which 

agent should process a task, the fidelity with which to process 

a task, etc. 

There has been a lot of research related to (a) software 

technology, such as, agent architecture, communication, 

migration, adaptation, learning, etc., and (b) networking, 

such as, QoS provisioning, fault-tolerance, dependability, 

robustness, etc. However there is very little research that 

combines the two and studies them from a systems engineering 

viewpoint. In this research we address that shortcoming. 

We focus on the strategic problem stated in the 

previous paragraph of designing a network infrastructure 

in terms of hardware to support a given society of agents 

and their interactions. This is with the understanding that 

in order to solve the tactical and operational problems effectively, 

the strategic problem must favor a network design 

that would ease tactical and operational decisions. 

We now present some of the related research with the 

understanding that due to space restrictions it is difficult to 

cite all relevant articles in the literature. Andreoli et al [2] 

consider a distributed software network infrastructure for 

agents performing search tasks (such as search engines). 

Optimization issues for MAS at software level such as 

load balancing using non-linear programming techniques is 

studied in Aiello et al [1] for a given hardware topology. In 

Hofmann et al [9], a mobile intelligent agent system is built 

under conditions of low bandwidth to show that it could 

improve the efficiency of military tactical operations and 

the mobile agents would outperform static agents. Multiagent 

hybrid systems that combine computational hardware 

and a large scale software residing on it, with an application 

to air-traffic management is studied in Tomlin et al 

[13]. In Kephart et al [11], one of the emerging research 

areas, namely considering a distributed information system 

as analogous to biological ecosystem and social systems, is 

presented in order to study their survivability. Cancho and 

Sole [7] consider a complex network and show that by op-

timizing simultaneously the link density and path distance 

in a graph (with a fixed set of nodes), leads to a scale-free 

topology which is robust to random attacks. 

The remainder of this paper is organized as follows. 

In Section 2 we describe the strategic problem in detail. 

Then in Section 3 we formulate the problem as a mathematical 

program. We discuss various methods to solve 

the mathematical program in Section 4. Then we describe 

numerical examples and results in Section 5. Finally we 

present our concluding remarks and directions for future 

work in Section 6. 

2 Problem Description 

We now present details of the strategic problem of designing 

a network infrastructure for a MAS. Distributed information 

systems (DIS) can be viewed as a reconfigurable 

network with (i) computational infrastructure forming the 

backbone, and (ii) agents residing on it and moving around, 

consuming resources and providing services under uncertain 

and often hostile conditions. Each agent, has access 

to different information and makes its own local decisions, 

but must work together with other agents for the achievement 

of a common, system-wide goal. In this research we 

consider a MAS and an underlying network infrastructure 

that can be modeled as a DIS. 

One of the key inputs that go into the network design 

problem is the agent interaction pattern. A typical 

agent interaction tree is depicted in Figure 1. In the figure, 

the agents are nodes and if there arcs connecting two 

nodes, then the corresponding agents interact. The agents 

also specify the bandwidth requirement for their interaction. 

Besides the bandwidth (and interaction graph), another 

input to the design problem is resource requirements 

such as CPU and memory from the host computer or server. 

Although the figure suggests a hierarchical network, the 

model does not require that. In addition, some or all of 

the agents can be identical (in terms of what they can do). 

Figure 1. Example of an agent logical network 

Given the inputs mentioned above and other inputs 

based on survivability requirements, the output of the design 

problem is a physical network of nodes and arcs, 

Figure 2. Agent logical network residing in a physical network 

where nodes signify processors such as computers and 

servers, and arcs signify links (it is not required that there 

be a single link between 2 nodes, however, we use the capacity 

of the bottleneck link and pretend as though it is a 

single link). The two extremes of design are if we place all 

agents in one node and if we place one agent in each node. 

In case of all agents in one node, the bottleneck would be 

resources, i.e. whether the CPU and memory requirements 

of all agents can be met. However if the agents are such 

that there is one agent per node, there would be a lot of 

time spent in communication between them. We assume 

that if two agents are on a single node, the available bandwidth 

for their interaction is infinite. In Figure 2, we take 

the logical network of agents (described in Figure 1) and 

build 4 nodes and three arcs to house the agents in a physical 

infrastructure. 

3 Robust Design: Problem Formulation 

In this section we formulate a robust design problem for 

DIS. As we have discussed previously in Section 2, a DIS 

consists of two critical components: computational hardware 

(processors and communication links) and a MAS residing 

on it. With this viewpoint we have the following 

robust design problem: given the software agents (MAS) 

with their interaction pattern and computational resource 

requirements, we want to decide (i) How much processing 

power to start with i.e., how many processors and the capacity 

of each processor (to be selected from a given a set of 

processors)? (ii) How to lay out the physical network structure, 

i.e., how to connect the processors and what should be 

the capacity of each link (to be selected from a given a set 

of bandwidths)? (iii) How to distribute agents on this network? 

The above decisions are to be made so that we can 

meet the survivability requirements and at the same time 

minimize the information infrastructure cost. We translate 

the survivability requirements into following “specifications” 

for the design problem: (a) Sufficient computational 

resources to start with and its balanced utilization;

(b) Small average path length and diameter, measuring the 

connectivity; (c) Resilience to complete node and link failures. 

Given the above specifications, it is clear that a robust 

design for DIS would be one with maximum possible computational 

resources and a fully connected backbone network. 

However, this would incur a very large infrastructure 

cost. This leads to the problem of optimally designing the 

backbone network and distributing agents on it such that it 

is fairy robust and at the same time cost effective. In order 

to systematically pose this trade-off as a mathematical 

programming model, we first give a formal description of 

various entities involved in the model. 

3.1 Agent Society, Nodes and Links 

The MAS or the agent society is described by the computational 

resource each agent consumes and their interaction 

pattern. Let N A be the total number of agents in the society, 

indexed as {1, 2, · · · , N A }. Let for an i th agent P a i 

denote the computational resource (CPU and Memory) it 

consumes and let Ba ij denote the bandwidth it uses, if it 

interacts with agent j. 

A node represents a computer with a given processing 

power (power can refer to CPU, Memory, etc.). Each agent 

in the given society has to be assigned to a node. As a result 

each node can be assigned one or more agents. For the 

agents which reside on the same node, the communication 

requirement is automatically satisfied. Let N max be the 

total number of nodes numbered {1, 2, · · · N max }, that is 

initially chosen to distribute the agents on. Let N i denote 

the decision variable such that, 

{ 1, if node i is selected from Nmax nodes 

N i = 

0, otherwise, 

for 1 ≤ i ≤ N max . Let P = {P 1 , P 2 , · · · , P Np } denote 

the set of available processing power for nodes with an associated 

cost set C(P ) = {C p1 , C p2 , · · · , C pNp } and P n ij 

a decision variable such that 

{ 

1, if i 

P n ij = 

th node uses processor with a power P j 

0, otherwise, 

for 1 ≤ i ≤ N max and 1 ≤ j ≤ N p . Furthermore, let 

A d = [A ij ] denote a matrix of the distribution of agents on 

the nodes, where A ij is 

{ 1, if agent i resides on node j 

A ij = 

0, otherwise, 

for 1 ≤ i ≤ N A and 1 ≤ j ≤ N max . It is assumed 

that there are no multiple links and no self-loops when we 

connect the nodes with communication links. Let X ij be 

the decision variable, such that 

{ 1, if there is link from node i to j and i ≠ j 

X ij = 

0, otherwise, 

for 1 ≤ i, j ≤ N max . The matrix, X = [X ij ], is symmetric 

as the links connecting the nodes form the communication 

pathways and hence are undirected. 

Consider the set V = {N i |N i ≠ 0, 1 ≤ i ≤ N max } 

of occupied nodes and the corresponding index set I = 

{i|N i ≠ 0, 1 ≤ i ≤ N max }. Let E = {X ij |X ij ≠ 

0 and i, j ∈ I, 1 ≤ i, j ≤ N max } . We shall denote 

by G = (V, E), the graph with V as the set of vertices and 

E as a set of undirected edges. Let l avg (G) be the average 

path length and D(G) be the diameter of G. Note that if G 

consists of disconnected components, then l avg (G) −→ ∞ 

and D(G) −→ ∞. Furthermore, we are only allowed to 

choose the capacity of links from an available set of bandwidths 

B = {B 1 , B 2 , · · · , B Nb } with an associated cost 

set C(B) = {C b1 , C b2 , · · · , C bNb }. Let Br ijl be the decision 

variable which is 1 if link X ij uses a capacity B l and 

0 otherwise, for, 1 ≤ i, j ≤ N max and 1 ≤ l ≤ N b . 

3.2 Problem Statement 

With the notations given in the previous section, let D = 

{N max , N i , P n ij , A ij , X ij , Br ijl } denote the set of decision 

variables (which are all binary). We can state the network 

design problem as follows: 

Objective: Let C denote the infrastructure cost, then 

we desire to 

min C = 

N∑ 

max 

i=1 

N p 

∑ 

C pj P n ij + 

j=1 

N∑ 

max 

i=1 

subject to the following constraints: 

1. Resource Choice Constraints 

∑N b 

l=1 

N p 

N∑ 

max 

j>i 

∑N b 

l=1 

C bl Br ijl , 

(1) 

∑ 

P n ij = N i , 1 ≤ i ≤ N max (2) 

j=1 

Br ijl = X ij , 1 ≤ i ≤ N max and i < j ≤ N max 

(3) 

The above constraints (2) and (3), restricts the choice 

of one type of processor and one type of bandwidth 

capacity for a node and a link respectively. 

2. Agent Distribution Constraints 

N∑ 

max 

j=1 

A ij = 1, 1 ≤ i ≤ N A (4) 

∑N A 

∑ 

A ij P a i + ∆ 1 (j) ≤ P n jl P l , 1 ≤ j ≤ N max 

i=1 

N p 

l=1 

∑N A 

∑N A 

A li A kj (Ba lk + Ba kl ) 

l=1 k=1 

(5) 

∑N b 

+∆ 2 (i, j) ≤ Br ijt B t (6) 

t=1 

for 1 ≤ i ≤ N max , i < j ≤ N max ,

where ∆ 1 (j) ≥ 0 and ∆ 2 (i, j) ≥ 0 are given constants, 

which can vary with the node and the link, respectively. 

The constraints (4), force that each agent is assigned 

to only one node. On the other hand the constraints 

(5) guarantee that the agents are assigned to only those 

nodes which have been selected and the processing capacity 

chosen for that node meets the computational 

requirements in terms of CPU for all the agents assigned 

to that node. Note that this constraint also leads 

to a well balanced utilization of CPU by the agents, to 

begin with. Similarly the constraints (6), are for the 

meeting the communication requirements in terms of 

bandwidth of the links between nodes. Also each of 

the constraint (6), forces that if two agents which communicate 

with each other reside on separate nodes, 

then a direct communication link exists between those 

nodes. The constants ∆ 1 (j) and ∆ 2 (i, j), provide 

for additional or redundant CPU and bandwidth in 

the network. This redundancy takes into consideration 

the additional computational resources that may 

be required due to factors like: variability in computational 

resource requirements by agents, complete or 

partial loss resources at nodes and links and migration 

of agents between nodes. It should be noted that the 

effect of these constants can be absorbed in the processing 

P a i and bandwidth Ba ij requirements of the 

agents and hereafter we would assume that this has 

been done. 

3. Connectivity Constraints 

X ij ≤ N i 1 ≤ i ≤ N max and 1 ≤ j ≤ N max 

(7) 

l avg (G) ≤ l max (8) 

D(G) ≤ D max , (9) 

where l max is the maximum allowable average path 

length and D max is the maximum allowable diameter 

of the network considered. The constraints (7) enforce 

that link exists only between nodes which have been 

selected. On the other hand, the constraints (8) and 

(9) are related to network performance and also guarantee 

that G is connected. In general, the constraints 

(8) and (9), cannot be expressed explicitly in terms of 

decision variables as equations, and have to be verified 

algorithmically. 

4 Solution Methodology 

The problem discussed in the previous section is similar 

in many respects, to the problems that often arise in the 

design of telecommunication networks [3], [4], [5]. For 

example, in [5], the authors consider the problem of “survivable 

network design” (SND), which seeks to design a 

minimum cost network with a given set of nodes and a set 

of possible edges between them, such that the connectivity 

requirement (which is specified as the minimum number 

of edge-disjoint paths needed between different nodes) is 

satisfied. The major distinction of our model from such 

formulations is that we consider infrastructural design and 

the distribution of agents on this network simultaneously in 

the strategic design phase. Note that: 

• The maximum number of nodes needed satisfies, 

N max ≤ N A , otherwise the optimization problem has 

no feasible solution, as the constraints (5) cannot be 

satisfied. Hence, we can always take N max = N A . 

• Our problem, is a generalization of the “bin-packing” 

problem [12]. Following distinctions of our optimization 

problem from the “bin-packing” problem can be 

noted. There are two types of bins: the processors and 

the network links and the agents are the objects to be 

chosen. The capacity for both type of bins are variable 

and can be selected from a given set, rather than being 

fixed. There is constraint between filling two types of 

bins i.e., as we fill the processors with agents, the bin 

which is the link connecting the processors also gets 

filled based on the agent distribution. Also, there are 

additional constraints related to the diameter and average 

path length (7)-(9), that should be satisfied. 

• Consider a special case of our optimization problem 

where Agents do not interact with each other i.e 

Ba ij = 0 (1 ≤ i, j ≤ N A ); There is only one processor 

with a capacity P and unit cost; There are no 

constraints on l avg (G) and D(G), i.e., D max −→ ∞ 

and l max −→ ∞. Under these conditions the problem 

reduces to the usual bin-packing problem, as follows: 

subject to 

min C = 

N∑ 

max 

i=1 

N i , (10) 

∑N A 

A ij = 1, 1 ≤ i ≤ N A , (11) 

j=1 

∑N A 

A ij P a i ≤ N j P, 1 ≤ j ≤ N A . (12) 

i=1 

The bin-packing problem is known to be NP-hard in the 

strong sense [12] . Since our problem is a generalization 

of the bin-packing problem it is also NP-hard in the strong 

sense. Given this we either need to develop heuristics or 

use evolutionary algorithms to obtain solutions. 

We have used Genetic Algorithms (GA) with the following 

important features: 

• Rather than using a binary encoding, we used a 

scheme of integer coding of the decision variables. 

• The initial pool of population is generated randomly, 

with one feasible solution. The feasible solution can

e obtained as follows. Start with N A nodes, assign 

each agent to a separate node and choose a lowest possible 

processing capacity from the available set C(P ) 

such that the processing requirements for each agent 

is satisfied. Connect the nodes which have agents that 

interact with each other and assign to that link a capacity 

with the lowest possible bandwidth from the 

available set C(B) such that the communication requirements 

are satisfied. 

• We have used NSGA2 [8, 10], as the GA solver. It 

has capability to automatically handle constraints. It 

uses a mean-centric crossover and uniform bounded 

mutation operators for real coded strings. 

5 GA Application Examples and Results 

Figure 4. GA Result: DIS Layout for MSC, C = 130 

the nature of branching in the hierarchical structure 

and the variation in processing and bandwidth requirements 

for agents. In Figure 3, each node in the 

tree is labeled by an agent number A i and its processing 

requirement P a i , whereas each link between 

the agents A i and A j (if they interact), is labeled by 

(Ba ij + Ba ji ), their communication requirement. 

Figure 3. A military supply chain as an agent society 

3. The restriction on the maximum allowable diameter 

D max and the average path length l max for the DIS 

are listed in Table 2. 

5.1 Inputs 

Following are inputs based on notation in Section 3.2: 

1. The cost structure for processing power and bandwidth 

used in examples is shown in the Table 1. 

2. The agent societies in the examples considered were 

generated randomly. The processing P a i and the 

bandwidth Ba ij requirements for the agents were 

sampled from uniform distribution. However, the 

structure of the agent societies in all the cases was restricted 

to be hierarchical. This is motivated by the 

fact that most of the organizations, like in Command 

and Control or in society have hierarchical structure. 

Note that the optimization problem and formulation 

we have considered is general enough to be applied to 

an agent society with any underlying structure. The 

agent societies, differ in number of agents (Table 2), 

Table 1. Processing Cost and Bandwidth Cost 

i 1 2 3 

P i 5 7 9 

C pi 1 2 3 

5.2 Output: GA Results 

i 1 2 

B i 3 6 

Cb i 1 2 

The costs C obtained by running GA have been tabulated 

below (Table 2), while the DIS layout is shown in Figure 

4, next to the corresponding agent society. In the figure, 

each node in the graph is labeled by the processing power 

P i followed by the agents which are assigned to that node, 

while each link is labeled by the bandwidth B i chosen for 

it. It should be noted that many agents can be assigned to a 

same node. For example, (Figure 4), agents A12 and A13 

are both assigned to a the node labeled P 9 : A12A13.

Table 2. GA Results 

Agent N A D max , GA Ratio 

Society No. l max C 

C 

N A 

1 4 2, 2 15 3.75 

2 7 ” 27 3.86 

3 10 5, 4 30 3.00 

4 12 ” 33 2.75 

5 15 ” 50 3.33 

6 19 ” 61 3.21 

7 24 ” 87 3.63 

8 33 ” 108 3.27 

9 40 ” 164 4.1 

10 50 ” 188 3.76 

The Table 2, shows that the optimal cost per agent 

ratio ( C 

N A 

) is fairly constant, with a small variation. This 

may be a result of the cost structure assumed and the particular 

instances of the agent societies considered. This observation, 

however, can have following implication: given 

a very large agent society like with N A = 5, 000 agents, we 

can decompose it into smaller agent societies, solve the optimization 

problem for each of the sub-societies and then 

combine them to solve the overall problem. Due to the 

constancy of the ratio 

C 

N A 

, this heuristic should lead to solutions 

which are fairly close to optimal. 

As a final example we consider one of the realistic 

agent societies which has been developed in the Ultra*Log 

Program [14], [6]. The society is shown below in Figure 3 

and represents a typical military supply chain (MSC). The 

structure of society is an exact replica of the true society, 

however the processing and band width requirements for 

the agents have been assigned randomly. The result obtained 

by GAs has been shown in the Figure 4. 

6 Conclusion and Future Work 

In this paper we have systematically studied the issue of 

survivability of DIS. Based on these we formulated a robust 

design problem for DIS. We showed that this problem 

is NP hard in strong sense and used GAs to obtain solutions 

for a number of example agent societies. We also considered 

a realistic agent society representing a military supply 

chain. We showed that our robust design problem formulation 

results in a fairly survivable DIS. 

Survivability of DIS is an emerging area and future 

research is possible in a number of varied directions. 

Refining the robust design problem we have posed and 

developing heuristic solution methodologies for it. Further 

research is required to build mechanisms for survivability 

against other types of attacks, such as security and DOS 

attacks. Most of the above stated problems are nothing 

new for biological systems which have routinely solved 

them for literally millions of years. Can we draw inspiration 

from the structures discovered in biology to solve 

problems of distributed systems? We believe that the quest 

for “open-ended” survivability for DIS can be achieved 

only by exporting biological mechanisms into software 

systems. 

Acknowledgements 

The authors acknowledge DARPA (Grant#: 

MDA972-1-1-0038 under Ultra*Log Program) and 

NSF (Grant#:ANI-0219747 under ITR program) for their 

generous support for this research. Special thanks to the 

anonymous reviewers for their comments and suggestions. 

References 

[1] W. Aiello, B. Awerbuch, B. M. Maggs, and S. Rao. Approximate 

load balancing on dynamic and asynchronous 

networks. In ACM Symposium on Theory of Computing, 

pages 632–641, 1993. 

[2] J. Andreoli, U. Borghoff, R. Pareschi, S. Bistarelli, U. Montatnari, 

and F. Rossi. Constraints and agents for a decentralized 

network infrastructure. In AAAI Workshop, Menlo 

Park, California, USA: AAAI Press., pages 39–44, 1997. 

[3] A. Balakrishnan and K. Altinkemer. Using hop-constrined 

model to generate alternative= communication network design. 

ORSA Journal on Computing, 4(2), 1992. 

[4] A. Balakrishnan, T. l. Magnanti, and P. Mirchandani. A 

dual-based algorithm for multi-level network design. Management 

Science, 40(5):567–581, 1994. 

[5] A. Balakrishnan, T. l. Magnanti, and P. Mirchandani. 

Connectivity-splitting models for survivable network design. 

Submitted, 2003. 

[6] M. Brinn and M. Greaves. Leveraging agent properties 

to assure survivability of distributed multi-agent 

systems. In https://docs.ultralog.net/dscgi/ds.py/Get/File- 

4088/AA03-SurvivabilityOfDMAS.pdf, 2002. 

[7] R. F. Cancho and R. V. Sole. Optimization in 

complex networks. In http://arxiv.org/PS cache/condmat/pdf/0111/0111222.pdf, 

2001. 

[8] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan. A 

fast and elitist multi-objective genetic algorithm-nsga-ii. In 

http://www.iitk.ac.in/kangal/pub.htm, 2000. 

[9] M. O. Hofmann, A. McGovern, and K. R. Whitebread. Mobile 

agents on the digital battlefield. In Proceedings of 

the 2nd International Conference on Autonomous Agents 

(Agents’98), pages 219–225, 1998. 

[10] KanGAL. Kanpur genetic algorithm laboratory. In 

http://www.iitk.ac.in/kangal/pub.htm. 

[11] J. O. Kephart, T. Hogg, and B. A. Huberman. Collective 

behavior of predictive agents. Physics D, 42:48–65, 1990. 

[12] A. Martello and P. Toth. Knapsack Problems Algorithms 

and Computer Implementations. John Wiley and Sons, 

1990. 

[13] C. Tomlin, G. Pappas, and S. Sastry. Conflict resolution for 

air traffic management: A case study in multi-agent hybrid 

systems. IEEE Trans. on Automat. Ctrl, 43(4), 1998. 

[14] ULTRALOG. A darpa program on logistics infromation 

system= survivability. In http://www.ultralog.net/.

D e p e n d a b l e A g e n t S y s t e m s 

Survivability of 

Multiagent-Based 

Supply Networks: A 

Topological Perspective 

Hari Prasad Thadakamalla, Usha Nandini Raghavan, Soundar Kumara, and 

Réka Albert, Pennsylvania State University 

Supply chains involve complex webs of interactions among suppliers, manufacturers, 

distributors, third-party logistics providers, retailers, and customers. 

You can improve a 

multiagent-based 

supply network’s 

survivability by 

concentrating on the 

topology and its 

interplay with 

functionalities. 

Although fairly simple business processes govern these individual entities, real-time 

capabilities and global Internet connectivity make today’s supply chains complex. 

Fluctuating demand patterns, increasing customer 

expectations, and competitive markets also add to 

their complexity. 

Supply networks are usually modeled as multiagent 

systems (MASs). 1 Because supply chain management 

must effectively coordinate among many 

different entities, a multiagent modeling framework 

based on explicit communication between these entities 

is a natural choice. 1 Furthermore, we can represent 

these multiagent systems as a complex network 

with entities as nodes and the interactions between 

them as edges. Here we explore the survivability (and 

hence dependability) of these MASs from the view 

of these complex supply networks. 

Today’s supply networks aren’t dependable—or 

survivable—in chaotic environments. For example, 

Figure 1 shows how mediocre a typical supply network’s 

reaction to a node or edge failure is compared 

to a network with built-in redundancy. 

Survivability is a critical factor in supply network 

design. Specifically, supply networks in dynamic 

environments, such as military supply chains during 

wartime, must be designed more for survivability 

than for cost effectiveness. The more survivable a 

network is, the more dependable it will be. 

We present a methodology for building survivable 

large-scale supply network topologies that can 

extend to other large-scale MASs. Building survivable 

topologies alone doesn’t, however, make an 

MAS dependable. To create survivable—and hence 

dependable—multiagent systems, we must also consider 

the interplay between network topology and 

node functionalities. 

A topological perspective 

To date, the survivability literature has emphasized 

network functionalities rather than topology. To be 

survivable, a supply network must adapt to a dynamic 

environment, withstand failures, and be flexible 

and highly responsive. These characteristics 

depend on not only node functionality but also the 

topology in which nodes operate. 

The components of survivability 

From a topological perspective, the following 

properties encompass survivability, and we denote 

them as survivability components. 

The first is robustness. A robust network can sustain 

the loss of some of its structure or functionalities and 

maintain connectedness under node failures, whether 

the failure is random or is a targeted attack. We measure 

robustness as the size of the network’s largest 

24 1541-1672/04/$20.00 © 2004 IEEE IEEE INTELLIGENT SYSTEMS 

Published by the IEEE Computer Society

connected component, in which a path exists 

between any pair of nodes in that component. 

The second is responsiveness. A responsive 

network provides timely services and 

effective navigation. Low characteristic path 

length (the average of the shortest path 

lengths from each node to every other node) 

leads to better responsiveness, which determines 

how quickly commodities or information 

proliferate throughout the network. 

The third is flexibility. This property depends 

on the presence of alternate paths. 

Good clustering properties ensure alternate 

paths to facilitate dynamic rerouting. The 

clustering coefficient, defined as the ratio 

between the number of edges among a node’s 

first neighbors and the total possible number 

of edges between them, characterizes the 

local order in a node’s neighborhood. 

The fourth is adaptivity. An adaptive network 

can rewire itself efficiently—that is, 

restructure or reorganize its topology on the 

basis of environmental shifts—to continue 

providing efficient performance. For example, 

if a supplier can’t reliably meet a customer’s 

demands, the customer should be 

able to choose another supplier. 

A typical supply chain with a tree-like or 

hierarchical structure lacks these four properties—the 

clustering coefficient is nearly 

zero, and the characteristic path length scales 

linearly with the number of nodes (or agents) 

N. In designing complex agent networks 

with built-in survivability, conventional optimization 

tools won’t work because of the 

problem’s extremely large scale. When networks 

were smaller, we could understand 

their overall behavior by concentrating on 

the individual components’ properties. But 

as networks expand, this becomes impossible, 

so we shift focus to the statistical properties 

of the collective behavior. 

Using topologies 

Studying complex networks such as protein 

interaction networks, regulatory networks, 

social networks of acquaintances, 

and information networks such as the Web 

is illuminating the principles that make these 

networks extremely resilient to their respective 

chaotic environments. The core principles 

extracted from this exploration will 

prove valuable in building robust models for 

survivable complex agent networks. 

Complex-network theory currently offers 

random-graph, small-world, and scale-free network 

topologies as likely candidates for survivable 

networks (see the sidebar “Complex 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

(a) 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

(b) 

Battalion 

FSB 

FSB 

Battalion 

Battalion 

FSB 

FSB 

Battalion 

Battalion 

Node 

failure 

Battalion 

Node 

failure 

Battalion 

Battalion 

Battalion 

Battalion 

Networks” for more on this topic). Evaluating 

these for survivability (see Figure 2), we find 

that no one topology consistently outperforms 

the others. For example, while small-world networks 

have better clustering properties, scalefree 

networks are significantly more robust to 

random attacks. So, we can’t directly use these 

Battalion 

Battalion 

MSB 

Battalion 

Battalion 

Battalion 

Battalion 

MSB 

Battalion 

Battalion 

Battalion 

FSB 

FSB 

Battalion 

Battalion 

FSB 

FSB 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Battalion 

Figure 1. How redundancy affects survivability. (a) A part of the multiagent system 

for military logistics modeled using the UltraLog (www.ultralog.net) program. This 

example models each entity, such as main support battalion, forward support battalion, 

and battalion, as a software agent. (We’ve changed the agents’ names for security 

reasons.) In the current scenario, MSBs send the supplies to the FSBs, who in turn 

forward these to battalions. (b) A modified military supply chain with some redundancy 

built into it. This network performs much better in the event of node failures and hence 

is more dependable than the first network. 

topologies to build supply networks. We can, 

however, use their evolution principles to build 

supply chain networks that perform well in all 

respects of the survivability components. 

Researchers have studied complex networks 

in part to find ways to design evolutionary 

algorithms for modeling networks 

SEPTEMBER/OCTOBER 2004 www.computer.org/intelligent 25

Complex Networks 

Social scientists, among the first to study complex networks 

extensively, focused on acquaintance networks, where nodes 

represent people and edges represent the acquaintances between 

them. Social psychologist Stanley Milgram posited the 

“six degrees of separation” theory that in the US, a person’s 

social network has an average acquaintance path length of six. 1 

This turns out to be a particular instance of the small-world 

property found in many real-world networks, which, despite 

their large size, have a relatively short path between any two 

nodes. 

An early effort to model complex networks introduced random 

graphs for modeling networks with no obvious pattern or 

structure. 2 A random graph consists of N nodes, and two nodes 

are connected with a connection probability p. Random graphs 

are statistically homogeneous because most nodes have a degree 

(that is, the number of edges incident on the node) close 

to the graph’s average degree, and significantly small and large 

node degrees are exponentially rare. 

However, studying the topologies of diverse large-scale networks 

found in nature reveals a more complex and unpredictable 

dynamic structure. Two measures quantifying network 

topology found to differ significantly in real networks are the 

degree distribution (the fraction of nodes with degree k) and 

the clustering coefficient. Later modeling efforts focused on 

trying to reproduce these properties. 3,4 Duncan Watts and 

Steven Strogatz introduced the concept of small-world networks 

to explain the high degree of transitivity (order) in complex 

networks. 5 The Watts-Strogatz model starts from a regular 

1D ring lattice on L nodes, where each node is joined to its 

first K neighbors. Then, with probability p, each edge is rewired 

with one end remaining the same and the other end 

chosen uniformly at random, without allowing multiple edges 

(more than one edge joining a pair of vertices) or loops (edges 

joining a node to itself). The resulting network is a regular lattice 

when p = 0 and a random graph when p = 1, because all 

edges are rewired. This network class displays a high clustering 

coefficient for most values of p, but as p → 1, it behaves like a 

random graph. 

Albert-László Barabási and Réka Albert later proposed an 

evolutionary model based on growth and preferential attachment 

leading to a network class, scale-free networks, with 

power law distribution. 6 Many real-world networks’ degree 

distribution follows a power law, fundamentally different 

from the peaked distribution observed in random graphs and 

small-world networks. Barabási and Albert argued that a 

static random graph of the Watts-Strogatz model fails to capture 

two important features of large-scale networks: their 

constant growth and the inherent selectivity in edge creation. 

Complex networks such as the Web, collaboration networks, 

or even biological networks are growing continuously with 

the creation of new Web pages, the birth of new individuals, 

and gene duplication and evolution. Moreover, unlike random 

networks where each node has the same chance of 

acquiring a new edge, new nodes entering the scale-free network 

don’t connect uniformly to existing nodes but attach 

preferentially to higher-degree nodes. This reasoning led 

Barabási and Albert to define two mechanisms: 

• Growth: Start with a small number of nodes—say, m 0 —and 

assume that every time a node enters the system, m edges 

are pointing from it, where m < m 0 . 

• Preferential attachment: Every time a new node enters the 

system, each edge of the newly connected node preferentially 

attaches to a node i with degree k i with the probability 

k 

Π i = i 

∑ j 

k j 

Research has shown that the second mechanism leads to a 

network with power-law degree distribution P(k) = k –γ with 

exponent γ = 3. Barabási and Albert dubbed these networks 

“scale free” because they lack a characteristic degree and have 

a broad tail of degree distribution. Following the proposal of 

the first scale-free model, researchers have introduced many 

more refined models, leading to a well-developed theory of 

evolving networks. 7 

Protein-to-protein interactions in metabolic and regulatory 

networks and other biological networks also show a striking 

ability to survive under extreme conditions. Most of these 

networks’ underlying properties resemble the three most 

familiar networks found in the literature (see Figure 1 in the 

article). 

Complex networks are also vulnerable to node or edge 

losses, which disrupt the paths between nodes or increase 

their length and make communication between them harder. 

In severe cases, an initially connected network breaks down 

into isolated components that can no longer communicate. 

Numerical and analytical studies of complex networks indicate 

that a network’s structure plays a major role in its response to 

node removal. For example, scale-free networks are more 

robust than random or small-world networks with respect to 

random node loss. 8 Large scale-free networks will tolerate the 

loss of many nodes yet maintain communication between 

those remaining. However, they’re sensitive to removal of the 

most-connected nodes (by a targeted attack on critical nodes, 

for example), breaking down into isolated pieces after losing 

just a small percentage of these nodes. 

References 

1. S. Milgram, “The Small World Problem,” Psychology Today, vol. 2, 

May 1967, pp. 60–67. 

2. P. Erdös and A. Renyi, “On Random Graphs I,” Publicationes Mathematicae, 

vol. 6, 1959, pp. 290–297. 

3. S.N. Dorogovtsev and J.F.F. Mendes, “Evolution of Networks,” 

Advances in Physics, vol. 51, no. 4, 2002, pp. 1079–1187. 

4. M.E.J. Newman, “The Structure and Function of Complex Networks,” 

SIAM Rev., vol. 45, no. 2, 2003, pp. 167–256. 

5. D.J. Watts and S.H. Strogatz, “Collective Dynamics of ‘Small-World’ 

Networks,” Nature, vol. 393, June 1998, pp. 440–442. 

6. A.-L. Barabási and R. Albert, “Emergence of Scaling in Random 

Networks,” Science, vol. 286, Oct. 1999, pp. 509–512. 

7. R. Albert and A.-L. Barabási, “Statistical Mechanics of Complex Networks,” 

Reviews of Modern Physics, Jan. 2002, pp. 47–97. 

8. R. Albert, H. Jeong, and A.-L Barabási, “Error and Attack Tolerance 

of Complex Networks,” Nature, July 2000, pp. 378–382. 

26 www.computer.org/intelligent IEEE INTELLIGENT SYSTEMS

Random 

Small-world 

Scale-free 

with distinct properties found in nature. A 

network’s evolutionary mechanism is designed 

such that the network’s inherent properties 

emerge owing to the mechanism. For 

example, small-world networks were designed 

to explain the high clustering coefficient 

found in many real-world networks, 

while the “rich get richer” phenomenon used 

in the Barabási-Albert model explains the 

scale-free distribution. 2 

Similarly, we seek to design supply networks 

with inherent survivability components 

(see Figure 3), obtaining these components by 

coining appropriate growth mechanisms. Of 

course, having all the aforementioned properties 

in a network might not be practically feasible—we’d 

likely have to negotiate trade-offs 

depending on the domain. Also, domain specificities 

might make it inefficient to incorporate 

all properties. For instance, in a supply 

network, we might not be able to rewire the 

edges as easily as we can in an information 

network, so we would concentrate more on 

obtaining other properties such as low characteristic 

path length, robustness to failures 

and attacks, and high clustering coefficients. 

So, the construction of these networks is 

domain specific. 

Establishing edges between network nodes 

is also domain specific. For instance, in a supply 

network, a retailer would likely prefer to 

have contact with other geographically convenient 

nodes (distributors, warehouses, and 

other retailers). At the same time, nodes in a 

file-sharing network would prefer to attach to 

other nodes known to locate or hold many 

shared files (that is, nodes of high degree). 

Obtaining the survivability 

components 

While evolving the network on the basis 

of domain constraints, we need to incorporate 

four traits into the growth model for 

obtaining good survivability components. 

The first is low characteristic path length. 

During network construction, establish a few 

long-range connections between nodes that 

require many steps to reach one from 

another. 

The second is good clustering. When two 

nodes A and B are connected, new edges 

from A should prefer to attach to neighbors 

of B, and vice versa. 

The third is robustness to random and targeted 

failure. Preferential attachment—where 

new nodes entering the network don’t connect 

uniformly to existing nodes but attach preferentially 

to higher-degree nodes (see the side- 

Degree 

distribution 

Characteristic 

path length 

Clustering 

coefficient 

Robustness 

to failures 

P(k) 

Poisson 

 

k 

Scales as 

log(N) 

p (the connection 

probability) 

Similar responses 

to both random 

and targeted 

attacks 

Peaked 

1.0 

0.8 

0.6 

0.4 

0.2 

0.0 

2 4 6 8 10 12 

k 

Scales linearly with N 

for small p. And for higher 

p scales as log(N) 

High, but as p → 1 

behaves like 

a random graph 

Similar response as 

random networks. 

This is because it has 

a degree distribution 

similar to random 

networks. 

P(k) 

1 

0.1 

0.01 

0.001 

0.0001 

Power law 

1 10 100 1,000 

k 

Scales as 

log(N)/log(logN)) 

((m–1)/2)*(log(N)/N) 

where m is the number of 

edges with which a 

node enters 

Highly resilient to random 

failures while being very 

sensitive to targeted 

attacks 

Figure 2. Comparing the survivability components of random, small-world, and 

scale-free networks. 

Manufacturer 

Warehouse 

Warehouse 

Warehouse 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Manufacturer 

Manufacturer 

Warehouse 

Warehouse 

Warehouse 

Figure 3. The transition from supply chain to a survivable supply network. 

Failed node 

Failed edge 

Alternate path 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 

Retailer 



Preferential attachment Random attachment Proposed attachment rules 

Figure 4. Snapshots of the modeled networks during their growth, where the nodes number 70. MSBs are green, FSBs are red, and 

battalions are blue. 

bar for more details)—leads to scale-free networks 

with very few critical and many not-socritical 

nodes. Here we measure a node’s criticality 

in terms of the number of edges incident 

on it. So, these networks are robust to random 

failures (the probability that a critical node fails 

is very small) but not to targeted attacks (attacking 

the very few critical nodes would devastate 

the network). Also, it’s not practically feasible 

to have all nodes play an equal role in the system—that 

is, be equally critical. Thus, the network 

should have a good balance of critical, 

not-so-critical, and noncritical nodes. 

The fourth is efficient rewiring. Rewiring 

edges in a network might or might not be feasible, 

depending on the domain. But where 

it is feasible, it should preserve the other 

three traits. 

Although complete graphs come equipped 

with good survivability components, they 

clearly aren’t cost effective. Allowing every 

agent in an agent system to communicate 

with every other agent uses system bandwidth 

inefficiently and could completely bog 

down the system. So the amount of redundancy 

results from a trade-off between cost 

and survivability. 

An illustration 

Suppose we want to build a topology for a 

military supply chain that must be survivable 

in wartime. First, we broadly classify the network 

nodes into three types: 

• Battalions prefer to attach to a highly connected 

node so that the supplies from different 

parts of the network will be transported 

to them in fewer steps. Battalions 

also require quick responses, so they prefer 

the subsequent links to attach to nodes at 

convenient shorter distances (in our model 

we considered a fixed distance of two). 

• A forward support battalion prefers to 

attach to highly connected nodes so that 

its supplies proliferate faster in the network. 

The supply range from an FSB goes 

up to a particular distance (at most three 

in our model). 

• A main support battalion also prefers to 

attach to a highly connected node to 

enable its supplies to proliferate faster in 

the network. We assume an unrestricted 

supply reach from an MSB, thus facilitating 

some long-range connections. 

In a conventional logistics network, the 

MSBs supply commodities (such as ammunitions, 

food, and fuel) to the FSBs, who in 

turn forward them to the battalions. Our 

approach doesn’t restrict node functionalities 

as such—for example, we assume that 

even a battalion can supply commodities to 

other battalions if necessary. 

In (number of nodes of degree > k) 

8 

7 

6 

5 

4 

3 

2 

1 

Model 1 

Model 2 

Model 3 

Characteristic path length 

5.6 

5.5 

5.4 

5.3 

5.2 

5.1 

5.0 

4.9 

4.8 

(a) 

0 

0 1 2 3 4 5 

In (degree k) 

(b) 

4.7 

6.5 7.0 7.5 

8.0 8.5 9.0 

Ln (number of nodes) 

Figure 5. How our proposed network performed: (a) the log-log of the degree distribution for all the three networks; 

(b) the characteristic path length of the proposed network against the log of the number of nodes. 


Growth mechanisms 

Start with a small number of nodes—say, 

m 0 —and assume that every time a node 

enters the system, m edges are pointing from 

it, where m < m 0 . Battalions, FSBs, and 

MSBs enter the system in a certain ratio 

l:m:n where l > m > n: 

• A battalion has one edge pointing from it 

and a second edge added with a probability 

p. 

• An FSB has three edges pointing from it. 

• An MSB has five edges pointing from it. 

The attachment rules applied depend on 

which node type enters the system: 

• For a battalion, the first edge attaches to a 

node i of degree k i with the probability 

ki 

Π i = 

∑ k 

. 

j 

j 

The second edge, which exists with a 

probability p, attaches to a randomly chosen 

node at a distance of two. 

• For an FSB, the first edge attaches to a 

node i of degree k i with the probability 

ki 

Π i = 

∑ k 

. 

j 

j 

The subsequent edges attach to a randomly 

chosen node at a distance of at most three. 

Table 1. Simulation results. 

Model 1 (random) Model 2 (preferential) Model 3 (proposed) 

Clustering coefficient 0.0038–0.0039 0.013–0.019 0.35–0.39 

Characteristic path length 5.26–5.36 4.09–4.25 4.69–4.79 

• For an MSB, each edge attaches preferentially 

to a node i with degree k i with the 

probability 

ki 

Π i = 

∑ k 

. 

j 

j 

Simulation and analysis 

Using this method, we built a network of 

1,000 nodes with l, m, and n being 25, 4, and 

1 (we obtained these values from the current 

configuration of the military logistics network 

used in the UltraLog program) and 

p = 1/2. We compared this network’s survivability 

with that of two other networks built 

using similar mechanisms except that one 

used purely preferential attachment rules 

(similar to scale-free networks) and the other 

used purely random attachment rules (similar 

to random networks) (see Figure 4). All 

three networks had an equal number of edges 

and nodes to ensure fair comparison. 

We refer to the networks built from random, 

preferential, and proposed attachment 

rules as Models 1, 2, and 3, respectively. As 

we noted earlier, a typical military supply 

chain (see Figure 1a) with a tree-like or hierarchical 

structure has deficient survivability 

components, making it vulnerable to both 

random and targeted attacks. Models 1, 2, 

and 3 outperform the typical supply network 

in all survivability components. 

Figure 5a shows the three models’ degree 

distribution. As expected, the preferentialattachment 

network has a heavier tail than 

the other two networks. We measured survivability 

components for all three networks. 

The clustering coefficient for Model 3 was 

the highest (see Table 1). The Model 3 attachment 

rules, especially those for battalions and 

FSBs, contribute implicitly to the clustering 

coefficient, unlike the attachment rules in the 

other models. 

The proposed network model’s characteristic 

path length measured between 4.69 and 4.79 

despite the network’s large size (1,000 nodes). 

This value puts it between the preferential and 

random attachment models. Also, as Figure 5b 

shows, the characteristic path length increases 

in the order of log(N) as N increases. Model 3 

clearly displays small-world behavior. 

To measure network robustness, we removed 

a set of nodes from the network and 

evaluated its resilience to disruptions. We 

considered two attacks types: random and targeted. 

To simulate random attacks, we removed 

a set of randomly chosen nodes; for 

targeted attacks, we removed a set of nodes 

selected strictly in order of decreasing node 

degree. To determine robustness, we measured 

how the size of each network’s largest 

connected component, characteristic path 

length, and maximum distance within the 

largest connected component changed as a 

function of the number of nodes removed. We 

expect that in a robust network the size of the 

largest connected component is a considerable 

fraction of N (usually O(N)), and the distances 

between nodes in the largest connected 

component don’t increase considerably. 

For random failures, Figure 6 shows that 

Model 3’s robustness nearly matches that of 

the preferential-attachment network (note that 

scale-free networks are highly resilient to ran- 

Size of the largest connected component 

1,000 

900 

800 

700 

600 

500 

400 

300 

200 

100 

0 

(a) 

Model 1 

Model 2 

Model 3 

0 20 40 60 80 

Percentage of nodes removed 

Average length in the largest 

connected component 

10 

9 

8 

7 

6 

5 

4 

3 

Maximum distance in the largest 


2 

1 

0 

0 20 40 60 80 

(b) Percentage of nodes removed (c) 

25 

20 

15 

10 

5 

0 

0 20 40 60 80 


Figure 6. Responses of the three networks to random attacks, plotted as (a) the size of the largest connected component, 

(b) characteristic path length, and (c) maximum distance in the largest connected component against the percentage of nodes 

removed from each network. 



1,200 

18 

45 

Model 1 

16 

40 

1,000 

Model 2 

Model 3 

14 

35 

800 

12 

30 

10 

25 

600 

8 

20 

400 

6 

15 

4 

10 

200 

2 

5 

0 

0 

0 

0 10 20 30 40 50 60 0 20 40 60 0 10 20 30 40 50 60 

(a) Percentage of nodes removed (b) Percentage of nodes removed (c) Percentage of nodes removed 

Size of the largest connected component 

Average length in the largest 


Figure 7. The three networks’ responses to targeted attacks, plotted as (a) the size of the largest connected component, 

(b) characteristic path length, and (c) maximum distance in the largest connected component against the percentage of nodes 

removed from each network. 

Maximum distance in the largest 


dom failures). Also, the decrease in the largest 

connected component’s size is linear with 

respect to the number of nodes removed, which 

corresponds to the slowest possible decrease. 

So, we can safely conclude that these networks 

T h e A u t h o r s 

are robust to random failures—most of the 

nodes in the network have a degree less than 

four, and removing smaller-degree nodes 

impacts the networks much less than removing 

high-degree nodes (called hubs). 

Hari Prasad Thadakamalla is a PhD student in the Department of Industrial 

and Manufacturing Engineering at Pennsylvania State University, University 

Park. His research interests include supply networks, search in complex networks, 

stochastic systems, and control of multiagent systems. He obtained 

his MS in industrial engineering from Penn State. Contact him at 

hpt102@psu.edu. 

Usha Nandini Raghavan is a PhD student in industrial and manufacturing 

engineering at Pennsylvania State University, University Park. Her research 

interests include supply chain management, graph theory, complex adaptive 

systems, and complex networks. She obtained her MSc in mathematics from 

the Indian Institute of Technology, Madras. Contact her at uxr102@psu.edu. 

Soundar Kumara is a Distinguished Professor of industrial and manufacturing 

engineering. He holds joint appointments with the Department of Computer 

Science and Engineering and School of Information Sciences and Technology 

at Pennsylvania State University. His research interests include 

complexity in logistics and manufacturing, software agents, neural networks, 

and chaos theory as applied to manufacturing process monitoring and diagnosis. 

He’s an elected active member of the International Institute of Production 

Research. Contact him at skumara@psu.edu. 

Réka Albert is an assistant professor of physics at Pennsylvania State University 

and is affiliated with the Huck Institutes of the Life Sciences. Her 

main research interest is modeling the organization and dynamics of complex 

networks. She received her PhD in physics from the University of Notre 

Dame. She is a member of the American Physical Society and the Society for 

Mathematical Biology. Contact her at ralbert@phys.psu.edu. 

These networks’ responses to targeted 

attacks are inferior compared to their resilience 

to random attacks (see Figure 7). The 

size of the largest component decreases much 

faster for the proposed network than for the 

other two networks, but the proposed network 

performs better on the other two robustness 

measures. That is, the distances in the connected 

component are considerably smaller 

when more than 10 percent of nodes are 

removed. 

We can improve robustness to targeted 

attacks by introducing constraints in the 

attachment rules. Here we assume that node 

type constrains its degree—that is, network 

MSBs, FSBs, and battalions can’t have more 

than m 1 , m 2 , and m 3 edges, respectively, incident 

on them. This is a reasonable assumption 

because in military logistics (or any orga- 

Sixe of the largest connected component 

1,000 

900 

800 

700 

Model 

m 1 = 4, m 2 = 10, m 3 = 25 

m 1 = 4, m 2 = 8, m 3 = 12 

m 1 = 3, m 2 = 6, m 3 = 10 

600 

500 

400 

300 

200 

100 

0 

0 10 20 30 40 50 60 


Figure 8. The proposed network’s 

responses to targeted attacks for 

different values of m 1 , m 2 , and m 3 . 


Table 2. The proposed network’s characteristic path 

length for different m 1 , m 2 , and m 3 values. 

Values of m 1 , m 2 , and m 3 

Characteristic path length 

m 1 = ∞, m 2 = ∞, m 3 = ∞ 4.4 

m 1 = 4, m 2 = 10, m 3 = 25 6.2 

m 1 = 4, m 2 = 8, m 3 = 12 7.1 

m 1 = 3, m 2 = 6, m 3 = 10 8.0 

nization’s logistics management, for that matter), 

the suppliers might not be able to cater to 

more than a certain number of battalions or 

other suppliers. Initial experiments (see Figure 

8) show that a network with these constraints 

displayed improved robustness to targeted 

attacks while not deviating much from 

the clustering coefficient. However, as we 

restrict how many links a node can receive, 

the network’s characteristic path length 

increases (see Table 2). Clearly a trade-off 

exists between robustness to targeted attacks 

and the average characteristic path length. 

The fourth measure of survivability, network 

adaptivity, relates more to 

node functionality than to 

topology. Node functionality 

should facilitate the ability to 

rewire. For example, if a supplier 

can’t fulfill a customer’s 

demands, the customer seeks 

an alternate supplier—that is, 

the edge connected to the supplier 

is rewired to be incident on another supplier. 

Our model rewires according to its 

attachment rules. We conjecture that in such 

a case, other survivability components (clustering 

coefficient, characteristic path length, 

and robustness) will be intact. But to make a 

stronger argument we need more analysis in 

this direction. 

The growth mechanism we describe is 

more like an illustration because 

real-world data aren’t available, but we can 

always modify it to incorporate domain 

constraints. For example, we’ve assumed 

that a new node can attach preferentially to 

any node in the network, which might not 

be a realistic assumption. If specific geographical 

constraints are known, we can 

modify our mechanism to make the new 

node entering the system attach preferentially 

only within a set of nodes that satisfy 

the constraints. 

Acknowledgments 

We thank the anonymous reviewers for their 

helpful comments. We acknowledge DARPA for 

funding this work under grant MDA972-01-1- 

0038 as part of the UltraLog program. 

References 

1. J.M. Swaminathan, S.F. Smith, and N.M. 

Sadeh, “Modeling Supply Chain Dynamics: 

A Multiagent Approach,” Decision Sciences, 

vol. 29, no. 3, 1998, pp. 607–632. 

2. A.-L. Barabási and R. Albert, “Emergence of 

Scaling in Random Networks,” Science, vol. 

286, Oct. 1999, pp. 509–512. 

Look to the Future 

IEEE Internet Computing reports 

emerging tools, technologies, 

and applications implemented through 

the Internet to support a worldwide 

computing environment. 

In 2004-2005, we’ll look at 

• Homeland Security 

• Internet Access to Scientific Data 

• Recovery-Oriented 

Approaches to Dependability 

• Information Discovery: 

Needles and Haystacks 

• Internet Media 

... and more! 

www.computer.org/internet/ 


Survivability of a Distributed Multi-Agent Application - A Performance Control 

Perspective 

Nathan Gnanasambandam, Seokcheon Lee, Soundar R.T. Kumara, Natarajan Gautam 

Pennsylvania State University 


{gsnathan, stonesky, skumara, ngautam}@psu.edu 

Wilbur Peng, Vikram Manikonda 

Intelligent Automation Inc. 

Rockville, MD 20855 

{wpeng, vikram}@i-a-i.com 

Marshall Brinn 

BBN Technologies 

Cambridge, MA 02138 

mbrinn@bbn.com 

Mark Greaves 

DARPA IXO 

Arlington, VA 22203 

mgreaves@darpa.mil 

Abstract 

Distributed Multi-Agent Systems (DMAS) such as supply 

chains functioning in highly dynamic environments need to 

achieve maximum overall utility during operation. The utility 

from maintaining performance is an important component 

of their survivability. This utility is often met by identifying 

trade-offs between quality of service and performance. 

To adaptively choose the operational settings for better utility, 

we propose an autonomous and scalable queueing theory 

based methodology to control the performance of a hierarchical 

network of distributed agents. By formulating 

the MAS as an open queueing network with multiple classes 

of traffic we evaluate the performance and subsequently the 

utility, from which we identify the control alternative for a 

localized, multi-tier zone. When the problem scales, another 

larger queueing network could be composed using 

zones as bu0ilding-blocks. This method advocates the systematic 

specification of the DMAS’s attributes to aid realtime 

translation of the DMAS into a queueing network. We 

prototype our framework in Cougaar and verify our results. 

1. Introduction 

Distributed multi-agent systems (DMAS), through adaptivity, 

have enormous potential to act as the “brains” behind 

numerous emerging applications such as computational 

grids, e-commerce hubs, supply chains and sensor 

networks [13]. The fundamental hallmark of all these applications 

is dynamic and stressful environmental conditions, 

of one type or the other, in which the MAS as a whole must 

survive albeit it suffers temporary or permanent damage. 

While the survival notion necessitates adaptivity to diverse 

conditions along the dimensions of performance, security 

and robustness, delivering the correct proportion of these 

quantities can be quite a challenge. From a performance 

standpoint, a survivable system can deliver excellent Quality 

of Service (QoS) even when stressed. A DMAS could be 

considered survivable if it can maintain at least x% of system 

capabilities and y% of system performance in the face 

of z% of infrastructure loss and wartime loads (x, y, z are 

user-defined) [7]. 

We address a piece of the survivability problem by building 

an autonomous performance control framework for the 

DMAS. It is desirable that the adaptation framework be 

generic and scalable especially when building large-scale 

DMAS such as UltraLog [2]. For this, one can utilize a 

methodology similar to Jung and Tambe [19], composing 

the bigger society of smaller building blocks (i.e. agent 

communities). Although Jung and Tambe [19] successfully 

employ strategies for co-operativeness and distributed 

POMDP to analyze performance, an increase in the number 

of variables in each agent can quickly render POMDP ineffective 

even in reasonable sized agent communities due to 

the state-space explosion problem. In [27], Rana and Stout 

identify data-flows in the agent network and model scala-

¡ ¡ ¢ £ ¤ ¥ ¦ £ § ¨ © ¥ 

¨ ¥ ¦ ¤ ¦ © ¥ § ¥ ¥ 

£ ¤ ¥ ¢ © ¥ 

© £ ¨ ¥ ¤ § ¢ ¥ 

© £ ¨ £¦ £ ¨ ¦ ¥ ¢ ¥ 

Figure 1. Operational Layers forming the MAS 

bility with Petri nets, but their focus is on identifying synchronization 

points, deadlocks and dependency constraints 

with coarse support for performance metrics relating to delays 

and processing times for the flows. Tesauro et al. [34] 

propose a real-time MAS-based approach for data centers 

that is self-optimizing based on application-specific utility. 

While [19, 27] motivate the need to estimate performance of 

large DMAS using a building block approach, [34] justifies 

the need to use domain specific utility whose basis should 

be the network’s service-level attributes such as delays, utilization 

and response times. 

We believe that by using queueing theory we can analyze 

data-flows within the agent community with greater granularity 

in terms of processing delays and network latencies 

and also capitalize on using a building block approach by 

restricting the model to the community. Queueing theory 

has been widely used in networks and operating systems 

[5]. However, the authors have not seen the application 

of queueing to MAS modeling and analysis. Since, agents 

lend themselves to being conveniently represented as a network 

of queues, we concentrate on engineering a queueing 

theory based adaptation (control) framework to enhance the 

application-level performance. 

Inherently, the DMAS can be visualized as a multilayered 

system as is depicted in Figure 1 . The top-most 

layer is where the application resides, usually conforming 

to some organization such as mesh, tree etc. The infrastructure 

layer not only abstracts away many of the complexities 

of the underlying resources (such as CPU, bandwidth), 

but more importantly provides services (such as Message 

Transport) and aiding agent-agent services (such as naming, 

directory etc.). The bottom most layer is where the 

actual computational resources, memory and bandwidth reside. 

Most studies in the literature do not make this distinction 

and as such control is not executed in a layered 

fashion. Some studies such as [35, 17], consider controlling 

attributes in the physical or infrastructural layers so 

that some properties (eg. robustness) could result and/or 

the facilities provided by these layers are taken advantage 

of. Often, this requires rewiring the physical layer, availability 

of a infrastructure level service or the ability of the 

application of share information with underlying layers in 

a timely fashion for control purposes. In this initial work, 

we consider control only due to application-level trade-offs 

such as quality of service versus performance and assume 

that infrastructure level services (such as load-balancing, 

priority scheduling) or physical level capabilities (such as 

rewiring) are not possible. While we intend to extend the 

approach to multi-layered control, it must be noted that it 

is not always possible for the application (or the application 

manager) to have access to all the underlying layers due 

to security reasons. In autonomic control of data centers, 

the application manager may have complete authority over 

parameters in the physical layer (servers, buffers, network), 

the infrastructure (middle-ware) and the applications. However, 

in DMAS scenarios, especially when dealing with mobile 

agents (as an application), trust between the layers is 

often partial forcing them to negotiate parameters through 

authorized channels. Hence, each layer must be capable of 

adapting with minimum cross-layer dependencies. 

Our contribution in this work is to combine queueing 

analysis and application-level control to engineer a generic 

framework that is capable of self-optimizing its domainspecific 

utility. Secondly, we provide a methodology for 

engineering a self-optimizing DMAS to assure applicationlevel 

survivability. While we see utility improvements by 

by adopting application-level adaptivity, we understand that 

further improvement may be gained by utilizing the adaptive 

capabilities of the underlying layers. 

Before we consider the details of our framework, we 

classify the performance control approaches in literature in 

Section 2. We present the details for our Cougaar based 

test-bed system in Section 3. The architectural details of 

our framework is provided in Section 4. We provide an empirical 

evaluation in Section 5 and finally conclude with discussions 

and future work in Section 6. 

2. Background and Motivation 

2.1 Approaches in Literature 

Because of the diversity of literature on control frameworks 

and performance evaluation, we examined a representative 

subset primarily on the basis of control objective, 

(component) interdependence and autonomy, generality, 

composability, real-time capability (off-line/on-line 

control) and layering in control architecture.

In some AI based approaches such as [32, 10], behavioral 

or rule based controllers are employed to make the 

system exhibit particular behavior based upon logical reasoning 

or learning. While performance is not the objective, 

layered learning is an interesting capability that may 

be helpful in a large scale MAS. Learning may be from a 

statistical sense as well where the parameters of a transfer 

function are learnt from empirical data to subsequently 

enforce feedback control [8]. Another architectural framework 

called MONAD [37], utilizes a hierarchical and distributed 

behavior-based control module, with immense flexibility 

through scripting for role and resource allocation, 

and co-ordination. While many these approaches favor 

the “sense-plan-act” or “sense and respond” paradigm and 

some partially support flexibility through scripting, some 

important unanswered questions are what happens when 

system size changes, can all axioms and behaviors be learnt 

a-priori and what is the performance impact of size (i.e. 

scalability)? 

Control theoretic approaches in software performance 

optimization are becoming important [22, 29], with software 

becoming increasingly more complex, multi-layered 

and having real-time requirements. However, because of 

the dynamic system boundaries, size, varying measures of 

performance and non-linearity in DMAS it is very complex 

to design a strict control theoretic control process [21]. 

Some approaches such as [21, 34] take the heuristic path, 

with occasional analogs to control theory, with an emphasis 

on application or domain-specific utility. Kokar et al. 

[22] refer to this utility as benefit function and elaborate on 

various analogs between software systems and traditional 

control systems. From the perspective of autonomic control 

of computer systems, Bennani and Menasce [4] study 

the robustness of self-management techniques for servers 

under highly variable workloads. Although queueing theory 

has been used in this work, any notion of components 

being distributed or agent-based seems to be absent. 

Furthermore, exponential smoothing or regression based 

load-forecasting may not be sufficient to address situations 

caused by wartime dynamics, catastrophic failure and distributed 

computing. Nevertheless, in our approach we have 

a notion of controlling a distributed application’s utility using 

queueing theory. 

Numerous market-based control mechanisms are available 

in literature such as [24, 9, 12, 6]. In market-based 

control systems, agents emulate buyers and sellers in a 

market acting only with locally available information yet 

helping us realize global behaviour for the community of 

agents. While these methods are very effective and offer 

desirable properties such as decentralization, autonomy and 

control hierarchy, they have been used for resource allocation 

[24, 9] and resource control [6]. The Challenger [9] 

system seeks to minimize mean flow time (job completion 

time - job origination time), the task is allocated to an agent 

providing least processing time. Load balancing is another 

application as applied by Ferguson et al. [12]. Resource allocation 

and load-balancing can be thought of as infrastructure 

level services, that agent frameworks such as Cougaar 

[1] provide and hence in our work we focus on applicationlevel 

performance and the associated utility to the DMAS. 

Using finite state machines, hybrid automata and their 

variants have been the foci of many research paths in agent 

control as in [11, 23]. The idea here is to utilize the states 

of the multi-agent system to represent, validate, evaluate, 

and choose plans that lead the system towards the goal. Often, 

the drawback here is that when the number of agents 

increase, the state-space approaches tend to become intractable. 

Heuristics have widely been used in controlling multiagent 

systems primarily in the following sense: searching 

and evaluating options based on domain knowledge and 

picking a course of action (maybe a compound action composed 

of a schedule of individual actions) eventually. The 

main idea in recent heuristics based control as exemplified 

by [36, 26, 31] is that schedules of actions are chosen based 

upon requirements such as costs, feasibilities for real-time 

contexts, complexity, quality etc. Opportunistic planning is 

an interesting idea as mentioned in Soto et al. [31] refers 

to the best-effort planning (maximum quality) considering 

available resources. These meta-heuristics offer very effective, 

special-purpose solutions to control agent behavior, 

however to be more flexible, we separate the performance 

evaluation and the domain-specific application utility computation. 

Given that we have a model for performance estimation 

(whose parameters and state-space are known), dynamic 

programming (DP) and its adaptive version - reinforcement 

learning (RL), and model predictive control (MPC) have 

been used to find the control policy [3, 33, 20, 28, 25]. 

Since the complexity of finding the optimal policy grows 

exponentially with the state space [3] and convergence has 

to be ensured in RL [33, 20], we take an MPC-like approach 

in our work for finding quick solutions in real-time. We discuss 

this further in Section 4. 

2.2 Related Work 

In large scale MAS applications, performance estimation 

and modeling itself can be a formidable task as illustrated 

by [16] in the UltraLog [2] context. UltraLog [2], 

built on Cougaar [1], uses for heuristic control a host of 

architectural features such as operating modes, conditions, 

and plays and play-books as described in [21]. Helsinger 

et al. [15] incorporate the aforementioned features into 

their closed-loop heuristic framework that balances the different 

dimensions of system survivability through targeted

© © ¤ ¢ ¨ ¤ 

£ ¦ ¨ © § 

¢ £ ¤ ¥ ¦ ¡ 

£ ¦ ¨ © § 

defense mechanisms, trade-offs and layered control actions. 

The importance of high-level, system specifications (interchangeably 

called TechSpecs, specification database, component 

database) has been emphasized in many places such 

as [18, 21, 14]. These specifications contain componentwise, 

static input/output behavior, operating requirements 

and control actions of agents along with domain measures 

of performance and computation methodologies [14]. 

Also, queueing network based methodologies for offline 

and design-time performance evaluation have been applied 

and validated in [14, 30]. Building on these ideas, we build 

a real-time framework with queueing based performance 

prediction capabilities. 

¤ ¨ 

£ ¦ ¨ © § 

(a) MAS building 

block: Community 

¡ ¢ £ ¤ ¥ ¦ § § ¨ £ © ¤ 

¡ ¢ £ ¤ ¥ ¦ § § ¨ £ © ¤ 

¡ ¢ £ ¤ ¥ ¦ § § ¨ £ © ¤ 

(b) Agent society formed by composing 

communities 


Being the top-most layer as an application, the survivability 

of a DMAS depends on its ability to leverage its 

knowledge of the domain, the system’s overall utility and 

available control-knobs. The utility of the application is the 

combined benefit along several conflicting (eg. completeness 

and timeliness [7, 2]) and/or independent (eg. confidentiality 

and correctness [7, 2]) dimensions, which the application 

tries to maximize in a best-effort sense through 

trade-offs. Understandably, in a distributed multi-agent 

setting, mechanisms to measure, monitor and control this 

multi-criteria utility function become hard and inefficient, 

especially under conditions of scale-up. Given that the application 

does not change its high-level goals, task-structure 

or functionality in real-time, it is beneficial to have a framework 

that assists in the choice of operational modes (eg. 

plan quality) that maximize the utility from performance. 

Hence, the research objective of this work is to design and 

develop a generic, real-time, self-controlling framework for 

DMAS, that utilizes a queueing network model for performance 

evaluation and a learned utility model to select an 

appropriate control alternative. 

2.4 Solution Methodology 

This research concentrates on adjusting the applicationlevel 

parameters or operating modes (opmodes for short) 

within the distributed agents to make an autonomous choice 

of operational parameters for agents in a reasonable-sized 

domain (called an agent community). The choice of opmodes 

is based on the perceived application-level utility of 

the combined system (i.e. the whole community) that current 

environmental conditions allow. We assume that the 

application’s utility depends on the choice of opmodes at 

the agents constituting the community because the opmodes 

directly affect the performance. A queueing network model 

is utilized to predict the impact of DMAS control settings 

and environmental conditions on steady-state performance 

Figure 2. MAS Community and Society 

(in terms of end-to-end delays in flows), which in turn is 

used to estimate the application-level utility. After evaluating 

and ranking several alternatives from among the feasible 

set of operational settings on the basis of utility, the best 

choice is picked. 

3. Overview of Application (CPE) Scenario 

The Continuous Planning and Execution (CPE) Society 

is a command and control (C2) MAS built on Cougaar 

(DARPA Agent Framework [1]) that serves as the test-bed 

for performance control. Designed as a building block for 

larger scale MAS, the primary CPE prototype consists of 

three tiers (Brigade, Battalion, Company) as shown in Figure 

2a. While the discussion is mainly with respect to the 

structure of CPE, the system can be grown by combining 

many CPE communities to form large agent societies as 

shown in Figure 2b. 

CPE embodies a complete military logistics scenario 

with agents emulating roles such as suppliers, consumers 

and controllers all functioning in a dynamic and hostile (destructive) 

external environment. Embedded in the hierarchical 

structure of CPE are both command and control, and 

superior-subordinate relationships. The subordinates compile 

sensor updates and furnish them to superiors. This 

enables the superiors to perform the designated function 

of creating plans (for maneuvering and supply) as well as 

control directives for downstream subordinates. Upon receipt 

of plans, the subordinates execute them. The supply 

agents replenish consumed resources periodically. This 

high level system definition is to be executed continuously 

by the application with maximum achievable performance 

in the presence of stresses that include temporary and catastrophic 

failure. Stresses associated with wartime situations 

cause the resource allocation (CPU, memory, bandwidth) 

and offered load (due to increased planning requirements)

¢ £ ¤ ¤ £ ¤ ¡ 

¤ ¨© § ¢ ¤ ¡ ¢ © ¡ ¢ £ £ ¢ 

 

< 

5 

1 

¢ £ ¤ ¤ £ ¤ ¡ 

¦ ¦ § ¨© ¡ ¨ ¤ £ ¢ 

¥ 

, > 

, = ; : = 

 

 

: ; , 

 

! " " # ! 

: = 

8 ) 

) ; < ; ) ; : 

$ % & ' % ( 

- . / . - 0 , 

/ % 2 / - 3 4 1 

3 6 - 4 3 / - 1 

A / 2 - 3 4 @ 

: 

? ; $ 

) > 

8 3 9 / + 2 . / 9 4 

: > ; < > 

, 

+ * 

2 2 . 6 7 5 

3 9 / 8 

) 

+ * 

Figure 3. Traffic flow within CPE 

to fluctuate immensely. 

As part of the application-level adaptivity features, a set 

of opmodes are built into the system. Opmodes allow individual 

tasks (such as plans, updates, control) to be executed 

at different qualities or to be processed at different rates. We 

assume that TechSpecs for the CPE application (similar to 

[14]) are available to be utilized by the control framework. 

Although, functionally CPE and UltraLog are unique, 

the same flavor of activities are reflected in both. Both of 

them share the same Cougaar infrastructure; execute planning 

in dynamic, distributed settings with similar QoS requirements; 

and are both one application with physically 

distributed components interconnected by task flows (as 

shown in Figure 3 in the case of CPE), wherein the individual 

utilities of the components contribute to the global 

survivability. 

4. Architecture of the Performance Control 

Framework 

The distributed performance control framework that 

accomplishes application-level survivability while operating 

amidst infrastructure/physical layer and environmental 

stresses is represented in Figure 4. This representation consists 

of activities, modules, knowledge repositories and information 

flow through a distributed collection of agents. 

The features for adaptivity are solely at the application level 

without considering infrastructure or physical level adaptivity 

such as dynamically allocating processor share or adjusting 

the buffer sizes. 

Figure 4. Architecture Overview 

Architecture Overview 

When the application is stressed by an amount S by the 

underlying layers (due to under-allocation of resources) 

and the environment (due to increased workloads during 

wartime conditions), the DMAS Controller has to examine 

all its performance-related variables from set X and the 

current overall performance P in order to adapt. The variables 

that need to be maintained are specified in the Tech- 

Specs and may include delays, time-stamps, utilizations and 

their statistics. They are collected in a distribution fashion 

through measurement points (MP ) which are “soft” storage 

containers residing inside the agents and contain information 

on what, when and how they should be measured. 

The DMAS Controller knows the set of flows F that traverse 

the network and the set of packet types T from the 

TechSpecs. With (F, T, X, C), where C is a suggestion 

based on prior effectiveness from the DMAS Controller, the 

Model Builder can select a suitable queueing model template 

Q. The Control Set Evaluator knows the current opmode 

set O as well as the set of possible opmodes, OS 

from TechSpecs. To evaluate the performance due to a candidate 

opmode set O ′ , the Control Set Evaluator uses the 

Queueing Model with a scaled set of operating conditions 

X ′ . Once the performance P ′ is estimated by the Queueing 

Model it can be cached in the performance database P DB 

and then sent to the Utility Calculator. The Utility Calculator 

computes the domain utility (U ′ ) due to (O ′ , P ′ ) 

and caches it in the utility database, UDB. Subsequently, 

the optimal operating mode O ∗ is identified and sent to the

DMAS Controller. The functional units of the architecture 

are distributed but for each community that forms part of 

a MAS society, O ∗ will be calculated by a single agent. 

We now examine the functionality and role offered by each 

component of the framework in greater detail. 

4.1 Self-Monitoring Capability 

Any system that wants to control itself should possess 

a clear specification of the scope of the variables it has to 

monitor. The TechSpecs is a distributed structure that supports 

this purpose by housing meta-data about all variables, 

X, that have to be monitored in different portions of the 

community (refer [14]). The data/statistics collected in a 

distributed way, is then aggregated to assist in control alternatives 

by the top-level controller that each community will 

possess. 

The attributes that need to be tracked are formulated 

in the form of measurement points (MP ). For example, 

one simple measurement could be specified as 

{what = delay, when = every packet, how = 

timestamp at receiving emd − timestamp at sending end } 

which is subsequenly stored in an MP . Each agent can 

look up its own TechSpecs and from time-to-time forward 

a measurement to its superior. The superior can analyze 

this information (eg. calculate statistics such as mean or 

variance) and/or add to this information and forward it 

again. We have measurement points for time-periods, timestamps, 

operating-modes, control and generic vector-based 

measurements. These measurement points can be chained 

for tracking information for a flow such that information is 

tagged-on at every point the flow traverses. For the sake of 

reliability, the information that is contained in these agents 

is replicated at several points, so that in the absence of packets 

reaching on time or not reaching at all, previously stored 

packets and their corresponding information can be utilized 

for control purposes. 

4.2 Self-Modeling Capability 

One of the key features of this framework is that it 

has the capability to choose a type of performance model 

for analysing the current system configuration from several 

queueing model templates provided. The type of model that 

is utilized is based on the accuracy, the computation time 

and the history of effectiveness of the model. For example, 

a simulation based queueing model may be very accurate 

but cannot evaluate enough alternatives in limited time, in 

which case an analytical model (such as BCMP [5], QNA 

[38]) is preferred. 

The inputs to the model builder are the flows that traverse 

the network (F ), the types of packets (T ) and the current 

configuration of the network. If at a given time, we know 

that there are n agents interconnected in a hierarchical fashion 

then the role of this unit is to represent that information 

in the required template format (Q). The current number 

of agents is known to the controller by tracking the measurement 

points. For example, if there is no response from 

an agent for a sufficient period of time, then for the purpose 

of modeling, the controller may assume the agent to 

be non-existent. In this way dynamic configurations can 

be handled. On the other hand, TechSpecs do mandate 

connections according to superior-subordinate relationships 

thereby maintaining the flow structure at all times. Once the 

modeling is complete, the MAS has to capability to analyze 

its current performance using the selected type of model. 

The MAS does have the flexibility, to choose another model 

template for a different iteration. 

4.3 Self-Evaluating Capability 

The evaluation capability, the first step in control, allows 

the MAS to examine its own performance under a given 

set of plausible conditions. This prediction of performance 

is used for the elimination of control alternatives that may 

lead to instabilities. Our notion of performance evaluation 

is similar to [34]. While Tesauro et al. [34] compute the 

resource level utility functions (based on the application 

manager’s knowledge of the system performance model) 

that can be combined to obtain a globally optimal allocation 

of resources, we predict the performance of the MAS 

as a function of its operating modes in real-time (within 

Queueing Model) and then use it to calculate its global utility 

(some more differences are pointed out in Section 4.4). 

By introducing a level of indirection, we may get some 

desirable properties because we separate an application’s 

domain-specific utility computation from performance prediction 

(or analysis). This theoretically enables us to predict 

the performance of any application whose TechSpecs are 

clearly defined and then compute the application-specific 

utility. In both cases, control alternatives are picked based 

on best-utility. We discuss the notion of control alternatives 

in Section 4.4. Also, our performance metrics (and hence 

utility) are based on service level attributes such as endto-end 

delay and latency, which is a desirable attribute of 

autonomic systems [34]. 

When plan, update and control tasks (as mentioned in 

Section 3) flow in this heterogeneous network of agents 

in predefined routes (called flows), the processing and wait 

times of tasks at various points in the network are not alike. 

This is because the configuration (number of agents allocated 

on a node), resource availability (load due to other 

contending software) and environmental conditions at each 

agent is different. In addition, the tasks themselves can be 

of varying qualities or fidelities that affects the time taken 

to process that task. Under these conditions, performance is

Symbol 

Table 1. Notation 

Description 

N Total # of nodes in the community 

λ ij Average arrival rate of class j at node i 

1/µ ijk Average processing time of class j at 

node i at quality k 

M total number of classes 

T i Routing probability matrix for class i 

W ijk Steady state waiting time for class j at 

node i at quality k 

Q ij Set of qualities at which a class j task 

can be processed at node i 

estimated on the basis of the end-to-end delay involved in a 

“sense-plan-respond” cycle. 

The primary performance prediction tool that we use are 

called Queueing Network Models (QNM) [5]. The QNM 

is the representation of the agent community in the queueing 

domain. As the first step of performance estimation, the 

agent community needs to be translated into a queueing network 

model. Table 1 provides the notations used is this section. 

Inputs and outputs at a node are regarded as tasks. The 

rate at which tasks of class j are received at node i is captured 

by the arrival rate (λ ij ). Actions by agents consume 

time, so they get abstracted as processing rates (µ ij ). Further, 

each task can be processed at a quality k ∈ Q ij , that 

causes the processing rates to be represented as µ ijk . Statistics 

of processing times are maintained at each agent in PDB 

to arrive at a linear regression model between quality k and 

µ ijk . Flows get associated with classes of traffic denoted 

by the index j. If a connection exists between two nodes, 

this is converted to a transition probability p ij , where i is 

the source and j is the target node. Typically, we consider 

flows originating from the environment, getting processed 

and exiting the network making the agent network an open 

queueing network [5]. Since we may typically have multiple 

flows through a single node, we consider multi-class 

queueing networks where the flows are associated with a 

class. Performance metrics such as delays for the “senseplan-respond” 

cycle is captured in terms of average waiting 

times, W ijk . As mentioned earlier, TechSpecs is a convenient 

place where information such as flows and Q ij can be 

embedded. 

The choice of QNM depends on the number of classes, 

arrival distribution and processing discipline as well as 

a suggestion C by the DMAS controller that makes this 

choice based upon history of prior effectiveness. Some analytical 

approaches to estimate performance can be found 

in [5, 38]. In the context of agent networks, Jackson and 

BCMP queueing networks to estimate the performance in 

[14]. By extending this work we support several templates 

of queueing models (such as BCMP [5], Whitt’s QNA [38], 

Jackson [5], M/G/1, a simulation) that can be utilized for 

performance prediction. 

4.4 Self-Controlling Capability 

In contrast to [34], we deal with optimization of the domain 

utility of a single MAS that is distributed, rather than 

allocating resources in an optimal fashion to multiple applications 

that have a good idea of their utility function 

(through policies). As mentioned before opmodes allow 

for trading-off quality of service (task quality and response 

time) and performance. We are assuming there is a maximum 

ceiling R on the amount of resources, and the available 

resources fluctuate depending on stresses S = S e +S a , 

where S e are the stresses from the environment (i.e. multiple 

contending applications, changes in the infrastructural 

or physical layers) and S a are the application stresses (i.e. 

increased tasks). The DMAS controller receives from 

the measurement points (MP ) a measurement of the actual 

performance P and a vector of other statistics (relating 

to X). Also at the top-level the overall utility (U) is 

U(P, S) = ∑ w n x n is known where x n is the actual utility 

component and w n is the associated weight specified by 

the user or another superior agent. We cannot change S, but 

we can adjust P to get better utility. Since P depends on 

O, which is a vector of opmodes collected from the community, 

we can use the QNM to find O ∗ and hence P ∗ that 

maximizes U(P, S) for a given S from within the set OS. 

In words, we find the vector of opmodes (O ∗ ) that maximizes 

domain utility at current S and opmodes O. This 

computation is performed in the Utility Calculator module 

using a learned utility model based on UDB. 

In addition to differences pointed out thus far, here are 

some more differences between this work and [34]: 

• Tesauro et al. [34] assume that the application manager 

in Unity has a model of system performance, which 

we do not assume. Although they allude to a modeler 

module, they do not explain the details of their performance 

model. We use a queueing network model 

that is constructed in real-time to estimate the performance 

for any set of opmodes O ′ by taking the current 

opmodes O and scaling them appropriately based on 

observed histories (X) to X ′ in the Control Set Evaluator. 

• Because of the interactions involved and complexity 

of performance modeling [19, 27], it may be timeconsuming 

to utilize statistical inferencing and learning 

mechanisms in real-time. This is why we use an 

analytical queueing network model to estimate performance 

quickly.

1000 

500 

0 

-500 

-1000 

0.2 0.4 0.6 0.8 

Default Policy 

Controlled 

Stress (S) 

Figure 5. Results Overview 

• Another difference is that in [34], they assume operating 

system support for being able to tune parameters 

such as buffer sizes and operating system settings 

which may not be true in many MAS-based situations 

because of mobility, security and real-time constraints. 

Besides, in addition to the estimation of performance, 

the queueing model may have the capability to eliminate 

instabilities from a queueing sense, which is not 

apparent in the other approach. 

• But most importantly, their work reflects a two level hierarchy 

where the resource manager mediates several 

application environments to obtain maximum utility to 

the data center. But our work is from the perspective 

of a single, self-optimizing application that is trying to 

be survivable by maximizing its own utility. 

Inspite of these differences, it is interesting to see that the 

self-controlling capability can be achieved, with or without 

explicit layering, in real-world applications. 

5. Empirical Evaluation on CPE Test-bed 

The aforementioned framework was implemented within 

CPE which we use as a test-bed for our experimentation. 

The main goal of this experimentation was to examine if 

application-level adaptivity led to any utility gains in the 

long run. The superior agents in CPE continuously plan 

maneuvers for their subordinates which get executed by the 

lower rung nodes. We subjected the entire distributed community 

to random stresses by simulating enemy clusters of 

varying sizes and arrival rates. These stresses translated into 

the need to perform the distributed “sense-plan-respond” 

more frequently causing increased load and traffic in the 

network of agents. The stresses were created by a worldagent 

whose main purpose was to simulate warlike dynamics 

within our test-bed. 

The CPE prototype consists of 14 agents spread across 

a physical layer of 6 CPUs. We utilized the prototype 

CPE framework to run 36 experiments at two stress levels 

(S = 0.25 and S = 0.75). There were three layers of 

hierarchy as shown in Figure 2a with a three-way branching 

at each level and one supply node. The community’s 

utility function was based on the achievement of real goals 

in military engagements such as terminating or damaging 

the enemy and reducing the penalty involved in consuming 

resources such as fuel or sustaining damage. To keep 

our queueing models simple, we assumed that the external 

arrival was Poisson while the service times were generally 

distributed. In order to cater to general arrival rates, the 

framework contains a QNA-based and a simulation-based 

model. Using this assumption a BCMP or M/G/1 queueing 

model could be selected by the framework for real-time 

performance estimation. The baseline for comparison was 

the do nothing policy (default) where we let the Cougaar infrastructure 

manage conditions of high load. Although our 

framework did better than any set of opmodes as shown in 

Figure 5 for the two stress modes, we show instantaneous 

and cumulative utility for two opmodes (Default A, B) in 

particular in Figure 6. We noticed that in the long run the 

framework enhanced the utility of the application as compared 

to the default policy. 

At both stress levels, the controlled scenario performed 

better that the default as shown in Figure 6. We did observe 

oscillations in the instantaneous utility and we attribute this 

to the impreciseness of the prediction of stresses. Stresses 

vary relatively fast in the order of seconds while the control 

granularity was of the order of minutes. Since this is a 

military engagement situation following no stress patterns, 

it is hard to cope with in the higher stress case. In contrast 

to MAS applications dealing with data centers where load 

can be attributed to time-of-day and other seasonal effects, 

it is not possible to get accurate load predictions for MAS 

applications simulating wartime loads. We think that this 

could be the reason why our utility falls in the latter case. 

In subsequent work, we intend to enhance Cougaar capability 

to be supportive of the application-layer by forcing it to 

guarantee some end-to-end delay requirements. 

6. Conclusions and Future Work 

In this paper, we were able to successfully control a realtime 

DMAS to achieve overall better utility in the long run, 

thus making the application survivable. Utility improvements 

were made through application-level trade-offs between 

quality of service and performance. We utilized a 

queueing network based framework for performance analysis 

and subsequently used a learned utility model for computing 

the overall benefit to the DMAS (i.e. community). 

While Tesauro et al. [34] employ a resource arbiter to maximize 

the combined utility of several application environments 

in a data center scenario, we focus on using queueing

-10 

20 

15 

10 

5 

0 

-5 

0 200 400 600 800 1000 1200 

time (sec.) 

Controlled Default A Default B 

1400 

1200 

1000 

800 

600 

400 

200 

0 

0 200 400 600 800 1000 1200 

time (sec.) 


(a) Instantaneous Utility (stress 0.25) 

(b) Cumulative Utility (stress 0.25) 

20 

15 

10 

5 

0 

0 200 400 600 800 1000 1200 

-5 

-10 

time (sec.) 


1200 

1000 

800 

600 

400 

200 

0 

0 200 400 600 800 1000 1200 

time (sec.) 


(c) Instantaneous Utility (stress 0.75) 

(d) Cumulative Utility (stress 0.75) 

Figure 6. Sample Results 

theory to maximize the utility from performance of a single 

distributed application given that is has been allocated 

some resources. We think that the approaches are complementary, 

with this study providing empirical evidence to 

support the observation by Jennings and Wooldridge in [18] 

that agents can be used to optimize distributed application 

environments, including themselves, through flexible highlevel 

(i.e. application-level) interactions. 

Furthermore, this work has resulted in a general architectural 

lesson. We believe that any distributed application 

would have flows of traffic and would require service 

level attributes such as response times, utilization or delays 

of components to be optimized. The paradigm that we 

have chosen can capture such quantities and help evaluate 

choices that may lead to better application utility. This concept 

of breaking the application into flows and allowing a 

real-time model-based predictor to steer the system into regions 

of higher utility is pretty generic in nature. 

While we are continuing the empirical evaluation, we 

keep the building blocks small to ensure scalability and 

to reduce interactions. We utilize TechSpecs to distribute 

knowledge and meta-data thus reemphazing the separation 

principle. Subsequently, we hope to broaden the layered 

control approach to encompass infrastructure-level control 

within the framework. Another avenue for improvement is 

to design self-protecting mechanisms so that the security 

aspect of the framework is reinforced. 


This work was performed under the DARPA UltraLog 

Grant#: MDA 972-01-1-0038. The authors wish to acknowledge 

DARPA for their generous support. 

References 

[1] Cougaar open source site. http://www.cougaar.org. 

DARPA. 

[2] Ultralog program site. http://dtsn.darpa.mil/ixo/. 


[3] A. G. Barto, S. J. Bradtke, and S. Singh. Learning to 

act using real-time dynamic programming. Artificial 

Intelligence, 72:81–138, 1995. 

[4] M. N. Bennani and D. A. Menasce. Assessing the 

robustness of self-managing computer systems under 

highly variable workloads. International Conference 

on Autonomic Computing, 2004. 

[5] G. Bolch, S. Greiner, H. de Meer, and K. S.Trivedi. 

Queueing Networks and Markov Chains: Modeling 

and Performance Evaluation with Computer Science 

Applications. John Wiley and Sons, Inc., 1998. 

[6] J. Bredin, D. Kotz, and D. Rus. Market-based resource 

control for mobile agents. Autonomous Agents, 1998. 


to assure survivability of distributed multi-agent sys-

tems. Proceedings of the Second Joint Conference on 

Autonomous Agents and Multi-Agent Systems, 2003. 

[8] T. Chao, F. Shan, and S. X. Yang. Modeling and design 

monitor using layered control architecture. Autonomous 

Agents and Multi-Agent Systems, 2002. 

[9] A. Chavaz, A. Moukas, and P. Maes. Challenger: 

A multi-agent systems for distributed resource allocation. 

Agents, 1997. 

[10] L. Chen, K. Bechkoum, and G. Clapworthy. A logical 

approach to high-level agent control. Agents, 2001. 

[11] A. E. Fallah-Seghrouchni, I. Degirmenciyan-Cartault, 

and F. Marc. Modelling, control and validation of 

multi-agent plans in dynamic context. Autonomous 


[12] D. Ferguson, Y. Yemini, and C. Nikolaou. Microeconomic 

algorithms for load balancing in distributed 

computer systems. Proceedings of the International 

Conference on Distributed Systems, 1988. 

[13] I. Foster, N. R. Jennings, and C. Kesselman. Brain 

meets brawn: Why grid and agents need each other. 

Autonomous Agents and Multi-Agent Systems, 2004. 

[14] N. Gnanasambandam, S. Lee, N. Gautam, S. R. T. 

Kumara, W. Peng, V. Manikonda, M. Brinn, and 

M. Greaves. Reliable mas performance prediction using 

queueing models. IEEE Multi-agent Security and 

Survivabilty Symposium, 2004. 

[15] A. Helsinger, K. Kleinmann, and M. Brinn. A framework 

to control emergent survivability of multi agent 

systems. Autonomous Agents and Multi-Agent Systems, 

2004. 

[16] A. Helsinger, R. Lazarus, W. Wright, and J. Zinky. 

Tools and techniques for performance measurement 

of large distributed multi-agent systems. Autonomous 


[17] Y. Hong and S. R. T. Kumara. Coordinating control 

decisions of software agents for adaptation to dynamic 

environments. Working Paper, Dept. of IME, Pennsylvania 

State University, University Park, PA, 2004. 

[18] N. R. Jennings and M. Wooldridge. Handbook of 

Agent Technology, chapter Agent-Oriented Software 

Engineering. AAAI/MIT Press, 2000. 

[19] H. Jung and M. Tambe. Performance models for large 

scale multi-agent systems: Using distributed pomdp 

building blocks. Proceedings of the Second Joint Conference 

on Autonomous Agents and Multi-Agent Systems, 

July 2003. 

[20] L. P. Kaelbling, M. L. Littman, and A. Moore. Reinforcement 

learning: A survey. Journal of Artificial 

Intelligence Research, 4:237–285, 1996. 

[21] K. Kleinmann, R. Lazarus, and R. Tomlinson. An infrastructure 

for adaptive control of multi-agent systems. 

IEEE Conference on Knowledge-Intensive 

Multi-Agent Systems, 2003. 

[22] M. M. Kokar, K. Baclawski, and Y. A. Eracar. Control 

theory-based foundations of self-controlling software. 

IEEE Intelligent Systems, pages 37–45, May/June 

1999. 

[23] K. C. Lee, W. H. Mansfield, and A. P. Sheth. A framework 

for controlling cooperative agents. IEEE Computer, 

1993. 

[24] T. W. Malone, R. Fikes, K.R.Grant, and M.T.Howard. 

Enterprise: A Market-like Task Scheduler for Distributed 

Computing Environments. Elsevier, Holland, 

1988. 

[25] M. Morari and J. H. Lee. Model predictive control: 

past, present and future. Computers and Chemical Engineering, 

23(4):667–682, 1999. 

[26] A. Raja, V. Lesser, and T. Wagner. Toward robust 

agent control in open environments. Agents, 2000. 

[27] O. F. Rana and K. Stout. What is scalabilty in multiagent 

systems? Proceedings of the Fourth International 

Conference on Autonomous Agents, 2000. 

[28] J. B. Rawlings. Tutorial overview of model predictive 

control. IEEE Control Systems, 20(3):38–52, 2000. 

[29] R. Sanz and K.-E. Arzen. Trends in software and control. 

IEEE Control Systems Magazine, June 2003. 

[30] F. Sheikh, J. Rolia, P. Garg, S. Frolund, and A. Shepard. 

Performance evaluation of a large scale distributed 

application design. World Congress on Systems 

Simulation, 1997. 

[31] I. Soto, M. Garijo, C. A. Iglesias, and M. Ramos. 

An agent architecture to fulfill real-time requirement. 

Agents, 2000. 

[32] P. Stone and M. Veloso. Using decision tree confidence 

factors for multi-agent control. Autonomous 

Agents, 1998. 

[33] R. S. Sutton, A. G. Barto, and R. J. Williams. Reinforcement 

learning is direct adaptive optimal control. 

IEEE Control Systems, 12(2):19–22, 1992. 

[34] G. Tesauro, D. M. Chess, W. E. Walsh, R. Das, 

I. Whalley, J. O. Kephart, and S. R. White. A multiagent 

systems approach to autonomic computing. Autonomous 


[35] H. P. Thadakamalla, U. N. Raghavan, S. R. T. Kumara, 

and R. Albert. Survivability of multi-agent supply 

networks: A topological perspective. IEEE Intelligent 

Systems: Dependable Agent Systems, 19(5):24– 

31, September/October 2004. 

[36] R. Vincent, B. Horling, V. Lesser, and T. Wagner. Implementing 

soft real-time agent control. Agents, 2001. 

[37] T. Vu, J. Go, G. Kaminka, M. Velosa, and B. Browning. 

Monad: A flexible architecture for multi-agent 

control. Autonomous Agents and Multi-Agent Systems, 

2003. 

[38] W. Whitt. The queueing network analyzer. The Bell 

System Technical Journal, 62(9):2779–2815, 1983.

Proceedings of the 1st Open Cougaar Conference 1 

Survivability through Implementation Alternatives 

in Large-scale Information Networks with Finite Load 

Seokcheon Lee and Soundar Kumara 

Department of Industrial and Manufacturing Engineering 



{stonesky, skumara}@psu.edu 

Abstract 

We study a large-scale information network, which is 

composed of distributed software components linked with 

each other through a task flow structure. The service 

provided by the network is to produce a global solution to 

a given problem, which is an aggregate solution of 

partial solutions from processing tasks. Quality of Service 

of this network is determined by the value of the global 

solution and time for generating the global solution. 

Survivability of the network is the capability to provide 

high Quality of Service by utilizing implementation 

alternatives as control actions, in the presence of 

accidental failures and malicious attacks. In this paper 

we develop an adaptive control mechanism to support 

survivability. We stress two desirable properties in 

designing the mechanism: scalability and predictability. 

To address adaptivity we model the stress environment 

indirectly by quantifying resource availability of the 

system. We build a mathematical programming model 

with the resource availability incorporated, which 

predicts Quality of Service as a function of control 

actions. By periodically solving the programming model 

and taking optimal control actions with recent resource 

availability, the system can be adaptive to the changing 

stress environment predictably. But, as the programming 

model becomes large-scale and complex, we agentify the 

components of the network from a control point of view 

so that the system can solve the large-scale programming 

model in a decentralized mode. We provide an auctionbased 

market as a decentralized coordination 

mechanism. 


Critical infrastructures become increasingly dependent 

on networked systems in many domains for automation or 

organizational integration. Though such infrastructure can 

improve the efficiency and effectiveness, these systems 

can be easily exposed to various adverse events such as 

accidental failures and malicious attacks [1]. Two metrics, 

namely survivability and scalability, can be used to 

determine the efficiency and effectiveness of these 

systems. Survivability is defined as “the capability of a 

system to fulfill its mission, in a timely manner, in the 

presence of attacks, failure, or accidents” [2]. One 

promising way to achieve survivability is through 

adaptivity: changing the system behavior to achieve the 

system goal in response to the changing environment [3]. 

One important consideration of an adaptation is 

predictability. Unpredictable adaptation can sometimes 

result in worse performance than without adaptation [4]. 

Scalability is defined as: “the ability of a solution to some 

problem to work when the size of the problem increases” 

(From Dictionary of Computing at 

http://wombat.doc.ic.ac.uk). As the size of networked 

systems grows scalability becomes a critical issue when 

developing practical software systems [5]. 

As software systems grow larger and more complex, 

component technology became one of the topmost topics 

in the computing community [6][7]. A component is a 

reusable program element, with which developers can 

build the systems needed by simply wiring all the 

components together. To support flexible usage of the 

components in various forms, the components must be 

independent, self-contained, and highly specialized. In 

component-based software systems, components interact 

with each other through a task flow structure with each 

component specialized for specific tasks. 

We study a large-scale information network, which is 

composed of distributed software components linked with 

each other through a task flow structure. A problem given 

to the network is decomposed into a set of tasks for some 

of software components and those tasks are propagated 

through the task flow structure. The service provided by 

the network is to produce a global solution to the given 

problem, which is an aggregate solution of partial 

solutions from processing tasks. Each component can 

process a task using one of available implementation 

alternatives, which trade off processing time and value of 

partial solution. Quality of Service (QoS) of this network 

is determined by the value of the global solution and time 

for generating the global solution. Survivability of the 

network is the capability to provide high QoS in the 

presence of accidental failures and malicious attacks. A


promising approach to deal with the large-scale systems is 

multiagent systems (MAS), we agentify the components 

in purely control point of view. In MAS, agents address 

the scalability issue by computing solutions locally and 

then using this information in a social way. In this paper 

we develop a multiagent-based adaptive control 

mechanism with scalability and predictability to support 

survivability of large-scale networks. 

Specifically, in Section 2, we discuss problem domain 

and in Section 3 formally define the problem in detail. 

We review previous control approaches in Section 4. We 

design an adaptive control mechanism in Section 5 and 

show empirical results in Section 6. Finally, we discuss 

implications and possible extensions of our work in 

Section 7. 

2. Problem domain 

The networks we study in this paper represent 

distributed and component-based architectures. As an 

instance, Cougaar (Cognitive Agent Architecture: 

http://www.cougaar.org) developed by DARPA (Defense 

Advanced Research Project Agency), follows such an 

architecture for building large-scale multiagent systems. 

Recently, there have been efforts to combine the 

technologies of agents and components to improve the 

way of building large-scale software systems [8][9][10]. 

While component technology focuses on reusability, 

agent technology focuses on processing complex tasks as 

a community. Cougaar is in line with this trend. In 

Cougaar a software system is comprises of agents and an 

agent of components (called plugins). The task flow 

structure in those systems is that of components as a 

combination of intra-agent and inter-agent task flows. As 

the agents in Cougaar can be distributed both from 

geographical and information content sense, the networks 

implemented in Cougaar have distributed and componentbased 

architecture. 

UltraLog (http://www.ultralog.net) networks are 

military supply chain planning systems implemented in 

Cougaar. Agents in those networks represent 

organizations in military supply chains. The objective of 

an UltraLog network is to provide appropriate logistics 

plan to a military operational plan. The system produces a 

logistics plan by decomposing the operational plan into 

logistics tasks and processing them through a task flow 

structure. The system makes initial planning for a given 

operation and continuous replanning in the execution 

mode to cope with logistics plan deviations or operational 

plan changes. As the scale of operation increases there 

can be thousands of agents working together to generate a 

logistics plan. 

Initial planning or replanning generates a logistics plan 

as a global solution, which is an aggregate of individual 

schedules built by plugins through their task flow 

structure. Each plugin can implement one of its available 

implementation alternatives which trade off processing 

time and quality of the schedule. Quality of service is 

determined by two metrics, quality of logistics plan and 

plan completion time. These two metrics directly affect 

the performance of the operation. 

Planning and replanning of UltraLog networks are the 

instances of the current research problem. An UltraLog 

network cannot work in isolation from outside world 

because they utilize external databases and users should 

be able to access the system. This inevitable connection to 

the outside makes the system exposed to malicious 

attacks in addition to accidental failure. Now, the 

question is how can we make this system survivable to 

generate high quality logistics plans in a timely manner in 

the presence of accidental failures and malicious attacks? 

3. Problem specification 

In this Section we formally define the problem by 

detailing the network model. We concentrate on 

computational CPU resources assuming that the system is 

computation-bounded. 

3.1. Network model 

We define four elements of the network to clarify its 

mechanics: network configuration, implementation 

alternatives, quality of service, and stress environment. 

Network configuration 

A network is composed of a set of agents A with each 

agent located in its own machine. Task flow structure of 

the network, which defines precedence relationship 

between agents, is an acyclic directed graph with each 

link assigned a positive real number. A link number l ij 

(i≠j) indicates the number of tasks generated for successor 

agent j when agent i processes a task in its queue. Once 

accumulated tasks for a successor agent becomes over 

one, the corresponding integer number of tasks are sent to 

the successor agent. By using real numbers we can 

represent wide range of task flow structure including noninteger 

aggregation and expansion. 

A problem given to a network is decomposed in terms 

of root tasks for some agents. And, those tasks are 

propagated through task flow structure. 

Implementation alternatives 

An agent can have multiple implementation 

alternatives to process a task. Different alternatives trade 

off CPU time and solution value with more CPU time


resulting in higher solution value. As we can find optimal 

mixed alternatives, an agent has a monotonically 

increasing convex function, say value function, with CPU 

time as a function of value. We call the value in the 

function as value mode that the agent can select as its 

decision variable. A value function is defined with three 

components as: 

〈 f i ( vi 

), vi(min) , vi(max) 

This function says that an agent i’s expected CPU time 

to process a task is f i (v i ) with a value mode v i and v i(min) ≤ 

v i ≤ v i(max) . 

Quality of service 

A problem given to the network is decomposed to root 

tasks for some agents and those tasks are propagated 

through task flow structure. The service provided by the 

network is to produce a global solution to the given 

problem, which is an aggregate solution of the partial 

solutions from processing tasks. QoS of the network is 

determined by the value of global solution and the cost of 

completion time for generating global solution. The value 

of global solution is the summation of partial solution 

values. And, the cost of completion time is determined by 

a cost function CCT(T), which is a monotonically 

increasing function with completion time T. Consider that 

v i d denotes the value mode used to process d th task and e i 

the number of tasks processed to completion by agent i. 

Then, QoS can be calculated as: 

Stress environment 

QoS 

ei 

= ∑∑ 

i∈ A d = 1 

d 

i 

〉 

v − CCT( 

T) 

Survivability stresses, such as accidental failures and 

malicious attacks, affect the system by consuming 

resources directly or indirectly through activating defense 

mechanisms as remedies against them. For example, 

“denial of service” attack consumes resources directly 

while relevant defense mechanism also consumes 

resource in terms of resistance, recognition, and recovery 

[1]. We consider both of survivability stresses and 

remedies as stress environment from the viewpoint of the 

agents in the network. 

The stress environment space is a high-dimensional 

and also evolving one [11][12]. But, as we concentrate on 

computational CPU resources a stress environment can be 

regarded as a set of threads residing in the machines of 

the network and sharing resources with the agents. The 

threads, say stressors, can have some priorities or weights 

for resource allocation under admission or can be stealing 

resources without admission. 

3.2. Problem definition 

In this paper we develop an adaptive control 

mechanism with scalability and predictability to support 

the survivability of large-scale networks. The system 

needs to adapt to the changing stress environment to 

provide high QoS utilizing implementation alternatives 

(v) as: 

arg max 

v 

QoS 

We discuss several characteristics of the problem that 

will be helpful in understanding the problem and 

developing appropriate control mechanism: 

• Large-scale network: The network can be large-scale 

as the number of agents and nodes increase with the 

scale of the given problem to the network. 

• Finite time horizon: The time horizon for a network 

to generate a global solution is finite. 

• Indecomposable QoS: QoS is not decomposable to 

individual elements’ performance because one of the 

two conflicting QoS elements is the completion time 

that is common throughout the network. 

• Complex dynamics: Agents interact with each other 

through task flow and with stressors through sharing 

resources. As those interactions are in parallel to 

control actions the dynamics of the system is 

intrinsically complex especially in large-scale 

networks. 

• Non-availability of statistics: Statistics such as arrival 

rates or service rates are not fixed or given. But, they 

are changing as the system evolves. In addition, the 

stress environment changes. 

4. Control approaches in dynamic systems 

In general in dynamic systems, centralized and 

decentralized control approaches are used. 

4.1. Centralized approaches 

There are three centralized control approaches, 

dynamic programming (DP), reinforcement learning 

(RL), and model predictive control (MPC). Dynamic 

programming (DP) solves optimality equation to produce 

reactive strategies in terms of optimal closed-loop control 

policy, which is a rule specifying optimal action as a 

function of state and time [13]. It assumes that the 

structure of dynamic model is fixed and the model 

parameters are known in advance. DP gives absolutely 

optimal policy but the complexity in solving optimality 

equation grows exponentially with the dimension of the 

state space. RL is an adaptive version of DP to develop a


policy in real-time when the model parameters are 

unknown [14][15]. This method takes longer time to 

converge than DP at the cost of exploration in addition to 

exploitation. 

In MPC, for each current state, an optimal open-loop 

control policy is designed for finite-time horizon by 

solving a static mathematical programming model based 

on an explicit process model [13][16][17][18][19]. The 

design process is repeated for the next observed state 

feedback forming a closed-loop policy reactive to each 

current system state. Though MPC does not give 

absolutely optimal policy in stochastic environment, it is 

easy to adapt to new contexts by explicitly handling 

objective function or constraints. But, it requires efforts to 

develop process models and has scalability problem. 

4.2. Decentralized approaches 

There are three decentralized control approaches, 

market-based approaches, insect-behavioral approaches, 

and, learning-based approaches. Market-based control 

works through the interaction of local agents in the same 

way as economic markets [20]. Agents trade with one 

another using a relatively simple mechanism, yet 

desirable global objectives can often be realized. These 

approaches are implemented in distributed processor 

allocation problems. Insect-behavioral approaches are 

inspired by effective and adaptive behavior of social 

insect colonies such as ants, bees, wasps, and termites 

[21]. An important and interesting behavior of ant 

colonies is their foraging behavior, in particular how ants 

can find the shortest paths between food sources and their 

nest. Algorithms based on the foraging behavior are 

implemented in routing problems in communication 

networks and shop floor. Similar to ant algorithms, wasp 

algorithms are proposed inspired by wasps’ task 

allocation behavior. Algorithms based on the task 

allocation behavior are implemented in routing problems 

in shop floor. Reinforcement learning can be used without 

prior knowledge of the system model. By making agents 

to learn through their experience the method can be used 

in decentralized mode. These approaches are 

implemented in routing problems in communication 

networks [22]. 

5. Control mechanism 

DP and RL have inefficiencies in terms of scalability 

and agility which are important considerations in our 

problem. In addition, the dynamic model in our problem 

is not fixed and partially known due to unpredictable 

stress environment. Decentralized approaches are scalable 

and robust, but they lack agility and optimality. We 

choose MPC-style approach considering its benefits with 

respect to complexity, optimality, and agility. However, 

we need to overcome scalability problem. 

5.1. Overall control procedure 

As we discussed we develop an adaptive control 

mechanism to provide high QoS to the changing stress 

environment while ensuring scalability and predictability. 

To address adaptivity we model the stress environment 

indirectly by quantifying resource availability of the 

system through sensors. We build a mathematical 

programming model with the resource availability 

incorporated, which predicts QoS as a function of control 

actions. By periodically solving the programming model 

and taking optimal control actions with recent resource 

availability, the system can be adaptive to the changing 

stress environment predictably. But, as the programming 

model can be large-scale, we provide a decentralized 

coordination mechanism to solve the large-scale 

programming model in a decentralized mode. 

5.2. Sensors 

We facilitate two different types of sensors, Load 

sensor and Resource sensor, which are located in each 

agent and measure statistics which form the coefficients 

in the mathematical programming model. 

Load sensor 

A load sensor measures future load L i of agent i, 

which is the number of tasks to be processed in the future. 

Initially, each agent identifies their future loads by 

combining its own root tasks and incoming tasks from its 

predecessor agents in the future. After identifying initial 

future loads, agents update them by counting down as 

they process tasks. 

Resource sensor 

A resource sensor measures resource availability, 

which is defined as the available fraction of resource 

when an agent requests the resource. In a given time 

window we define two measurements to calculate this 

statistic, request time and execution time. Request time is 

the time duration that an agent requests resource, which is 

the duration for which queue length (including one in 

service) is more than zero. Execution time is the time 

duration for which an agent actually utilizes resource. An 

agent i’s resource availability in between two subsequent 

control points (k-1, k) is calculated as: 

RA 

( k−1, 

k) 

i 

execution time in ( k −1, 

k) 

= 

. 

request time in ( k −1, 

k)


5.3. Mathematical programming model 

Agents can estimate their resource availability in the 

future using observed resource availability in the past. An 

agent i estimates its resource availability in the future RA i 

f 

using observed resource availability in the last control 

period. Service time to process a task can be directly 

predicted as a function of value mode by incorporating 

the estimation as: 

i 

i 

f 

i 

f ( v ) / RA . 

Based on this we build a mathematical programming 

model. Consider completion time as T and current time as 

t. An agent’s optimal mode is a pure mode common to all 

the tasks because of the convexity of value function. 

When agents use pure modes such that their total service 

times are less than or equal to T-t, each agent can 

complete its tasks approximately by T because in worst 

case tasks will arrive at a constant rate, L i /(T-t). In other 

words, the completion time is dominantly determined by 

bottleneck agents with maximal total service times for 

their future loads, that is: 

f 

i 

T − t ≈ Max [ L * f ( v ) / RA ] . 

i∈A 

i 

So, given completion time T each agent can select a 

maximal mode so that total service time is less than or 

equal to T. That is, it is optimal for each agent to select a 

mode maximizing: 

subject to 

i 

L i * v i 

f 

i 

L * f ( v ) / RA ≤ T − t . 

i 

i 

i 

Through the optimality condition we can formulate the 

control problem through a mathematical programming 

model that maximizes QoS by trading off the value of 

solution and the cost of completion time as: 

Select v i ’s and T satisfying: 

Max 

s. 

t. 

∑ 

i∈A 

L * f ( v ) / RA 

v 

i 

L * v − CCT ( T ) 

i 

i 

i(min) 

i 

i 

≤ v ≤ v 

i 

f 

i 

i(max) 

≤ T − t 

i 

for all i ∈ A 


security. As we discussed earlier our effort is to support 

survivability. If information is revealed to others directly 

it is not survivable in the viewpoint of information 

security. So, decentralization will also help survivability 

with respect to information security. 

One branch of distributed control approaches is that of 

decentralizing structured mathematical programming 

models. In this branch there are two popular methods, 

decomposition methods and auction/bidding algorithms. 

We decentralize the mathematical programming model 

through a non-iterative auction mechanism, so called 

multiple-unit auction with variable supply [23]. In this 

auction a seller may be able and willing to adjust the 

supply as a function of bidding. In the programming 

model we have built, all the agents are coupled with each 

other. But, it has a typical structure, where the objective 

function and constraints are separable if one variable T is 

fixed. This characteristic makes it possible to convert the 

model into an auction. The completion time T is an 

unbounded resource and the supply can be adjusted as a 

function of bidding. 

In the designed auction for the programming model, 

agents bid for T and the seller decides T* based on the 

bids by maximizing its utility considering the cost. But, 

the seller supplies so that minimum requirements of the 

agents are fulfilled. After the seller broadcasts T*, agents 

select their optimal value modes by maximizing their 

utility. 

 

Agents’ bids 

〈 b 

i( T ), Ti 

(min)〉 

b ( T ) = L * f 

i 

i 

= L * v 

i 

−1 

i 

f 

( T − t)* 

RAi 

( 

) 

L 

i(max) 

f 

Ti (min) 

= Li 

* fi( 

vi 

(min))/ 

RAi 

+ t 

 

Max 

s. 

t. 

 

Seller’s decision problem 

∑ 

i 

i∈A 

b ( T) 

− CCT( 

T) 

T ≥ Max( 

T 

i∈A 

i(min) 

Agents’ decision 

) 

i 

if T ≤ L * f ( v 

else 

i 

i 

i(max) 

)/ RA 

f 

i 

+ t 

5.4. Decentralization 

The next question is how to decentralize the 

mathematical programming model for scalability and 

robustness. In addition to these properties 

decentralization will give a byproduct, information 

v 

f 

* 1 

i 

= − 

i 

= v 

( T 

( 

i(max) 

* 

f 

− t) * RAi 

) 

L 

i 

if T 

else 

* 

≤ L * f ( v 

i 

i 

i(max) 

) / RA 

f 

i 

+ t


The auction mechanism described as a decentralized 

coordination incorporates a centralized seller. As a 

centralized auction can still exhibit problems in terms of 

scalability and robustness we introduce a hierarchical 

auction mechanism. Suppose that T a * is optimal of agent 

group a and T b * is optimal of agent group b. If a ⊂ b, then 

we can say that: 

* * 

a T b 

T ≤ . 

Through this property we can convert the auction 

mechanism into a hierarchical one, in which there are 

multiple auction markets that are structured 

hierarchically. Each auction solves its decision problem 

based on the bids from the agents or subordinate auctions 

and makes a bid to its superior auction with T larger than 

and equal to its optimal completion time. This 

hierarchical structure makes improvements with respect 

to scalability and robustness compared to central auction 

mechanism. Scalability improves because bids and 

decisions are distributed to multiple auctions in the 

hierarchical framework. And, robustness improves 

because there is no single point of failure. 

6. Empirical result 

We ran several experiments to validate the proposed 

control approach through discrete-event simulation. 

6.1. Experimental design 

The network is composed of fifteen agents with a 

convergent structure as in figure 1, in which each link is 

assigned 1 (l ij ). In this network each agent in the lowest 

position has 200 root tasks. Each of the agents has the 

same linear value function and the cost of completion 

time is linear as described in the figure. 

A 2 

A 4 A 5 

A 1 

 

CCT(T) = 4T 

Figure 1. Experimental network configuration 

To observe adaptive behavior we assign weight w i to 

an agent and w′ i to a stressor residing in a same machine 

with the agent for proportional resource share between 

them. A stressor, which has infinite work (continuously 

A 3 

A 8 A 9 A 10 A 11 

A 6 A 7 

A 12 A 13 A 14 

A 15 

200 200 200 200 200 200 200 200 

requiring resources), can impose different levels of stress 

on the agent directly by changing w′ i . When it is zero there 

is no stress, and as it increases the stress level increases. 

We implement our stress environment simply by using 

Weighted Round-Robin scheduling, in which each thread 

gets a number of quanta in proportion to its weight. 

We set up four different experimental conditions as in 

table 1. In stressed conditions we stress agent A 4 in the 

middle of run. And, the distribution of CPU time in value 

function can be deterministic or exponential. While using 

stochastic value function we ran 5 experiments. 

Table 1. Experimental conditions 

Condition Stress Value function 

Con1 W/o Stress Deterministic 

Con2 W/o Stress Exponential 

Con3 W/Stress Deterministic 

Con4 W/Stress Exponential 

* Control period: 100 

* w i : 0.1, w′ 4 : 1 in (500, 1000) 

We use three different control modes for each 

experimental condition. Table 2 shows the control modes 

we used for experimentation. AC represents the adaptive 

control mechanism we have developed. 

Table 2. Control modes for experimentation 

Control mode 

Description 

FL 

Fixed lowest value mode 

FH 

Fixed highest value mode 

AC 

Adaptive control 

6.2. Results 

Experimental results are summarized in table 3. The 

proposed adaptive control showed significant advantages 

compared to non-adaptive cases in all different 

conditions. 

Table 3. Experimental results 

FL FH AC 

T V QoS T V QoS T V QoS 

Con1 1656 13558 6934 6313 30643 5391 1663 22898 16245 

Con2 1652 13547 6942 6302 30643 5435 1723 22982 16089 

Con3 1656 13558 6934 6313 30643 5391 1966 23401 15539 

Con4 1652 13547 6942 6371 30643 5159 2024 23495 15401 

* T: Completion time 

* V: Value of solution 

The adaptive behaviors under proposed control 

mechanism are shown in figures 2 and 3 for deterministic 

case, and figures 4 and 5 for stochastic case. These 

represent time series of decision variables at each control 

point. Under stress the system changes its behavior


adaptively to the new environment. And, once the stress 

is removed the system adapts again. 

4000 

3500 

mode 

6.0 

5.0 

4.0 

3.0 

A 8 A 2 

A 1 

A 4 

optimal T 

3000 

2500 

2000 

1500 

1000 

500 

2.0 

0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

time 

mode 

optimal T 

1.0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

time 

Figure 2. Adaptive value mode under Con3 

4000 

3500 

3000 

2500 

2000 

1500 

1000 

500 

0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

time 

Figure 3. Adaptive optimal T under Con3 

A 8 

6.0 

A 2 

5.0 

4.0 

A 1 

3.0 

A 4 

2.0 

1.0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

time 

Figure 4. Adaptive value mode under Con4 

Figure 5. Adaptive optimal T under Con4 

7. Summary and conclusions 

A typical information network emerges as a result of 

automation or organizational integration. These networks 

are large-scale with distributed and component-based 

architecture. As such networks can be easily exposed to 

various adverse events such as accidental failures and 

malicious attacks, there is a need to study survivability of 

the networks. 

In this paper we studied the emerging networks to 

support survivability by utilizing implementation 

alternatives. By adopting MPC-style approach 

considering its benefits with respect to complexity, 

optimality, and agility, we developed an adaptive control 

mechanism with scalability and predictability. To address 

adaptivity we modeled the stress environment indirectly 

by quantifying resource availability of the system. We 

built a mathematical programming model with the 

resource availability incorporated, which predicts QoS as 

a function of control actions. By periodically solving the 

programming model and taking optimal control actions 

with recent resource availability, the system could be 

adaptive to the changing stress environment predictably. 

But, as the programming model can be large-scale and 

complex, we agentified the components of the network 

from control point of view so that the system can solve 

the large-scale programming model in a decentralized 

mode. We provided a hierarchical auction mechanism as 

a coordination mechanism. We showed the effectiveness 

of our approach regarding to QoS and adaptivity in 

different experimental conditions. 

Our approach can be extended for the network 

configurations where there are multiple agents in a 

machine sharing resources together. In this case we have 

a good opportunity to improve the system performance by 

appropriately allocating resources to the agents. 

To implement the proposed control mechanism in 

information networks such as UltraLog network, we need 

to devise several things which are discussed in


developing the mechanism. Each component should have 

value function and sensors. And, to coordinate the 

components through hierarchical auction market, sellers 

need to be built with appropriate optimization algorithms. 

To provide necessary information to auction market 

components and sellers should be able to make bids. As 

the system makes periodic decisions a seller at the top of 

hierarchy may send market opening messages to market 

participants periodically. 


Support for this research was provided by DARPA 

(Grant#: MDA972-01-1-0038) under the UltraLog 

program. We thank Dr. Mark Greaves (DARPA), 

Marshall Brinn, Beth DePass, and Aaron Helsinger (all 

from BBN) for their suggestions in this work. 

References 

[1] S. Jha, J. M. Wing, “Survivability analysis of networked 

systems”, 23rd international conference on Software 

engineering, pp. 307-317, 2001 

[2] R. Ellison, D. Fisher, H. Lipson, T. Longstaff, and N. Mead, 

“Survivable network systems: An emerging discipline”, 

Technical Report CMU/SEI-97-153, Software Engineering 

Institute, Carnegie Mellon University, 1997 

[3] J. E. Eggleston, S. Jamin, T. P. Kelly, J. K. MacKie-Mason, 

W. E. Walsh, and M. P. Wellman, “Survivability through 

Market-Based Adaptivity: The MARX Project”, DARPA 

Information Survivability Conference and Exposition, 2000 

[4] S. Bowers, L. Delcambre, D. Maier, C. Cowan, P. Wagle, D. 

McNamee, A. L. Meur, and H. Hinton, “Applying Adaptation 

Spaces to Support Quality of Service and Survivability”, 

DARPA Information Survivability Conference and Exposition, 

2000 

[5] O. F. Rana and K. Stout, “What is scalability in multi-agent 

systems?”, Fourth International Conference on Autonomous 

Agents, 2000 

[6] B. Meyer, “On to components”, IEEE Computer, vol. 32, no. 

1, pp. 139-140, 1999 

[7] P. Clements, “From Subroutine to Subsystems: Component- 

Based Software Development”, In Alan W. Brown, editor, 

Component Based Software Engineering, IEEE Computer 

Society Press, pp. 3-6, 1996 

[8] F. M. T. Brazier, C. M. Jonker, and J. Treur, “Principles of 

Component-Based Design of Intelligent Agents”, Data and 

Knowledge Engineering, vol. 41, no. 1, pp. 1-28, 2002 

[9] H. J. Goradia and J. M. Vidal, “Building blocks for agent 

design”, Fourth International Workshop on Agent-Oriented 

Software Engineering, pp. 17-30, 2003 

[10] R. Krutisch, P. Meier, and M. Wirsing, “The 

AgentComponent approach, combining agents and 

components”, Net.objectDays, 2003 

[11] A. P. Moore, R. J. Ellison, and R. C. Linger, “Attack 

Modeling for Information Security and Survivability”, 

Technical Note CMU/SEI-2001-TN-001, Software Engineering 

Institute, Carnegie Mellon University, 2001 

[12] F. Moberg, “Security Analysis of an Information System 

Using an Attack Tree-based Methodology”, Master’s Thesis, 

Automation Engineering Program, Chalmers University of 

Technology, 2000 

[13] G. Barto, S. J. Bradtke, and S. P. Singh, “Learning to act 

using real-time dynamic programming”, Artificial Intelligence, 

vol. 72, pp. 81-138, 1995 

[14] R. S. Sutton, A. G. Barto, and R. J. Williams, 

“Reinforcement learning is direct adaptive optimal control”, 

IEEE Control Systems, vol. 12, no. 2, pp. 19-22, 1992 

[15] L. P. Kaelbling, M. L. Littman, and A. W. Moore, 

“Reinforcement learning: A survey”, Journal of Artificial 

Intelligence Research, vol. 4, pp. 237-285, 1996 

[16] J. B. Rawlings, “Tutorial overview of model predictive 

control”, IEEE Control Systems, vol. 20, no. 3, pp. 38-52, 2000 

[17] M. Morari and J. H. Lee, “Model predictive control: past, 

present and future”, Computers and Chemical Engineering, vol. 

23, no. 4, pp. 667-682, 1999 

[18] M. Nikolaou, “Model predictive Controllers: A Critical 

Synthesis of Theory and Industrial Needs”, Advances in 

Chemical Engineering Series, Academic Press, 2001 

[19] S. J. Qin and T. A. Badgwell, “A survey of industrial model 

predictive technology”, Control Engineering Practice, vol. 11, 

pp. 733-764, 2003 

[20] S. Clearwater, Market-Based Control: A Paradigm for 

Distributed Resource Allocation, World Scientific Publishing, 

1996 

[21] E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm 

Intelligence: From Natural to Artificial Systems, Oxford 

University Press, 1999 

[22] S. Kumar, “Confidence based dual reinforcement Q- 

routing: An on-line adaptive network routing algorithm”, 

Technical Report AI98-267, Department of Computer Sciences, 

The University of Texas at Austin, 1998 

[23] Y. Lengwiler, “The multiple unit auction with variable 

supply”, Economic Theory, vol. 14, pp. 373-392, 1999

Proceedings of the 1st Open Cougaar Conference 9

1 

SITUATION IDENTIFICATION USING DYNAMIC PARAMETERS IN 

COMPLEX AGENT-BASED PLANNING SYSTEMS 

SEOKCHEON LEE, N. GAUTAM, S. KUMARA, Y. HONG, H. GUPTA, 

A. SURANA, V. NARAYANAN, H. THADAKAMALLA, M. BRINN, M. 

GREAVES 

Department of Industrial Engineering 



ABSTRACT 

Survivability of multi-agent systems is a critical problem. Real-life systems are 

constantly subject to environmental stresses. These include scalability, 

robustness and security stresses. It is important that a multi-agent system adapts 

itself to varying stresses and still operates within acceptable performance 

regions. Such an adaptivity comprises of identifying the state of the agents, 

relating them to stress situations, and then invoking control rules (policies). In 

this paper, we study a supply chain planning implemented in COUGAAR 

(Cognitive Agent Architecture) developed by DARPA (Defense Advanced 

Research Project Agency), and develop a methodology to identify behavior 

parameters, and relate those parameters to stress situations. Experimentally we 

verify the proposed method. 

1. INTRODUCTION 

Survivability of multi-agent systems is a critical problem. Real-life systems are 

inherently distributed and are constantly subject to environmental and internal stresses. 

These include scalability, robustness and security stresses. It is important that a multiagent 

system adapts itself to varying stresses and still operates within an acceptable 

performance region. Such an adaptivity comprises of identifying the state of the agents, 

relating them to stress situation, and then invoking control rules (policies). One of the 

fundamental problems is agent state (behavior) identification. 

In this paper, we study a supply chain planning society called Small Supply Chain 

(SSC) implemented in COUGAAR (Cognitive Agent Architecture) developed by 

DARPA (Defense Advanced Research Project Agency), and develop a methodology for 

behavior parameter identification, and relating it to stress situations. The two important 

steps in our methodology are: 1. Identify the most discriminable behavior parameter set 

for situation identification, 2. Apply it to situation identification. To identify the most 

discriminable behavior parameter set we collect the time series data from one of the 

agents in SSC (TAO) and compute 38 statistical and deterministic parameters to represent 

the collected time series. In essence, these 38 parameters are the features of agent state. In 

our earlier work (Ranjan et al., 2002) we prove that SSC shows chaotic behavior from an 

inventory fluctuation point of view and computed chaos indicators (which we call as 

deterministic parameters without loss of generality). Though we compute 38 different 

parameters, next question we address is whether all these are really useful and necessary 

for identifying several stress situations. So, we develop a discriminability index and 

identify the most discriminable behavior parameter set based on this index as a

2 

representative parameter set for identifying several stress situations. Using those 

parameters we develop a nearest neighbor classification based method to identify stress 

situations. 

2. SSC (SMALL SUPPLY CHAIN) SOCIETY 

SSC is a COUGAAR society for supply chain planning composed of 26 agents. 

Each agent generates logistics plan depending on its relative position in the supply chain. 

TAO is an important agent of the SSC and we have selected it to test our schema. Figure 

1 shows the detailed view. In TAO GenerateProjection Tasks are expanded to Supply 

Tasks, which are for internal consumption. Each Supply Task is expanded to Withdrawal 

Task, which is allocated to inventory asset. Supply Tasks are also transferred from other 

agents. They are expanded to Withdrawal Tasks, which are allocated to inventory asset. 

MaintainInventory Tasks, which are for the maintenance of inventory assets in TAO, are 

expanded to Supply Tasks. Each Supply Task is allocated to other agents. 

MaintainInventory 

ProjectSupply 

Supply 

TAO 

GenerateProjection 

ProjectSupply 

Supply 

ProjectWithdrawl 

Withdrawl 

Inventory Asset 

Figure 1. TAO in SSC 

3. STRESSES AND BEHAVIOR 

For the sake of analysis we have parameterized the stress situations and system 

behavior. 

3.1 Stress 

Stress refers to survivability stress and includes scalability, security, and robustness 

stresses. Scalability is defined as the ability of a solution to a problem to work when the 

size of the problem increases. And, survivability (regarding security and robustness) is 

defined as as the capability of a system to fulfill its mission, in a timely manner, in the 

presence of attacks, failures, or accidents (Ellison et al., 1997). There can be diverse 

stress situations, but in this paper we consider stress situations formed by two scalability 

stress types given below: 

• Problem Complexity: Problem complexity is determined by the complexity of 

the planning task. This includes many aspects and we have chosen one of the stress types, 

called OpTempo of each agent. OpTempo defines operation tempo. 

• Query Frequency: Each agent provides query service for its planning 

information to human operators. We have chosen query frequency (# of query request per 

second) to each agent as one of stress types. 

Although SSC society is composed of 26 agents there are only 8 agents that are 

directly affected by OpTempo. We define stress levels: Low/Medium/High. So, the size 

of our stress situation space becomes 3 34 .

3 

3.2 Behavior 

In SSC society an agent’s behavior can be described by its Task groups’ behaviors. 

Behaviors can be represented by time series. We define four different time series (Task 

arrival, Time to solution sorted by generation sequence, Time to solution sorted by 

completion sequence, and Queue length). A time series may be characterized using 

deterministic and statistical parameters as shown in Table 1. 

Deterministic characterization makes it possible to handle non-stationary, nonperiodic, 

irregular time series, including chaotic deterministic time series. In this study 

we use five different deterministic behavior parameters. In a deterministic dynamical 

system since the dynamics of a system are unknown, we cannot reconstruct the original 

attractor that gave rise to the observed time series. Instead, we seek the embedding space 

where we can reconstruct an attractor from the scalar data that preserves the invariant 

characteristics of the original unknown attractor using delay coordinates proposed by 

Packard et al. (1980) and justified by Taken (1981). Average mutual information has 

been suggested to choose time delay coordinates by Fraser and Swinney (1986). And, 

Schuster (1989) proposed nearest neighbor algorithm to base the choice of the embedding 

dimension. Local dimension has been used to define the number of dynamical variables 

that are active in the embedding dimension (1998). The most popular measure of an 

attractor’s dimension is the correlation dimension, first defined by Grassberger and 

Procaccia (1983). And, a method to measure the largest Lyapunov exponent, sensitivity 

to initial condition as a measure of chaotic dynamics, is proposed by Wolf et al. (1985). 

We have systematically studied the use of the methods from the literature and computed 

38 different behavioral parameters to characterize the four time series we have 

considered. These 38 parameters are shown in Table 1. 

Statistical 

Parameters 

Deterministic 

Parameters 

Table 1. Behavioral parameters 

Time Series 

Task Arrival 

Time to Solution Time to Solution 

(Generation) (Completion) 

# of events 

# of events 

Average 

Average 

Minimum 

Minimum 

Maximum 

Maximum 

Radius 

Radius 

Variance 

Variance 

ami 

e_dim 

l_dim 

c_dim 

l_exp 

ami 

e_dim 

l_dim 

c_dim 

l_exp 

ami 

e_dim 

l_dim 

c_dim 

l_exp 

ami: average mutual information, e_dim: embedding dimension, l_dim: local dimension, 

c_dim: correlation dimension, l_exp: lyapunov exponent 

Queue Length 

# of events 

Average 

Minimum 

Maximum 

Radius 

Variance 

ami 

e_dim 

l_dim 

c_dim 

l_exp 

4. EXPERIMENTATION AND RESULTS 

We ran several simulations of SSC to identify the most discriminable behavior 

parameter set.

4 

4.1 Experimental configuration 

SSC 

TAO 

Behavior 

Stressor 

Stress 

Situation 

Database 

Parameter 

Generation 

Parameter 

Table 

Online Experimentation 

Figure 2. Experimental configuration 

Offline Analysis 

In this experimentation we store event data from TAO and the parameters of stress 

situation from stressor into an online database, and then from the database we construct 

the parameter table with stress parameters and behavior parameters as in the Fig. 2. The 

experimental matrix is shown in Table 2. 

Table 2. Experimental matrix 

TestID OpTempo Query Repetition 

PRE001 Low to all agents Low to all agents 10 

PRE002 High to all agents Low to all agents 10 

PRE003 Medium to all agents Low to all agents 10 

PRE004 Medium to all agents High to all agents 10 

4.2 Results 

Reduction of stress space 

Figure 3. shows an example of ‘# of events’ parameter in each experiment repeated 

10 times in four different stress conditions. We identified the stresses that have no 

significant effects on the society’s behavior by comparing the behavior parameters under 

different conditions. The result shows: 

• No significant difference between Low and Medium of OpTempo stress 

• No significant effect of query frequency stress 

# of Events from Task Arrival 

1120 

1110 

1100 

# of events 

1090 

1080 

1070 

1060 

1050 

PRE001 PRE002 PRE003 PRE004 

1040 

1 

3 

5 

7 

9 

11 

13 

15 

17 

19 

21 

23 

25 

27 

29 

31 

33 

35 

37 

39 

Experiment 

Figure 3. Comparison of a behavior parameter in different stress conditions 

This leads to the reduction in the stress space to 2 8 (OpTempo Low/High for 8 

agents) from 3 34 . 

Discriminability of behavior parameters

5 

All the behavior parameters may not be equally good in helping the classification of 

stress situations. Therefore, there is a need for a measure of discriminating power of each 

of the behavior parameters. We call this as discriminability index (DI). DI can be 

represented as the ratio between sensitivity to the stress situations and random variation 

defined as: 

Discriminability Index (DI) = [∑(µ-µi) 2 /n] / [∑(si 2 )/n] = ∑ (µ-µi) 2 / ∑ (si 2 ) (1) 

µ : Average of parameter values 

µi : Average of parameter values from ith condition 

si : Standard deviation of parameter values from ith condition 

n : Number of conditions 

We ranked those 38 behavior parameters using the DI. Top 5 are as shown in Table 

3. As shown in the table ‘# of events’ from task arrival time series was the most 

discriminable behavior parameter. Because this parameter is sensitive to the different 

stress situations and has small variation in the same stress situations the DI is relatively 

larger than those of other parameters. 

Table 3. Discriminability index (DI) of behavior parameters 

Rank DI Time Series Behavior Parameter 

1 2477 Task arrival # of events 

2 6 Time to solution Variance 

3 5 Time to solution Radius 

4 4 Time to solution Average 

5 4 Time to solution Maximum 

5. SITUATION IDENTIFICATION 

Results from preliminary experimentation showed that ‘# of events’ from task 

arrival time series (# of tasks) is the most discriminable behavior parameter in our stress 

space. So, assuming that the input to an agent affects the output depending on that agent’s 

stress situation we can identify OpTempo of an agent by using four features of ‘# of 

tasks’ as shown in Fig. 4. 

ProjectSupply 

Supply 

Agent 

(OpTempo) 

# of ProjectSupply from Outside / # of Supply from Outside 

# of ProjectSupply to Outside / # of Supply to Outside 

Figure 4. Features for situation identification 

ProjectSupply 

Supply 

OpTempo 

We performed an initial design of experiments and constructed a database of the 

behavior parameters from 100 experiments. Each agent’s OpTempo is randomly chosen 

and the parameters are computed and stored in the database. Given a new experimental 

data we select the nearest neighbor from the base database by using the Euclidean 

distance between feature vectors. The stress level of the nearest neighbor is used for 

stress estimation. We estimated the stress level for 100 new experimental data using this

6 

approach. The results of estimation are shown in Table 4. Half of agents identified the 

stress successfully although the other half didn’t. 

Table 4. Stress estimation result 

Stress Correct estimation Stress Correct estimation 

OpTempo of agent 1 54% OpTempo of agent 5 100% 




6. CONCLUSIONS 

In this paper, we developed a methodology for extracting features from time series’ 

of an agent-based supply chain planning society (behavior parameters) and relating it to 

stress situations. We identified ‘# of tasks’ as the most discriminable behavior parameter 

of our 38 statistical and deterministic parameters in our stress space. Using this parameter 

we validated the method’s ability to identify stress situation using nearest neighbor 

classification. Although our analysis showed deterministic parameters don’t have the 

ability to identify stress situations in our stress space it is possible that they can be good 

indicators under other stress space such as security and robustness stresses. 

ACKNOWLEDGEMENTS 

Support for this research was provided by DARPA (Grant#: MDA 972-01-1-0563) under 

the UltraLog program. 

REFERENCES 

Abarbanel, H. D. I., Gilpin, M. E., Rotenberg, M., 1998, Analysis of Observed Chaotic Data, 

Springer. 

Ellison, R. J., Fisher, D. A., Linger, R. C., Lipson, H. F., Longstaff, T., Mead, N. R., 1997, 

“Survivable Network Systems, An Emerging Discipline”, Technical Report CMU/SEI-97-153, 

Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA. 

Fraser, A. M., and Swinney, H., 1986, “Independent coordinates for strange attractors from mutual 

information”, Physical Review A, Vol. 33, pp. 1134 – 1140. 

Grassberger, P., and Procaccia, I., 1983, “Characterization of Strange Attractors”, Physical Review 

Letters, Vol. 50, pp. 346. 

Grassberger, P., and Procaccia, I., 1983, “Characterization of Strange Attractors”, Physica D, Vol. 9, 

pp. 189 – 208. 

Packard, N. H, Crutchfield, J. P., Farmer, J. D., and Shaw, R. S., 1980, “Geometry from a Time 

Series”, Physical Review Letters, Vol. 45, pp. 712. 

Ranjan, P., Kumara, S., Surana, A., Manikonda, V., Greaves, M., Peng, W., 2002, “Decision Making 

in Logistics: A Chaos Theory Based Analysis”, AAAI Spring Symposium, Technical Report SS-02- 

03, pp. 130-136. 

Schuster, H. G., 1989, Deterministic Chaos: An Introduction, Verlagsgesellshaft, Weinheim. 

Taken, F., 1981, “Detecting strange attractors in turbulence”, Dynamical Systems and Turbulence, 

pp. 366 - 381, Springer, Berlin. 

Wolf, A., Swift, J. B., Swinney, H. L., and Vastano, J., 1985, “Determining Lyapunov Exponents 

from a Time Series”, Physica D, Vol. 16, pp. 285 – 317.

Estimating Global Stress Environment by Observing Local 

Behavior in Distributed Multiagent Systems 

Seokcheon Lee and Soundar Kumara 

Department of Industrial and Manufacturing Engineering, 


University Park, PA 16802 USA 

Abstract—A multiagent system can be considered survivable 

if it adapts itself to varying stresses without considerable 

performance degradation. Such an adaptivity comprises of 

identifying the behavior of the agents in a society, relating them 

to stress situations, and then invoking control rules. This 

problem is a hard one, especially in distributed multiagent 

systems wherein the agent behaviors tend to be nonlinear and 

dynamic. In this paper, we study a supply chain planning 

system implemented in COUGAAR (Cognitive Agent 

Architecture) and develop a methodology for identifying the 

behavior of agents through their behavioral parameters, and 

relating those parameters to stress situations. One important 

aspect of our approach is that we identify the stress situations 

of agents in the society by observing local behavior of one 

representative agent. This approach is motivated by the fact 

that a local time series can have the information of the 

dynamics of the entire system in deterministic dynamical 

systems. We validate our approach empirically through 

identifying the stress situations using k-nearest neighbor 

algorithm based on the behavioral parameters. 

S 

I. INTRODUCTION 

urvivability is defined as “the capability of a system to 

fulfill its mission, in a timely manner, in the presence of 

attacks, failures, or accidents” [1]. This definition considers 

security and robustness stresses as components of the stress 

environment. With the increasing size of networked systems 

scalability becomes a critical issue for a system to fulfill its 

mission [2]. We argue that scalability is also an important 

component of survivability and hence the stress 

environment. In this paper we consider only scalability 

stress in dealing with survivability. 

Survivability of multiagent systems is a critical problem. 

As infrastructures become large-scale and increasingly 

dependent on networked systems for automation or 

organizational integration, this capability becomes more and 

more important. Real-life systems are inherently distributed 

This work was supported in part by DARPA under Grant MDA 972-01- 

1-0038. 

S. Lee is with the Department of Industrial and Manufacturing 

Engineering, The Pennsylvania State University, University Park, PA 16802 

USA (phone: 814-863-4799; fax: 814-863-4745; e-mail: 

stonesky@psu.edu). 

S. Kumara is with the Department of Industrial and Manufacturing 


USA (e-mail: skumara@psu.edu). 

and are constantly subject to environmental and internal 

stresses. Hence, it is important that a multiagent system 

adapts itself to varying stresses and maintains its 

performance within the acceptable bounds of performance. 

The three important constituents of adaptivity are: agent 

behavior identification, mapping the agent behavior to the 

environment (stresses) and invoking the appropriate control 

rules (policies). 

In this paper, we study a supply chain planning system, 

Small Supply Chain (SSC) society implemented in 

COUGAAR (Cognitive Agent Architecture: 

http://www.cougaar.org) as an example system. We develop 

a methodology to identify the stress situations of the agents 

in the society by observing local behavior of one 

representative agent (called TAO). This information can 

subsequently be used to devise and invoke control policies. 

The two important steps in our methodology are: 1. Extract 

meaningful behavioral parameters for situation 

identification, 2. Apply these parameters to situation 

identification. We collect time series data from TAO and 

compute 38 statistical and deterministic parameters to 

represent its behavior. In essence, these 38 parameters are 

the features of the behavior of the society as we can assume, 

from the theory of deterministic dynamic systems [3], [4], 

that the behavior of TAO has the information of the 

dynamics of the entire system. All the 38 different 

parameters are not equally important and independent. We 

therefore develop a discriminability index of the parameters 

based on which, extract meaningful behavioral parameters. 

Using those selected parameters we develop a k-nearest 

neighbor classification based method to identify stress 

situations of agents in the society. 

The organization of the paper is as follows. In section II 

we discuss the SSC society. In section III, we parameterize 

stress situations and behavior. In section IV we analyze the 

results from preliminary experimentation to build the 

methodology. In section V we implement our approach to 

identify the stress situations. Finally, in section VI, we 

conclude our work. 

II. SSC (SMALL SUPPLY CHAIN) SOCIETY 

SSC is a COUGAAR society for military supply chain 

planning composed of 26 agents with 17 agents working for

actual planning. COUGAAR has distributed and 

component-based architecture in which agents are 

geographically distributed and process their specific types of 

tasks. The objective of the SSC society is to generate a 

logistics plan for a given military operation. Each agent, 

representing an organization of military supply chain, 

processes tasks received from other agents or generated 

internally. Those tasks are allocated to assets after 

expanding or aggregating. The allocations in an agent 

trigger generating tasks to its supplier agents to refill the 

assets. When tasks from customers are allocated in a 

supplier agent, the results are fed back to the customer 

agents. Fig. 1 shows the task flow structure of the SSC 

society. TAO (Agent 3), which provides direct logistics 

support to combat units (Agents 1 and 2), is an important 

agent of the SSC society with respect to its relationship with 

other agents and amount of tasks. We have selected it as a 

representative agent to test our schema. 

1 

2 

14 

III. STRESS AND BEHAVIOR 

For analysis purposes we parameterize the stress and 

behavioral space. 

A. Stress 

TAO 

3 

15 

4 

5 6 7 

There are diverse survivability stresses with respect to 

scalability, security, and robustness [5]–[7]. For 

implementation purpose, we consider three types of 

scalability stresses as follows: 

- Network topology: We consider one aspect of scalability 

stress as adding or removing an agent(s) to TAO in the 

existing topology. Theoretically we can randomly add or 

remove any agent. However, we consider agent 1 and TAO 

together. The three stress levels we impose on TAO are 

through removing agent 1, having one agent 1 connected 

and adding one more agent of agent 1 type. 

- Problem Complexity: Problem complexity is determined 

by the complexity of the planning tasks. This includes many 

aspects and we have chosen OpTempo to implement this 

stress type, which represents the tempo of military 

operations. We define three stress levels of OpTempo for 

each of the 16 agents other than agent 1 as Low, Medium 

8 

9 

16 17 

Fig. 1. SSC society 

10 

11 

12 

13 

and High. 

- User Query: Each agent provides query service for its 

planning information to human operators. We have chosen 

query frequency to implement this stress type, which is the 

number of query requests per second. We define three stress 

levels of query frequency for each of the 16 agents other 

than agent 1as Low, Medium and High. 

The size of the stress space is very large. Combining these 

three types of stresses the size of stress space becomes 3 33 

(3*3 16 *3 16 ). 

B. Behavior 

In SSC society an agent’s behavior can be abstracted by 

observing the agent’s task processing. We define four 

different time series related to the agent’s task processing as 

follows: 

- Task arrival: Task inter-arrival times from other agents 

as well as TAO itself 

- Time to solution sorted by generation sequence: Time 

durations taken to complete a task from its generation, 

sorted by generation sequence 

- Time to solution sorted by completion sequence: Time 

durations taken to complete a task from its generation, 

sorted by completion sequence 

- Queue length: Number of tasks that are waiting for 

processing 

A time series can be characterized using deterministic and 

statistical parameters. We have systematically studied the 

use of the methods from the literature and computed 38 

different behavioral parameters to characterize the four time 

series we have considered. These 38 parameters, composed 

of 18 statistical and 20 deterministic parameters (from 

dynamical systems theory), are shown in Table I. These 

represent the features of agent’s behavior. 

Task 

Arrival 

# of events 

Average 

Minimum 

Maximum 

Radius 

Variance 

AMI 

E_Dim 

L_Dim 

C_Dim 

L_Exp 

TABLE I 

BEHAVIORAL PARAMETERS 

Time to Solution 

(Generation) 

AMI 

E_Dim 

L_Dim 

C_Dim 

L_Exp 

Time Series 

# of events 

Average 

Minimum 

Maximum 

Radius 

Variance 

Time to Solution 

(Completion) 

AMI 

E_Dim 

L_Dim 

C_Dim 

L_Exp 

Queue 

Length 

# of events 

Average 

Minimum 

Maximum 

Radius 

Variance 

AMI 

E_Dim 

L_Dim 

C_Dim 

L_Exp 

AMI: Average Mutual Information, E_Dim: Embedding Dimension, 

L_Dim: Local Dimension, C_Dim: Correlation Dimension, L_Exp: 

Lyapunov Exponent 

Deterministic characterization makes it possible to handle 

non-stationary, non-periodic, irregular time series, including

chaotic deterministic time series. In this paper we use five 

different deterministic behavioral parameters. In a 

deterministic dynamical system since the dynamics of a 

system are unknown, we cannot reconstruct the original 

attractor that gives rise to the observed time series. Instead, 

we seek the embedding space where we can reconstruct an 

attractor from the scalar data that preserves the invariant 

characteristics of the original unknown attractor using delay 

coordinates [3], [4]. This motivates us to characterize the 

system dynamics of the society by observing local behavior. 

Average mutual information has been suggested to select 

time delay coordinates [8]. Nearest neighbor algorithm to 

base the choice of the embedding dimension is proposed in 

[9]. Local dimension has been used to define the number of 

dynamical variables that are active in the embedding 

dimension [10]. The most popular measure of an attractor’s 

dimension is the correlation dimension [11], [12]. In [13] a 

method to measure the largest Lyapunov exponent, 

sensitivity to initial condition as a measure of chaotic 

dynamics, is proposed. As these parameters are well 

documented in the references we have given, we do not 

undertake a detailed explanation. 

TABLE II 

EXPERIMENTAL MATRIX 

TestID OpTempo Query Replication 

PRE001 Low to all agents Low to all agents 10 

PRE002 High to all agents Low to all agents 10 

PRE003 Medium to all agents Low to all agents 10 

PRE004 Medium to all agents High to all agents 10 

For all experiments the number of agent 1 is one 

B. Results 

1) Reduction of stress space: We identified the stress 

situations that have no significant effects on the system 

dynamics by analyzing behavioral parameters. Fig. 3 shows 

an example of ‘# of events’ parameter from ‘Task Arrival’ 

time series in four different stress conditions. By analyzing 

all 38 parameters systematically we concluded that: 

- There is no significant difference between Low and 

Medium levels of OpTempo stress. 

- There is no significant effect of query frequency stress. 

This analysis leads to the reduction of the stress space to 

3*2 16 (the number of agent 1: 0/1/2, OpTempo for each of 

16 agents other than agent 1: Low/High) from 3 33 . 

IV. PRELIMINARY EXPERIMENTATION 

We ran several experiments to reduce the stress space by 

removing ineffective stress situations (stresses which do not 

change the existing behavior of a given agent). In addition 

we use the experiments to extract meaningful behavioral 

parameters from the 38 behavioral parameters we computed. 

In the following we undertake a detailed explanation. 

A. Experimental configuration 

# of events 

1120 

1110 

1100 

1090 

1080 

1070 

1060 

1050 

1040 

PRE001 PRE002 PRE003 PRE004 

SSC 

TAO 

1030 

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 

Experiment 

Stressor 

Stress 

Situation 

Behavior 

Online Experimentation 

Database 

Parameter 

Generation 

Fig. 2. Experimental configuration 

Parameter 

Table 

Offline Analysis 

In this experimentation we store event data from TAO 

and the stress parameters from stressor (i.e., injector of 

stresses) into an online database, and then from the database 

we construct the parameter table with stress parameters and 

behavioral parameters as shown in Fig. 2. The experimental 

matrix in this preliminary experimentation is shown in Table 

II. There are four different experimental conditions with 

different OpTempo and query frequency levels. We replicate 

each condition ten times. 

Fig. 3. Variance of a behavioral parameter 

2) Discriminability of behavioral parameters: All the 38 

behavioral parameters may not be equally good in helping 

the identification of stress situations. It is important to select 

good parameters, especially when we use deterministic 

parameters because they are computationally expensive. 

Therefore, there is a need for a measure of discriminating 

power of the parameters. We developed an index to measure 

the discriminating power. We call this as DI 

(Discriminability Index). DI is represented as the ratio 

between sensitivity to the stress situations and random 

variation defined as in (1). 

DI = 

∑ ( 

∑ 

2 

µ − µ i ) / n 

= 

2 

s / n 

i 

∑ ( 

∑ 

µ − µ ) 

s 

2 

i 

µ : Average of parameter values 

µ i : Average of parameter values in i th condition 

s i : Standard deviation of parameter values in i th condition 

n : Number of conditions 

i 

2 

(1)

A DI value greater than one implies that the particular 

parameter can help in discriminating between the situations 

(more discrimination power). We calculated DI values for 

38 behavioral parameters and selected those parameters with 

DI values larger than one. This resulted in 10 parameters, 

comprising of eight statistical and two deterministic 

parameters, as shown in Table III. ‘# of events’ from ‘Task 

Arrival’ and ‘Time to Solution’ was the most discriminating 

behavioral parameter. Note that ‘# of events’ is the same for 

both time series’ as arrived tasks are processed. 

TABLE III 

DISCRIMINABILITY INDEX OF BEHAVIORAL PARAMETERS 

Rank DI Time Series Parameter 

1 2477.5 Task Arrival/Time to Solution # of events 

2 5.7 Time to Solution(G) Variance 

3 5.1 Time to Solution(G) Radius 

4 4.4 Time to Solution(G) Average 

5 4.2 Time to Solution(G) Maximum 

6 2.9 Queue length # of events 

7 2.8 Queue length Maximum 

8 2.2 Queue length AMI 

9 1.2 Queue length Average 

10 1.1 Time to Solution(C) L_Expo 

(G): Generation, (C): Completion 

V. SITUATION IDENTIFICATION 

Results from preliminary experimentation shows that 10 

of the 38 behavioral parameters have better discriminating 

power in the stress space. By using them as features we 

identify the stress situations using k-nearest neighbor 

classification algorithm. 

A. k-nearest neighbor algorithm 

k-nearest neighbor algorithm, one of the instance-based 

learning methods, is conceptually straightforward. The 

summary reported here is based on [14]. In this algorithm 

learning is simply storing the training instances, in which 

each instance corresponds to a point in the n-dimensional 

feature space. Given a new query instance k nearest 

neighbors are retrieved from memory and used to classify 

the new query instance. The nearest neighbors of an instance 

are defined in terms of the standard Euclidean distance. One 

problem in this algorithm is the sensitivity to noise axes in 

high dimensional problems. One possible solution would be 

to normalize each feature. However, normalization does not 

resolve this problem because Euclidean distance can become 

very noisy for high dimensional problems where only a few 

of the features carry classification information. The solution 

to this problem is to modify the Euclidean metric by a set of 

weights that represents the information content or goodness 

of each feature. Therefore, given a set of weights w the 

distance between two normalized instances x i and x j with m 

features can be calculated as in (2). 

m 

∑ 

d( x , x ) = w[ 

k]( 

x [ k] 

− x [ k]) 

(2) 

i 

j 

k = 1 

B. Empirical results 

i 

We performed 200 experiments with the same 

experimental configuration as in Fig. 2 to construct the 

database of training instances. Each training instance is 

represented with the 10 behavioral parameters. In this 

experimentation each agent’s OpTempo (Low/High) and the 

number of agent 1 (0/1/2) are randomly chosen. Given a 

new instance we select 20 nearest neighbors (10% of the 

population of training instances) from the database and 

estimate the stress situations of the agents in the society by 

using those neighbors. To address the effectiveness of DI we 

use 12 different sets of weights to calculate the distance. In 

the first 10 sets only one parameter is considered with other 

parameters’ weights equal to zero. We weight the 

parameters equally in the 11th set and proportionally to DI 

in the 12th set. 

We estimated the stress situations for 100 new instances 

using those 12 different sets of weights. To eliminate the 

noise from those agents that has no significant effect on the 

behavioral parameters, we removed the agents from our 

analysis that we cannot identify more than 2/3 by using any 

weight set. Through this procedure only 8 agents are 

selected. The results of correct estimation from different 

weight sets are shown in Fig. 4. 

Correctness (%) 

75 

70 

65 

60 

55 

50 

1 2 3 4 5 6 7 8 9 10 

On the whole, performance of behavioral parameters to 

identify the stress situations is quite correlated with DI of 

the parameters. As a parameter’s DI ranked high its 

estimation accuracy is also high. And, when weighted 

equally the performance is in the middle. But, when we 

weight proportionally to DI the performance becomes the 

highest. This result demonstrates the effectiveness of DI to 

get the goodness of the behavioral parameters for situation 

identification. Fig. 5 shows the performance for each agent 

when we weight proportionally to DI. As agents located 

farther from the TAO the performance becomes degraded. 

j 

DI Rank 

2 

Weighted w ith DI 

Weighted equally 

Fig. 4. Correct estimation using different weight sets

100% 

1 

95% 

2 

14 15 

64% 

TAO 4 

64% 

62% 

5 6 7 

59% 

VI. CONCLUSIONS 

In this paper, we developed a methodology for extracting 

features by characterizing the time series’ and relating it to 

stress situations in distributed multiagent systems. One 

important aspect of our approach is that we identify the 

stress situations of the agents in the society by observing 

local behavior of one representative agent. This approach is 

motivated by the fact that a local time series can have the 

information of the dynamics of entire system in 

deterministic dynamic systems. It is important to identify the 

situations of other agents when agents are interdependent in 

networked systems. 

When we have a large society we will be able to predict 

the stress levels in some other agents in the society. This 

helps in invoking an appropriate control policy. For example 

by studying the local behavior of TAO during certain time, 

we may be able to estimate that agent 11’s OpTempo is high 

with 62% accuracy. This may need us to reduce the amount 

of tasks to the agent as high OpTempo requires more 

computational resource. 

To extract meaningful behavioral parameters we collected 

the time series data from a representative agent and 

computed 38 statistical and deterministic parameters to 

represent its behavior. Discriminability Index defined by us 

in this paper as a measure of the discriminating power of the 

parameters seems to be a promising direction for agent 

behavior estimation. Using those selected parameters we 

validated our approach through identifying the stress 

situations using k-nearest neighbor algorithm with the index 

values as weights. Although our analysis showed that 

deterministic parameters don’t have significant ability to 

identify stress situations in our stress space, it is possible 

that they can be good indicators under other stress space 

such as security and robustness stresses. 

8 

9 

16 17 

52% 

10 

62% 

Fig. 5. Correct estimation with proportional weights to DI 

11 

12 

13 

[2] O. F. Rana and K. Stout, “What is scalability in multi-agent systems?” 

in Proc. 4th Int. Conf. Autonomous Agents, 2000, pp. 56–63. 

[3] N. H. Packard, J. P. Crutchfield, J. D. Farmer, and R. S. Shaw, 

“Geometry from a time series,” Physical Review Letters, vol. 45, pp. 

712–716, 1980. 

[4] F. Taken, “Detecting strange attractors in turbulence,” in Dynamical 

Systems and Turbulence, D. Rand and L.-S. Young, Eds. Springer: 

Berlin, 1981, pp. 366–381. 

[5] A. P. Moore, R. J. Ellison, and R. C. Linger, “Attack modeling for 

information security and survivability,” Software Engineering 

Institute, Carnegie Mellon University, Pittsburg, PA, Tech. Note 

CMU/SEI-2001-TN-001, 2001. 

[6] F. Moberg, “Security analysis of an information system using an 

attack tree-based methodology,” M.S. thesis, Automation Engineering 

Program, Chalmers University of Technology, Sweden, 2000. 

[7] S. Jha and J. M. Wing, “Survivability analysis of networked systems,” 

in Proc. 23rd Int. Conf. Software engineering, 2001, pp. 307–317. 

[8] A. M. Fraser and H. Swinney, “Independent coordinates for strange 

attractors from mutual information,” Physical Review A, vol. 33, pp. 

1134–1140, 1986. 

[9] H. G. Schuster, Deterministic Chaos: An Introduction, 

Verlagsgesellshaft: Weinheim, 1989. 

[10] H. D. I. Abarbanel, M. E. Gilpin, and M. Rotenberg, Analysis of 

Observed Chaotic Data, Springer: New York, 1998. 

[11] P. Grassberger and I. Procaccia, “Characterization of strange 

attractors,” Physical Review Letters, vol. 50, pp. 346–349, 1983. 

[12] P. Grassberger and I. Procaccia, “Characterization of strange 

attractors,” Physica D, vol. 9, pp. 189–208, 1983. 

[13] A. Wolf, J. B. Swift, H. L. Swinney, and J. Vastano, “Determining 

Lyapunov exponents from a time series,” Physica D, vol. 16, pp. 285– 

317, 1985. 

[14] T. M. Mitchell, Machine Learning, MaGraw-Hill, pp. 230–236, 1997. 

REFERENCES 

[1] R. Ellison, D. Fisher, H. Lipson, T. Longstaff, and N. Mead, 

“Survivable network systems: An emerging discipline,” Software 

Engineering Institute, Carnegie Mellon University, Pittsburg, PA, 

Tech. Rep. CMU/SEI-97-153, 1997.

Using Predictors to Improve the Robustness of Multi-Agent Systems: Design 

and Implementation in Cougaar 

† Himanshu Gupta, ‡ Yunho Hong, ‡ Hari Prasad Thadakamalla, † Vikram Manikonda, ‡ Soundar Kumara and † Wilbur 

Peng 

† Intelligent Automation Incorporated 

7519 Standish Place, Suite 200, Rockville, MD – 20855 

{hgupta, vikram, wpeng}@i-a-i.com 

‡ 

Industrial and Manufacturing Engineering 

310 Leonhard Building, The Pennsylvania State University, University Park, PA 16802 

{yyh101, hpt102, skumara}@psu.edu 

Abstract 

In this paper we discuss the use of predictors as a means 

to improve the robustness of a multi-agent system in the 

event of information attacks that might result in a 

communication loss between agents. We focus on an 

adaptive logistics application developed under 

DARPA’s Ultralog program using the Cougaar Agent 

infrastructure. The objective of the predictors is to 

estimate key “state variables” such as demand, 

inventory etc. in the absence of communication, and 

allow logistics planning and execution to continue when 

communication resources are limited or lost. Prediction 

schemes based on a model-based linear state estimation 

and moving averages are discussed. A generalized 

software implementation of the predictors as plugins 

within a Cougaar agent, and approaches to reconcile 

any errors between the “estimated” and “actual” states 

when communication is restored is also discussed. 

Experimental results based on the implementation of the 

predictors in a logistics society subject to simulated 

communication losses and variable changes to the 

operational plan are presented. 


Agent-based technology provides a natural solution 

for inherently complex, distributed and decentralized 

systems, where a desired solution emerges as set of 

autonomous, interacting entities, execute/optimize their 

individual/group behavior in a dynamically changing 

environment. Adaptive logistics is one such example. In 

this setting, agents represent logistics entities such as the 

Units of Action (UA), Forward Support Battalions 

(FSB), Brigades, and Companies etc. These agents, 

distributed across various physical and logical 

boundaries, collaborate to perform logistics sustainment 

operations such as forecasting logistics consumption 

trends, identifying potential shortfalls, planning, 

executing, monitoring and re-planning logistics 

operations in a dynamically changing environment. 

When deployed in a battlefield environment, the agent 

infrastructure is subject to several stresses such as 

wartime loads (e.g. CPU stressors due to variable loads), 

information attacks (e.g. denial of service, 

communication loss, reduced bandwidth) and kinetic 

attacks (e.g. loss of hardware resources). For successful 

deployment, the overall agent infrastructure needs to be 

robust and resilient to these stresses/attacks. 

In this paper we discuss the use of predictors as a 

means to achieve robust behavior in the event of 

information attacks that might result in a communication 

loss between agents. We use an adaptive logistics 

application developed under DARPAs Ultralog program 

using the Cougaar agent infrastructure [9] as a testbed to 

motivate, implement and test the predictor designs. Two 

prediction schemes are discussed. The first is based on a 

linear state estimation approach that models agent

interactions as a dynamical system and the second is 

based on moving averages. 

The paper is organized as follows. In Section 2, we 

discuss the modeling approach adopted to build the 

predictors. In Section 3, software implementation of the 

predictors as plugins within a Cougaar agent is 

discussed. In Section 4, we discuss experimental results 

based on the implementation of the predictors on a 

logistics system built on Cougaar. Approaches adopted 

to tune the predictors based on historical data is also 

discussed. Section 5 discusses conclusions and possible 

future areas of research and development. 

2 Predictor Design and Algorithms 

In this section we discuss the predictor algorithms 

that were implemented in Cougaar. Before we discuss 

the technical details of the algorithms we present a brief 

description of the logistics application domain, as some 

aspects of the design and implementation are specific to 

the application. 

2.1 Logistics Scenario 

The Cougaar multi-agent society considered in this 

effort is the Full society that was developed as a part of 

DARPA’s Ultralog Program (See [1] for more details) 

The Full is a military supply chain logistics society that 

consists of many different supply classes. Each agent in 

the society represents a military unit performing a 

certain logistics operation in the supply chain. For e.g. 

the TRANSCOM agent represents the transportation 

command authority for the US military. It issues 

directives to its subordinate units regarding the 

transportation to be provided to a particular agent for a 

particular type of shipment. Figure 1 shows the 

organizational structure of the prototype Full society. 

Figure 1. Full society hierarchical structure 

There are five main supply chain threads in the 

prototype military logistics society. They are (i) 

Ammunition Supply Chain (ii) Petroleum, Oil and 

Lubricants Supply Chain (BulkPOL and PackagedPOL) 

(iii) Subsistence Supply Chain (Food, Water) (iv) Repair 

Parts Supply Chain; and (v) Transportation Supply 

Chain 

Within each supply chain there exists a customersupplier 

relationship between various agents. A 

customer makes requests for various items (POL, 

ammunition etc) to its supplier and the supplier in turn 

attempts to meet these demands based on its current 

inventory, or forwards the requests up the supply chain 

hierarchy. Thus, depending on its positioning in the 

hierarchy, a supplier can also be the customer for 

another agent. 

Figure 2 shows a part of the supply chain. Here, FSB 

is a supplier. ARBN and INFBN are customers of FSB 

(Note: FSB is also a customer of MSB.) These agents 

send demand requests to the FSB, which are managed 

by its Inventory Manager. Based on the operation plan, 

Optempo and current inventory each customer (agent) 

requests items from its supplier. 

Figure 2. Predictor Implementation in Agent 

Network

2.2 Role of Predictors 

As mentioned earlier, when deployed in a battlefield 

this agent society is subject to several stresses related to 

varying wartime loads, kinetic and information warfare. 

These stresses may result in node failures, denial of 

service and other network related faults that will result 

in the lack of communications between agents. In this 

decentralized application the inability of customer/ 

suppliers to make/meet requests can significantly impact 

the performance of the various operational units. 

In this setting, predictors can play an important role 

in maintaining the supply chain connectivity, while 

network related faults are restored. The predictors can 

provide the ability to approximate the “expected 

behavior” by continuing to make appropriate 

demand/supply projections. 

We focus on two classes of predictors (i) a customer 

predictor that resides at the supplier agent and estimates 

the customer’s demand when communications are lost 

(ii) a supplier predictor inserted at the customer agent 

that predicts the allocation results for the tasks generated 

by the supplier. As shown in Figure 2, the customer 

predictor residing on FSB forecasts each customer’s 

(ARBN and INFBN) demand when communications are 

lost. In a similar fashion the supplier predictor residing 

on INFBN agent predicts the supplier’s behavior. These 

agents use the predicted values and continue execution 

of their functionality. 

Depending on the accuracy of the predictors, 

typically predicted states are not identical to the actual 

states. Thus when communications are restored, and 

actual demands/supply values are available, any errors 

between estimated and actual values will need to be 

resolved. This process, termed as “reconciliation”, 

requires any surplus tasks to be rescinded and new tasks 

added for any shortfalls. The predictors in turn will need 

to update their models based on available data. 

2.3 Predictor Design 

2.3.1 Customer Predictor 

The customer predictor is implemented in the form of 

two plugins. One plugin is used during the planning 

mode where it collects the data about the customersupplier 

relationship and items involved. It also collects 

the Optempo of these items. Another plugin is used 

during the execution mode. This Plugin monitors the 

demand from the customer and predicts the demand 

when there is a communication loss. Figure 3 shows the 

framework of the customer predictor during the 

execution mode. 

Figure 3. Customer Predictor 

2.3.2 Supplier Predictor 

This predictor is also built in the form of two plugins. 

One plugin resides at the supplier and other resides at 

the customer. The Plugin at the supplier periodically 

sends the snapshots of the inventory levels for each item 

in all the supply classes to the Plugin at the customer. 

The plugin at the customer uses this information for 

predicting the allocation results of the demand task. The 

design of the supplier predictor is represented 

diagrammatically in Figure 4. 

Figure 4. Supplier Predictor 

2.4 Predictor Algorithms 

Based on the nature of the supply chain dynamics 

(uncertainty in demand, model complexity), duration of 

communication loss, computational requirements 

different approaches for the predictors were investigated. 

These ranged from dynamical systems, to classification 

theory to traditional forecasting [3,4,6,7,8]. While some 

of our research [5] and prototypes indicates that we may 

get better prediction results by using a non parametric 

method such as support vector machine, radial basis 

function neural network etc., from an 

implementation/computational perspective this becomes 

impractical. The primary reason is that as the society 

size scales it becomes increasingly difficult to generate 

historical patterns for each agent and classify its

ehavior states under varying system configurations and 

environment. Given the practical nature of the logistics 

application we adopted a more generic computationally 

inexpensive approach based on moving averages and 

linear-model based estimation schemes. In this section 

the design and implementation of the two schemes are 

discussed. 

2.4.1 Model-based State Estimator 

This approach is motivated in part by traditional 

approaches to estimation such as a Linear Kalman filter 

[2], where the system state is estimated by propagating 

the state using a model of the system and updating the 

state using data-based in actual state measurements. 

The implementation adopted here is a fairly simplistic 

one but has shown to perform significantly well in a 

large number of settings. 

Recall, that a Cougaar-based logistics society 

operates in two modes – a planning mode and an 

execution mode. In the planning mode, based on the 

anticipated operational plan and Optempo a logistics 

plan is generated from projected demands from each of 

the agents. In the execution mode this plan is executed, 

and modifications and re-planning are done as needed 

based on actual demands. 

The approach used in the model-based state estimator 

is to use “plan” time information as the ‘best” estimate 

for the state (demand) in the event that actual state 

information is not available. 

In this approach, plan-time data is used to build a 

linear estimator model for the system of the form 

x(k+1|k) = x(k) + u(k) (1) 

Where x(k) is the demand at time k, and u(k) is the 

request for the “change in demand” at time k. Since 

during the planning mode x(k) is available at each time 

step, in this approach u(k) is explicitly computed as 

u(k) = x(k+1) – x(k) (2) 

for each time step and saved. Thus, based on data 

available during plan generation a model with state and 

inputs for each time step is built. During the execution 

mode, as time evolves the estimator projects the demand 

for the next time step using (1) and corrects its estimate 

based on actual demand as follows 

xˆ( 

k + 1| k + 1) = x( 

k + 1| k) 

+ 

(3) 

K( 

x( 

k + 1| k) 

− xm( 

k + 1| k + 1)) 

xˆ 

Here denoted the estimated state, K is the filter 

gain and xm is the actual (measured) demand. K is 

computed offline based on historical data and execution 

demand error covariances. In the event of a 

communication loss at time j the demand (x cl ) for the 

next time step is projected as 

x cl ( j + 1) = xˆ( 

j | j) 

+ u( 

j) 

(4) 

Thus using the above model, the customer/suppliers 

continue to execute their functionality using the 

estimated states, until communication is restored. At this 

point any differences between the estimated demands 

and the actual demands during the communication loss 

are reconciled and supplies/demand are adjusted 

accordingly. 

2.4.2 Moving Average Model 

In the moving approach the forecasted demand for 

the day t, denoted by F t , for the time window i, is given 

by 

t 

∑ − 1 

− 

Dk 

k = t i 

Ft 

= 

i 

where, D k is the demand for the k th day. For example, 

let the time window be 4. Then forecasted demand for 

the day 10 is given by 

F 

10 

= 

9 

∑ 

k = 6 

4 

D 

k 

( D 

= 

6 

+ D7 

+ D8 

4 

+ D ) 

We evaluate the effectiveness of the forecasting 

method with the following two error criteria which are 

chosen depending on the requirements of the system. 

The corresponding optimal time window i is calculated 

using these error criteria. 

Error 1: Difference between average of demands and 

average of forecasted values 

t 

∑ Dk 

− ∑ Fk 

k 

k 

Et 

= 

t 

t 

Where the time window t = 1,2, 3, … , D t denotes the 

demand at day t, F t denotes the forecasted value for 

day t and E t denotes the error at day t. 

Error Criteria 2: Difference between daily demand and 

daily forecast value 

E t ′ = |D t – F t | 

9

3 Implementing Predictors as Cougaar 

Plugins 

A generalized predictor framework was adopted to 

implement predictors in Cougaar to ensure component 

reusability with minimal code replication. In the adopted 

approach each algorithm does not need a separate 

implementation but extends the predictor 

implementation interface to make use of the data 

collection and other functionalities. The predictors were 

implemented as plugins providing a lightweight 

implementation capability for different agents without 

any risk of corrupting or jamming other services 

provided by the architecture. The predictors are coupled 

with the application domain, i.e., in our case, the 

logistics application, and hence are not part of the 

Cougaar core release but available with the logistics 

functionality package. 

3.1 The Generalized Predictor Framework 

The implementation of predictor framework has the 

following features and services: 

• It has a set of two plugins, one for the planning 

mode and other for the execution mode. Each 

plugin differs in the type of tasks it subscribes to 

and the task processing logic. 

• The plugins automatically identify the customers 

for a given agent in which the plugin is inserted. 

Thus the predictor does not need to know in 

advance what agents represent its customers. 

• The plugins automatically identify the supply 

classes (for e.g. Ammunition, Subsistence etc.) and 

their respective items for each of its customers that 

the supplier predictor serves. 

• The plan time plugin subscribes to demand 

projection tasks and generates a demand/day model 

for each unique customer-supplyclass-item 

relationship. It publishes the data structure or the 

model to the blackboard. 

• The execution time plugin subscribes to actual 

demand tasks and generates demand/day quantity 

values for each unique customer-supplyclass-item 

relationship. 

• The execution time plugin subscribes to the model 

generated by the plan time plugin to update the 

model with actual demand values (this features is 

used by the linear state estimator approach). The 

plugin does not subscribe to the model when other 

approaches such as the Moving Average is used. 

• Different algorithm implementations can be hooked 

into the execution time plugin to do prediction and 

updates on the model. 

• A predictor servlet implementation that can turn the 

predictor ON, OFF or SLEEP manually. This 

feature is used during testing the event of low CPU 

availability, memory usage etc. SLEEP mode refers 

to a passive data collection mode with no 

communication loss whereas OFF mode refers to 

completely shutting down the predictor. 

• The execution time plugin can access the 

communication loss object to automatically change 

the predictor mode to ON/SLEEP. 

• The framework is rehydration compatible. This 

enables the agent to store the data model and 

current state values to keep functioning normally 

when rehydrated. 

The above framework is robust and generic enough 

to be plugged into different agents and offers a plug and 

play mechanism to hook up different algorithms for 

prediction. It should be noted that in the current 

implementation the type of algorithm (model-based, 

average-based etc) cannot be chosen at run time and is 

implemented as a rule in the society definition. Future 

work involves algorithm selection as a run-time 

capability. 

3.2 Reconciliation Mechanism 

Once the communications are back up, a 

reconciliation mechanism has been developed that 

resolves the differences between the actual and 

predicted values to avoid any overages or shortages. 

Furthermore, as during run-time actual demands are 

often available for some period into the future, the 

mechanism only uses estimated demands for those days 

(after communication loss) where actual demand was 

not available. The impact of reconciling the predictions 

with the actual demand after communication is restored 

is significant since it eliminates/reduces the cascading 

effect/bullwhip effect up the supply chain. Also, it 

reduces the re-planning of tasks and resources which 

might have resulted due to shortages or overages. 

4 Experimental Results 

4.1 Customer Predictor using Model-based 

State Estimation 

For our implementation and analysis, we consider 

two agents FSB as a customer and MSB as the supplier. 

MSB provides FSB support for different supply classes 

viz., Ammunition, Fuel, Subsistence and Consumable 

that in turn have various types and items. There are in 

total more than 100 items that are supplied by MSB to 

FSB. .

Extensive testing and validation of the algorithms 

was performed on a number of societies with varying 

number of agents. The results obtained with the linear 

estimator approach seem very encouraging across 

different supply classes. Due to space limitations we 

would show results for few of these items 1 . Figure 5 

shows actual demand and predicted demand values for 

an Ammunition item. We can see that the predicted 

values are very close to the actual values and match with 

the reorder periods of the actual demand. The 

communications were cut for 12 days (day 43 -55) 

during which we can still see the predictions, but no 

actual demand tasks. In Figure 6 (for Subsistence item) 

observe that the predicted demand catches up with the 

actual demand. The initial error is due to the initial 

model inaccuracies that are reduced as the model is 

updated with observed data. 

Figure 6. Actual vs. Predicted values for 

Subsistence item 

Figure 5. Actual vs. Predicted values for Ammo 

item 

Figure 7 and Figure 8 show the planned demand 

(model), actual demand, predicted demand without 

communication loss and predicted demand with 

communication loss for BulkPOL item and Subsistence 

item respectively. As actual demand values roll in, the 

measurement equations reduce the error causing the 

model to closely mimic the actual demand pattern. With 

communication loss, the predictor uses the last updated 

value to forecast demand. 

Figure 7. With & W/o Comm. Loss Prediction 

for BulkPOL item 

1 Some of the experimental results in this paper are shown for Tiny 

and Small societies, which are smaller versions of Full society. Note 

that since the agents are generic, the demand patterns are similar in all 

the societies. 

Figure 8. With & W/o Comm. Loss Prediction 

for Subsistence item

4.2 Moving Average Model Results 

Table 1 shows some typical data collected for the 

moving average model based predictor. Each column 

shows the demand sent to the supplier (MSB) from the 

customer (FSB). On each execution day the customer 

sends the demand for about 20 days ahead. Suppose 

there is a communication loss on day 51, the predictor 

then forecasts the demand on day 52. 

The graphs below (Figure 9, Figure 10 and Figure 

11) show some of the results for the moving average 

based predictor. These results are a few representative 

examples of several runs performed across a number of 

Cougaar societies for various supply classes. The results 

show that the forecasted demand is quite close to the 

original demand thus validating the methodology of the 

predictor. 

Table 1. Demand Requests from FSB to 

MSB 

Figure 9. Comparison of the forecasted 

demand with the actual demand for BulkPOL in 

Small society 


demand with the actual demand for BulkPOL in 

Tiny society 


demand with the actual demand for 

Ammunition in Tiny society

5 Conclusions and Future Research 

The generic predictor framework provides core 

functionality to Cougaar, making Cougaar a more 

survivable agent infrastructure. The predictor plugins 

can be invoked by any agent participating in the 

logistics supply process. Different algorithms can be 

hooked into the framework to use the data and other 

predictor services, hence eliminating the need to write 

predictor plugins from scratch. Initial studies show that 

the estimator models work well for items across 

different supply classes with the prediction almost 

mimicking the actual demand values. However, due to 

the low variability and uncertainty of the observed 

demand, the performance of predictors has not been 

extensively tested. Testing with variable demand is 

currently in progress. Furthermore as a certain class of 

predictors seems to perform better for a particular class 

of data, hybrid approaches to intelligently selecting the 

predictor algorithms based on data-type and demand are 

being investigated. One such approach is to use a 

SMART predictor (Figure 12) as explained below. Here 

a smart predictor would monitor the demand coming 

from the customers and choose which method should be 

used during the communication loss. 

We observe that each method (Model based state 

estimator and Moving-average) gives good forecasts for 

certain types of data. Thus by building a SMART 

predictor which chooses the type of predictor to be used 

depending on the situation would result in better 

forecasts. 

Brinn and Beth DePass for their support, comments and 

insightful discussions. We would also like to thank Lora 

Goldston for her support in the development of the 

reconciliation code and in the testing and integration of 

the predictor algorithms. 

7 References 

1. Ultra*Log Adaptive Logistics Defense Team Plan, 

Revised version 2.0, 2003. 

2. Welch .G, Bishop., G. An introduction to Kalman filter., 

Department of Computer Science, University of North 

Carolina at Chapel Hill, Chapel Hill, TR 95-041, March 

11 2002. 

3. John Moody and Christian J. Darken, Fast learning in 

networks of locally-tuned processing units, Neural 

Computation 1, 281-294, 1989. 

4. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for 

AdaBoost. Machine Learning, 42(3):287-320, March 

2001. 

5. Y.Hong, N.Gautam, S.R.T.Kumara, A.Surana, H.Gupta, 

S.Lee, V.Narayanan, H.Thadakamalla, M. Greaves, M. 

Brinn, Survivability of Complex System – Support Vector 

Machine Based Approach Conf., Artificial Neural 

Networks in Engineering (ANNIE) 2002. 

6. Osuna, E. E., Freund R. and Girosi, F., 1997, Support 

Vector Machines: Training and Applications, Technical 

Report AIM-1602, MIT A.I. Lab. 

7. Vapnik, V. N., 1998, Statistical Learning Theory, John 

wiley & sons, Inc, New York. 

8. Burges, C. J. C., 1998, A Tutorial on Support Vector 

Machines for Pattern Recognition, Knowledge Discovery 

and Data Mining, Vol. 2, No. 2, pp. 121-167. 

9. Cougaar Website (www.cougaar.org) 

Figure 12. Description of SMART Predictor 

6 Acknowledgements 

This research was performed under the DARPA 

Ultralog effort and was supported by DARPA grant 

MDA972-1-1-0038 and Contract 2087-IAI-ARPA-0038. 

We would like to thank Dr. Mark Greaves, Marshall

1 

SURVIVABILITY OF COMPLEX SYSTEM – SUPPORT VECTOR 

MACHINE BASED APPROACH 

Y, HONG, N. GAUTAM, S. R. T. KUMARA, A. SURANA, H. GUPTA, 

S. LEE, V. NARAYANAN, H. THADAKAMALLA 

The Dept. of Industrial Engineering, The Pennsylvania State University, 


M.BRINN 

BBN Technologies, Cambridge, MA 

M. GREAVES 

DARPA IXO/JLTO, Arlington, VA 22203 

ABSTRACT 

Logistic systems which are inherently distributed, in general can be classified 

as complex systems. Survivability of these systems under varying environment 

conditions is of paramount importance. Different environmental conditions in 

which the logistic system resides are translated into several stresses. These in 

turn will manifest as internal stresses. Logistic systems can be modeled as a 

collection of software agents. Each agent’s behavior is a result of the stresses 

imposed. Predicting the agents’ collective behavior is of paramount importance 

to ensure survivability. Analytical modeling of such systems becomes very 

difficult, albeit impossible. In this paper, we study a supply chain in which a 

real life scenario is used. We implement the supply chain in Cougaar 

(Cognitive Agent Architecture developed by DARPA) and develop a predictor, 

based on Support Vector Machine. We report our methodology and results with 

real-life experiments and stress scenarios. 

INTRODUCTION 

Logistic systems can be classified as complex systems (Choi et al., 2001, 

Baranger, http://necsi.org/projects/baranger/cce.pdf). Logistic systems have 

many components such as suppliers and distributors at several stages. These 

components are distributed geographically but interdependent. At each 

component some form of nonlinear decision making process goes on. Typically 

the system would respond in a stable manner to external disturbances. But due to 

information delay, inherent feedback structure and nonlinear components 

unstable phenomena can arise which may ultimately manifest as chaotic 

behavior. Efficient resource allocation and collective oscillations (of say 

inventory levels) are some examples of emergent behavior shown by supply 

chains. They have structure at many scales, each component itself represents a 

simple supply chain. The components compete due to resource limitation but 

collobarate/cooperate to maximize their gains which is another characteristic 

feature of a complex system. 

The survivability of logistic systems under varying environmental 

conditions is of paramount importance. Survivability is going to be itself an 

emergent property of a logistic system and it represents the ability of the system

2 

to function critically even under adverse conditions. We refer to these adverse 

conditions as stresses. In order to improve the survivability, agents should detect 

stresses and take appropriate actions so that they can adapt to stress conditions. 

Due to lack of analytical tools for predicting emergent behavior of a 

complex system from its component behavior, simulation is primary tool of 

designing and optimizing them. In this paper, we would like to show how an 

agent learns to detect stresses as the first step towards improving survivability. 

We implemented the supply chain in COUGAAR (Cognitive Agent Architecture 

developed by DARPA) as a simulation model. Through an extensive design of 

experiments we subjected the supply chain to various stress conditions and made 

the agents learn to predict them using Support Vector Machines. 

THE SMALL SUPPLY CHAIN (SSC) 

We built a multi-agent system for a small supply chain using Cougaar 

version 8.6.0 (http://www.cougaar.org). Cougaar is an open source multi-agent 

architecture and is appropriate for modelling large-scale distributed supply chain 

management applications. We call our supply chain system ‘the Small Supply 

Chain (SSC)’. 

Each agent in SSC represents an individual organization such as a retailer 

and a supplier in the supply chain. Figure 1 represents demand flows in this 

small supply chain. 

Supplier 1 

Factory 1 

Warehouse 1 

Supplier 2 

Warehouse 2 

Retailer 1 

Retailer 2 

Distribution Center 1 Whole Saler 1 

Retailer 3 

Distribution Center 2 

Factory 2 

Supplier 3 

Figure 1. Demand flows in the Small Supply Chain (SSC) 

STRESS TYPES AND LEVELS 

After some preliminary experiments and observations we used the 

following stress conditions to show our approach. 

Stress 1. Changing OPTEMPO. The SSC works according to a Logistics 

Plan. The plan for each agent is prespecified. Every activity of each agent has an 

OPTEMPO value which represents the level of the activity. Changing 

OPTEMPOs can result in a different plan. OPTEMPO can have one of the three 

values, ‘low’, ‘medium’ and ‘high’. 

Stress 2. Adding and Dropping agents. Dropping agents can represent 

situations such as communication loss due to physical accidents or cyber attacks. 

When a retailer agent is dropped, its supporting agent will not receive tasks from 

the dropped agent and its retailer agents will not receive responses from the

3 

dropped agent. These changes will affect planning significantly. By adding new 

retailer agents, we can evaluate how sensitive the SSC is to scalability. The 

addition of new retailer agent increases a load to the other supplier agents. 

PREDICTORS 

In Cougaar, every agent has its own blackboard. During logistics planning, 

the intermediate planning results are continuously accumulated on that 

blackboard. Therefore, by observing the blackboard we can recognize the 

planning state. Our idea is to detect stresses by observing the blackboard. Each 

agent should have the ability to detect the stresses coming from outside so that it 

can make a decision autonomously to handle the stresses. 

In this work, for each agent we build a separate supervised learning model. 

Many types of task classes are instantiated on the blackboard. The collection of 

the number of tasks of each type represents the state of the agent. A task is a 

java class of Cougaar which represents a logistic requirement or activity. Tasks 

are generated successively along the supply chain starting from the tasks of the 

retailers. The learning model takes the state of the agent as input feature and 

predicts the corresponding stress type and level. The pattern recognition model - 

predictor - is built using the Support Vector Machine. 

In order to prepare training and test data, the blackboard of each agent is 

monitored and data is stored into a database during experiments by a monotoring 

facility which consists of a specialized Plugin and a separate server machine. 

The Plugin is a java class provided by Cougaar. The pattern recognition model is 

trained by the data from the database off-line. 

SUPPORT VECTOR MACHINES (SVM) 

A Support Vector Machine is a pattern recognition method. It has been 

popular since the mid-90s because of its theoretical clearness and good 

performance. Many pattern recognition applications have been reported since 

this theory was developed by Vapnik (Műller, et al., 2001), which also 

exemplify its superiority over similar such techniques. Moghaddam and Yang 

(2002) applied SVM to the appearance-based gender classification and showed 

that it is superior to other classifiers-nearest neighbor, radial basis function 

classifier. Liang and Lin (2001) showed SVM has better performance than 

conventional neural network in detection of delayed gastric emptying. For an 

exhaustive review, we recommend the reader to (Burges, 1998), (Chapelle et al., 

1999) and (Műller et al., 2001). 

The main idea of SVM is to separate the classes with a surface that 

maximizes the margin between them. This is an approximate implementation of 

the Structural Risk Minimization induction principle (Osuna, et al., 1997). To 

construct a classifier for a given data set, a SVM solves a quadratic 

programming problem with each variable corresponding to a data point. When 

the size of the data set is large, it requires special techniques such as 

decompositions to handle the large number of variables. Basically, the SVM is a 

linear classifier. Thus in order to handle a dataset which is not separable by a 

linear function, inner-product kernel functions are used. The role of the innerproduct 

kernel functions is to convert an inner product of low dimensional data

4 

points into a corresponding inner product in high dimensional space without 

actual mapping. The principle of the mapping is based on Mercer’s theorem 

(Vapnik 1998). By doing so, the SVM overcomes non linear-separable cases. 

The selection of kernel functions depends on the problem. We should 

choose an appropriate function by performing experiments. Other control 

parameters for the SVM are the extra cost for errors (represented as C), the loss 

sensitivity constant (ε insensitivity ) and the maximum number of iterations. The extra 

cost for errors, C, is a cost assigned to training errors in non linear-separable 

cases. A larger C corresponds to assigning a higher penalty to errors (Burges 

1998). The loss sensitivity constant (ε insensitivity ) represents the allowable error 

range for the prediction values. 

SVMs are basically developed as binary classifiers. Currently a lot of 

research is being done in the area of multi-class SVM. We use BSVM 2.0 which 

is the multi-class SVM program suggested by Hsu and Lin (2002). 

EXPERIMENT CONDITIONS 

We simulated the SSC under various stress conditions. 12 Stress conditions 

are made through the combination of the following factors; The number of 

retailer 1(zero, one, two), OPTEMPO of retailer 2 (LOW, HIGH), OPTEMPO 

of retailer 3 (LOW, HIGH). 

219 data sets were used for training and 94 data sets were used for testing 

the prediction power from a total of 313 data sets. The conditions and number of 

experiments are shown in the table 1. 

Condition Retailer 1 Retailer 2 Retailer 3 Training Test Total 

1 Zero LOW LOW 25 11 36 

2 Zero HIGH LOW 19 8 27 

3 Zero LOW HIGH 18 8 26 

4 Zero HIGH HIGH 17 7 24 

5 One LOW LOW 29 13 42 

6 One HIGH LOW 17 7 24 

7 One LOW HIGH 18 7 25 

8 One HIGH HIGH 18 8 26 

9 Two LOW LOW 21 9 30 

10 Two HIGH LOW 12 5 17 

11 Two LOW HIGH 13 6 19 

12 Two HIGH HIGH 12 5 17 

Grand Total - - - 219 94 313 

Table 1. The stress condition and number of experiments 

TRAINING 

Through preliminary studies apart from the experiments tabulated above we 

found that all the stress conditions do not affect all the agents. Thus, we 

prepared different classification definitions for the training set depending on the 

agent (See the table 2). 

As the classification definitions are different for different agents, the input 

features are also different. The tasks used as input features in each agent are 

shown in table 3.

5 

The option ‘multi-class bound-constrained support vector classification’ in 

BSVM 2.0 is selected. For others, we use default options of BSVM 2.0 such as 

the radial basis function with gamma = 1/(the number of input features). 

Agent name 

Retailer 2, 

Warehouse 2 

Retailer 1 

Factory 2, 

Supplier 2 

Warehouse 1, 

Factory 1 

Retailer 3, 

Distribution Center 2 

Distribution Center 1, 

Supplier 1 

Wholesaler 1 

Class definition 

Class 1: condition 1,3,5,7,9,11 (Retailer 2 LOW) 

Class 2: condition 2,4,6,8,10,12 (Retailer 2 HIGH) 

Class 1: condition 1,2,3,4 (zero Retailer 1) 

Class 2: condition 5,6,7,8 (one Retailer 1) 

Class 3: condition 9,10,11,12 (two Retailer 1) 

Class 1: condition 1,3 (Retailer 2 LOW at zero Retailer 1) 

Class 2: condition 2,4 (Retailer 2 HIGH at zero Retailer 1) 

Class 3: condition 5,6,7,8,9,10,11,12 (All other cases) 

12 Classes, Regard each condition as one class 

Class 1: condition 1,2,5,6,9,10 (LOW Retailer 3) 

Class 2: condition 3,4,7,8,11,12 (HIGH Retailer 3) 

Class 1: 1,2 (LOW Retailer 3 at zero Retailer 1) 

Class 2: 3,4 (HIGH Retailer 3 at zero Retailer 1) 

Class 3: 5,6 (LOW Retailer 3 at one Retailer 1) 

Class 4: 7,8 (HIGH Retailer 3 at one Retailer 1) 

Class 5: 9,10 (LOW Retailer 3 at two Retailer 1) 

Class 6: 11,12 (HIGH Retailer 3 at two Retailer 1) 

Class 1: condition 9,10,11,12 (Two Retailer 1) 

Class 2: condition 1,2,3,4,5,6,7,8 (other conditions) 

Table 2. The class definition by agents 

Agent Features Agent Features 

Retailer 2 PS, PW Warehouse 1 PS, PW, OS 

Distribution Center 1 W, OPS, OS Factory 1 TP, W, OPS, OS 

Retailer 3 PS, PW Supplier 1 TR, TP, OTP 

Distribution Center 2 PS, PW Retailer 1 PS, PW 

Factory 2 TP Warehouse 2 TP, W 

Supplier 2 OTP Wholesaler 1 S 

* PS = ProjectSupply, PW = ProjectWithdraw, W =Withdraw, TP = Transport, 

TR = Transit, S = Supply, OPS = ProjectSupply coming from outside, 

OS = Supply coming from outside, OTP = Transport coming from outside 

Table 3. The input features by agents 

Agent Success rate Agent Success rate 

Retailer 2 100% Warehouse 1 100% 

Retailer 1 100% Distribution Center 1 100% 

Retailer 3 100% Factory 1 22.34% 

Distribution Center 2 100% Supplier 1 40.43% 

Factory 2 100% Warehouse 2 84.04% 

Supplier 2 100% Wholesaler 1 86.17% 

Table 4. The success rate to classify the stress condition at each agent 

RESULTS 

The table 4 contains the test results. Overall performance is good. In 

addition, we can see the agent near the retailers in the supply chain can detect

6 

stresses well. The Warehouse 1 agent can detect exactly all the stress types even 

though it is far from retailers (see Fig 1.). 

CONCLUSIONS 

We have shown an effective application of pattern recognition model for 

detecting stresses by observing the internal state of each agent. Each agent has a 

SVM since the influence of the same stress on different agents can be different. 

Some agents near the retailers can detect stresses very well. However, it is hard 

to detect the influence of the stress on agents which are far from retailers. 

Overall performance of predictor of each agent is good. Constructing the 

capability for stress detection is the first step towards improving the 

survivability of a multi-agent system. This result is important because we can 

pursue further research on how we can dampen the effect of stresses based on 

the result of this study. Based on current detected state each agent can change 

their behaviors - ordering or planning - to adapt to stress conditions without 

serious performance degradation of overall supply chain. In addition, our 

approach is generally useful because it is very hard to model a complex system 

analytically. 

ACKNOWLEDGEMENTS 

Support for this research was provided by DARPA (Grant#: MDA 972-01- 

1-0563) under the UltraLog program. 

REFERENCES 

Baranger, Michel, “Chaos, Complexity, and Entropy – A physics talk for non-physicists,” MIT- 

CTP-3112, http://necsi.org/projects/baranger/cce.pdf. 

Burges, C. J. C., 1998, “A Tutorial on Support Vector Machines for Pattern Recognition,” 

Knowledge Discovery and Data Mining, Vol. 2, No. 2, pp. 121-167. 

Chapelle, O., Haffner, P. and Vapnik, V. N., 1999, “Support Vector Machines for Histogram-Based 

Image Classification,” IEEE Transactions on Neural Networks, Vol. 10, No. 5, pp. 1055-1064. 

Choi, T., Dooley, K. and Rungtusanatham, M, 2000, " Supply Networks and Complex Adaptive 

Systems: control versus emergence,” Journal of Operations Management, Vol. 19, pp 351-366. 

Dooley, K., 2002, “Simulation Research Methods,” Companion to Organizations, Joel Baum (ed.), 

London: Blackwell, pp. 829-848. 

Hsu, Chih-Wei and Lin, Chih-Jen, 2002, “A Comparison of Methods for Multiclass Support Vector 

Machines,” IEEE Transactions on Neural Networks, Vol. 13, No. 2, pp. 415-425. 

Liang, H. and Lin, Z., 2001, “Detection of Delayed Gastric Emptying from Electrogastrograms with 

Support Vector Machine,” IEEE Transactions on Biomedical Engineering, Vol. 13, No. 2, pp. 

415-425. 

Moghaddam, B. and Yang, M., 2002, “Learning Gender with Support Faces,” IEEE Transactions on 

Pattern Analysis and Machine Intelligence, Vol. 24, No. 5, pp. 707-711. 

Műller, K., Mika, S., Rätch, G., Tsuda, K. and Schőlkopf B., 2001, “A Introduction to Kernel-Based 

Learning Algorithms,” IEEE Transactions on Neural Networks, Vol. 12, No. 2, pp. 181-202. 

Osuna, E. E., Freund R. and Girosi, F., 1997, Support Vector Machines: Training and Applications, 

Technical Report AIM-1602, MIT A.I. Lab. 

Vapnik, V. N., 1998, Statistical Learning Theory, John wiley & sons, Inc, New York.

A Framework for Performance Control of 

Distributed Autonomous Agents 

Nathan Gnanasambandam, Seokcheon Lee, Soundar R.T. Kumara and Natarajan Gautam 

310 Leonhard Building, The Pennsylvania State University, University Park, PA, 16802, USA 

Abstract 

We propose an autonomous and scalable queueing theory-based methodology to control the performance of a hierarchical network 

of distributed agents. Multi-agent systems (MAS) such as supply chains functioning in highly dynamic environments 

need to achieve maximum overall utility during operation. Hence, the objective of the control framework is to identify the 

trade-offs between quality and performance and adaptively choose the operational settings to posture the MAS for better utility. 

By formulating the MAS as an open queueing network with multiple classes of traffic we evaluate the performance and 

subsequently the utility, from which we identify the control alternative for a localized, multi-tier zone. 

Keywords: Queueing Network, Multi-Agent Systems, Performance control. 


With the growing view of agent-oriented software sytems [1] and the increased deployment of distributed multi-agent systems 

(DMAS) for numerous emerging applications such as computational grids, e-commerce hubs, supply chains and sensor networks, 

we are faced with large-scale distributed agents whose performance needs to be estimated and controlled. Often times, 

these DMAS operate in dynamic and stressful environmental conditions, of one type or the other, in which the MAS as whole 

must survive. While survival notion necessitates adaptivity to diverse conditions along the dimensions of performance, security 

and robustness, delivering the correct proportion of these quantities can be quite a challenge. In this paper, we address a piece 

of this problem by building an autonomous performance control framework for MAS. 

While building large multi-agent societies (such as UltraLog [2]), it is desirable that the associated adaptation framework 

be generic and scalable. One way to do this is to utilize a methodology similar to Jung and Tambe [3], where the bigger 

society is composed of smaller building blocks, in this case, corresponding to communities of agents. Although, strategies for 

co-operativeness and distributed POMDP to have been utilized to analyze performance in [3], an increase in the number of 

variables in each agent can quickly render POMDP ineffective even in reasonable sized agent communities due to the statespace 

explosion problem. In [4], Rana and Stout identify data-flows in the agent network and model scalability with Petri 

nets, but their focus is on identifying synchronization points, deadlocks and dependency constraints with coarse support for 

performance metrics relating to delays and processing times for the flows. In a recent architecture for autonomic computing, 

Tesauro et al. [5] build a real-time MAS-based framework that is self-optimizing based on application-specific utility. While 

[3, 4] motivate the need to estimate performance of large DMAS using a building block approach, [5] justifies the need to use 

domain specific utility whose basis should be the network’s service-level attributes such as delays, utilization and response 

times. 

We believe that by using queueing theory we can analyze data-flows within the agent community with greater granularity in 

terms of processing delays and network latencies and also capitalize on using a building block approach by restricting the model 

to the agent community. Queueing theory has been widely used in networks and operating systems [6]. However, the authors 

have not seen the application of queueing to MAS modeling and analysis. Since, agents lend themselves to being conveniently 

represented as a network of queues, we concentrate on engineering a queueing theory based adaptation (control) framework to 

enhance the application-level performance. 

Inherently, the DMAS can be visualized as a multi-layered system as is depicted in Figure 1a. The top-most layer is where 

the application resides, usually conforming to some organization such as mesh, tree etc. The infrastructure layer not only 

abstracts away many of the complexities of the underlying resources (such as CPU, bandwidth), but more importantly provides 

services (such as Message Transport) and aiding agent-agent services (such as naming, directory etc.). The bottom most layer 

is where the actual computational resources, memory and bandwidth reside. Most studies in the literature do not make this 

distinction and as such control is not executed in a layered fashion. Some studies such as [7, 8], consider controlling attributes 

in the physical or infrastructural layers so that some properties (eg. survivability) could result and/or the facilities provided by 

these layers are taken advantage of. Often, this requires rewiring the physical layer, availability of a infrastructure level service 

or the ability of the application to share information with underlying layers in a timely fashion for control purposes. In this 

work, we consider control only due to application-level trade-offs such as quality of service versus performance and assume that 

infrastructure level services (such as load-balancing, priority scheduling) or physical level capabilities (such as rewiring) are not 

possible. This does not exclude the possibility that in future we can combine all approaches to achieve a multi-layered control.

© £ ¨ ¥ ¤ § ¢ ¥ 

© £ ¨ £¦ £ ¨ ¦ ¥ ¢ ¥ 

¢ £ ¤ ¤ £ ¤ ¡ 

¤ ¨© § ¢ ¤ ¡ ¢ © ¡ ¢ £ £ ¢ 

 

< 

) 

+ * 

5 

1 

 

! " " # ! 

3 6 - 4 3 / - 1 

A / 2 - 3 4 @ 

2 2 . 6 7 5 

3 9 / 8 

¢ £ ¤ ¤ £ ¤ ¡ 

¦ ¦ § ¨© ¡ ¨ ¤ £ ¢ 

¥ 

, > 

, = ; : = 

- . / . - 0 , 

/ % 2 / - 3 4 1 

, 

+ * 

Our contribution in this work is to combine queueing analysis and application-level control to engineer a generic framework 

that is capable of self-optimizing its domain-specific utility. 

¡ ¡ ¢ £ ¤ ¥ ¦ £ § ¨ © ¥ 

 

 

: ; , 

: = 

8 ) 

) ; < ; ) ; : 

¨ ¥ ¦ ¤ ¦ © ¥ § ¥ ¥ 

$ % & ' % ( 

: 

? ; $ 

8 3 9 / + 2 . / 9 4 

) > 

£ ¤ ¥ ¢ © ¥ 

: > ; < > 

(a) Operational Layers 

(b) Framework Architecture 

Figure 1: MAS framework 


Typically, the top-most layer in the computing infrastructure (here the DMAS-based application) possesses maximum transparency 

to system’s overall utility, control-knobs and domain knowledge. The utility of the application is the combined benefit 

along several conflicting (eg. completeness and timeliness [9, 2]) and/or independent (eg. confidentiality and correctness [9, 2]) 

dimensions, which the application tries to maximize in a best-effort sense through trade-offs. Understandably, in a distributed 

multi-agent setting, mechanisms to measure, monitor and control this multi-criteria utility function become hard and inefficient, 

especially under conditions of scale-up. Given that the application does not change its high-level goals, task-structure or 

functionality in real-time, it is beneficial to have a framework that assists in the choice of operational modes (or opmodes) in a 

distributed way. Hence, the research objective of this work is to design and develop a generic, real-time framework for DMAS, 

that utilizes a queueing network model for performance evaluation and a learned utility model to select an appropriate control 

alternative. 

1.2 Solution Methodology 

The focus of this research is to adjust the application-level parameters or opmodes within the distributed agents to make an 

autonomous choice of operational parameters for agents in a reasonable-sized domain (called an agent community). The choice 

of opmodes is based on the perceived application-level utility of the combined system (i.e. the whole community) that current 

environmental conditions allow. We assume that the application’s utility depends of the choice of opmodes at the agents 

constituting the community because the opmodes directly affect the performance. A queueing network model is utilized to 

predict the impact of DMAS control settings and environmental conditions on steady-state performance (in terms of end-to-end 

delays in tasks), which in turn is used to estimate the application-level utility. After evaluating and ranking several alternatives 

from among the feasible set of operational settings on the basis of utility, the best choice is picked. 

2 Architecture of the Performance Control Framework 

We implement the performance control framework for the Continuous Planning and Execution (CPE) Society which is a command 

and control MAS built on Cougaar (DARPA Agent Platform [10]). While we describe the functionality of the components 

of the framework (Figure 1b) in this section, we highlight the autonomic capabilities that are built into the system. 

2.1 Overview of Application (CPE) Scenario 

In our set-up, the primary building block consists of three tiers in the application layer. CPE embodies a complete military 

logistics scenario with agents emulating roles such as suppliers, consumers and controllers all functioning in a dynamic and 

hostile (destructive) external environment. Embedded in the hierarchical structure of CPE are both command and control, 

and superior-subordinate relationships. The subordinates compile sensor updates and furnish them to superiors. This enables 

the superiors to perform the designated function of creating plans (for maneuvering and supply) as well as control directives

for downstream subordinates. Upon receipt of plans, the subordinates execute them. The supply agents replenish consumed 

resources periodically. This high level system definition is the functionality of CPE that it seeks to perform repeatedly with 

maximum utility while residing in the application layer.As part of the application-level adaptivity features, a set of opmodes 

are built into the system. Opmodes allow individual tasks (such as plans, updates, control) to be executed at different qualities 

or to be processed at different rates. We assume that TechSpecs for the CPE scenario are available to be utilized by the control 

framework. The framework that accomplishes the aforementioned goal of CPE in a distributed fashion while performing at a 

maximum possible level of utility is represented in Figure 1b. 

2.2 Self-Monitoring Capability 

Any system that wants to control itself should possess a clear specification of the scope of the variables it has to monitor. The 

TechSpecs is a distributed structure that supports this purpose by housing all variables, X, that have to be monitored in different 

portions of the community (or sub-system). The data/statistics collected in a distributed way, is then aggregated to assist in 

control alternatives by the top-level controller that each community will possess. 

The attributes that need to be tracked are formulated in the form of measurement points (MP ). The measurement points are 

“soft” storage containers residing inside the agents and contain information on what, where and how frequently they should 

be measured. Each agent can look up its own TechSpecs and from time-to-time forward that to its superior. The superior can 

analyse this information (eg. calculate statistics such as delay, delay-jitter) and/or add to this information and forward it again. 

We have measurement points for time-periods, time-stamps, operating-modes, control and generic vector-based measurements. 

These measurement points can be chained for tracking information for a flow such that information is tagged-on at every point 

the flow traverses. For the sake of reliability, the information that is contained in these agents is replicated at several points, so 

that when packets do not reach on time or not reach at all, previously stored packets can be utilized for control purposes. 

2.3 Self-Modeling Capability 

One of the key features of this framework is that it has the capability to choose a type of model for representing itself for the 

purpose of performance evaluation. The system is equipped with several queueing model templates that it can utilize to analyze 

the system with. The type of model that is utilized at any given moment is based on accuracy, computation time and history of 

effectiveness. For example, a simulation based queueing model may be very accurate but cannot complete evaluating enough 

alternatives in limited time, in which case an analytical model (such as BCMP, QNA [11]) is preferred. 

The inputs to the model builder are the flows that traverse the network (F ), the types of packets (T ) and the current configuration 

of the network. If at a given time, we know that there are n agents interconnected in a hierarchical fashion then the role of 

this unit is to represent that information in the required template format (Q). The current number of agents is known to the 

controller by tracking the measurement points. For example, if there is no response from an agent for a sufficient period of time, 

then for the purpose of modeling, the controller may assume the agent to be non-existent. In this way dynamic configurations 

can be handled. On the other hand, TechSpecs do mandate connections according to superior-subordinate relationships thereby 

maintaining the flow structure at all times. Once the modeling is complete, the MAS has to capability to analyze its current 

performance using the selected type of model. The MAS does have the flexibility, to choose another model template for a 

different iteration. 

2.4 Self-Evaluating Capability 

The evaluation capability, the first step in control, allows the MAS to examine its own performance under a given set of plausible 

conditions. This prediction of performance is used for the elimination of control alternatives that may lead to instabilities. Our 

notion of performance evaluation is similar to [5]. While Tesauro et al. [5] compute the resource level utility functions (based 

on the application manager’s knowledge of system performance) that can be combined to obtain a globally optimal allocation of 

resources, we predict the performance of the MAS as a function of its operating modes in real-time (within Queueing Model) and 

then use it to calculate its global utility. By introducing a level of indirection, we may get some desirable properties (explained 

in Section 4.2) because we separate an application’s domain-specific utility computation from performance prediction (or 

analysis). This theoretically enables us to predict the performance of any application whose TechSpecs are clearly defined and 

then compute the application-specific utility. In both cases, control alternatives are picked based on best-utility. We discuss 

the notion of control alternatives in Section 2.5. Also, our performance metrics (and hence utility) are based on service level 

attributes such as end-to-end delay and latency, which is a desirable attribute of autonomic systems [5]. 

When plan, update and control tasks (as mentioned in Section 2.1) flow in this heterogeneous network of agents in predefined 

routes (called flows), the processing and wait times of tasks at various points in the network are not alike. This is because 

the configuration (number of agents allocated on a node), resource availability (load due to other contending software) and 

environmental conditions at each agent is different. In addition, the tasks themselves can be of varying qualities or fidelities 

that affects the time taken to process that task. Under these conditions, performance is estimated on the basis of the end-to-end 

delay involved in a “sense-plan-respond” cycle.

Table 1: Notation 

Symbol 

Description 

N 

Total # of nodes in the community 

λ ij 

Average arrival rate of class j at node i 

1/µ ijk Average processing time of class j at node i at quality k 

M 

Total number of classes 

T i 

Routing probability matrix for class i 

W ijk 

Steady state waiting time for class j at node i at quality k 

Set of qualities at which a class j task can be processed at node i 

Q ij 

The primary performance prediction tool that we use are called Queueing Network Models (QNM) [6]. The QNM is the representation 

of the agent community in the queueing domain. As the first step of performance estimation, the agent community 

needs to be translated into a queueing network model. Table 1 provides the notations used is this section. Inputs and outputs 

at a node are regarded as tasks. The rate at which tasks of class j are received at node i is captured by the arrival rate (λ ij ). 

Actions by agents consume time, so they get abstracted as processing rates (µ ij ). Further, each task can be processed at a 

quality k ∈ Q ij , that causes the processing rates to be represented as µ ijk . Statistics of processing times are maintained at each 

agent in the Performance Database (PDB) to arrive at a linear regression model between quality k and µ ijk . Flows get associated 

with classes of traffic denoted by the index j. If a connection exists between two nodes, this is converted to a transition 

probability p ij , where i is the source and j is the target node. Typically, we consider flows originating from the environment, 

getting processed and exiting the network making the agent network an open queueing network [6]. Since we may typically 

have multiple flows through a single node, we consider multi-class queueing networks where the flows are associated with a 

class. Performance metrics such as delays for the “sense-plan-respond” cycle is captured in terms of average waiting times, 

W ijk . As mentioned earlier, TechSpecs is a convenient place where information such as flows and Q ij can be embedded. 

The choice of QNM depends on the number of classes, arrival distribution and processing discipline as well as a suggestion 

C by the DMAS controller that makes this choice based upon history of prior effectiveness. Some analytical approaches to 

estimate performance can be found in [6, 11]. In the context of agent networks, Jackson and BCMP queueing networks have 

been applied to estimate the performance in [12]. By extending this work we provide several templates of queueing models 

(such as BCMP, Whitt’s QNA [11], Jackson, M/G/1, a simulation) that can be utilized for performance prediction. 

2.5 Self-Controlling Capability 

In contrast to [5], we deal with optimization of the domain utility of a MAS that is distributed, rather than allocating resources in 

an optimal fashion to multiple applications that have a good idea of their utility function (through policies). As mentioned before 

opmodes allow for trading-off quality of service (task quality and response time) and performance. We are assuming there is a 

maximum ceiling R on the amount of resources, and the available resources fluctuate depending on stresses S = S e +S a , where 

S e are the stresses from the environment (i.e. multiple contending applications, changes in the infrastructural or physical layers) 

and S a are the application stresses (i.e. increased tasks). The DMAS controller receives from MP (measurement points) a 

measurement of the actual performance P and a vector of other statistics (X) about task processing times. Also at the top-level 

the overall utility (U) is U(P, S) = ∑ w n x n is known where x n is the actual utility component and w n is the associated 

weight. We cannot change S, but we can adjust P to get better utility. Since P depends on O, which is a vector of opmodes 

collected from the community, we can use the QNM to find O ∗ and hence P ∗ that maximizes U(P, S) for a given S. In words, 

we find the vector of opmodes (O ∗ ) that maximizes domain utility at current S and update O. This computation is performed in 

the Utility Calculator module using a utility model that is learned and stored in the Utility Database (UDB). This formulation 

although independently found matches the self-optimization notion in [5]. But some differences exist as follows. Tesauro et al. 

[5] assume that the system’s knowledge includes a performance model, which we do not assume. We use a queueing network 

model to estimate the performance in real-time for any set of opmodes O ′ by taking the current set of opmodes O and scaling 

them appropriately based on observed histories (X) to X ′ in the Control Set Evaluator. Also, we deal with a single MAS with 

an overall utility function for the entire distributed functionality (within the community). Because of the interactions involved 

and complexity of performance modeling[3, 4], it may be time-consuming to utilize inferencing and learning mechanisms in 

real-time. This is why we use an analytical queueing network to get the performance estimate quickly. Another difference is 

that in [5], they assume operating system support which may not be true in many MAS-based situations because of mobility, 

security and real-time constraints. Furthermore, in addition to the estimation of performance, the queueing model may have 

the capability to eliminate instabilities from a queueing sense, which is not apparent in the other approach. But inspite of these 

differences, it is interesting to see that self-controlling capability can be achieved, with or without explicit layering, in a couple 

of real-world applications.

1000 

500 

0 

0.2 0.4 0.6 0.8 

-500 

-1000 

Default Policy 

Controlled 

Stress (S) 

Figure 2: Results overview 

20 

15 

10 

5 

0 

0 200 400 600 800 1000 1200 

-5 

-10 

time (sec.) 


1400 

1200 

1000 

800 

600 

400 

200 

0 

0 200 400 600 800 1000 1200 

time (sec.) 


(a) Instantaneous Utility (stress 0.25) 

(b) Cumulative Utility (stress 0.25) 

20 

15 

10 

5 

0 

0 200 400 600 800 1000 1200 

-5 

-10 

time (sec.) 


1200 

1000 

800 

600 

400 

200 

0 

0 200 400 600 800 1000 1200 

time (sec.) 


(c) Instantaneous Utility (stress 0.75) 

(d) Cumulative Utility (stress 0.75) 

Figure 3: Sample results 

3 Empirical Evaluation on CPE Test-bed 

We utilized the prototype CPE framework to run 36 experiments at two stress levels (S = 0.25 and 0.75). The scenario 

consisted of 14 agents, besides a world agent that created random scenarios in military logistics for the agents to react to. There 

were three layers of hierarchy with a three-way branching at each level and one supply node. The community’s utility function 

was based on the achievement of real goals in military engagements such as terminating or damaging the enemy and reducing 

the penalty involved in consuming resources such as fuel or sustaining damage. We also assumed for the model selection 

process that the external arrival was Poisson and the service times were exponentially distributed. In order to cater to general 

arrival rates, our framework contains a QNA- and simulation-based model. Using this assumption a BCMP or M/G/1 queueing 

model could be utilized. We used the Cougaar based default control without additional support from our framework as the 

baseline (denoted as Default A and Default B) and found that controlling the agent community using our framework (denoted 

as controlled) was beneficial in the long run. The overview of the results is provided in Figure 2. 

At both stress levels, the controlled scenario performed better that the default as shown in Figure 3. We did observe oscillations 

in the instantaneous utility and we attribute this to the impreciseness of the prediction of stresses. Stresses vary relatively fast 

in the order of seconds while the control granularity was of the order of minutes. Since this is a military engagement situation 

following no pre-determined stress patterns, it is hard to cope with in the higher stress case. We think that this could be the 

reason why our utility falls in the latter case.

4 Conclusions and Future Work 

4.1 Conclusions 

In this paper, we were able to successfully control a real-time MAS to achieve overall better utility in the long run using 

application-level trade-offs between quality of service and performance. We utilized a queueing network based framework for 

performance analysis and subsequently used a learned utility model for computing the overall benefit to the MAS (i.e. community). 

While Tesauro et al. [5] have found a similar construction to improve utility in multiple applications, we concentrated 

on optimizing the utility of a single distributed application using queueing theory. We think that the approaches are complementary, 

with this study providing empirical evidence to support the observation in [1] that agents can be used to optimize 

distributed application environments, including themselves, through flexible high-level (i.e. application-level) interactions. 

4.2 Discussion and Future Work 

We believe that keeping the building-blocks small and the number of interactions (between performance and utility models) 

minimal may assist in making the framework more flexible and scalable. For example, if system size increases, we can consider 

a superior agent or human user to be at the next higher level controlling the weights in the utility function without affecting the 

performance model. The larger system with supervisory control would then be analyzed using another higher-level QNM or a 

network of networks. TechSpecs has assisted this effort to a large extent, re-emphasizing the well-founded separation principle 

(separating knowledge/policy and mechanism) in the computing field. While we think that the aforementioned architectural 

principles have been well-utilized, we hope to broaden the layered control approach to encompass the infrastructural-level 

control into the framework. Another avenue for improvement is to design self-protecting mechanisms within our framework so 

that the security aspect of the framework is reinforced. 

Acknowledgments 

The work described here was performed under the DARPA UltraLog Grant#: MDA972-1-1-0038. The authors wish to acknowledge 

DARPA for their generous support. 

References 

[1] Jennings, N. R. and Wooldridge, M., 2000, “Agent-Oriented Software Engineering”, Handbook of Agent Technology, 

AAAI/MIT Press. 

[2] UltraLog Program Site. www.ultralog.net. DARPA. 

[3] Jung, H., and Tambe, M., 2003, “Performance Models for Large Scale Multi-Agent Systems”, Proceedings of the Seocnd 

Joint Conference on Autonomous Agents and Multi-Agnet Systems. 

[4] Rana, O. F., and Stout, O., “What is Scalability in Mult-Agent Systems”, 2000, Proceedings of the Fourth International 

Conference on Autonomous Agents. 

[5] Tesauro, G., Chess, D. M., Walsh, W. E., Das, R., Whalley, I., Kephart, J. O., and White, S. R., 2004, “A Multi-Agent 

Systems Approach to Autonomic Computing”, Autonomous Agents and Multi-Agent Systems. 

[6] Bolch, G., de Meer, H., Greiner, S., and Trivedi, K. S., 1998, Queueing Networks and Markov Chains: Modeling and 

Performance Evaluation with Computer Science Applications. John Wiley and Sons. 

[7] Thadakamalla, H. P., Raghavan, U. N., Kumara, S. R. T., and Albert, R., 2004, “Survivability of Multi-Agent Systems - A 

Topological Perspective”, IEEE Intelligent Systems: Dependable Agent Systems, vol. 19, no. 5, pp. 24-31, Sep/Oct 2004. 

[8] Hong, Y., and Kumara, S. R. T., 2004, “Coordinating Control Decisions of Software Agents for Adaptation to Dynamic 

Environments”, Working Paper, Marcus Department of Industrial and Manufacturing Engineering, Pennsylvania State 

Univerity, University Park, PA. 

[9] Brinn, M., and Greaves, M., 2003, “Leveraging Agent Properties to Assure Survivability of Distributed Multi-Agent 

Systems”, in the Proceedings of the Second Joint Conference on Autonomous Agents and Multi-Agent Systems. 

[10] Cougaar Open Source Site. www.cougaar.org. DARPA. 

[11] Whitt, W., 1983, “The Queueing Network Analyzer”, The Bell System Technical Journal, vol. 62, no. 9, pp. 2779-2815. 

[12] Gnanasambandam, N., Lee, S., Gautam, N., Kumara, S. R. T., Peng, W., Manikonda, V., Brinn, M., and Greaves, M., 

2004, “Reliable MAS Performance Prediction Using Queueing Models”, IEEE Multi-Agent Security and Survivability 

Symposium.

An Autonomous Performance Control Framework for 

Distributed Multi-Agent Systems: A Queueing Theory 

Based Approach 

Nathan 

Gnanasambandam 

gsnathan@psu.edu 

Seokcheon Lee 

stonesky@psu.edu 


310 Leonhard Building 


Soundar R.T. Kumara 

skumara@psu.edu 

ABSTRACT 

Distributed Multi-Agent Systems (DMAS) such as supply chains 

functioning in highly dynamic environments need to achieve maximum 

overall utility during operation. The utility from maintaining 

performance is an important component of their survivability. 

This utility is often met by identifying trade-offs between quality 

of service and performance. To adaptively choose the operational 

settings for better utility, we propose an autonomous and scalable 

queueing theory based methodology to control the performance of 

a hierarchical network of distributed agents. 

Categories and Subject Descriptors 

C.4 [Performance of Systems]: Design studies, modeling techniques, 

performance attributes 

General Terms 

Performance 

Keywords 

Multi-Agent Systems, Survivability, Queueing Models 

1. INTRODUCTION 

With the emerging popularity of distributed multi-agent systems 

as application platforms, it is necessary that they survive dynamic 

and stressful environmental conditions, even partial permanent damage. 

While the survival notion necessitates adaptivity to diverse 

conditions along the dimensions of performance, security and robustness, 

delivering the correct proportion of these quantities can 

be quite a challenge. From a performance standpoint, a survivable 

system can deliver excellent Quality of Service (QoS) even when 

stressed. A DMAS could be considered survivable if it can maintain 

at least x% of system capabilities and y% of system perfor- 

Permission to make digital or hard copies of all or part of this work for 

personal or classroom use is granted without fee provided that copies are 

not made or distributed for profit or commercial advantage and that copies 

bear this notice and the full citation on the first page. To copy otherwise, to 

republish, to post on servers or to redistribute to lists, requires prior specific 

permission and/or a fee. 

AAMAS’05, July 25-29, 2005, Utrecht, Netherlands. 

Copyright 2005 ACM 1-59593-094-9/05/0007 ...$5.00. 

mance in the face of z% of infrastructure loss and wartime loads 

(x, y, z are user-defined) [1]. 

We address a piece of the survivability problem by building an 

autonomous performance control framework for the DMAS drawing 

on the idea of composing the bigger society of smaller building 

blocks (i.e. agent communities) [3]. Identifying data-flowsinthe 

agent network (similar to [4]) and utilizing the network’s servicelevel 

attributes such as delays, utilization and response times as a 

basis for its utility (like in [5]) we build a self-optimizing framework 

for DMAS. We believe that by using queueing theory we can 

analyze data-flows within the agent community as a network of 

queues with greater granularity in terms of processing delays and 

network latencies and also capitalize on using a building block approach 

by restricting the model to the community. We contribute 

by engineering a queueing theory based adaptation (control) framework 

to enhance the performance of the application layer, which 

inherently can be visualized as residing over the infrastructure (logical 

layer or middle-ware) and the physical layer (resources such as 

CPU, bandwidth). 

2. FRAMEWORK ARCHITECTURE 

Building on the ideas of high-level system specifications (or Tech- 

Specs) and utilizing queueing network models (QNMs) for performance 

estimation as in [2] we build a real-time framework for 

application-level survivability. This framework is represented in 

Figure 1 and consists of activities, modules, knowledge repositories 

and information flow through a distributed collection of agents. 

2.1 Architecture Overview 

When the DMAS is stressed by an amount S by the underlying 

layers (due to under-allocation of resources) and the environment 

(due to increased workloads during wartime conditions), the 

DMAS Controller has to examine all its performance-related variables 

from set X and the current overall performance P in order 

to adapt. The variables that need to be maintained are specified in 

the TechSpecs and may include delays, time-stamps, utilization and 

their statistics. They are collected in a distributed fashion through 

the measurement points MP which are “soft” storage containers 

residing inside the agents and contain information on what, where 

and how frequently they should be measured. The DMAS Controller 

knows the set of flows F that traverse the network and the set 

of packet types T from the TechSpecs. With {F, T, X, C}, where 

C is a suggestion from the DMAS Controller, the Model Builder 

can select a suitable queueing model template Q. The Control Set

O, U 

Stresses 

Physical/Infrastructure Layer 

X 

F,T 

Se 

TechSpecs 

P 

DB 

MP 

OS 

Model Builder 

Q 

P,X 

C 

DMAS 

Controller 

S,P,O 

Control Set 

Evaluator 

O',X' 

Queueing 

Model 

P' 

Stresses 

Application / User 

U*,O* 

U' 

Sa 

Utility 

Calculator 

Figure 1: Architecture Overview 

Evaluator knows the current operating mode (opmode) set O as 

well as the set of possible opmodes, OS from TechSpecs. To evaluate 

the performance due to a candidate opmode set O ′ , the Control 

Set Evaluator uses the Queueing Model with a scaled set of operating 

conditions X ′ . Once the performance P ′ 

is estimated by 

the Queueing Model it can be cached in the performance database 

PDB and then sent to the Utility Calculator. The Utility Calculator 

computes the domain utility due to (O ′ ,P ′ ) and caches it in 

the utility database, UDB. Subsequently, the optimal opmode set 

O ∗ is identified and sent to the DMAS Controller. The functional 

units of the architecture are distributed but for each community that 

forms part of a MAS society, O ∗ will be calculated by a single 

agent. We now examine the capabilities of the framework. 

2.1.1 Self-Monitoring Capability 

TechSpecs acts as a distributed structure that contains meta-data 

about all variables, X, that have to be monitored in different portions 

of the community. The data/statistics collected in a distributed 

way, is then aggregated to assist in control alternatives by the toplevel 

controller that each community possesses. Each agent can 

look up its own TechSpecs and from time-to-time forward a measurement 

to its superior. The superior can analyse this information 

(eg. calculate statistics such as delay, delay-jitter) and/or add to this 

information and forward it again. 

2.1.2 Self-Modeling Capability 

One of the key features of this framework is that it has the capability 

to choose a type of model for representing itself for the 

purpose of performance evaluation. The system is equipped with 

several queueing model templates that it can utilize to analyze the 

system configuration with. The inputs to the Model Builder are the 

flows that traverse the network (F ), the types of packets (T ) and 

the current configuration of the network. Given we know that there 

are n agents interconnected in a hierarchical fashion, this unit represents 

the information in the required template format (Q) which 

is subsequently used to analyze the current performance. 

O* 

U 

DB 

2.1.3 Self-Evaluating Capability 

The evaluation capability allows the MAS to examine its own 

performance under a given set of plausible conditions. This prediction 

of performance is used for the elimination of control alternatives 

that may lead to instabilities. Given that a variety of tasks 

traverse the heterogeneous network of agents in predefined routes 

(called flows), the processing and wait times of tasks at various 

points in the network are not alike because of dissimilar configurations, 

resource availabilities and/or environmental stresses. Under 

these conditions, performance is evaluated in terms of end-to-end 

delays for the “sense-plan-respond” cycles. 

2.1.4 Self-Controlling Capability 

Since tasks can be processed at various pre-defined qualities, opmodes 

allow for trading-off quality of service (task quality) for performance 

(end-to-end response time). The available resources fluctuate 

depending on stresses S = S e +S a, where S e are the stresses 

from the environment (i.e. multiple contending applications) and 

S a are the application stresses (i.e. increased tasks). Using current 

measured performance P and the measured stress S the DMAS 

Controller relates the overall utility (U) asU(P, S) = P w nx n 

where x n is the actual utility component and w n is the associated 

weight specified by the user. To adjust P to get the best achievable 

utility under S, the following is done. Since P depends on 

O, which is a vector of opmodes collected from the community, we 

can use the QNM to find O ∗ and hence P ∗ that maximizes U(P, S) 

for a given S from within the set OS. In words, we find the vector 

of opmodes (O ∗ ) that maximizes domain utility at current S. The 

utility computation is performed in the Utility Calculator module 

using a learned utility model based on UDB. 

3. CONCLUSIONS 

We combined queueing analysis and application-level control to 

engineer a generic framework that is capable of self-optimizing 

its domain-specific utility to assure application-level survivability. 

While application-level adaptivity yields improvement in utility further 

gains are possible by leveraging underlying layers. 

4. ADDITIONAL AUTHORS 

Additional Authors: Natarajan Gautam (Pennsylvania State University, 

email: ngautam@psu.edu), Wilbur Peng and Vikram 

Manikonda (IAI Inc., email: wpeng,vikram@i-a-i.com), Marshall 

Brinn (BBN Technologies, email: mbrinn@bbn.com) and 

Mark Greaves (DARPA IXO, email: mgreaves@darpa.mil). 

5. REFERENCES 

[1] M. Brinn and M. Greaves. Leveraging agent properties to 

assure survivability of distributed multi-agent systems. 

Proceedings of the Second Joint Conference on Autonomous 


[2] N. Gnanasambandam, S. Lee, N. Gautam, S. R. T. Kumara, 

W. Peng, V. Manikonda, M. Brinn, and M. Greaves. Reliable 

mas performance prediction using queueing models. IEEE 

Multi-agent Security and Survivabilty Symposium, 2004. 

[3] H. Jung and M. Tambe. Performance models for large scale 

multi-agent systems: Using distributed pomdp building 

blocks. Proceedings of the Second Joint Conference on 

Autonomous Agents and Multi-Agent Systems, July 2003. 

[4] O. F. Rana and K. Stout. What is scalabilty in multi-agent 

systems? Proceedings of the Fourth International Conference 

on Autonomous Agents, 2000. 

[5] G. Tesauro, D. M. Chess, W. E. Walsh, R. Das, I. Whalley, 

J. O. Kephart, and S. R. White. A multi-agent systems 

approach to autonomic computing. Autonomous Agents and 

Multi-Agent Systems, 2004.

Manuscript for IEEE Transactions on Automatic Control 1 

ADAPTIVE CONTROL FOR LARGE-SCALE INFORMATION NETWORKS THROUGH 

ALTERNATIVE ALGORITHMS TO SUPPORT SURVIVABILITY * 

Seokcheon Lee † and Soundar Kumara ‡ 

†‡ Department of Industrial & Manufacturing Engineering, The Pennsylvania State University, 


† Phone: 814-863-4799; Fax: 814-863-4745; E-mail: stonesky@psu.edu 

‡ Corresponding author. Phone: 814-863-2359; Fax: 814-863-4745; E-mail: skumara@psu.edu 

ABSTRACT 

As modern networks can be easily exposed to various adverse events such as malicious 

attacks and accidental failures, there is a need to study their survivability. We study a large-scale 

information network composed of distributed software components linked together through a 

task flow structure. The service provided by the network is to produce a global solution to a 

given problem, which is an aggregate solution of partial solutions of individual tasks. Quality of 

service of the network is determined by the value of global solution and the time taken for 

generating global solution. In this paper we design an adaptive control mechanism along the 

lines of model predictive control to support the survivability of such networks by utilizing 

alternative algorithms. To address adaptivity we model stress environment by quantifying 

resource availability through sensors. We build a mathematical programming model with the 

resource availability incorporated, which predicts quality of service as a function of alternative 

algorithms. The programming model is decentralized through an auction market without any 

degradation of the solution optimality. By periodically opening the auction market, the system 

can achieve desirable performance adaptive to changing stress environments while assuring 

scalability property. We verify the designed control mechanism empirically. 

Key Words: Adaptive control, survivability, alternative algorithms, scalability 

* This work is supported in part by DARPA under Grant MDA 972-01-1-0038.



Critical infrastructures become increasingly dependent on networked systems in many 

domains for automation or organizational integration. Though such infrastructure can improve 

the efficiency and effectiveness, these systems can be easily exposed to various adverse events 

such as malicious attacks and accidental failures [1]. Two metrics, namely survivability and 

scalability, can be used to determine the efficiency and effectiveness of these systems. 

Survivability is defined as “the capability of a system to fulfill its mission, in a timely manner, in 

the presence of attacks, failure, or accidents” [2]. One promising way to achieve survivability is 

through adaptivity: changing the system behavior to achieve the system goal in response to the 

changing environment [3]. But, unpredictable adaptation can sometimes result in worse 

performance than without adaptation [4]. Scalability is defined as: “the ability of a solution to 

some problem to work when the size of the problem increases” (From Dictionary of Computing 

at http://wombat.doc.ic.ac.uk). As the size of networked systems grows, scalability becomes a 

critical issue when developing practical software systems [5]. 

As software systems grow larger and more complex, component technology has become one 

of the important research topics in the computing community [6][7]. A component is a reusable 

program element, with which developers can build systems needed by simply defining their 

specific roles and wiring them together. In networks with component-based architecture, each 

component is highly specialized for specific tasks. Another emerging technology is adaptive 

software [8][9]. Adaptive software has alternative algorithms for the same numerical problem 

and a switching function for selecting the best algorithm in response to environmental changes. 

As modern operating environments are highly dynamic, adaptive software becomes an important 

tool to achieve portable high performance.


We study a large-scale information network, which is composed of distributed software 

components linked together through a task flow structure. A problem given to the network is 

decomposed in terms of root tasks for some components and those tasks are propagated through 

a task flow structure to other components. As a problem can be decomposed with respect to 

space, time, or both, a component can have multiple root tasks that can be considered 

independent and identical in their nature. The service provided by the network is to produce a 

global solution to a given problem, which is an aggregate of partial solutions of individual tasks. 

Each component can have alternative algorithms to process a task which trade off processing 

time and value of partial solution. Quality of Service (QoS) of the network is determined by the 

value of global solution and time for generating global solution (i.e., completion time). 

Survivability of the network is the capability to provide high QoS in the presence of adverse 

events such as malicious attacks and accidental failures. In this paper we design an adaptive 

control mechanism to support the survivability of such networks by utilizing alternative 

algorithms. 

The organization of this paper is as follows. In Section 2 we discuss problem domain and in 

Section 3 formally define the problem in detail. We design control mechanism in Sections 4 

through 7 and show empirical results in Section 8. Finally, we discuss implications and possible 

extensions of our work in Section 9. 

2. Problem domain 

The networks we study represent distributed and component-based architectures for providing 

a solution to a given problem. A problem is decomposed in terms of root tasks and solved by 

distributed components through a task flow structure. As a problem can be decomposed with


respect to space, time, or both, a component can have multiple root tasks that can be considered 

independent and identical in their nature. When the size of a problem becomes large, the size of 

the network as well as the number of tasks for each component can be large. One can imagine 

wide range of scientific and engineering problems that can be solved with such architectures. 

Cougaar (Cognitive Agent Architecture: http://www.cougaar.org) developed by DARPA 

(Defense Advanced Research Project Agency), is such an architecture for building large-scale 

multi-agent systems. Recently, there have been efforts to combine the technologies of agents and 

components to improve building large-scale software systems [10]-[12]. While component 

technology focuses on reusability, agent technology focuses on processing complex tasks as a 

community. Cougaar is in line with this trend. In Cougaar a software system comprises of agents 

and an agent of components (called plugins). The task flow structure in those systems is that of 

components as a combination of intra-agent and inter-agent task flows. As the agents in Cougaar 

can be distributed both from geographical and information content sense, the networks 

implemented in Cougaar have distributed and component-based architecture. 

UltraLog (http://www.ultralog.net) networks are military supply chain planning systems 

implemented in Cougaar [13]-[17]. Each agent in these networks represents an organization of 

military supply chain and has a set of components specialized for each functionality (allocation, 

expansion, inventory management, etc) and class (ammunition, water, fuel, etc). The objective of 

an UltraLog network is to provide an appropriate logistics plan for a given military operational 

plan. A logistics plan is a global solution which is an aggregate of individual schedules built by 

components. An operational plan is decomposed into logistics requirements of each thread for 

each agent, and a requirement is further decomposed into root tasks (one task per day) for a 

designated component. As a result, a component can have hundreds of root tasks depending on


the horizon of an operation and thousands of tasks to process as the root tasks are propagated. As 

the scale of operation increases there can be thousands of agents (tens of thousands of 

components) working together to generate a logistics plan. The system makes initial planning 

and continuous replanning to cope with logistics plan deviations or operational plan changes. 

Initial planning and replanning are the instances of the current research problem. 

QoS of these networks is determined by the quality of logistics plan (value of solution) and 

(plan) completion time. These two metrics directly affect the performance of an operation. As the 

networks are working in a military environment, they are especially vulnerable to malicious 

attacks and accidental failures. Now, the question is how can we make this system survivable to 

generate high quality logistics plans in a timely manner in the presence of such adverse events? 


In this section we formally define the problem by detailing network configuration, control 

action, and stress environment. We focus on computational CPU resources assuming that the 

system is computation-bounded. 

3.1 Network configuration 

A network is composed of a set of components A and each component resides in its own 

machine 1 . Task flow structure of the network, which defines precedence relationship between 

components, is an arbitrary directed acyclic graph. A problem given to the network is 

decomposed in terms of root tasks for some components and those tasks are propagated through 

the task flow structure. Each component processes one of the tasks in its queue (which has root 

1 For simplicity we consider the cases where there is one component in a machine. Though the designed control 

mechanism is also applicable to resource sharing environments, we may need to consider resource allocation in 

addition as will be discussed in Section 9.


tasks as well as tasks from predecessor components) and then sends it to successor components. 

We denote the number of root tasks of component i as rt i . Fig. 1 shows an example network 

composed of four components. Each of A 1 and A 2 has 100 root tasks. A 3 and A 4 have no root 

tasks but they have 200 and 100 tasks respectively from the corresponding predecessors. 

 

 

A 1 

A 3 

100 

0 

 

A 2 A 4 

100 

0 

Fig. 1. An example network 

3.2 Control action 

A component can use one of alternative algorithms to process a task. Different alternatives 

trade off CPU time and value of solution with more CPU time resulting in higher solution value. 

As one can find optimal mixed alternatives, a component has a monotonically increasing convex 

function, say value function, with CPU time as a function of value. We call the value in the 

function as value mode that a component can select as its decision variable. A value function is 

defined with three elements as 

〈 f i ( vi 

), vi(min), 

vi(max)〉 

as shown in Fig. 1. This function indicates 

that component i’s expected CPU time 2 to process a task is f i (v i ) with a value mode v i and v i(min) ≤ 

v i ≤ v i(max) . We assume that components cannot change the mode for a task in process. 

3.3 Stress environment 

Survivability stresses such as malicious attacks and accidental failures, affect the system by 

directly consuming resources or indirectly invoking defense mechanisms as remedies. For 

2 The distribution of CPU time can be arbitrary though we use only expected CPU time.


example, “Denial of Service” attack consumes resources directly while relevant defense 

mechanism also consumes resources in terms of resistance, recognition, and recovery [1]. We 

consider both survivability stresses and remedies as stress environment from the viewpoint of 

components. The space of stress environment is high-dimensional and also evolving [18][19]. 

But, as we concentrate on CPU resources, a stress environment can be considered as a set of 

threads residing in the machines of the network and sharing resources with components. The 

threads, say stressors, may have admission to access resources or be stealing resources without 

admission. 

3.4 Problem definition 

The service provided by the network is to produce a global solution to a given problem, which 

is an aggregate of partial solutions of individual tasks. QoS of the network is determined by the 

value of global solution and the cost of completion time. The value of global solution is the 

summation of partial solution values, and the cost of completion time is determined by a cost 

function CCT(T) which is a monotonically increasing function with completion time T. We 

assume that the solution values and cost are represented in a common unit 3 . Consider v d i as the 

value mode used to process d th task by component i and e i the number of tasks processed by 

component i to the completion. Then, the control objective is to maximize QoS by utilizing 

alternative algorithms (v) as in (2). As stated earlier, we design an adaptive control mechanism to 

achieve the objective for supporting the survivability of large-scale information networks. 

arg max 

v 

e 

i 

∑∑ 

i∈ A d = 1 

v 

d 

i 

− CCT(T ) 

(2) 

3 Relative importance can be considered by scaling the functions and it results in the same function structures.


4. Overall control procedure 

There are two representative optimal control approaches in dynamic systems: Dynamic 

Programming (DP) and Model Predictive Control (MPC). Though DP gives optimal closed-loop 

policy it has inefficiencies in dealing with large-scale systems especially when systems are 

working in finite time horizon [20]-[22]. In MPC, for each current state, an optimal open-loop 

control policy is designed for finite-time horizon by solving a static mathematical programming 

model [23]-[26]. The design process is repeated for the next observed state feedback forming a 

closed-loop policy reactive to each current system state. Though MPC does not give absolutely 

optimal policy in stochastic environments, the periodic design process alleviates the impacts of 

stochasticity and it is easy to adapt to new contexts by explicitly handling objective function or 

constraints. 

Considering the characteristic of the current problem, we choose MPC framework. Our 

networks are large-scale working in finite time horizon and need to adapt to unpredictable stress 

environment. Therefore, under MPC framework, we develop an adaptive control mechanism as 

depicted in Fig. 2. First, to address adaptivity we model stress environment by quantifying 

resource availability through sensors. Second, we build a mathematical programming model with 

the resource availability incorporated, which predicts QoS as a function of alternative algorithms. 

Third, we provide an auction market as a decentralized coordination mechanism for solving the 

programming model. By periodically opening the auction market, the system can achieve 

desirable performance adaptive to changing stress environment while assuring scalability 

property. We define sensors and build a mathematical programming model in Section 5, and 

refine it based on stability analysis in Section 6. The refined programming model is decentralized 

in Section 7.


Stress 

Environment Modeling 

Sensor Design 

Sensor 

Component 

Sensor 

Component 

Sensor 

Component 

Mathematical 

Programming 

Periodic Auctioning 

Auction 

Decentralized 

Coordination 

Fig. 2. Overall control procedure 

5. Mathematical programming model 

In this section we define sensors and build a mathematical programming model under MPC 

framework. 

5.1 Sensors 

Each component monitors its operating environment through a sensor. The sensor measures 

resource availability MRA i (t), which is defined as available fraction of a resource when a 

component i requests that resource in the last control period at control point t. There are two 

quantities to extract this measurement, which are request time and execution time. Request time 

is the duration for which the component requests resource or equivalently queue length 

(including a task in service) is more than zero. Execution time is the duration for which the 

component utilizes the resource. If control period is SW, resource sensor calculates MRA i (t) as:


( t MRA i 

execution time in ( t − SW ,t ) 

) = . (3) 

request time in ( t − SW ,t ) 

5.2 Mathematical programming model 

A component can estimate its resource availability in the future by using observed resource 

availability in the past. So, by incorporating the estimation, the component can induce service 

time per task as a function of value mode as: 

f ( v ) / MRA ( t ) . (4) 

i 

i 

Now, consider current time as t and estimate the completion time T by assuming that each 

component uses a mode common to all the tasks (i.e. pure strategy). We will discuss the 

optimality of the pure strategy later in this subsection. 

In a task flow structure where each component processes only one task after its predecessors 

complete their tasks, the completion time will be the length of the longest path (i.e., critical path) 

as widely studied in project management literature. However, as the number of tasks increases, a 

bottleneck component, which has maximal total service time, becomes dominating the 

completion time. As each component in our networks can have large number of tasks to process 

rather than just one, the completion time T can be estimated as: 

i 

T − t ≈ Max [ R ( t ) + L ( t ) f ( v )] / MRA ( t ) , (5) 

i∈A 

i 

in which R i (t) denotes remaining CPU time for a task in process and L i (t) the number of 

remaining tasks excluding a task in process. After identifying initial number of tasks L i (0) as in 

(6) where i denotes the immediate predecessors of component i, each component updates it by 

counting down as they process tasks. 

i 

∑ 

a∈i 

i 

i 

L i (0 ) = rti 

+ La(0 

) 

(6) 

i


So, given completion time T it is optimal for each component i to select a mode by the 

following: 

Max L ( t ) 

i v i 

(7) 

subject to 

[ i i i i 

i 

R ( t ) + L ( t ) f ( v )] / MRA ( t ) ≤ T − t . (8) 

Consequently, the programming model can be formulated in a straightforward way as in (9), 

named naïve decision model. The model maximizes QoS by trading off the value of solution and 

the cost of completion time. 

 

Naïve decision model 

Max 

s.t. 

∑ 

i∈A 

[ R ( t ) + L ( t ) f ( v )] / MRA( t ) ≤ T − t 

v 

L ( t )v 

i 

i 

i(min) 

≤ v 

i 

i 

− CCT(T ) 

i 

≤ v 

i 

i(max) 

i 



(9) 

The naïve decision model maximizes QoS as if all the tasks of each component are available 

in its queue at current time t. That is, a network under the naïve decision model can achieve a 

performance close to the optimal performance of an ideal network with maximal task availability 

when L i (t)s are large. As any mixed strategy (i.e. using different modes for processing tasks) 

cannot perform better in the ideal network due to the convexity of value functions, it is optimal 

for each component to use a pure strategy. We will refine the model so that it can be applicable 

even though L i (t)s are small in the next section.


6. Model refinement 

In this section we analyze system behavior under the naïve decision model and refine it to 

eliminate undesirable behavioral properties. 

6.1 Analysis of system behavior 

To analyze the system behavior under the naïve decision model, we made experimentation 

using discrete-event simulation. There are five components in the system linked serially as in 

Fig. 3. Component A 1 in the lowest position is assigned 100 root tasks. Components have a 

common deterministic value function and the cost of completion time is a linear increasing 

function as indicated in the figure. There is no stress in the system and components measure 

MRA i (t) equal to 1 all the time. The system makes decision every 100 time units (i.e., SW=100) 

by solving the naïve decision model. 

100 

A 1 

A 2 A 3 A 4 

A 5 

 

CCT(T) = 10T 

Fig. 3. An example network for stability analysis 

Fig. 4 and 5 show the resultant behavior of the system, in which the decisions T * and v * i are 

divergent. The divergent behavior indicates that there is inefficiency in the naïve decision model 

and system performance can be improved if we eliminate this inefficiency. The divergent 

behavior is due to the inaccurate prediction of the naïve decision model. In the example network 

the system (or A 5 ) can complete at T * when A 4 completes before T * . As each component is trying 

to complete at T * without considering its position in the task flow structure, the components


cannot receive tasks in time from their predecessors. This inaccuracy leads to changing the 

decisions in the subsequent decision points resulting in the divergent behavior. 

1150 

1120 

Optimal T 

1090 

1060 

1030 

1000 

0 200 400 600 800 1000 1200 

Time 

Fig. 4. Behavior of T * under naïve decision model 

40.0 

A4 

38.0 

Mode 

36.0 

34.0 

A5 

A3 

32.0 

A2 

30.0 

A1 

0 200 400 600 800 1000 1200 

Time 

Fig. 5. Behavior of v i * under naïve decision model 

6.2 Model refinement 

To stabilize the system behavior we need to reinforce the naïve decision model by taking into 

account the components’ positions in the task flow structure. For this purpose, we define Depth


D i (t) as a quantitative representation of the component’s position. D i (t) is the required time gap 

between the system’s and the component’s completion times at time t. Each component needs to 

complete at less than or equal to T-D i (t) to keep the completion time T. Components without 

successors have depth equal to 0 but components with successors have positive depths. A 

component a can keep its depth if its predecessors’ depth is D a (t) plus its total service time for 

the last arriving tasks in the worst case. So, a component i’s depth to keep the depths of its all 

successors is the maximal of the required depths from its successors represented as: 

D i( t ) = max[ Da( t ) + f a( va 

) / MRAa 

( t )] , (10) 

a∈i 

∑ 

b∈a 

in which i denotes successors of component i and a predecessors of component a. 

Though it is possible to refine the naïve decision model by incorporating the depths as 

variables, the model complexity increases because each component’s constraint will be 

intertwined with the decision variables of all connected components. So, we simply estimate 

components’ depths through the decisions used at the last control point. At each control point 

each successor informs its predecessors of required depths and each predecessor chooses the 

maximal one as its depth. As a result, we can consider the depth as constant rather than variable 

so that the refined model has no increase in complexity. We call the refined model in (11) as 

stable decision model. If we don’t consider the depth, i.e., D i (t)=0, the model becomes equivalent 

to the naïve decision model. Also, the stable decision model becomes an exact CPM/PERT 

formulation as a special case found in project management literature when one consider D i (t) as 

variable.


 

Stable decision model 

Max 

s.t. 

∑ 

i∈A 

[ R ( t ) + L ( t ) f ( v )] / MRA( t ) ≤ T − t − D ( t ) 

v 

L ( t )v 

i 

i 

i(min) 

≤ v 

i 

i 

− CCT(T ) 

i 

≤ v 

i 

i(max) 

i 

i 

for all 

for all 

i ∈ A 

i ∈ A 

(11) 

6.3 System behavior under stable decision model 

To observe system behavior under the stable decision model, we experimented with the 

example network described in Fig. 3. Fig. 6 and 7 show the resultant behavior of the system, in 

which the decisions T * and v * i are stable. The stability indicates that the inefficiency of the naïve 

decision model is removed as a result of improved prediction accuracy. 

1150 

1120 

Optimal T 

1090 

1060 

1030 

1000 

0 200 400 600 800 1000 1200 

Time 

Fig. 6. Behavior of T * under stable decision model


31.0 

30.8 

30.6 

Mode 

30.4 

30.2 

A1 

A2 

A3 

A4 

30.0 

A5 

0 200 400 600 800 1000 1200 

Time 

Fig. 7. Behavior of v i * under stable decision model 

The effects of the stability on performance are shown in Table 1. QoS is improved 

significantly when using the stable decision model. Improved prediction accuracy made the 

system behaving stable and consequently performing better. 

Table 1. The effects of stability on performance 

Decision model 

Naïve 

Stable 

T V QoS T V QoS 

1171 15289 3583 1082 15104 4282 

T: Completion time, V: Value of solution 

7. Decentralization 

The next question is how to decentralize the programming model. Centralized control 

mechanisms scale badly, due to the rapid increase of computational and communicational 

overheads with system size. Single point failure will often lead to failure of the complete system 

leading to a non-robust network. Decentralization can address these issues by distributing the 

computations and communications to multiple entities. In addition to these properties


decentralization will give a byproduct, information security. As we discussed earlier our effort is 

to support survivability. If information is revealed to others directly information security will be 

in question. In this section we decentralize the programming model through an auction market. 

7.1 Two-tier auction market 

There are two popular methods of decentralizing structured programming models: 

decomposition methods and auction/bidding algorithms. Considering the compatible structure of 

the programming model, we decentralize it through a non-iterative auction mechanism, so called 

multiple-unit auction with variable supply [27]. In this auction a seller may be able and willing 

to adjust the supply as a function of bidding. In the programming model we have built, all 

components are coupled with each other. However, the objective function and constraints are 

separable if one variable T is fixed. This characteristic makes it possible to solve the model 

through an auctioning process for T. The completion time T is an unbounded resource and the 

supply can be adjusted as a function of bidding. To design the auction market we define a seller 

which determines T * based on the bids from the components. We call this auction market as 

Two-tier Auctioning Model. 

We define T i as available resource of component i which is required minimally to the amount 

of T i(min) as in (12) and maximally T i(max) as in (13). 

T 

T 

= [ R ( t ) L ( t ) f ( v )] / MRA ( t ) 

(12) 

i (min) i + 

i 

i 

i(min) 

= [ R ( t ) L ( t ) f ( v )] / MRA ( t ) 

(13) 

i (max) i + 

i 

Each component bids to the seller with maximal value as a function of T as in (14). The seller 

decides T * based on the bids by considering CCT(T) as in (15). After the seller broadcasts T * , 

each component selects its optimal value mode in the limits T * as in (16). Though this auctioning 

i 

i(max) 

i 

i


process gives an equivalent solution to the centralized programming model, it gives more 

benefits as communications and computations are distributed to multiple market participants. 

 

Two-tier auctioning model 

Component’s bid 

b (T ) = −∞ 

i 

= L ( t )v 

i 

= L ( t ) f 

i 

i(max) 

−1 

i 

Seller’s decision 

Max ∑bi ( T ) − CCT ( T ) 

i∈A 

Component’s decision 

(T − t − D i( t ))MRA i( t ) − R i( t ) 

( 

) 

L ( t ) 

i 

if T − t − D ( t ) < T 

if T − t − D ( t ) > T 

else 

i 

i 

i(min) 

i(max) 

(14) 

(15) 

v 

* 

i 

= v 

i(max) 

* 

i 

if 

T 

* 

− t − D 

i 

> T 

i(max) 

− 1 (T − t − D i( t ))MRA i( t ) − R i( t ) 

(16) 

= fi 

( 

) else 

L ( t ) 

7.2 Multi-tier auction market 

Though the designed auctioning process is decentralized it incorporates a centralized seller 

which needs to coordinate all the components. As the centralized auction can still exhibit 

problems in terms of scalability and robustness we introduce a multi-tier auction market. 

Suppose there are two component groups a and b with a ⊂ b, and denote S a as a set of optimal 

completion time solutions of group a and S b of group b. Then, the maximal of S b is greater than 

or equal to the maximal of S a as in (17). 

max S 

a 

≤ max S if a ⊂ b 

(17) 

b 

Proof. Suppose it is not true, that is, T a =max S a > T b =max S b . Then, for group b,


∑ 

i∈b 

≡ 

≡ [ 

b (T 

i 

∑ 

i∈a 

∑ 

i∈a 

i 

i 

b 

b (T 

) − CCT(T 

b 

b (T 

) + 

b 

∑ 

i∉a 

And, for group a, 

∑ 

i∈a 

b (T 

i 

b 

) > 

b 

) − CCT(T 

i 

b 

b (T 

) − CCT(T 

b 

b 

) ≤ 

∑ 

i∈b 

b (T 

) − CCT(T 

i 

)] − [ 

∑ 

i∈a 

∑ 

i∈a 

b (T 

So, the inequality in (20) should hold. 

∑ 

i∉a 

a 

∑ 

i∉a 

b 

i 

a 

) − CCT(T 

b 

i 

) > 

b (T 

a 

a 

∑ 

i∈a 

i 

a 

) 

b (T 

a 

) + 

) − CCT(T 

) − CCT(T 

a 

a 

∑ 

i∉a 

)] > 

b (T 

i 

∑ 

i∉a 

a 

i 

) − CCT(T 

b (T 

a 

) − 

∑ 

i∉a 

a 

b (T 

i 

) 

b 

. (18) 

) 

) . (19) 

b i (T ) < bi 

(T ) 

(20) 

But, this inequality is not possible because b i (T) is an increasing function with T. 

 

Through this property the two-tier auctioning model can be transformed into a multi-tier 

model, in which there are multiple brokers arbitrating components and the seller. A broker bids 

to its superior broker or seller for T≥T s(m) as in (21), in which T s(m) denotes the maximal of 

optimal completion time solutions of a group s(m) and s(m) subordinate components and brokers 

of broker m. In this way, the search space becomes reduced as the bidding process goes to the 

superior. In this multi-tier auctioning model communications and computations are more 

distributed through the brokers overcoming the problems of the two-tier model. 

 

Multi-tier auctioning model 

Broker’s bid 

b 

m 

(T ) = −∞ 

= 

∑ 

a 

a∈s( 

m ) 

b (T ) 

if T < max{arg max 

else 

T 

∑ 

a 

a∈s( 

m ) 

b (T ) − CCT(T )} 

(21)


8. Empirical results 

We ran several experiments using discrete-event simulation to validate the designed control 

mechanism. Though we use a small network in the experimentation for validation purpose, the 

decentralized model, especially, can handle much larger networks. 

8.1 Experimental design 

The experimental network is composed of fifteen components with a tree structure as shown 

in Fig. 8. Each component in the lowest position has 200 root tasks. Also, all the components 

have a common linear value function and the cost of completion time is linear increasing 

function as indicated in the figure. 

A 1 

A 2 

 

CCT(T) = 4T 

A 3 

A 15 

A 4 A 5 

A 8 A 9 A 10 A 11 

A 6 A 7 

A 12 A 13 A 14 

200 200 200 200 200 200 200 200 

Fig. 8. Experimental network configuration 

We set up four different experimental conditions as shown in Table 2. There can be stressors 

which share resources with components. We assign weight w i to a component i and w i ′ to a 

stressor sharing resource with component i. A stressor, which has infinite work (continuously 

requiring resource), can impose different levels of stress on the component directly by changing 

w i ′. When it is zero there is no stress, and as it increases the stress level increases. We implement 

the stress environment by using a weighted round-robin scheduling, in which CPU time received


by each thread in a round is equal to its assigned weight. Also, the distribution of CPU time can 

be deterministic or stochastic. While using stochastic value function we repeat 5 experiments. 


Condition Stress f i (v i ) 

Con1 Unstressed Deterministic 

Con2 Unstressed Exponential 

Con3 Stressed Deterministic 

Con4 Stressed Exponential 

w i =0.1for all i∈A, w A4 ′ =1 in 500≤t≤1000 for Con3 and Con4 

Initial value mode: (2, 5, 5, 3 for A 4 to A 15 ) 

We use four different control policies for each experimental condition as shown in Table 3. 

FL and FH use fixed value modes over time. AC-X policies represent the adaptive control 

mechanism we have designed. In AC-N the system is controlled under naïve decision model 

while in AC-S under stable decision model. When using adaptive policies the system makes 

decision every 100 time units (i.e., SW=100). 

Table 3. Control policies used for experimentation 

Control policy 

FL 

FH 

AC-N 

AC-S 

Description 

Fixed with lowest value mode 

Fixed with highest value mode 

Adaptive control under naïve decision model 

Adaptive control under stable decision model 

8.2 Results 

Numerical results from the experimentation are summarized in Table 4. The adaptive control 

policies show significant advantages compared to non-adaptive ones in all different conditions. 

But, the benefit of AC-S is not clear in the numerical results. Though AC-S outperforms AC-N 

in deterministic environments, AC-N outperforms AC-S in stochastic environments. This means 

that AC-S cannot guarantee better performance especially in stochastic environments. But, we


can say that AC-S is a robust policy to keep the system from behaving divergently and degrading 

performance significantly as have shown in the previous stability analysis. 


Control Policy 

FL FH AC-N AC-S 

Condition T V QoS T V QoS T V QoS T V QoS 

Con1 1656 13558 6934 6313 30643 5391 1663 22898 16245 1656 22884 16259 

Con2 1652 13547 6942 6302 30643 5435 1723 22982 16089 1728 22959 16046 

Con3 1656 13558 6934 6313 30643 5391 1966 23401 15539 1965 23403 15542 

Con4 1652 13547 6942 6371 30643 5159 2024 23495 15401 2007 23406 15376 

T: Completion time, V: Value of solution 

Fig. 9 shows the behavior of T * under adaptive control policies in unstressed environments. In 

the deterministic environment the system behaves stable under AC-S while diverging under AC- 

N. But, it is not valid in the stochastic environment because the system seems more stable under 

AC-N. This might explain partially why AC-S does not perform better in stochastic 

environments. 

1720 

1710 

Optimal T 

1700 

1690 

1680 

1670 

1660 

1650 

AC-S (Con2) 

AC-N (Con2) 

AC-S (Con1) 

AC-N (Con1) 

1640 

0 200 400 600 800 1000 1200 1400 1600 1800 

Time 

Fig. 9. Behavior of T * in unstressed environments


The system controlled by adaptive control policies is naturally adaptive to changing 

environments as components monitor their environments and incorporate them into the decision 

process. As shown in Fig. 10 and 11 for deterministic case and Fig. 12 and 13 for stochastic case, 

when environment changes the system adapts to the new environment. 

4000 

3000 

Optimal T 

2000 

1000 

0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

Time 

Fig. 10. Adaptive behavior of T * in deterministic environment (Con3) 

6.0 

A 8 

A 4 

5.0 

A 2 

Mode 

4.0 

3.0 

A 1 

2.0 

1.0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

Time 

Fig. 11. Adaptive behavior of v i * in deterministic environment (Con3)


4000 

3000 

Optimal T 

2000 

1000 

0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

Time 

Fig. 12. Adaptive behavior of T * in stochastic environment (Con4) 

6.0 

A 8 

A 4 

5.0 

A 2 

4.0 

A 1 

Mode 

3.0 

2.0 

1.0 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 

Time 

Fig. 13. Adaptive behavior of v i * in stochastic environment (Con4) 

9. Conclusions 

A typical information network emerges as a result of automation or organizational integration, 

which is large-scale with distributed and component-based architecture. In this paper we 

developed an adaptive control mechanism to support the survivability of such networks by


utilizing alternative algorithms. We designed an auction market which coordinates the 

components of a network. Each component bids based on its measured resource availability and 

optimal decisions are made through a multi-tier auctioning process. By periodically opening the 

auction market, the system can achieve desirable performance adaptive to changing stress 

environment while assuring scalability property. 

Our work can be extended by considering more general network configurations. There can be 

multiple components in a machine sharing resources together. In such resource sharing 

environments, we have an opportunity to improve system performance by appropriately 

allocating resources. Though the designed control mechanism is applicable to the resource 

sharing environments, it would be desirable to explore an appropriate control mechanism by 

incorporating the resource allocation in addition. 

References 

[1] S. Jha and J. M. Wing, “Survivability analysis of networked systems,” in Proc. 23rd Int. 

Conf. Software engineering, 2001, pp. 307-317. 

[2] R. Ellison, D. Fisher, H. Lipson, T. Longstaff, and N. Mead, “Survivable network systems: 

An emerging discipline,” Software Engineering Institute, Carnegie Mellon University, 

Pittsburg, PA, Tech. Rep. CMU/SEI-97-153, 1997. 

[3] J. E. Eggleston, S. Jamin, T. P. Kelly, J. K. MacKie-Mason, W. E. Walsh, and M. P. 

Wellman, “Survivability through market-based adaptivity: The MARX project,” in Proc. 

DARPA Information Survivability Conference and Exposition, 2000, pp. 145-156. 

[4] S. Bowers, L. Delcambre, D. Maier, C. Cowan, P. Wagle, D. McNamee, A. L. Meur, and H. 

Hinton, “Applying adaptation spaces to support quality of service and survivability,” in


Proc. DARPA Information Survivability Conference and Exposition, 2000, pp. 271-283. 

[5] O. F. Rana and K. Stout, “What is scalability in multi-agent systems?,” in Proc. 4th Int. 

Conf. Autonomous Agents, 2000, pp. 56-63. 

[6] B. Meyer, “On to components,” IEEE Computer, vol. 32, no. 1, pp. 139-140, 1999. 

[7] P. Clements, “From subroutine to subsystems: Component-based software development,” in 

Component Based Software Engineering, A. W. Brown, Ed. IEEE Computer Society Press, 

pp. 3-6, 1996. 

[8] M. O. McCracken, A. Snavely, and A. Malony, “Performance modeling for dynamic 

algorithm selection,” in Proc. Int. Conf. Computational Science, 2003, pp. 749-758. 

[9] P. Oreizy, M. M. Gorlick, R. N. Taylor, D. Heimbigner, G. Johnson, N. Medvidovic, A. 

Quilici, D. S. Rosenblum, and A. L. Wolf, “An architecture-based approach to self-adaptive 

software,” IEEE Intelligent Systems, vol. 14, no. 3, pp. 54-62, 1999. 

[10] F. M. T. Brazier, C. M. Jonker, and J. Treur, “Principles of component-based design of 

intelligent agents,” Data and Knowledge Engineering, vol. 41, no. 1, pp. 1-28, 2002. 

[11] H. J. Goradia and J. M. Vidal, “Building blocks for agent design,” in Proc. 4th Int. 

Workshop on Agent-Oriented Software Engineering, 2003, pp. 17-30. 

[12] R. Krutisch, P. Meier, and M. Wirsing, “The AgentComponent approach, combining agents 

and components,” in Proc. 1st German Conf. Multiagent System Technologies, 2003, pp. 1- 

12. 

[13] D. Moore, W. Wright, and R. Kilmer, “Control surfaces for Cougaar,” in Proc. First Open 

Cougaar Conference, 2004, pp. 37-44. 

[14] W. Peng, V. Manikonda, and S. Kumara, “Understanding agent societies using distributed 

monitoring and profiling,” in Proc. First Open Cougaar Conference, 2004, pp. 53-60.


[15] H. Gupta, Y. Hong, H. P. Thadakamalla, V. Manikonda, S. Kumara, and W. Peng, “Using 

predictors to improve the robustness of multi-agent systems: Design and implementation in 

Cougaar,” in Proc. First Open Cougaar Conference, 2004, pp. 81-88. 

[16] D. Moore, A. Helsinger, and D. Wells, “Deconfliction in ultra-large MAS: Issues and a 

potential architecture,” in Proc. First Open Cougaar Conference, 2004, pp. 125-133. 

[17] R. D. Snyder and D. C. Mackenzie, “Cougaar agent communities,” in Proc. First Open 


[18] A. P. Moore, R. J. Ellison, and R. C. Linger, “Attack modeling for information security and 

survivability,” Software Engineering Institute, Carnegie Mellon University, Pittsburg, PA, 

Tech. Note CMU/SEI-2001-TN-001, 2001. 

[19] F. Moberg, “Security analysis of an information system using an attack tree-based 

methodology,” M.S. thesis, Automation Engineering Program, Chalmers University of 

Technology, Sweden, 2000. 

[20] G. Barto, S. J. Bradtke, and S. P. Singh, “Learning to act using real-time dynamic 

programming,” Artificial Intelligence, vol. 72, no. 1-2, pp. 81-138, 1995. 

[21] R. S. Sutton, A. G. Barto, and R. J. Williams, “Reinforcement learning is direct adaptive 

optimal control,” IEEE Control Systems, vol. 12, no. 2, pp. 19-22, 1992. 

[22] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement learning: A survey,” J. 

Artificial Intelligence Research, vol. 4, pp. 237-285, 1996. 

[23] J. B. Rawlings, “Tutorial overview of model predictive control,” IEEE Control Systems, vol. 

20, no. 3, pp. 38-52, 2000. 

[24] M. Morari and J. H. Lee, “Model predictive control: Past, present and future,” Computers 

and Chemical Engineering, vol. 23, no. 4, pp. 667-682, 1999.


[25] M. Nikolaou, “Model predictive controllers: A critical synthesis of theory and industrial 

needs,” in Advances in Chemical Engineering Series, Academic Press, 2001. 

[26] S. J. Qin and T. A. Badgwell, “A survey of industrial model predictive technology,” Control 

Engineering Practice, vol. 11, pp. 733-764, 2003. 

[27] Y. Lengwiler, “The multiple unit auction with variable supply,” Economic Theory, vol. 14, 

no. 2, pp. 373-392, 1999.

1 

Self-Organizing Resource Allocation for 

Minimizing Completion Time in Large-Scale 

Distributed Information Networks 

Seokcheon Lee, Soundar Kumara, and Natarajan Gautam 

Abstract—As information networks grow larger in size due to 

automation or organizational integration, it is important to 

provide simple decision-making mechanisms for each entity or 

groups of entities that will lead to desirable global performance. 

In this paper, we study a large-scale information network 

consisting of distributed software components linked together 

through a task flow structure and design a resource control 

mechanism for minimizing completion time. We define load index 

which represents component’s workload. When resources are 

allocated locally proportional to the load index, the network can 

maximize the utilization of distributed resources and achieve 

optimal performance in the limit of large number of tasks. 

Coordinated resource allocation throughout the network emerges 

as a result of using the load index as global information. To clarify 

the obscurity of “large number of tasks” we provide a quantitative 

criterion for the adequacy of the proportional resource allocation 

for a given network. By periodically allocating resources under 

the framework of model predictive control, a closed-loop policy 

reactive to each current system state is formed. The designed 

resource control mechanism has several emergent properties that 

can be found in many self-organizing systems such as social or 

biological systems. Though it is localized requiring almost no 

computation, it realizes desirable global performance adaptive to 

changing environments. 

Index Terms—Distributed information networks, emergence, 

resource allocation, scalability. 

C 

I. INTRODUCTION 

ritical infrastructures are increasingly becoming dependent 

on networked systems in many domains due to automation 

or organizational integration. The growth in complexity and 

size of software systems is leading to the increasing importance 

of distributed and component-based architectures. Distributed 

computing aims at using computing power of machines 

Manuscript received June 24, 2005. This work was supported in part by 

DARPA under Grant MDA 972-01-1-0038. 

S. Lee is with the Department of Industrial and Manufacturing Engineering, 

The Pennsylvania State University, University Park, PA 16802 USA (phone: 

814-863-4799; fax: 814-863-4745; e-mail: stonesky@psu.edu). 

S. Kumara is with the Department of Industrial and Manufacturing 


USA (e-mail: skumara@psu.edu). 

N. Gautam is with the Department of Industrial and Manufacturing 


USA (e-mail: ngautam@psu.edu). 

connected by a network. When a task requires intensive 

computation, it becomes natural choice to achieve high 

performance. A component is a reusable program element. 

Component technology utilizes the components so that 

developers can build systems needed by simply defining their 

specific roles and wiring them together [1][2]. In networks with 

component-based architecture, each component is highly 

specialized for specific tasks. 

We study a large-scale information network (with respect to 

the number of components as well as machines) comprising of 

distributed software components linked together through a task 

flow structure. A problem given to the network is decomposed 

in terms of root tasks for some components and those tasks are 

propagated through a task flow structure to other components. 

As a problem can be decomposed with respect to space, time, or 

both, a component can have multiple root tasks that can be 

considered independent and identical in their nature. The 

service provided by the network is to produce a global solution 

to the given problem, which is an aggregation of the partial 

solutions of individual tasks. Quality of Service (QoS) of the 

network is determined by the time for generating the global 

solution, i.e. completion time. For a given topology, 

components are sharing resources and the network can control 

its behavior through resource allocation. In specific, we address 

allocating resources of each machine to the components 

residing on that machine. In this paper we develop a resource 

control mechanism of such networks for minimizing 

completion time. 

Many self-organizing systems such as social and biological 

systems exhibit emergent properties. Though entities act with a 

simple mechanism without central authority, these systems are 

adaptive and desirable global performance can often be 

realized. The control mechanism designed in this paper has 

such properties so that it can be applicable to large-scale 

networks working in a dynamic environment. Scalability, 

defined as “the ability of a solution to some problem to work 

when the size of the problem increases” (From Dictionary of 

Computing at http://wombat.doc.ic.ac.uk), becomes a critical 

issue when developing practical software systems as the size of 

networks grows [3]. We also provide a criterion by which one 

can evaluate if the emergent properties hold for a given 

network. 

The organization of this paper is as follows. In Section II we

2 

discuss problem domain and in Section III formally define the 

problem in detail. After designing resource control mechanism 

in Sections IV and V, we show empirical results in Section VI. 

Finally, we conclude our work in Section VII. 

replanning to cope with logistics plan deviations or operational 

plan changes. Initial planning and replanning are the instances 

of the current research problem. Plan completion time of such 

networks directly affects the performance of military operation. 

II. PROBLEM DOMAIN 

The networks we study represent distributed and 

component-based architectures for providing a solution for a 

given problem. A problem is decomposed in terms of root tasks 

and solved by distributed components through a task flow 

structure. As a problem can be decomposed with respect to 

space, time, or both, a component can have multiple root tasks 

that can be considered independent and identical in their nature. 

When the size of a problem becomes large, the size of the 

network as well as the number of tasks for each component can 

be large. One can imagine wide range of scientific and 

engineering problems that can be solved with such 

architectures. 

Cougaar (Cognitive Agent Architecture: 

http://www.cougaar.org) developed by DARPA (Defense 

Advanced Research Project Agency), is such an architecture 

for building large-scale multi-agent systems. Recently, there 

have been efforts to combine the technologies of agents and 

components to improve building large-scale software systems 

[4]-[6]. While component technology focuses on reusability, 

agent technology focuses on processing complex tasks as a 

community. Cougaar is in line with this trend. In Cougaar a 

software system comprises of agents and an agent of 

components (called plugins). The task flow structure in those 

systems is that of components as a combination of intra-agent 

and inter-agent task flows. As the agents in Cougaar can be 

distributed both from geographical and information content 

sense, the networks implemented in Cougaar have distributed 

and component-based architecture. 

UltraLog (http://www.ultralog.net) networks are military 

supply chain planning systems implemented in Cougaar 

[7]-[11]. Each agent in these networks represents an 

organization of military supply chain and has a set of 

components specialized for each functionality (allocation, 

expansion, inventory management, etc) and class (ammunition, 

water, fuel, etc). The objective of an UltraLog network is to 

provide an appropriate logistics plan for a given military 

operational plan. A logistics plan is a global solution which is 

an aggregate of individual schedules built by components. An 

operational plan is decomposed into logistics requirements of 

each thread for each agent, and a requirement is further 

decomposed into root tasks (one task per day) for a designated 

component. As a result, a component can have hundreds of root 

tasks depending on the horizon of an operation and thousands 

of tasks to process as the root tasks are propagated. As the scale 

of operation increases there can be thousands of agents (tens of 

thousands of components) in hundreds of machines working 

together to generate a logistics plan. 

An UltraLog network makes initial planning and continuous 

III. PROBLEM SPECIFICATION 

In this section we formally define the problem in a general 

form by detailing the network model and resource allocation. 

We concentrate on computational CPU resources assuming that 

the system is computation-bounded. 

A. Network Model 

A network is composed of a set of components A and a set of 

nodes (i.e., machines) N. K n denotes a set of components that 

reside in node n sharing the node’s CPU resource. Task flow 

structure of the network, which defines precedence relationship 

between components, is an arbitrary directed acyclic graph. A 

problem given to the network is decomposed in terms of root 

tasks for some components and those tasks are propagated 

through the task flow structure. Each component processes one 

of the tasks in its queue (which has root tasks as well as tasks 

from predecessor components) and then sends it to successor 

components. We denote the number of root tasks and expected 

CPU time 1 per task of component i as respectively. Fig. 

1 shows an example network in which there are four 

components residing in three nodes. Components A 1 and A 2 

resides in N 1 and each of them has 100 root tasks. A 3 in N 2 and 

A 4 in N 3 have no root tasks, but each of them has 100 tasks from 

the corresponding predecessors, namely A 1 and A 2 

respectively. 

 

A 1 

 

A 3 

N 2 

 

 

A 2 A 4 

N 3 

Fig. 1. An example network. The network is composed of four components in 

three nodes and the performance can depend on the resource allocation of node 

N 1 . 

B. Resource Allocation 

When there are multiple components in a node, the network 

needs to control its behavior through resource allocation. In the 

example network, node N 1 has two components and the system 

performance can depend on its resource allocation to these two 

components. There are several CPU scheduling algorithms for 

allocating a CPU resource amongst multiple threads. Among 

the scheduling algorithms, proportional CPU share (PS) 

scheduling is known for its simplicity, flexibility, and fairness 

[12]. In PS scheduling threads are assigned weights and 

resource shares are determined proportional to the weights 

1 The distribution of CPU time can be arbitrary though we use only expected 

CPU time.

3 

[13]. Excess CPU time from some threads is allocated fairly to 

other threads. There are many PS scheduling algorithms such as 

Weighted Round-Robin scheduling, Lottery scheduling, and 

Stride scheduling [14]-[16]. 

We adopt PS scheduling as resource allocation scheme 

because of its generality in addition to the benefits mentioned 

above. We define resource allocation variable set w = {w i (t): 

i∈A, t≥0} in which w i (t) is a non-negative weight of component 

i at time t. If total managed weight of a node n is ω n , the 

boundary condition for assigning weights over time can be 

described as: 

∑ 

i∈K 

C. Problem Definition 

n 

w ( t ) = ω where w ( t ) ≥ 0 . (1) 

i 

n 

The service provided by a network is to produce a global 

solution to a given problem, which is an aggregate solution of 

partial solutions of individual tasks. QoS is determined by 

completion time taken to generate the global solution. In this 

paper we develop a resource control mechanism to minimize 

the completion time T though resource allocation (w) as in (2). 

arg min 

w 

T 

i 

(2) 

has a set of serial operations and each operation should be 

processed on a specific machine. A job shop scheduling 

problem is sequencing the operations in each machine by 

satisfying a set of job precedence constraints such that the 

completion time is minimized. Our problem can be exactly 

transformed into such a job shop scheduling problem. However, 

scheduling problems are in general intractable. Though the job 

shop scheduling problem is polynomially solvable when there 

are two machines and each job has two operations, it becomes 

NP-hard on the number of jobs even if the number of machines 

or operations is more than two [24][25]. Considering that the 

task flow structure of our networks is arbitrary, our scheduling 

problem is NP-hard on the number of components in general 

and the increase of the number of tasks imposes additional 

complexity. Moreover, there can be large number of nodes in 

our networks. 

Though it is possible to use some available heuristic 

algorithms from the job shop scheduling problem, our 

scheduling problem has a particular characteristic, i.e., the 

number of tasks for each component can be large. Though the 

increase of the number of tasks adds more complexity, it can 

also give us great opportunity to develop an efficient heuristic 

solution. So, we analyze the impacts of the largeness on the 

optimal scheduling in the course of developing a resource 

control mechanism. 

IV. OVERALL SOLUTION METHODOLOGY 

There are two representative optimal control approaches in 

dynamic systems: Dynamic Programming (DP) and Model 

Predictive Control (MPC). Though DP gives optimal 

closed-loop policy it has inefficiencies in dealing with 

large-scale systems especially when systems are working in 

finite time horizon [17]-[19]. In MPC, for each current state, an 

optimal open-loop control policy is designed for finite-time 

horizon by solving a static mathematical programming model 

[20]-[23]. The design process is repeated for the next observed 

state feedback forming a closed-loop policy reactive to each 

current system state. Though MPC does not give absolute 

optimal policy in stochastic environments, the periodic design 

process alleviates the impacts of stochasticity. Considering the 

characteristic of our problem, we choose MPC framework. Our 

networks are large-scale and work in finite time horizon. So, 

we need to build a mathematical programming model. 

The mathematical programming model is essentially a 

scheduling problem formulation. There are a variety of 

formulations and algorithms available for diverse scheduling 

problems in the context of multiprocessor, manufacturing, and 

project management. In general, a scheduling problem is to 

allocate limited resources to a set of tasks to optimize a specific 

objective. One widely studied objective is completion time 

(also called makespan in the manufacturing literature) as in the 

problem we have considered. Though it is not easy to find a 

problem exactly same as ours, it is possible to convert our 

problem into one of the scheduling problems. For example, in a 

job shop, there are a set of jobs and a set of machines. Each job 

V. RESOURCE CONTROL MECHANISM 

In this section we develop a resource control mechanism 

under MPC framework. After exemplifying the effects of 

resource allocation, we develop a resource control mechanism 

by characterizing an optimal open-loop resource allocation 

policy in the limit of large number of tasks, and providing a 

quantitative criterion for the largeness. For theoretical analysis, 

we assume a hypothetical weighted round-robin server for CPU 

scheduling though it is not strictly required in practice as will 

be discussed. The hypothetical server has idealized fairness as 

the CPU time received by each thread in a round is infinitesimal 

and proportional to the weight of the thread. 

A. Effects of Resource Allocation 

The completion time T is the time taken to generate the 

global solution, i.e., to process all the tasks of a network. We 

denote T n as the completion time taken to process all the tasks 

of node n and T i of component i. Then, the relationships as in 

(3) hold. 

T 

= Max T = Max T T = Max T . (3) 

n∈N 

n 

i∈A 

i , n 

i 

i∈K 

n 

A component’s instantaneous resource availability RA i (t) is 

the available fraction of a resource when the component 

requests the resource at time t. Service time S i (t) is the time 

taken to process a task at time t and has a relationship with 

RA i (t) as:

4 

t Si 

t 

∫ + ( ) 

i ) 

t 

When RA i (t) remains constant S i (t) becomes: 

RA ( τ dτ 

= P . (4) 

i 

Pi 

Si ( t) 

= . (5) 

RA ( t) 

Now, consider the example network in Fig. 1. In the network 

only N 1 has the chance to allocate its resource as it has two 

residing components. T N1 is invariant to resource allocation and 

equal to 300 (=100*1+100*2). But, T A1 and T A2 can vary 

depending on the resource allocation of N 1 . When the resource 

is allocated equally to the components, both RA A1 (t) and RA A2 (t) 

are equal to 0.5 initially. As A 1 completes at t=200 

(=100*1/0.5), A 2 starts utilizing the resource fully from then, 

i.e. RA A2 (t)=1 for t≥200. So, A 2 completes 50 tasks at t=200 

(=50*2/0.5) and remaining 50 tasks at t=300 (=200+50*2/1). 

A 3 completes at t=202 (=200+1*2/1) because task inter-arrival 

time from A 1 is equal to its service time. As A 4 ’s service time is 

less than task inter-arrival time (=4) for t≤200, A 4 completes 49 

tasks at t=200 with one task in queue arriving at t=200. From 

t=200 task inter-arrival time from A 2 becomes reduced to 2 

which is less than A 4 ’s service time. So, tasks become 

accumulated till t=300 and A 4 completes at t=353 

(=200+51*3/1). In this way we trace exact system behavior 

under three resource allocation strategies as shown in Fig. 2. 

RA 

1 

4/5 

2/3 

1/2 

1/3 

1/5 

1:1 

1:2 

1:4 

0 50 100 150 200 250 

w A1 : w A2 

1 : 1 1 : 2 1 : 4 

T A1 200 300 300 

T A2 300 300 250 

T A3 202 302 352 

T A4 353 303 302.5 

T 353 303 352 

(a) Completion time 

300 Time 

The network cannot complete at less than t=300 because 

each of N 1 and N 3 requires 300 CPU time. When the resource is 

allocated with 1:2 ratio, the completion time T is minimal close 

to 300. The ratio is proportional to each component’s total 

required CPU time, i.e., 1:2 ≡ 100*1:100*2. One interesting 

question is whether the proportional allocation can give the best 

performance even if the successors have different parameters. 

i 

RA 

1 

1/2 

1/3 

1/5 

0 50 100 150 200 250 300 Time 

(b) Resource availability of A 1 (c) Resource availability of A 2 

Fig. 2. Effects of resource allocation. Depending on the resource allocation of 

node N 1 , each of components A 1 and A 2 follows different resource availability 

profile as in (b) and (c). Consequently, the difference results in different 

completion times as in (a). 

4/5 

2/3 

1:1 

1:2 

1:4 

The answer is yes. If a component A 1 is allocated more resource 

than the proportional allocation, T A3 is dominated by the 

maximal of T A1 and A 3 ’s total CPU time. But, the first quantity 

is less than T N1 and the second quantity is an invariant. So, 

allocating more resource than the proportional allocation 

cannot help reducing the completion time of the network. 

However, if a component is allocated less resource than the 

proportional allocation, its successor’s task inter-arrival time is 

stepwise decreasing. As a result, the successor underutilizes 

resource and can complete later than under the proportional 

allocation. Therefore, the proportional allocation leads the 

network to efficiently utilize distributed resources and 

consequently helps minimizing the completion time of the 

network, though it is localized independent of the successors’ 

parameters. 

B. Optimal Open-loop Policy 

To generalize the arguments for arbitrary network 

configurations, we define Load Index LI i which represents 

component i’s total CPU time required to process its tasks. As a 

component needs to process its own root tasks as well as 

incoming tasks from its predecessors, its number of tasks L i is 

identified as in (6) where i denotes the immediate predecessors 

of component i. Then, LI i is represented as in (7). 

∑ 

L = rt + L 

(6) 

i 

i 

i 

a∈i 

LI = L P 

To provide theoretical foundation of optimal resource 

allocation policy, we convert a network into a network with 

tasks having infinitesimal processing times. Each root task is 

divided into r infinitesimal tasks and each P i is replaced with 

P i /r. Then, the load index of each component is the same as the 

original network but tasks are infinitesimal. We denote the 

completion time of the network with infinitesimal tasks as T´. 

Also, we define a term called task availability as an indicator of 

relative preference for task arrival patterns. An arrival pattern 

gives higher task availability than another if cumulative 

number of arrived tasks is larger or equal over time. A 

component prefers a task arrival pattern with higher task 

availability as it can utilize more resource. Consider a network 

and reconfigure it such that all components have their tasks in 

their queues at t=0. Each component has maximal task 

availability in the reconfigured network and the completion time 

of the reconfigured network forms the lower bound T LB of a 

network’s completion time T given by: 

T 

LB 

= Max 

n∈N 

i 

i 

∑ 

i∈ 

K n 

a 

LI 

i 

(7) 

. (8) 

Theorem 1. T´ equals to T LB when each node allocates its 

resource proportional to its residing components’ load 

indices as:

5 

LI i 

w i( t ) = wi 

= ω n( i ) for all t ≥ 0 , (9) 

LI 

∑ 

p∈K 

n( 

i ) 

p 

T 

LB 

s 

ωn 

+ ωn 

= Max 

n∈N 

ω 

n 

s 

∑ 

i∈ 

K n 

LI 

i 

(12) 

where n(i) denotes a node in which component i resides. 

Theorem 2. T s´ equals to T s LB under proportional allocation. 

Proof. RA i (t) is more than or equal to assigned weight proportion 

as: 

w i ( t ) 

RA ( t ) ≥ for t ≥ 0 . (10) 

ω 

i 

n( i ) 

Proof. RA i (t) becomes: 

RA i( t ) ≥ 

ω 

w ( t ) 

n( i ) 

i 

+ ω 

s 

n( i ) 

for t ≥ 0 . (13) 

Suppose a component i receives its tasks at a constant interval 

of T LB /L i . Then, under proportional allocation, S i (t) is less 

than or equal to T LB /L i over time as shown in (11). 

P 

i 

= 

∫ 

wi 

= 

ω 

n( i ) 

LB 

T 

⇒ 

L 

i 

t+ 

Si( t ) 

t 

S ( t ) = 

i 

RA ( τ )dτ 

≥ 

≥ S ( t ) 

i 

i 

LI 

∑ 

i 

LI 

p∈K 

n( i ) 

p 

∫ 

for t ≥ 0 

t+ 

Si( 

t 

t ) 

LI 

Si( t ) ≥ 

T 

w i( t ) 

dτ 

ω 

n( i ) 

i 

LB 

S ( t ) 

i 

(11) 

So, any component can complete by T LB and generate tasks at 

a constant interval of T LB /L i from t=T LB /L i (first task 

generation time) under proportional allocation when it 

receives tasks at a constant interval of T LB /L i from t=0 (first 

task arrival time). As tasks are infinitesimal and root tasks 

increase task availability, each component can receive 

infinitesimal tasks at a constant interval in 0≤t≤T LB or more 

preferably, and complete at less than or equal to T LB . So, the 

network completes at T LB under proportional allocation. 

From Theorem 1 we can conjecture that a network can 

achieve a performance close to T LB under proportional 

allocation in the limit of large number of tasks. We propose the 

proportional allocation as an optimal resource allocation 

policy. Though the proportional allocation is localized, the 

network can maximize the utilization of distributed resources 

and achieve desirable performance. Coordinated resource 

allocation throughout the network emerges as a result of using 

the load index as global information. If nodes do not follow the 

proportional allocation policy, some components can receive 

their tasks less preferably resulting in underutilization and 

consequently increased completion time as have shown in the 

previous subsection. 

Another important property of the proportional allocation 

policy is that it is itself adaptive. Suppose there are some 

stressors sharing resources with the components. We denote 

ω s n as the amount of shared resource by a stressor in node n. 

Then, the lower bound performance T LB s under stress is given 

by (12). We denote the completion time under stress as T s´. 

Then, (11) results in (14) under proportional allocation. 

LB 

Ts 

L 

i 

≥ S ( t ) for t ≥ 0 

(14) 

i 

Therefore, the network completes at T LB s under proportional 

allocation. 

Theorem 2 depicts that the proportional allocation policy is 

optimal independent of the stress environments. Though we do 

not consider them explicitly, the policy gives lower bound 

performance adaptively. This characteristic is especially 

important when the system is vulnerable to unpredictable stress 

environments. Modern networked systems can be easily 

exposed to various adverse events such as accidental failures 

and malicious attacks, and the space of stress environment is 

high-dimensional and also evolving [26]-[28]. 

C. Adequacy criterion 

The arguments we have made hold in the limit of large 

number of tasks. As the term “large” is obscure we need to give 

it a concrete definition. We define it with an adequacy criterion, 

by which one can evaluate if the desirable properties of the 

proportional allocation hold for a given network. For this 

purpose we characterize upper bound performance of a 

network under proportional allocation. 

Theorem 3. Under proportional allocation a network’s upper 

bound T UB of completion time T is given by: 

T 

UB 

= T 

LB 

+ Max Max 

e∈E 

j∈Se 

∑ 

i∈j 

[ P 

∑ 

LI 

i 

p∈K 

n( 

i) 

p 

/ LI 

i 

] , (15) 

where E denotes a set of components which have no successor 

and S e a set of task paths to component e. A task path to 

component e is a set of components in a path from a 

component with no predecessor to component e and does not 

include component e. 

Proof. From (11) we can induce the lowest upper bound S i UB of 

S i (t) as:

6 

S 

UB 

i 

∑ 

= P LI / LI . (16) 

i 

p∈K 

n( 

i) 

So, a component i can complete by T LB and generate tasks at 

a constant interval of T LB /L i from t=S i UB when it receives 

tasks at a constant interval of T LB /L i from t=0. Now, consider 

component i’s successor s which has only one predecessor. 

As the successor receives tasks at a constant interval of T LB /L s 

from t=S i UB or more preferably, it can complete by S i UB +T LB . 

So, a component e∈E (with no successor) can receive tasks at 

a constant interval of T LB /L e from maximal task traveling time 

to the component of: 

Max 

j∈S 

e 

∑ 

i∈ 

j 

S 

p 

UB 

i 

i 

(17) 

(note that a path j does not include component e) or more 

preferably so that its completion time T e is bounded as: 

T 

e 

≤ T 

LB 

+ Max 

j∈S 

e 

∑ 

i∈ 

j 

S 

UB 

i 

. (18) 

And, the upper bound of T is the maximal of the bounds. 

Though we formulated the upper bound performance 

without considering stress environments, one can easily modify 

it so that the upper bound performance can reflect the stress 

environments (if each ω n s is identifiable or assumable). The 

adequacy criterion is defined as the ratio between T LB and T UB 

as in (19). When the criterion is close to one, a network can 

achieve the lower bound performance using the proportion 

allocation policy. Typically, the criterion converges to one as 

each L i increases. However, as the criterion approaches zero, 

the policy become more and more inadequate. The example 

network in Fig. 1 is quite adequate because the network’s 

adequacy is 0.99 (300/303). 

LB 

 

T 

Adequacy = (19) 

UB 

T 

So far, we assumed a hypothetical weighted round-robin 

server which is difficult to realize in practice. But, our 

arguments do not seem to be invalid because they are based on 

worst-case analysis and quantum size is relatively infinitesimal 

compared to working horizon in reality. 

D. Resource control mechanism 

Once a network has an appropriate adequacy over a certain 

level (depending on the nature of the network), the proportional 

allocation is deployed periodically under MPC framework. 

Consider current time as t. To update load index as the system 

moves on, we slightly modify it to represent total CPU time for 

the remaining tasks as: 

LI ( t ) = R ( t ) + L ( t ) P , (20) 

i 

i 

in which R i (t) denotes remaining CPU time for a task in process 

and L i (t) the number of remaining tasks excluding a task in 

process. After identifying initial number of tasks L i (0)=L i , each 

component updates it by counting down as they process tasks. 

Periodically, a resource manager of each node collects current 

LI i (t)s from residing components and allocates resource 

proportional to the indices as in (21). As the resource allocation 

policy is purely localized there is no need for synchronization 

between nodes. The designed resource control mechanism is 

scalable as each node can make decisions independent of 

others while requiring almost no computation. 

w 

i 

i ( t) 

ωn( 

i) 

∑ LI p ( t) 

p∈K 

n( 

i) 

i 

i 

LI ( t) 

= (21) 

VI. EMPIRICAL RESULTS 

We ran several experiments using discrete-event simulation 

to validate the designed resource control mechanism. 

A. Experimental design 

The experimental network is composed of eight components 

in four nodes as in Fig. 3. Two components are sharing a 

resource in N 3 and four components in N 4 . Also, ω n is 1 for all 

n∈N and CPU is allocated using a weighted round-robin 

scheduling in which CPU time received by each component in a 

round is equal to its assigned weight. 

N 1 N 2 

A 1 

N 3 

A 3 A 4 

N 4 

A 5 A 6 A 7 A 8 

Fig. 3. Experimental network configuration. The network is composed of 

eight components in four nodes and the performance can depend on the 

resource allocation of nodes N 3 and N 4 . 

We set up ten different experimental conditions as shown in 

Table I. We vary the number of root tasks rt i and CPU time per 

task P i , and the distribution of P i can be deterministic or 

exponentially distributed. While using stochastic distribution 

we repeat 5 experiments. 

We use three different resource control policies for each 

experimental condition. Table II shows these control policies. 

In round-robin allocation policy (RR) the components in each 

node are assigned equal weights over time. PA-O and PA-C use 

the proportional allocation policy in open-loop and closed-loop 

A 2

7 

respectively. In PA-O resources are allocated only at t=0 and 

kept over time while in PA-C periodically (every 100 time 

units). PA-C is the resource control mechanism we have 

designed. 

B. Results 

TABLE I 

EXPERIMENTAL CONDITIONS 

Condition Distribution of P i rt i P i 

Con1-1 Deterministic [000 000 000 000 [04 12 04 08 

Con1-2 Exponential 200 200 200 200] 02 02 02 02] 









TABLE II 

CONTROL POLICIES FOR EXPERIMENTATION 


Description 

RR 

Round-Robin allocation 

PA-O Proportional allocation - Open loop 

PA-C Proportional allocation - Closed loop 

Numerical results from the experimentation are shown in 

Table III. Lower and upper bounds are calculated for each 

experimental condition. The network adequacy of each 

condition is close to one and the proportional allocation policy 

can be used effectively for all the conditions. 

Proportional allocation policies (PA-O and PA-C) shows 

significant advantages compared to round-robin allocation in 

all the different conditions. The completion time T under 

proportional allocation is bounded to T UB and close to T LB in all 

deterministic conditions (note that the performance of PA-O 

and PA-C is the same in deterministic environments), 

supporting the effectiveness of the resource allocation policy. 

Though T UB does not work accurately in stochastic 

environments, the performance improves close to T LB when the 

proportional allocation is implemented in closed-loop. The 

periodic design process alleviates the impacts of stochasticity. 

So, we can conclude that the designed control mechanism can 

be effectively used even in stochastic environments for the 

networks with high adequacy. 

The performance differences can be reasoned from resource 

utilization as discussed earlier. A node with maximal total CPU 

time needs to utilize its resource almost fully to achieve a 

performance close to T LB . For example, N 2 is such a node in 

Con1-1 (deterministic) and Con1-2 (stochastic). Resource 

utilization profiles of N 2 are shown in Fig. 4 for Con1-1 and 

Fig. 5 for Con1-2, in which a data point corresponds to the 

amount of utilized resource during a control period (100 time 

units). In deterministic environment (Con1-1), N 2 utilizes its 

resource almost fully under both proportional allocation 

Utilization (%) 

100 

80 

60 

40 

20 

PA-O 

RR 

PA-C 

0 

0 1000 2000 3000 4000 5000 6000 

Fig. 4. Resource utilization of N 2 in Con1-1. In a deterministic environment, 

N 2 utilizes its resource almost fully under both proportional allocation policies 

(PA-O, PA-C) while underutilizing in initial stage under round-robin 

allocation policy (RR). 

Utilization (%) 

100 

80 

60 

40 

20 

PA-O 

RR 

Time 

0 

0 1000 2000 3000 4000 5000 6000 

Fig. 5. Resource utilization of N 2 in Con1-2. In a stochastic environment, N 2 

utilizes its resource more under proportional allocation policies (PA-O, PA-C) 

compared to round-robin allocation policy (RR), and resource utilization 

under closed-loop policy (PA-C) is larger than under open-loop policy 

(PA-O). 

Time 

PA-C 

TABLE III 

EXPERIMENTAL RESULTS 


RR PA-O PA-C 

T LB T UB Adequacy T T LB /T T T LB /T T T LB /T 

Con1-1 4800 4820 0.996 5619 0.854 4820 0.996 4820 0.996 

Con1-2 4800 4820 0.996 5618 0.854 5021 0.956 4939 0.972 

Con2-1 7200 7230 0.996 7612 0.946 7200 1.000 7200 1.000 

Con2-2 7200 7230 0.996 7679 0.938 7323 0.983 7252 0.993 

Con3-1 6000 6073 0.988 6412 0.936 6012 0.998 6012 0.998 

Con3-2 6000 6073 0.988 6408 0.936 6193 0.969 6013 0.998 

Con4-1 7200 7228 0.996 7200 1.000 7200 1.000 7200 1.000 

Con4-2 7200 7228 0.996 7231 0.996 7109 1.013 7169 1.004 

Con5-1 7200 7220 0.997 7810 0.922 7210 0.999 7210 0.999 

Con5-2 7200 7220 0.997 7979 0.902 7351 0.979 7319 0.984

8 

policies while underutilizing in initial stage under round-robin 

allocation. In stochastic environment (Con1-2), resource 

utilization profiles under proportional allocation policies 

become different. Though both policies give more utilization 

compared to round-robin allocation, resource utilization under 

closed-loop policy is larger than under open-loop policy. Such 

differences of resource utilization result in the performance 

differences in Table III. The designed control mechanism helps 

maximizing the utilization of distributed resources so as to 

achieve desirable performance. 

VII. CONCLUSIONS 

A typical information network emerges as a result of 

automation or organizational integration, which is large-scale 

with distributed and component-based architecture. In this 

paper we designed a resource control mechanism of such 

networks for minimizing completion time. The designed 

resource control mechanism has several desirable properties. 

First, it is localized as each node can make decisions 

independent of others. Second, it requires almost no 

computation. Third, nevertheless the network can achieve a 

desirable performance. Fourth, it is itself adaptive to the stress 

environments without explicit considerations. Such emergent 

properties can be found in many self-organizing systems such 

as social or biological systems. Though entities act with a 

simple mechanism without central authority, desirable global 

performance can often be realized. When a large-scale network 

is working in a dynamic environment under the designed 

control mechanism, it is really a self-organizing system. 

REFERENCES 

[1] B. Meyer, “On to components”, IEEE Computer, vol. 32, no. 1, pp. 

139-140, 1999. 

[2] P. Clements, “From subroutine to subsystems: Component-based 

software development,” in Component Based Software Engineering, A. 

W. Brown, Ed. IEEE Computer Society Press, 1996, pp. 3-6. 

[3] O. F. Rana and K. Stout, “What is scalability in multi-agent systems?,” in 

Proc. 4th Int. Conf. Autonomous Agents, 2000, pp. 56-63. 

[4] F. M. T. Brazier, C. M. Jonker, and J. Treur, “Principles of 

component-based design of intelligent agents,” Data and Knowledge 

Engineering, vol. 41, no. 1, pp. 1-28, 2002. 

[5] H. J. Goradia and J. M. Vidal, “Building blocks for agent design,” in Proc. 

4th Int. Workshop on Agent-Oriented Software Engineering, 2003, pp. 

17-30. 

[6] R. Krutisch, P. Meier, and M. Wirsing, “The AgentComponent approach, 

combining agents and components,” in Proc. 1st German Conf. 

Multiagent Sys. Technologies, 2003, pp. 1-12. 

[7] D. Moore, W. Wright, and R. Kilmer, “Control surfaces for Cougaar,” in 

Proc. First Open Cougaar Conference, 2004, pp. 37-44. 

[8] W. Peng, V. Manikonda, and S. Kumara, “Understanding agent societies 

using distributed monitoring and profiling,” in Proc. First Open Cougaar 

Conference, 2004, pp. 53-60. 

[9] H. Gupta, Y. Hong, H. P. Thadakamalla, V. Manikonda, S. Kumara, and 

W. Peng, “Using predictors to improve the robustness of multi-agent 

systems: Design and implementation in Cougaar,” in Proc. First Open 


[10] D. Moore, A. Helsinger, and D. Wells, “Deconfliction in ultra-large MAS: 

Issues and a potential architecture,” in Proc. First Open Cougaar 

Conference, 2004, pp. 125-133. 

[11] R. D. Snyder and D. C. Mackenzie, “Cougaar agent communities,” in 

Proc. First Open Cougaar Conference, 2004, pp. 143-147. 

[12] J. Regehr, “Some guidelines for proportional share CPU scheduling in 

general-purpose operating systems,” Presented as a work in progress at 

22nd IEEE Real-Time Systems Symposium, London, UK, Dec. 3-6, 2001. 

[13] I. Stoica, H. Abdel-Wahab, J. Gehrke, K. Jeffay, S. K. Baruah, and C. G. 

Plexton, “A proportional share resource allocation algorithm for 

real-time, time-shared systems,” in Proc. 17th IEEE Real-Time Systems 

Symposium, 1996, pp. 288-299. 

[14] C. A. Waldspurger and W. E. Weihl, “Lottery scheduling: Flexible 

proportional-share resource management,” in Proc. First Symposium on 

Operating System Design and Implementation, 1994, pp. 1-11. 

[15] C. Waldspurger and W. Weihl, “Stride scheduling: Deterministic 

proportional-share resource management,” Lab. for Computer Science, 

Massachusetts Institute of Technology, Cambridge, MA, Tech. Rep. 

MIT/LCS/TM-528, 1995. 

[16] C. Waldspurger, “Lottery and stride scheduling: Flexible proportional 

share resource management,” Ph.D. dissertation, Lab. for Computer 

Science, Massachusetts Institute of Technology, Cambridge, MA, 1995. 

[17] G. Barto, S. J. Bradtke, and S. P. Singh, “Learning to act using real-time 

dynamic programming,” Artificial Intelligence, vol. 72, pp. 81-138, 1995. 

[18] R. S. Sutton, A. G. Barto, and R. J. Williams, “Reinforcement learning is 

direct adaptive optimal control,” IEEE Control Systems, vol. 12, no. 2, pp. 

19-22, 1992. 

[19] L. P. Kaelbling, M. L. Littman, and A. W. Moore, “Reinforcement 

learning: A survey,” Journal of Artificial Intelligence Research, vol. 4, 

pp. 237-285, 1996. 

[20] J. B. Rawlings, “Tutorial overview of model predictive control,” IEEE 

Control Systems, vol. 20, no. 3, pp. 38-52, 2000. 

[21] M. Morari and J. H. Lee, “Model predictive control: Past, present and 

future,” Computers and Chemical Engineering, vol. 23, no. 4, pp. 

667-682, 1999. 

[22] M. Nikolaou, “Model predictive controllers: A critical synthesis of theory 

and industrial needs,” Advances in Chemical Engineering Series, 

Academic Press, 2001. 

[23] S. J. Qin and T. A. Badgwell, “A survey of industrial model predictive 

technology,” Control Engineering Practice, vol. 11, pp. 733-764, 2003. 

[24] T. Gonzalez and S. Sahni, “Flowshop and jobshop schedules: Complexity 

and approximation,” Operations Research, vol. 26, pp. 36-52, 1978. 

[25] J. Lenstra, A. R. Kan, and P. Brucker, “Complexity of machine scheduling 

problems,” Annals of Discrete Mathematics, vol. 1, pp. 343-362, 1977. 

[26] S. Jha and J. M. Wing, “Survivability analysis of networked systems,” in 

Proc. 23rd Int. Conf. Software engineering, 2001, pp. 307-317. 

[27] A. P. Moore, R. J. Ellison, and R. C. Linger, “Attack modeling for 

information security and survivability,” Software Engineering Institute, 

Carnegie Mellon University, Pittsburg, PA, Tech. Note 

CMU/SEI-2001-TN-001, 2001. 

[28] F. Moberg, “Security analysis of an information system using an attack 

tree-based methodology,” M.S. thesis, Automation Engineering Program, 

Chalmers University of Technology, Sweden, 2000.

Efficient Method of Quantifying Minimal Completion Time for Component- 

Based Service Networks: Network Topology and Resource Allocation * 





{stonesky, skumara, ngautam}@psu.edu 

ABSTRACT 

In a grid service environment, it is important to be able to agilely quantify the quality of 

service achievable by each alternative composition of resources and services. This capability 

is an essential driver to not only efficiently utilizing the resources and services, but also 

promoting the virtual economy. In this paper, we develop such a method of quantifying the 

minimal completion time for component-based service networks whose task flow structure is 

a combination of intra-service and inter-service task flows. The performance of the network 

is a function of network topology and resource allocation. Network topology assigns 

components to available machines and resource allocation allocates the resources of each 

machine to the residing components. Though similar problems can be found in the 

multiprocessor scheduling literature, our problem is different especially because a component 

in our networks can have multiple tasks to process, i.e. a component can process tasks in 

parallel with its successor or predecessor components. The designed method incorporates the 

fact that the components in a network can be considered independent under a certain resource 

allocation policy when the number of tasks of each component is large. 

Index Terms: Multiprocessor systems, sequencing and scheduling, network topology, 

modeling and prediction, optimization 


Individual systems are becoming interoperable in virtue of several enabling technologies. The 

Grid technology provides inexpensive access to large computational resources across 

institutional boundaries [1]. Services can be composed over the Internet via Web Service 

technology creating enormous opportunities for automation of business processes [2]. OGSA 

(Open Grid Services Architecture: http://www.globus.org/ogsa/) defines a grid system 

* This work was supported, in part, by DARPA (Grant#: MDA972-01-1-0038) under the UltraLog program. 

1

architecture based on both the Grid and Web Service technologies. The Grid Service enables the 

integration of resources and services across distributed, heterogeneous, dynamic virtual 

organizations [3]. Cost and quality considerations may force large number of customers to look 

for resources and services via such an architecture to deal with their own computing problems. 

Ubiquitous computing technology embeds computers in various objects and places for sensing 

and controlling environments [4]. As this technology is becoming realized and gives rise to 

complex computing problems, the use of such an architecture might be inevitable. 

In a grid service environment, a problem is processed by composing multiple resources and 

services. As there can be several alternative compositions of resources and services for a given 

problem, virtual markets will play a critical role in coordinating huge amount of economic 

entities such as customers, service providers, and resource providers. There are various market 

mechanisms such as OCEAN [5], Compute Power Market [6], and Nimord/G [7], proposed for 

the large-scale virtual economy. However, one essential enabler of such markets is the ability to 

agilely quantify the quality of service (QoS) achievable by each alternative. Without such a 

capability, the alternatives cannot be valuated in a timely manner and the virtual economy will 

fail to efficiently utilize the resources and services. 

There can be various ways of defining QoS depending on the nature of the problems. We 

consider a class of problems whose QoS is determined by completion time for generating a 

solution. The completion time (also called makespan) is one of the most widely studied 

objectives for diverse scheduling problems in the context of multiprocessor, manufacturing, and 

project management. Regarding to problem solving structure we adopt component-based 

architecture as a general framework. A component is a reusable program element. Component 

technology utilizes the components so that developers can build systems needed by simply 

2

defining their specific roles and wiring them together [8][9]. In service networks with 

component-based architecture, each component is highly specialized for specific tasks and task 

flow structure between components is a combination of intra-service and inter-service task flows. 

A problem given to such a network is decomposed in terms of root tasks for some components 

and those tasks are propagated through a task flow structure to other components. As a problem 

can be decomposed with respect to space, time, or both, a component can have multiple root 

tasks that can be considered independent and identical in their nature. One can imagine wide 

range of scientific and engineering problems that can be solved by such a network. 

In this paper, we develop an efficient method of quantifying the minimal completion time for 

the component-based service networks. For a given set of resources and services, the 

performance can vary depending on the way of utilizing distributed heterogeneous resources. 

Network topology assigns components to available machines with a set of constraints. The 

components of a web service may not be separable to different machines and a web service may 

be allowed to specific machines. Though mobile code provides a great flexibility for creating 

distributed systems there are technical challenges such as security to fulfill its promise [10]-[13]. 

Given a network topology, there can be multiple components in a machine sharing the machine’s 

resources together. So, resource allocation can play an important role in controlling the 

performance of a network. These two control facilities determine the performance of a network 

and the minimal completion time represents achievable QoS by a set of resources and services. 

Similar problems can be found in the multiprocessor scheduling literature 1 . There is a set of 

components with a task flow structure between them and each component without predecessors 

has one root task. Each component processes exactly one task only after all of its predecessors 

complete their tasks. A multiprocessor scheduling is composed of an assignment of components 

1 We adapt the terms used in the multiprocessor scheduling to our context throughout this paper. 

3

to machines (network topology) and a sequence of components for each machine (resource 

allocation). However, our problem is different especially because a component in our networks 

can have multiple tasks to process, i.e., a component can process tasks in parallel with its 

successors or predecessors. The easiest multiprocessor scheduling problem is when components 

are independent, i.e., there is no task flow between components. However, this problem is known 

as NP-complete [14][15]. Considering that the task flow structure of our networks is arbitrary 

and each component can have multiple tasks to process, our scheduling problem is even harder. 

In this context, the method designed in this paper is a heuristic which is applicable to the 

cases where the number of tasks to be processed by each component is large. Though the 

increase of the number of tasks adds more complexity, it can give us great opportunity to 

develop an efficient heuristic. Also, our method addresses resource reservation. When different 

applications share resources together, their performance can be guaranteed through the resource 

reservation. The method quantifies the minimal completion time by incorporating the resource 

reservations of other applications and also enables to make the resource reservations for the 

service network under consideration. 

The organization of this paper is as follows. In section 2 we formally define the problem in 

detail. After designing the method in Sections 3, we show empirical results in Section 4. Finally, 

we discuss implications and possible extensions of our work in Section 5. 

2. Problem statement 

In this section we formally define the problem by detailing component-based service network, 

network topology, and resource allocation. We focus on computational CPU resources assuming 

that the system is computation-bounded. 

4

2.1 Component-based service network 

A network is composed of a set I = {i: i∈I} of components and a task flow structure between 

them. Task flow structure of the network, which defines precedence relationship between 

components, is an arbitrary directed acyclic graph. A problem given to a network is decomposed 

in terms of root tasks for some components and those tasks are propagated through a task flow 

structure. Each component processes one of the tasks in its queue (which has root tasks as well as 

tasks from predecessor components) and then sends it to successor components. We denote the 

number of root tasks of component i as rt i . There is a set K = {k: k∈K} of available machines and 

P i (k) represents CPU time per task of component i at machine k reflecting computation speed 

difference between machines. 

Fig. 1 shows an example network composed of four components in three machines. In the 

figure denotes rt i and P i (k) at the residing machine respectively. Components I 1 and I 2 are 

residing in machine K 1 and each of them has 100 root tasks. I 3 in K 2 and I 4 in K 3 have no root 

tasks but they have 200 and 100 tasks from the corresponding predecessors. 

 

I 1 

 

I 3 

K 2 

K 3 

 

I 2 I 4 

K 1 

Fig. 1. An example network composed of four components in three machines. denotes the 

number of root tasks and CPU time per task at the residing machine. 

2.2 Network topology 

Considering that the components of a web service may not be separable to different machines, 

we define a set J = {j: j∈J} of clusters and denote the components of a cluster j as M j . Each 

5

component is a member of one of the clusters and the components in a cluster should be assigned 

to the same machine. Each cluster can be assigned to a set of machines and we denote the 

assignable machine set of cluster j as N j . We define topology variable set X = {x jk : j∈J, k∈K} in 

which x jk is 1 if cluster j is assigned to machine k and 0 otherwise. The constraints of topology 

variables are as in (1). 

 

Network topology constraints 

∑ 

k∈N 

∑ 

k∉N 

x 

jk 

j 

j 

x 

x 

jk 

jk 

= 1 

= 0 

∈{0,1} 

for all 

for all 

for all 

j ∈ J 

j ∈ J 

j ∈ J 

and 

k ∈ K 

(1) 

2.3 Resource allocation 

When there are multiple components in a machine, a network can control its behavior through 

resource allocation. In the example network, machine K 1 has two components and the system 

performance depends on its resource allocation to these two components. There are several CPU 

scheduling algorithms for allocating a CPU resource amongst multiple threads. Among the 

scheduling algorithms, proportional CPU share (PS) scheduling is known for its simplicity, 

flexibility, and fairness [16]. In PS scheduling threads are assigned weights and resource shares 

are determined proportional to the weights [17]. Excess CPU time from some threads is allocated 

fairly to other threads. There are many PS scheduling algorithms such as Weighted Round-Robin 

scheduling, Lottery scheduling, and Stride scheduling [18]-[20]. 

We adopt PS scheduling as resource allocation scheme because of its generality in addition to 

the benefits mentioned above. We define resource allocation variable set w = {w i (t): i∈I, t≥0} in 

which w i (t) is a non-negative weight of component i at time t. We denote the components 

6

assigned to machine k as S I[k] and the clusters assigned to machine k as S J[k] . If ω k a of total 

managed weight ω k is available to assign in machine k (i.e. ω k -ω k a 

is reserved by other 

applications), the constraints of resource allocation variables for a given topology are as in (2). 

 

Resource allocation constraints 

∑ 

i∈S 

I 

[ k ] 

a 

w i( t ) ≤ ω k for all k ∈ K 

(2) 


As the completion time T is a function of network topology (X) and resource allocation (w), 

the objective is to quantify the minimal completion time T * 

represented in (3) with the 

constraints of (1) and (2). 

T 

* 

= Min T 

X ,w 

. (3) 

3. Minimal completion time 

As stated earlier, we design a method of quantifying the minimal completion time by limiting 

to the cases where the number of tasks to be processed by each component is large. In this 

section, we investigate the impacts of the largeness on the optimal resource allocation for a given 

topology. Then, we formulate the problem by incorporating network topology and provide a 

heuristic algorithm for solving the problem formulation. 

3.1 Optimal resource allocation 

For a given topology, we define Load Index LI i which represents component i’s total CPU 

time required to process its tasks. As a component needs to process its own root tasks as well as 

incoming tasks from its predecessors, its number of tasks L i is identified as in (4), where i 

7

denotes the immediate predecessors of component i. Then, by denoting CPU time per task in the 

given topology as P i , LI i is represented as in (5). 

∑ 

L i = rti 

+ La 

(4) 

i 

a∈i 

LI = L P . (5) 

To provide theoretical foundation of optimal resource allocation, we convert a network into a 

network with infinitesimal tasks. Each root task is divided into r infinitesimal tasks and each P i is 

replaced with P i /r. Then, the load index of each component is the same as the original network 

but tasks are infinitesimal. We denote the completion time of the network with infinitesimal 

tasks as T´. Also, we define a term called task availability as an indicator of relative preference 

for task arrival patterns. A component’s task availability for an arrival pattern is higher than for 

another if cumulative number of arrived tasks is larger or equal over time. A component prefers a 

task arrival pattern with higher task availability as it can utilize more resource. Consider a 

network and reconfigure it such that all components have their tasks in their queues at t=0. Each 

component has maximal task availability in the reconfigured network and the completion time of 

the reconfigured network forms the lower bound T LB of a network’s completion time T given by: 

i 

i 

T 

LB 

ωk 

= Max ∑ LI i 

k∈K 

a . (6) 

ω 

k i∈ 

S I 

[ k ] 

Then, assuming a hypothetical weighted round-robin server 2 for CPU scheduling, T´ equals to 

T LB when each machine allocates resource to the residing components according to (7), where 

k(i) denotes a machine in which component i resides. 

2 The hypothetical server has idealized fairness as the CPU time received by each thread in a round is infinitesimal 

and proportional to the weight of the thread. This assumption is reasonable because quantum size is relatively 

infinitesimal compared to working horizon in reality. 

8

LI i 

wi 

( t ) ≥ ω 

k( i ) 

for all i ∈ I and t ≥ 0 

LB 

(7) 

T 

Proof. A component’s instantaneous resource availability RA i (t), which is the available fraction 

of a resource when the component requests the resource at time t, is more than or equal to 

assigned weight proportion as: 

w i( t ) 

RA i( t ) ≥ for t ≥ 0 . (8) 

ω 

k( i ) 

Service time S i (t) is the time taken to process a task at time t and has a relationship with RA i (t) 

as: 

∫ 

t+ Si 

( t) 

RAi 

( τ ) dτ 

= Pi 

. (9) 

t 

Suppose a component i receives its tasks at a constant interval of T LB /L i . Then, under the 

resource allocation in (7), S i (t) is less than or equal to T LB /L i over time as shown in (10). 

i 

Pi 

= 

∫ 

RA i ( τ )dτ 

≥ 

∫ 

T 

⇒ 

L 

LB 

i 

t+ 

S ( t ) 

t 

≥ S ( t ) 

i 

t+ 

S ( 

t 

i 

t ) 

wi 

( t ) LI 

dτ 

≥ 

ω T 

k( 

i ) 

i 

LB 

S ( t ) 

i 

(10) 

So, any component can complete by T LB and generate tasks at a constant interval of T LB /L i 

from t=T LB /L i (first task generation time) under the resource allocation in (7) when it receives 

tasks at a constant interval of T LB /L i from t=0 (first task arrival time). As tasks are infinitesimal 

and root tasks increase task availability, each component can receive infinitesimal tasks at a 

constant interval in 0≤t≤T LB or more preferably, and complete at less than or equal to T LB . So, 

the network completes at T LB . 

 

So, a network can achieve a performance close to T LB under this resource allocation in the 

9

limit of large number of tasks. If machines do not follow this resource allocation, some 

components can receive their tasks less preferably than constant interval resulting in 

underutilization and consequently increased completion time. The minimal weights required to 

achieve T LB are constants over time as in (11) and the summation of these weights for each 

machine forms the required amount ω r k of resource reservation in the machine as in (12). Note 

that ω r k is less than or equal to ω a k satisfying the resource allocation constraints in (2). 

 

Constant resource allocation 

w 

i 

LI i 

= ω 

k( i ) 

for all i ∈ I and t ≥ 0 

LB 

(11) 

T 

 

Resource reservation 

r ω 

ω k = k LI i for all k ∈ K 

LB ∑ 

(12) 

T 

i∈ 

S I 

[ k ] 

3.2 Optimal network topology 

As CPU time per task is machine-dependent we rewrite the load index as a function of 

machine as: 

LIi ( k) 

= Li 

Pi 

( k) 

. (13) 

Considering that the components in a cluster cannot be assigned to separate machines, we define 

Cluster Load Index CLI j (k) as: 

∑ 

CLI j ( k) 

= LIi 

( k) 

. (14) 

Then, under the constant resource allocation, the completion time for a given topology can be 

estimated by: 

i∈ 

M j 

10

ω 

∑ 

k 

Max CLI j ( k ) 

k∈K 

a 

. (15) 

ωk 

j∈ 

Consequently, the minimal completion time T * can be formulated as in (16) by incorporating 

topology variables and constraints in (1). 

S J [ 

k ] 

 

Topology problem formulation 

T 

* 

s.t. 

ω 

= Min Max 

k∈K 

ω 

∑ 

k∈N 

∑ 

k∉N 

x 

jk 

j 

j 

x 

x 

jk 

jk 

= 1 

= 0 

∈{0,1} 

k 

a 

k 

∑ 

j∈J 

CLI 

j 

( k )x 

jk 

for all j ∈ J 

for all j ∈ J 

for all j ∈ J and 

k ∈ K 

(16) 

The formulation has a simplistic form because it is completely separated from resource 

allocation variables. As a result, the formulation can be mapped into the easiest multiprocessor 

scheduling problem, i.e., an assignment of independent clusters to machines. As discussed, this 

problem is NP-complete and there are diverse heuristic algorithms available in the literature. 

Eleven heuristics were selected and examined with various problem configurations in [21]. They 

are Opportunistic Load Balancing, Minimum Execution Time, Minimum Completion Time, 

Min-min, Max-min, Duplex, Genetic Algorithm, Simulated Annealing, Genetic Simulated 

Annealing, Tabu, and A * . Though Genetic Algorithm always gave the best performance, if 

algorithm execution time is also considered, it was shown that the simple Min-min heuristic 

performs well in comparison to others. So, we recommend the Min-min heuristic as an algorithm 

for solving the problem formulation. By adapting to our context the Min-min heuristic is as 

follows. 

11

Min-min heuristic algorithm 

Step 1: Initialize a set of all unassigned clusters, U←J, and current machine-level completion 

times, mc(k)←0 for all k∈K. 

Step 2: Compute the minimal completion time after assignment for each unassigned cluster, 

ωk 

M={ min [ CLI j ( k ) + mc( k )] 

k∈N 

a 

: j∈U}. 

j ω 

k 

Step 3: Select the minimal from M, mmc←min M, and find corresponding cluster and 

machine, c and m respectively. 

Step 4: Assign c to m and update mc(m), mc(m)←mc(m)+mmc. 

Step 5: Remove c from U. 

Step 6: If U=∅ then go to step 7. Otherwise go to step 2. 

Step 7: T * ← max mc( k ) . 

k∈K 


We ran several experiments through discrete-event simulation to validate the designed 

method. Though we have not considered stochasticity so far, this empirical study will support the 

effectiveness of the method even in stochastic environments. 

4.1 Network description 

The network is composed of eight components in four clusters as in Table 1. Task flow 

structure between components is described in Fig. 2. There are three available machines {K 1 , K 2 , 

K 3 } with ω k =ω a k =1 for all k, and each cluster is assignable to any machine. 

12

Table 1. Experimental network parameters 

Component rt i P i (k) a Cluster 

I 1 0 4 J 1 

I 2 0 12 J 2 

I 3 0 4 J 3 

I 4 0 8 J 3 

I 5 200 2 J 4 

I 6 200 2 J 4 

I 7 200 2 J 4 

I 8 200 2 J 4 

a 

for all k∈K 

I 1 

I 3 I 4 

I 5 I 6 I 7 

I 2 

I 8 

Fig. 2. Experimental task flow structure between eight components. The components are members 

of three clusters and each cluster is assignable to any of three machines. 

4.2 Performance evaluation 

The Min-min heuristic algorithm gives T * =4800 and the resulting topology is as in Fig. 3(b). 

The heuristic solution is equivalent to the exact solution of (16) in this experimental network. 

K 1 

K 2 K 1 

K 2 

I 1 

I 1 

I 3 I 4 

I 3 I 4 

K 3 

I 2 

I 8 

K 3 

I 2 

(b) Optimal topology 

I 5 I 6 I 7 I 8 

I 5 I 6 I 7 

(a) Non-optimal topology 

Fig. 3. Experimental network topologies. In (a), clusters J 1 and J 3 are assigned to machine K 1 , J 2 to 

K 2 , and J 4 to K 3 . In (b), J 4 is reassigned to K 1 and J 3 to K 3 . 

13

We set up eight different experimental conditions by combining three independent factors as 

shown in Table 2. We use two different network topologies as in Fig. 3, which are non-optimal 

and optimal topologies. Two resource allocation policies are used: round-robin allocation and 

constant allocation. In round-robin allocation the components in each machine are assigned equal 

weights and in constant allocation according to the components’ load indices as in (11). To 

implement PS scheduling we use a weighted round-robin scheduling in which CPU time 

received by each component in a round is equal to its assigned weight. Also, the distribution of 

P i (k) can be deterministic or stochastic. While using stochastic distribution we repeat 5 

experiments. 

Table 2. Experimental design 

Condition Topology Resource allocation P i (k) 

Con1 Non-optimal Round-Robin Deterministic 

Con2 Non-optimal Round-Robin Exponential 

Con3 Non-optimal 

Constant Deterministic 

Con4 N on-optimal Constant Exponential 

Con5 Optimal Round-Robin Deterministic 

Con6 Optimal Round-Robin Exponential 

Con7 Optimal Constant Deterministic 

Con8 Optimal Constant Exponential 

Numerical results fr om the experimentation are shown in Table 3. The last two conditions 

(Con7 and Con8), which use the optimal network topology and constant resource allocation, 

gives a performance close to T * and outperforms other conditions significantly. Also, constant 

allocation for both non-optimal (Con3 and Con4) and optimal (Con7 and Con8) topologies, gives 

a performance superior to round-robin allocation and close to lower bound performance T LB in 

both deterministic and stochastic environments. These facts support the optimality of the 

constant resource allocation and consequently the validity of the method of quantifying the 

minimal completion time. 

14


Condition T LB T * Actual T % a 

Con1 6400 4800 7215 150.3 

Con2 6400 4800 7314 152.4 

Con3 6400 4800 6416 133.7 

Con4 6400 4800 6404 133.4 

Con5 4800 4800 5619 117. 1 

Con6 4800 4800 5645 117.6 

Con7 4800 4800 4820 100.4 

Con8 4800 4800 4899 102.1 

a A / T 

* 

ctual T 

4.3 Resource reservation 

The resource reservations required in the optimal topology are [ω r K1 =0.667, ω r K2 =1, ω r K3 =1] 

computed from (12). Our argument is that the network can achieve the optimal performance T * 

with these reservations even though unreserved resources are allocated to other applications. To 

validate this, we use eleven different reservations for machine K 1 as shown in Table 4. In each 

condition, ω r K1 is allocated proportional to the load indices of the residing components and 

unreserved resources are assigned to an application which has infinite work (continuously 

requiring resources). The numerical results are shown in Table 4 and Fig. 4 to 5. In overall, the 

completion time decreases as ω r K1 increases. However, when ω r K1 is greater than 0.667, there is no 

significant advantage in deterministic environment. In contrast, the threshold in stochastic 

environment is somewhere between 0.667 and 0.7. Considering that the other applications may 

not require resources continuously, such a slight difference (≤ 0.033) does not seem to be 

significant. 

Table 4. The effects of resource reservation 

Actual T 

r Deterministic Exponential 

ω K1 

P i (k) P i (k) 

0.1 32284 34502 

0.2 16132 16605 

0.3 10746 11069 

15

0. 4 8057 8237 

0.5 6439 6635 

0.6 5369 5503 

0 .667 4829 5135 

0.7 4827 4941 

0.8 4824 4965 

0.9 4822 4984 

1.0 4820 4946 

35000 

30000 

25000 

Actual T 

20000 

15000 

10000 

5000 

0 

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 


Fig. 4. The effects of resource reservation in deterministic environment. When the resource 

reservation in K 1 is greater than 0.667, there is no significant decrease of completion time. 

35000 

30000 

25000 

Actual T 

20000 

15000 

10000 

5000 

0 

0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 


Fig. 5. The effects of resource reservation in stochastic environment. When the resource reservation 

in K 1 is greater than somewhere between 0.667 and 0.7, there is no significant decrease of 

completion time. 

16


The simple Min-min heuristic algorithm was proposed as a method of quantifying the 

minimal completion time for the component-based service networks. A network can achieve the 

performance under 

the constant resource allocation in the limit of large number of tasks. Also, 

the 

performance can be guaranteed with the resource reservations we have formulated. The 

designed method is efficient enough to satisfy the requirements for the use in a grid service 

environment. In spite of its simplicity, the method can quantify the quality of service effectively. 

The virtual markets driven by such methods will make timely transactions with desirable 

surpluses leading to productive virtual economy. 

Our work can be extended by taking into account alternative algorithms. Each component can 

have alternative algorithms to process a task which trade off processing time and quality of 

solution. While network topology and resource allocation try to efficiently utilize limited 

resources, alternative algorithms can change the amount of required resources. As modern 

operating environments are highly dynamic, alternative algorithms becomes an important tool to 

achieve portable high performance [22][23]. Quality of service is determined by not only 

completion time but also quality of solution. The question is how to quantify the optimal quality 

of service that can be provided by such a network. 

References 

[1] I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. San 

Francisco: Morgan Kaufmann Publishers, 1999. 

[2] R. Hamadi and B. Benatallah, “A Petri net-based model for web service composition,” in 

Proc. 14th Australasian Database Conf. Database technologies, Adelaide, Australia, 2003, 

17

pp. 191-200. 

[3] I. Foster, C. Kesselman, J. M. Nick, and S. Tuecke, “Grid services for distributed system 

integration,” IEEE Computer, vol. 35, no. 6, pp. 37-46, 2002. 

[4] M. Weiser, “The computer for the 21st century,” Scientific American, vol. 265, no. 3, pp. 

94-104, 1991. 

[5] P. Padala, C. Harrison, N. Pelfort, E. Jansen, M. P. Frank, and C. Chokkareddy, “OCEAN: 

The open computation exchange and arbitration network, A market approach to meta 

computing,” in 

Proc. 2nd Int. Symp. Parallel and Distributed Computing, 2003, pp. 185- 

192. 

[6] 

R. Buyya and S. Vazhkudai, “Compute power market: Towards a market-oriented grid,” in 

Proc. First IEEE/ACM Int. Symp. Cluster Computing and the Grid, 2001, pp.574-581. 

[7] R. Buyya, D. Abramson, and J. Giddy, “Nimrod/G: An architecture for a resource 

management and scheduling system in a global computational grid,” in Proc. 4th Int. Conf. 

High Performance Computing in Asia-Pacific Region, 2000, pp. 283-289. 

[8] B. Meyer, “On to components”, IEEE Computer, vol. 32, no. 1, pp. 139-140, 1999. 

[9] 

P. Clements, “From subroutine to subsystems: Component-based software development,” in 


1996, pp. 3-6. 

[10] D. B. Lange, “Mobile objects and mobile agents: The future of distributed computing?,” in 

Proc. 12th European Conf. Object-Oriented Programming, 1998, pp. 1-12. 

[11] D. Schoder and T. Eymann, “The real challenges of mobile agents,” Communications of the 

ACM, vol. 43, no. 6, 2000, pp.111-112. 

[12] D. B. Lange and M. Oshima, “Seven good reasons for mobile agents,” Communications of 

18

the ACM, vol. 42, no. 3, 1999, pp. 88-89. 

[13] D. Chess, C. Harrison, and A. Kershenbaum, “Mobile agents: Are they a good idea?,” in 

Mobile Object Systems: Towards the Programmable Internet, Lecture Notes in Computer 

Science, vol. 1222, J. Vitek and C. Tschudin, Eds. Springer-Verlag, 1997, pp. 25–47. 

[14] O. H. Ibarra and C. E. Kim, “Heuristic algorithms for scheduling independent tasks on 

nonidentical processors,” Journal of the Association for Computing Machinery, vol. 24, no. 

2, pp. 280-289, 1977. 

[15] D. Fernandez-Baca, “Allocating modules to processors in a distributed system,” IEEE 

Transactions on Software Engineering, vol. 15, no. 11, pp. 1427-1436, 1989. 

[16] J. Regehr, “Some guidelines for proportional share CPU scheduling in general-purpose 

operating systems,” Presented as a work in progress at 22nd IEEE Real-Time Systems 

Symposium, London, UK, Dec. 3-6, 2001. 

[17] I. Stoica, H. Abdel-Wahab, J. Gehrke, K. Jeffay, S. K. Baruah, and C. G. Plexton, “A 

proportional share resource allocation algorithm for real-time, time-shared systems,” in 

Proc. 17th IEEE Real-Time Systems Symposium, 1996, pp. 288-299. 

[18] C. A. Waldspurger and W. E. Weihl, “Lottery scheduling: Flexible proportional-share 

resource management,” in Proc. First Symposium on Operating System Design and 

Implementation, 1994, pp. 1-11. 

[19] C. Waldspurger and W. Weihl, “Stride scheduling: Deterministic proportional-share 

resource management,” Lab. for Computer Science, Massachusetts Institute of Technology, 

Cambridge, MA, Tech. Rep. MIT/LCS/TM-528, 1995. 

[20] C. Waldspurger, “Lottery and stride scheduling: Flexible proportional share resource 

management,” Ph.D. dissertation, Lab. for Computer Science, Massachusetts Institute of 

19

Technology, Cambridge, MA, 1995. 

[21] T. D. Braun, H. J. Siegel, N. Beck, L. L. Bölöni, M. Maheswaran, A. I. Reuther, J. P. 

Robertson, M. D. Theys, B. Yao, D. Hensgen, and R. F. Freund, “A comparison of eleven 

static heuristics for mapping a class of independent tasks onto heterogeneous distributed 

computing systems,” Journal of Parallel and Distributed Computing, vol. 61, pp. 810-837, 

2001. 






20

Manuscript for IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED SYSTEMS 1 

MARKET-BASED MODEL PREDICTIVE CONTROL FOR LARGE-SCALE 

INFORMATION NETWORKS: COMPLETION TIME AND VALUE OF SOLUTION 





{stonesky, skumara, ngautam}@psu.edu 

ABSTRACT 

There are several important properties of modern software systems. They tend to be largescale 

with distributed and component-based architectures. Also, dynamic nature of operating 

environments leads them to utilize alternative algorithms. However, on the other hand, these 

properties make it hard to provide appropriate control mechanisms due to the increased 

complexity. Components are sharing resources and each component can have alternative 

algorithms. As a result, the behavior of a software system can be controlled through resource 

allocation as well as algorithm selection. This novel control problem is worthy of investigation in 

order to double the benefits of those properties. In this paper we design a scalable control 

mechanism for such systems. The quality of service we are considering is a product of the value 

of solution and the time for generating solution for a given problem. We build a mathematical 

programming model that trade off these two conflicting objectives and decentralize the model 

through an auction market. By periodically opening the auction market for each existing system 

state, a closed-loop policy is formed. We verify the designed control mechanism empirically. 

Index Terms: Distributed applications, modeling and prediction, optimization, scalability



The growth in complexity and size of software systems due to automation or organizational 

integration is leading to the increasing importance of distributed and component-based 

architectures. Distributed computing aims at using computing power of machines connected by a 

network. When a task requires intensive computation, it becomes natural choice to achieve high 

performance. A component is a reusable program element. Component technology utilizes the 

components so that developers can build systems needed by simply defining their specific roles 

and wiring them together [1][2]. In networks with component-based architecture, each 

component is highly specialized for specific tasks. Another emerging technology is adaptive 

software [3][4]. Adaptive software has alternative algorithms for the same numerical problem 

and a switching function for selecting the best algorithm in response to environmental changes. 

As modern operating environments are highly dynamic, adaptive software becomes an important 

tool to achieve portable high performance. 

We study a large-scale information network (with respect to the number of components as 

well as machines) comprising of distributed software components linked together through a task 

flow structure. A problem given to the network is decomposed in terms of root tasks for some 

components and those tasks are propagated through a task flow structure to other components. 

As a problem can be decomposed with respect to space, time, or both, a component can have 

multiple root tasks that can be considered independent and identical in their nature. The service 

provided by the network is to produce a global solution to a given problem, which is an 

aggregate of partial solutions of individual tasks. Each component can have alternative 

algorithms to process a task which trade off processing time and value of partial solution. Quality 

of Service (QoS) of the network is determined by the value of global solution and the time for


generating global solution (i.e., completion time). For a given topology, the network can control 

its behavior by utilizing two different kinds of control actions: algorithm selection and resource 

allocation. While resource allocation tries to efficiently utilize limited resources, algorithm 

selection can change the amount of required resources. The resource allocation we are addressing 

here, is allocating resources of each machine to the residing components for a given topology. As 

problems are decomposed in various ways depending on their nature and size, and their QoS 

functions are context-dependent, the network needs to provide adaptive solutions to given 

problems by utilizing such control actions. 

One can imagine wide range of scientific and engineering problems that can be solved by 

such a network. UltraLog (http://www.ultralog.net) networks, implemented in Cougaar 

(Cognitive Agent Architecture: http://www.cougaar.org) developed by DARPA (Defense 

Advanced Research Project Agency), are the instances [5]-[9]. Each agent in these networks 

represents an organization of military supply chain and has a set of components specialized for 

each functionality (allocation, expansion, inventory management, etc) and class (ammunition, 

water, fuel, etc). The objective of an UltraLog network is to provide an appropriate logistics plan 

for a given military operational plan. A logistics plan is a global solution which is an aggregate 

of individual schedules built by components. An operational plan is decomposed into logistics 

requirements of each thread for each agent, and a requirement is further decomposed into root 

tasks (one task per day) for a designated component. As a result, a component can have hundreds 

of root tasks depending on the horizon of an operation and thousands of tasks to process as the 

root tasks are propagated. As the scale of operation increases there can be thousands of agents 

(tens of thousands of components) in hundreds of machines working together to generate a 

logistics plan. QoS of these networks is determined by the quality of logistics plan (value of


solution) and (plan) completion time. These two metrics directly affect the performance of the 

operation. 

In this paper we design a control mechanism for such novel networks. We stress scalability 

with respect to computational complexity as well as communicational overhead, as an important 

consideration of the control mechanism for its practical use. The control mechanism should be 

able to supply appropriate control policy in a timely manner even though the size of the network 

is large. Such a property is important especially when completion time is an explicit 

consideration as in our control problem. However, the property is hard to achieve in general if 

one pursues exactly optimal policy. Therefore, we design a scalable control mechanism by 

sacrificing some amount of optimality in a systematic way as follows. 

First, we adopt Model Predictive Control (MPC) as our control framework. In MPC, for each 

current state, an optimal open-loop control policy is designed for finite-time horizon by solving a 

static mathematical programming model [10]-[13]. The design process is repeated for the next 

observed state feedback forming a closed-loop policy reactive to each current system state. 

Though MPC does not give absolutely optimal policy in stochastic environments, the periodic 

design process alleviates the impacts of stochasticity. Note that technologies such as Dynamic 

Programming are not efficient in terms of computational complexity as they try to give optimal 

closed-loop control policy. Second, under MPC framework, we build a heuristic programming 

model due to computational complexity. The heuristic model is solvable in polynomial time and 

its solution converges to the solution of exact model in the limit of large number of tasks. Third, 

we provide a decentralized coordination mechanism for solving the programming model. 

Computations and communications are distributed to multiple entities through an auction market 

while giving a solution equivalent to the solution of the programming model.


The organization of this paper is as follows. In Section 2 we formally define the problem in 

detail. After designing the control mechanism in Sections 3 and 4, we show empirical results in 

Section 5. Finally, we discuss implications and possible extensions of our work in Section 6. 


In this section we formally define the control problem by detailing network configuration and 

control actions. We focus on computational CPU resources assuming that the system is 

computation-bounded. 

2.1 Network configuration 

A network is composed of a set of components A and a set of nodes (i.e., machines) N. K n 

denotes a set of components that reside in node n sharing the node’s CPU resource. Task flow 

structure of the network, which defines precedence relationship between components, is an 

arbitrary directed acyclic graph. A problem given to the network is decomposed in terms of root 

tasks for some components and those tasks are propagated through the task flow structure. Each 

component processes one of the tasks in its queue (which has root tasks as well as tasks from 

predecessor components) and then sends it to successor components. We denote the number of 

root tasks of component i as rt i . Fig. 1 shows an example network in which there are four 

components residing in three nodes. Components A 1 and A 2 reside in N 1 and each of them has 

100 root tasks. A 3 in N 2 and A 4 in N 3 have no root tasks, but they have 200 and 100 tasks 

respectively from the corresponding predecessors.


 

A 1 

A 3 

N 2 

N 1 

100 

0 

 

A 2 A 4 

N 3 

100 

0 

Fig. 1. An example network 

2.2 Control actions 

The network can utilize two different kinds of control actions in controlling its behavior: 

algorithm selection and resource allocation. 

Algorithm selection 

A component can use one of alternative algorithms to process a task. Different alternatives 

trade off CPU time and value of solution with more CPU time resulting in higher solution value. 

As one can find optimal mixed alternatives, a component has a monotonically increasing 

piecewise-linear convex function, say value function, with CPU time as a function of value. We 

call the value in the function as value mode that a component can select as its decision variable. 

A value function is defined with three elements as f v ), v v 〉 as shown in Fig. 1. 

〈 i ( i i(min), 

i(max) 

This function indicates that component i’s expected CPU time 1 to process a task is f i (v i ) with a 

value mode v i and v i(min) ≤ v i ≤ v i(max) . We assume that components cannot change the mode for a 

task in process. 

Resource allocation 

When there are multiple components in a node, the network needs to control its behavior 

through resource allocation. In the example network, node N 1 has two components and the 

1 The distribution of CPU time can be arbitrary though we use only expected CPU time.


system performance can depend on its resource allocation to these two components. There are 

several CPU scheduling algorithms for allocating a CPU resource amongst multiple threads. 

Among the scheduling algorithms, proportional CPU share (PS) scheduling is known for its 

simplicity, flexibility, and fairness [14]. In PS scheduling threads are assigned weights and 

resource shares are determined proportional to the weights [15]. Excess CPU time from some 

threads is allocated fairly to other threads. There are many PS scheduling algorithms such as 

Weighted Round-Robin scheduling, Lottery scheduling, and Stride scheduling [16]-[18]. We 

adopt PS scheduling as resource allocation scheme because of its generality in addition to the 

benefits mentioned above. We define resource allocation variable set w = {w i (t): i∈A, t≥0} in 

which w i (t) is a non-negative weight of component i at time t. If total managed weight of a node 

n is ω n , the boundary condition for assigning weights over time can be described as: 

∑ 

i∈K 

n 

wi 

( t) 

= ωn 

where wi 

( t) 

≥ 0 . (1) 


The service provided by the network is to produce a global solution to a given problem, which 

is an aggregate of partial solutions of individual tasks. QoS of the network is determined by the 

value of global solution and the cost of completion time. The value of global solution is the 

summation of partial solution values, and the cost of completion time is determined by a cost 

function CCT(T) which is a monotonically increasing function with completion time T. We 

assume that the solution values and cost are represented in a common unit 2 . Consider v d i as the 

value mode used to process d th task by component i and e i the number of tasks processed by 

component i to the completion. Then, the control objective is to maximize QoS by utilizing 

2 Relative importance can be considered by scaling the functions and it results in the same function structures.


algorithm selection (v) and resource allocation (w) as in (2). As stated earlier, we design a 

scalable control mechanism to achieve the objective in the framework of MPC by building a 

mathematical programming model and decentralizing it. 

arg max 

v,w 

e 

i 

∑∑ 

i∈ A d = 1 

v 

d 

i 

− CCT(T ) 

(2) 

3. Mathematical programming model 

The mathematical programming model is essentially a scheduling problem formulation. There 

are a variety of formulations and algorithms available for diverse scheduling problems in the 

context of multiprocessor, manufacturing, and project management. In general, a scheduling 

problem is allocating limited resources to a set of tasks to optimize a specific objective. One 

widely studied objective is completion time (also called makespan in the manufacturing 

literature) as the problem we have considered. Though it is not easy to find a problem exactly 

same as ours, it is possible to convert our problem into one of the scheduling problems. For 

example, in job shop, there are a set of jobs and a set of machines. Each job has a set of serial 

operations and each operation should be processed on a specific machine. A job shop scheduling 

problem is sequencing the operations in each machine by satisfying a set of job precedence 

constraints such that the completion time is minimized. When we assign a value mode to each 

task, our problem can be exactly transformed into a job shop scheduling problem. However, 

scheduling problems are in general intractable. Though the job shop scheduling problem is 

polynomially solvable when there are two machines and each job has two operations, it becomes 

NP-hard on the number of jobs even if the number of machines or operations is more than two 

[19][20]. Considering that the task flow structure of our networks is arbitrary, our scheduling


problem is NP-hard on the number of components in general. The increase of the number of 

tasks and consideration of alternative algorithms make the problem even harder. Moreover, there 

can be large number of nodes in our networks. 

Though it may be possible to use some available heuristic algorithms from the job shop 

scheduling problem by taking into account alternative algorithms, our scheduling problem has a 

particular characteristic, i.e., the number of tasks for each component can be large. Though the 

increase of the number of tasks adds more complexity, it can also lead us to develop an efficient 

heuristic programming model. In this section, we characterize an optimal resource allocation by 

analyzing the impacts of the largeness and subsequently build a mathematical programming 

model solvable in polynomial time. 

3.1 Optimal resource allocation 

Consider current time t=0 and assume that each component uses a value mode common to all 

the tasks (i.e. pure strategy). We will discuss the optimality of the pure strategy later in this 

subsection. We define Load Index LI i which represents component i’s total CPU time required to 

process its tasks. As a component needs to process its own root tasks as well as incoming tasks 

from its predecessors, its number of tasks L i is identified as in (3), where i denotes the immediate 

predecessors of component i. Then, LI i is represented as in (4). 

∑ 

L i = rti 

+ La 

(3) 

a∈i 

LI = L f v ) 

(4) 

i 

To provide theoretical foundation of optimal resource allocation policy, we convert a network 

into a network with tasks having infinitesimal processing times. Each root task is divided into r 

infinitesimal tasks and each f i (v i ) is replaced with f i (v i )/r. Then, the load index of each component 

i 

i ( i


is the same as the original network but tasks are infinitesimal. We denote the completion time of 

the network with infinitesimal tasks as T´. Also, we define a term called task availability as an 

indicator of relative preference for task arrival patterns. An arrival pattern gives higher task 

availability than another if cumulative number of arrived tasks is larger or equal over time. A 

component prefers a task arrival pattern with higher task availability as it can utilize more 

resource. Consider a network and reconfigure it such that all components have their tasks in their 

queues at t=0. Each component has maximal task availability in the reconfigured network and the 

completion time of the reconfigured network forms the lower bound T LB 

of a network’s 

completion time T given by: 

LB 

n∈N 

∑ 

T = Max LI i . (5) 

i∈ 

K n 

For theoretical analysis, we assume a hypothetical weighted round-robin server for CPU 

scheduling though it is not strictly required in practice as will be discussed. The hypothetical 

server has idealized fairness as the CPU time received by each thread in a round is infinitesimal 

and proportional to the weight of the thread. 

Theorem 1. T´ equals to T LB when each node allocates its resource proportional to its residing 

components’ load indices as: 

LI i 

wi 

( t) 

= wi 

= ω n( 

i) 

for all i ∈ A and t ≥ 0 , (6) 

LI 

∑ 

p∈K 

n( 

i) 

where n(i) denotes a node in which component i resides. 

p 

Proof. A component’s instantaneous resource availability RA i (t), which is the available fraction 

of a resource when the component requests the resource at time t, is more than or equal to 

assigned weight proportion as:


w i( t ) 

RA i( t ) ≥ for t ≥ 0 . (7) 

ω 

n( i ) 

Service time S i (t) is the time taken to process a task at time t and has a relationship with RA i (t) 

as: 

t Si( 

∫ + 

t 

t ) 

RA ( τ )dτ 

= 

i 

f 

i 

( v 

i 

). (8) 

Suppose a component i receives its tasks at a constant interval of T LB /L i . Then, under 

proportional allocation, S i (t) is less than or equal to T LB /L i over time as shown in (9). 

f 

= 

i 

( v 

i 

p∈K 

) = 

LI 

∑ 

n( 

i 

LI 

i ) 

∫ 

p 

t+ 

S ( 

t 

i 

t ) 

RA ( τ )dτ 

≥ 

LI 

Si( t ) ≥ 

T 

i 

i 

LB 

S ( t ) 

i 

∫ 

t+ 

S ( 

t 

i 

t ) 

T 

⇒ 

L 

w i( t ) w 

dτ 

= 

ω ω 

LB 

i 

n( 

i ) 

≥ S ( t ) 

i 

n( 

i 

i ) 

S ( t ) 

i 

for t ≥ 0 

(9) 

So, any component can complete by T LB and generate tasks at a constant interval of T LB /L i 

from t=T LB /L i (first task generation time) under proportional allocation when it receives tasks at 

a constant interval of T LB /L i from t=0 (first task arrival time). As tasks are infinitesimal and root 

tasks increase task availability, each component can receive infinitesimal tasks at a constant 

interval in 0≤t≤T LB or more preferably, and complete at less than or equal to T LB . So, the 

network completes at T LB under proportional allocation. 

 

From Theorem 1 we can conjecture that a network can achieve a performance close to T LB 

under proportional allocation in the limit of large number of tasks. If nodes do not follow the 

proportional allocation policy, some components can receive their tasks less preferably than 

constant interval resulting in underutilization and consequently increased completion time. Also, 

it is optimal for each component to use a pure strategy. Each component’s optimal strategy in the


network with maximal task availability is a pure strategy due to the convexity of value functions, 

and a network can achieve the optimal performance under proportional allocation. Though we 

assumed a hypothetical weighted round-robin server which is difficult to realize in practice, the 

arguments do not seem to be invalid because they are based on worst-case analysis and quantum 

size is relatively infinitesimal compared to working horizon in reality. 

3.2 Programming model 

As discussed, each component’s optimal strategy is a pure strategy and the completion time T 

is close to T LB under proportional resource allocation in the limit of large number of tasks. Now, 

consider current time as t. To update load index as the system moves on, we slightly modify it to 

represent the total CPU time for the remaining tasks as: 

LI t) 

= R ( t) 

+ L ( t) 

f ( v ) , (10) 

i ( i i i i 

in which R i (t) denotes remaining CPU time for a task in process and L i (t) the number of 

remaining tasks excluding a task in process. After identifying initial number of tasks L i (0)=L i , 

each component updates it by counting down as they process tasks. 

Then, under proportional resource allocation, the completion time T can be estimated as: 

n∈N 

∑ 

T − t ≈ Max [ Ri 

( t) 

+ Li 

( t) 

fi 

( vi 

)] . (11) 

i∈ 

K n 

The estimation leads to building a programming model in a straightforward way. Given 

completion time T it is optimal for a node n to select a mode by the following: 

∑ 

Max Li ( t ) v i 

(12) 

i∈K 

n 

subject to 

∑ 

i∈ 

K n 

[ Ri 

( t) 

+ Li 

( t) 

fi 

( vi 

)] ≤ T − t . (13)


Consequently, the programming model can be formulated with two sub-models: optimization 

model as in (14) and resource allocation model as in (15). The optimization model maximizes 

QoS by trading off the value of solution and the cost of completion time, and the resource 

allocation model allocates resources proportional to the load indices of residing components 

based on the solution of (14). 

 

Programming model 

Max 

s.t. 

∑ 

i∈A 

∑ 

i∈K 

v 

L ( t )v 

n 

i 

i(min) 

[ R ( t ) + L ( t 

i 

≤ v 

i 

i 

− CCT(T ) 

≤ v 

i 

) f 

i(max) 

i 

( v )] ≤ T − t 

i 

for all 

for all 

n ∈ N 

i ∈ A 

(14) 

w 

* 

i 

= 

* 

Ri 

( t) 

+ Li 

( t) 

fi 

( vi 

) 

ω 

* n( 

i) 

∑[ 

R p ( t) 

+ L p ( t) 

f p ( v p )] 

(15) 

p∈K 

n( 

i) 

The optimal QoS from (14) with t=0 forms a QoS upper bound QoS UB and a network can 

achieve a performance close to QoS UB in the limit of large number of tasks. The programming 

model is efficient in terms of complexity because the two different kinds of control actions are 

completely separated. It is solvable in polynomial time as will be discussed in the next section. 

4. Decentralization 

The next question is how to decentralize the mathematical programming model. Centralized 

control mechanisms scale badly, due to the rapid increase of computational and communicational 

overheads with system size. Single point failure of the controller will often lead to failure of the 

complete system leading to non-robust network. Decentralization can address these issues by


distributing the computations and communications to multiple entities in the system. There are 

two popular methods of decentralizing structured programming models: decomposition methods 

and auction/bidding algorithms. Considering the compatible structure of the programming 

model, we decentralize it through a non-iterative auction mechanism, so called multiple-unit 

auction with variable supply [21]. In this auction a seller may be able and willing to adjust the 

supply as a function of bidding. 

4.1 Auction market design 

In the programming model we have built, all nodes and components are coupled with each 

other. However, it has a typical structure, where objective function and constraints are separable 

to each node if one variable T is fixed. This characteristic makes it possible to solve the model 

through an auctioning process for T. The completion time T is an unbounded resource and the 

supply can be adjusted as a function of bidding. 

To design the auction market we define two different types of participants in addition to the 

components: Seller and Resource Manager. There is one seller in the system which determines 

T * based on the bids from resource managers. A resource manager of each node manages the 

resource of the node and arbitrates between its components and the seller. 

We define T i as available resource of component i which is required minimally to the amount 

of T i(min) as in (16) and maximally T i(max) as in (17). 

T 

T 

= [ R ( t ) L ( t ) f ( v )] 

(16) 

i (min) i + 

i 

i 

i(min) 

= [ R ( t ) L ( t ) f ( v )] 

(17) 

i (max) i + 

i 

i 

i(max) 

A component i bids to its resource manager with maximal value as a function of T i as in (18). 

The resource manager bids to the seller with maximal total value of its components as a function 

of T based on the bids from its components as in (19). The seller decides T * based on the bids


from resource managers by taking into account the cost of T as in (20). After the seller 

broadcasts T * , each resource manager decides T * i and w * i as in (21) and (22). In (21) T * i is less 

than or equal to the maximally required resource T i(max) so that the resource can be allocated 

proportional to the components’ load indices. Each component selects optimal value mode in the 

limit of T * i as in (23). This auctioning process gives an equivalent solution to the programming 

model. 

 

Auctioning model 

Component’s bid 

b (T ) = −∞ 

i 

i 

= L ( t )v 

i 

= L ( t ) f 

i 

i(max) 

−1 

i 

(Ti 

− Ri( t ) 

( ) 

L ( t ) 

i 

if T 

if T 

else 

i 

i 

< T 

> T 

i(min) 

i(max) 

(18) 

Resource manager’s bid 

b (T ) = −∞ 

n 

= 

∑ 

i∈K 

n 

b (T 

i 

= Max { 

i∈K 

i(max) 

∑ 

n 

) 

b (T 

i 

i 

) : 

∑ 

i∈K 

n 

T 

i 

≤ T − t } 

if T < 

if T > 

else 

∑ 

i∈K 

i∈K 

n 

∑ 

n 

T 

T 

i(min) 

i(max) 

(19) 

Seller’s decision 

T 

* 

= argmax ∑bn 

( T ) − CCT ( T ) 

T 

n∈N 

(20) 

Resource manager’s decision 

{T 

* 

i 

: i ∈ K n } = arg max { bi 

(Ti 

) : Ti 

≤ min(T − t, Ti(max) 

)} 

(21) 

{ T :i∈K 

} 

i 

n 

∑ 

i∈K 

n 

∑ 

i∈K 

n 

* 

∑ 

i∈K 

n


w 

* 

i 

= 

T 

p∈K 

* 

i 

∑ 

T 

n( 

i) 

* 

p 

ω 

n( 

i) 

(22) 

Component’s decision 

v 

* 

i 

* 

1 ( Ti 

− Ri 

( t) 

= f 

− i ( ) 

(23) 

L ( t) 

i 

4.2 Analysis 

Resource manager’s bidding function b n (T) in (19) can be composed referring to the solution 

algorithm of fractional knapsack problem. In the fractional knapsack problem, there are multiple 

items that can be broken into fractions. Given unit weight and unit value of each item, the 

problem is to determine the amount of each item so as to maximize total value subject to a 

weight capacity. The fractional knapsack problem can be easily solved by a greedy algorithm, 

i.e., take as much as possible of the item that is the most valuable per unit weight until the 

capacity is reached. Similarly, b n (T) can be composed using a greedy algorithm. As b i (T i ) in (18) 

is a piecewise-linear increasing concave function, take the most valuable piece per unit T i among 

the first available pieces until all pieces are taken. This greedy algorithm leads building the 

resource manager’s bidding function in O(|K n | 2 ), where |X| denotes the cardinality of set X. 

Similarly, resource manager’s decision problem in (21) can be solved in O(|K n | 2 ), using the 

greedy algorithm except that (fractional) pieces are taken until a capacity is reached. So, the 

complexity of all resource managers’ local problems is O(|A| 2 ) in the worst case when |N|=1. 

The seller’s decision problem in (20) is simply a single variable problem, which can be solved 

using diverse search methods depending on the structure of objective function. As each b n (T) is 

piecewise-linear increasing concave function, ∑b n (T) is also a piecewise-linear increasing 

concave function and its number of pieces is proportional to |A|. To compose ∑b n (T) from each


b n (T), sort the starting T of each piece in ascending order (O(|A|log|A|)), and, for each T, 

summate from each b n (T) by moving on to corresponding pieces (O(|A||N|)). So, the seller can 

compose ∑b n (T) in O(|A| 2 ) in worst case when |N|=|A|. Once ∑b n (T) is composed, the 

complexity of the decision problem is proportional to the number of pieces and solvable in 

O(|A|). So, the seller’s decision problem is solvable in O(|A| 2 ). The complexity of other local 

problems such as (18), (22), and (23) is O(|A|). 

Therefore, if the auctioning model is solved in a centralized controller, it is solvable in 

O(|A| 2 ). That is, the complexity of the programming model is O(|A| 2 ). However, the auctioning 

model improves scalability as computations and communications are distributed to multiple 

market participants. Components as well as resource managers are solving their local problems 

in parallel rather than sequentially. This parallel processing reduces the time taken to solve the 

programming model. In addition, the participants communicate locally in terms of bids rather 

than all details to a centralized controller. 


We ran several experiments using discrete-event simulation to validate the designed control 

mechanism. Though we use a small network in the experimentation for validation purpose, the 

decentralized model, especially, can handle much larger networks. 

5.1 Experimental design 

The experimental network is composed of sixteen components in seven nodes as shown in 

Fig. 2. Each component in the lowest position has root tasks as indicated in the figure. The value 

function is for A 7 and A 8 , and for others. Also, ω n is 1 for all n∈N and


CPU is allocated using a weighted round-robin scheduling in which CPU time received by each 

component in a round is equal to its assigned weight. 

N 1 N 2 N 3 

N 4 

A 1 A 2 

A 3 A 4 

N 5 

A 5 A 6 

N 6 

A 7 A 8 

A 9 A 10 A 11 A 12 A 13 A 14 A 15 A 16 

200 200 200 200 400 400 400 400 

N 7 

Fig. 2. Experimental network configuration 

We set up six different experimental conditions as shown in Table 1. We vary the cost of 

completion time and the distribution of CPU time can be deterministic or exponential. While 

using stochastic value function we repeat 5 experiments. QoS UB is calculated from (14) with t=0 

for each condition as shown in the table. 


Condition CCT(T) f i (v i ) QoS UB 

Con1-1 0.5T Deterministic 30000 

Con1-2 0.5T Exponential 30000 





We use ten different control policies for each experimental condition as shown in Table 2. 

First eight control policies (FX-XX) use fixed value modes over time. In predictive control 

policies (PC-XX) components selects value modes by solving the optimization model in (14). In 

round-robin resource allocation (XX-RR) the components in each node are assigned equal


weights and in proportional allocation (XX-PA) proportional to the components’ load indices as 

in (15). PC-PA is the control policy corresponding to the programming model we have 

developed. The system makes decision every 100 time units. 

Table 2. Control policies used for experimentation 


F2-RR 

F2-PA 

F3-RR 

F3-PA 

F4-RR 

F4-PA 

F5-RR 

F5-PA 

PC-RR 

PC-PA 

Description 

v i = 2 for all i with round-robin allocation 

v i = 2 for all i with proportional allocation 







Predictive control with round-robin allocation 

Predictive control with proportional allocation 

5.2 Results 

Numerical results from the experimentation are shown in Table 3. PC-PA gives the best 

performance close to QoS UB in all different conditions. As the cost of completion time increases 

the system under PC-PA completes earlier as a result of trading off between the value of solution 

and the cost of completion time. There can be seen many cases in which the value of solution 

under PC-PA is even larger in spite of less completion time. It is because the programming 

model gives the maximal value of solution for a given completion time. 

Though both PC-PA and PC-RR choose value modes by solving the optimization model in 

(14), PC-RR gives worse performance because the optimization model is built presuming 

proportional resource allocation. Proportional allocation shows significant advantages compared 

to round-robin allocation in all thirty instances of comparison. The superiority supports the 

optimality of proportional resource allocation and consequently the effectiveness of the 

programming model.



Control Policy 

F2-RR F2-PA F3-RR F3-PA F4-RR F4-PA F5-RR F5-PA PC-RR PC-PA 

T 5614 4814 8019 7219 10423 9624 12828 12028 12828 12028 

Con1-1 

V 14400 14400 21600 21600 28800 28800 36000 36000 36000 36000 

QoS 11593 11993 17590 17990 23588 23988 29585 29985 29585 29985 

% 0.386 0.400 0.586 0.600 0.786 0.800 0.986 1.000 0.986 1.000 

T 5592 4993 8093 7114 10356 9593 12846 11885 12846 11885 

Con1-2 

V 14400 14400 21600 21600 28800 28800 36000 36000 36000 36000 

QoS 11604 11903 17553 18043 23622 24004 29577 30058 29577 30058 

% 0.387 0.397 0.585 0.601 0.787 0.800 0.986 1.002 0.986 1.002 

T 5614 4814 8019 7219 10423 9624 12828 12028 11282 9742 

Con2-1 

V 14400 14400 21600 21600 28800 28800 36000 36000 34283 33700 

QoS 5980 7179 9572 10771 13164 14364 16757 17956 17359 19087 

% 0.311 0.374 0.499 0.561 0.686 0.748 0.873 0.935 0.904 0.994 

T 5592 4993 8093 7114 10356 9593 12846 11885 11313 10062 

Con2-2 

V 14400 14400 21600 21600 28800 28800 36000 36000 34183 33845 

QoS 6011 6910 9460 10929 13266 14411 16731 18173 17214 18752 

% 0.313 0.360 0.493 0.569 0.691 0.751 0.871 0.947 0.897 0.977 

T 5614 4814 8019 7219 10423 9624 12828 12028 6171 4881 

Con3-1 

V 14400 14400 21600 21600 28800 28800 36000 36000 25354 24055 

QoS 366 2365 1553 3553 2741 4740 3928 5928 9927 11853 

% 0.031 0.197 0.129 0.296 0.228 0.395 0.327 0.494 0.827 0.988 

T 5592 4993 8093 7114 10356 9593 12846 11885 6309 5089 

Con3-2 

V 14400 14400 21600 21600 28800 28800 36000 36000 25593 24277 

QoS 419 1917 1367 3816 2910 4818 3886 6289 9820 11554 

% 0.035 0.160 0.114 0.318 0.243 0.402 0.324 0.524 0.818 0.963 

T: Completion time, V: Value of solution, %: QoS/QoS UB 


The increasing complexity of modern software systems gives rise to the needs for more 

sophisticated but scalable control mechanisms. In this paper we designed such a control 

mechanism for an emerging information network. The network is large-scale with distributed 

and component-based architectures, and its behavior can be controlled by algorithm selection 

and resource allocation. In the designed control mechanism, an auction market coordinates the 

components of a network to produce optimal decisions and the market opens periodically for 

each current system state.


Our work can be extended by providing adaptivity to changing stress environments. As the 

modern systems can be easily exposed to various adverse events such as accidental failures and 

malicious attacks, there is a need to adapt to such environments. Because the adverse events 

affect the system by limiting available resources, it would be possible to model such 

environments by quantifying the resource availability of the system through appropriate sensors. 


The authors acknowledge the support for this research provided by DARPA (Grant#: 

MDA972-01-1-0038) under the UltraLog program. 

References 

[1] B. Meyer, “On to components”, IEEE Computer, vol. 32, no. 1, pp. 139-140, 1999. 

[2] P. Clements, “From subroutine to subsystems: Component-based software development,” in 


1996, pp. 3-6. 






[5] D. Moore, W. Wright, and R. Kilmer, “Control surfaces for Cougaar,” in Proc. First Open 


[6] W. Peng, V. Manikonda, and S. Kumara, “Understanding agent societies using distributed


monitoring and profiling,” in Proc. First Open Cougaar Conference, 2004, pp. 53-60. 

[7] H. Gupta, Y. Hong, H. P. Thadakamalla, V. Manikonda, S. Kumara, and W. Peng, “Using 

predictors to improve the robustness of multi-agent systems: Design and implementation in 

Cougaar,” in Proc. First Open Cougaar Conference, 2004, pp. 81-88. 

[8] D. Moore, A. Helsinger, and D. Wells, “Deconfliction in ultra-large MAS: Issues and a 

potential architecture,” in Proc. First Open Cougaar Conference, 2004, pp. 125-133. 

[9] R. D. Snyder and D. C. Mackenzie, “Cougaar agent communities,” in Proc. First Open 


[10] J. B. Rawlings, “Tutorial overview of model predictive control,” IEEE Control Systems, vol. 

20, no. 3, pp. 38-52, 2000. 

[11] M. Morari and J. H. Lee, “Model predictive control: Past, present and future,” Computers 

and Chemical Engineering, vol. 23, no. 4, pp. 667-682, 1999. 

[12] M. Nikolaou, “Model predictive controllers: A critical synthesis of theory and industrial 

needs,” Advances in Chemical Engineering Series, Academic Press, 2001. 

[13] S. J. Qin and T. A. Badgwell, “A survey of industrial model predictive technology,” Control 

Engineering Practice, vol. 11, pp. 733-764, 2003. 

[14] J. Regehr, “Some guidelines for proportional share CPU scheduling in general-purpose 

operating systems,” Presented as a work in progress at 22nd IEEE Real-Time Systems 

Symposium, London, UK, Dec. 3-6, 2001. 

[15] I. Stoica, H. Abdel-Wahab, J. Gehrke, K. Jeffay, S. K. Baruah, and C. G. Plexton, “A 

proportional share resource allocation algorithm for real-time, time-shared systems,” in 

Proc. 17th IEEE Real-Time Systems Symposium, 1996, pp. 288-299. 

[16] C. A. Waldspurger and W. E. Weihl, “Lottery scheduling: Flexible proportional-share


resource management,” in Proc. First Symposium on Operating System Design and 

Implementation, 1994, pp. 1-11. 

[17] C. Waldspurger and W. Weihl, “Stride scheduling: Deterministic proportional-share 

resource management,” Lab. for Computer Science, Massachusetts Institute of Technology, 

Cambridge, MA, Tech. Rep. MIT/LCS/TM-528, 1995. 

[18] C. Waldspurger, “Lottery and stride scheduling: Flexible proportional share resource 

management,” Ph.D. dissertation, Lab. for Computer Science, Massachusetts Institute of 

Technology, Cambridge, MA, 1995. 

[19] T. Gonzalez and S. Sahni, “Flowshop and jobshop schedules: Complexity and 

approximation,” Operations Research, vol. 26, pp. 36-52, 1978. 

[20] J. Lenstra, A. R. Kan, and P. Brucker, “Complexity of machine scheduling problems,” 

Annals of Discrete Mathematics, vol. 1, pp. 343-362, 1977. 

[21] Y. Lengwiler, “The multiple unit auction with variable supply,” Economic Theory, vol. 14, 

no. 2, pp. 373-392, 1999.

Coordinating Control Decisions of Software Agents for Adaptation to Dynamic 

Environments 

Y. Hong 1 , S. R. T. Kumara 1 

1 Harold and Inge Marcus Department of Industrial and Manufacturing Engineering 

The Pennsylvania State University, University Park, PA, 16802, USA 

Abstract 

We suggest a design for an infrastructure-level load control mechanism of a multiagent system, Cougaar. The 

purpose of control is to strengthen the robustness of a software multiagent system with respect to load 

balancing such that the system can keep working without disastrous performance degradation even under 

occasional harsh running environments. Resource control in multiagent systems is carried out mainly by 

agent’s self-control, which makes the control problem very difficult. We suggest a hierarchical control 

structure in order to reduce complexity of control while inducing coherent movement of agents. 

Keywords: 

load balancing, hierarchical control, multi-agent system 

1 INTRODUCTION 

Multiagent systems have significant advantages in the 

development of complex distributed software system [1]. 

Agents are naturally matched to components in complex 

systems. Therefore, complicated interactions among the 

subcomponents can be represented by agent interactions. 

Due to the modularity and autonomy of agents, the 

application could be composed by assembling the agents. 

The multiagent systems are flexible in design. Partial 

changes in the system could be localized for a few agents 

without affecting the rest of the system. Thus, constructing 

or altering a large software system could become easier 

with agent technology. 

In addition to the advantages in designing and constructing 

a large system, robustness is also an important factor for a 

multiagent system to be a good software construction 

technology. Robustness of a software represents “the 

ability of software to react appropriately to abnormal 

circumstances” [2]. Like many biological or man-made 

systems, through feedback controls and redundancy of 

components (agents), the software system can also cope 

with uncertainties in dynamic environments and improve its 

robustness at the expense of increasing complexity [3][4]. 

The time varying computational load could be one of 

threats to robustness. A sudden excessive workload could 

degrade performance to an extent in which the system 

cannot meet minimum requirements on response time. 

This is in specific very critical for real time applications. 

Because agent systems are distributed and decentralized, 

it is hard to build a control mechanism by which agents can 

adapt to the changing environments effectively and 

coherently. In order to resolve this problem, we suggest an 

infrastructure-level load control mechanism for a 

multiagent system, Cougaar. The reason we consider 

infrastructure level control mechanism is that the 

application developers’ efforts to secure robustness of 

software with respect to the load control could be much 

reduced. Multiagent systems such as Cougaar [5] and 

Jade [6] provide many infrastructure level services, which 

save the application developers efforts required to build 

basic functions of the multiagent system. Load control 

function can be included in the infrastructure and its 

necessity has been emphasized [7]. Infrastructure can hide 

the complexity of controlling resource allocation such that 

application developers tune the performance using highlevel 

abstract parameters for load control. 

2. LOAD BALANCING IN MULTIAGENT SYSTEMS 

In multiagent systems, system functions are decomposed 

into software agents. Agents carry out system functions by 

exchanging services with each other [7]. Agents have their 

own work and specialize in a specific service. Agents 

request some service from another agent who is 

specialized in that service. Providing the service requires 

the use of some computational resource such as CPU 

time. Agents are distributed on multiple machines, which 

are connected through communication networks. More 

than one agent can be on a machine and share the CPU 

time. The frequency of service request of each agent is 

time varying depending on real world, which the application 

deals with. 

Considerable research has been done on dynamic load 

balancing for computer clustering. However, we cannot 

apply this directly to a multiagent system [7]. As noted by 

chow and kwok [7], multiagent systems (MAS) are different 

from computer clustering with respect to load balancing. 

Firstly, in MAS, agents are continuously running while in 

computer clustering, jobs submitted by users are killed 

after completion. Secondly, communications between 

agents in multiagent system are highly variable, whereas, 

communications between jobs usually has static patterns. 

Another difference, which is not pointed out by chow and 

kwok, is that agents could proactively manage their 

workload.

Load balancing issues have not been paid much attention 

in MAS studies [7][8]. There are few papers in multiagent 

load balancing. Schaerf et. al. [9] studies how an agent can 

adapt to the environment. They separated the resources 

from the agents. In their model, agents assign their jobs to 

these resources. Using reinforcement learning, they 

showed that agents could adapt to each other under fixed 

or even for dynamical loads. Chow and kwok [7] devise an 

agent reallocation algorithm, called ‘Comet’ algorithm, 

which select agents to be moved to other machines. 

Agents are distributed on multiple machines. The comet 

algorithm chooses agents based on credits, which are 

continuously evaluated for each agent. The agent with low 

credit will be moved. The credit will decrease as the 

agent’s workload increases or the agent has more 

communication with other agents on other machines. We 

consider a similar agent system environment with chow 

and kwok. However, we added a feature of agent’s selfregulation 

on the workload. 

3 QUEUEING MODEL FOR WORKLOAD DYNAMICS 

We conjecture that workload dynamics could be modeled 

as a queueing system. A service request from outside or 

other agents could be seen as a customer in a queueing 

system. While a request is being served, the later incoming 

service requests will wait in the queue. We consider a 

situation in which agents have multiple alternative 

algorithms to provide their service. Those algorithms trade 

off between computation time and quality of solution. 

Thus, depending on the workload in the queue, an agent 

can choose an optimal algorithm to improve the overall 

performance measure. This is similar to anytime algorithm 

composition [10]. In anytime algorithm, we have to 

determine time duration in which the algorithm solves the 

problem. Here, we assume that problem solving time is not 

predetermined. Instead, it is a statistical characteristic of 

the algorithm. From the queueing model perspective, this 

could be seen as a service rate control problem [11]. 

Multiagent system infrastructure can have a facility, where 

each machine works as a server by assigning 

computational resources (run right) to the agents for CPU 

time-sharing. We call the server as a node. This could be 

seen as a polling model, which has been used to model for 

time-sharing in a computer operating system or link 

sharing in a communication network. The node could give 

priority to a certain agent by visiting the agent more 

frequently. The node could monitor the amount of workload 

or arrival rate of service requests through agents. Based 

on the detected changes, it can change the priority of the 

agent. 

Imbalance among machines can be controlled by 

reallocation of agents from a high loaded machine to a low 

loaded machine for better performance. However, in this 

paper, we considered only agent and machine level 

control. 

4 DECENTRALIZED CONTROL 

In view of the above mentioned workload dynamics, load 

control in multiagent systems could be seen as a 

decentralized stochastic control problem [12]. The 

decentralized control system consists of multiple control 

posts. They locally sense and control some part of the 

system they take charge of. However, their controls 

influence the system dynamics collectively. Thus, in order 

to operate the system optimally, decisions taken by 

controllers should be compatible and coherent. The 

information sharing between agents is treated as the main 

issue. In order for the controllers to obtain global optimal 

control decisions, exchange of all the local information is 

inevitable. Each controller makes a decision by solving a 

larger problem in which other agents’ movement is 

considered. However, it is unrealistic in the case of 

multiagent systems because of the long communication 

time. Finding optimal controls might be very difficult due to 

the size of the problem. It is difficult or almost impossible to 

find a purely decentralized optimal control policy for a 

multiagent system in this way. There are few systems in 

which locally made decisions could be globally compatible 

[13]. However this is very limited to some specific problem 

only. Thus, we need a control structure in which each 

control component (for example, an agent) takes control 

decisions by communicating only with closely connected 

components, while well coordinated decisions could be 

generated. In this paper, we suggest a hierarchical control 

structure which aims at achieving the above mentioned 

expectations. 

5 HIERARCHICAL CONTROL 

5.1 General Description 

In order to manage the complexity of large-scale problems, 

hierarchical control approach have been studied in various 

areas [14][15]. For multiagent systems, hierarchical control 

has been adopted as an intermediate form between 

centralized and decentralized control as a tradeoff between 

the advantages of the two approaches [14]. Hierarchical 

control can reduce the computation of gathering 

information and finding an optimal control than centralized 

control. On the other hand, it has better coordination 

capabilities than decentralized control. 

We consider three levels of hierarchy – the entire system, 

nodes and agents. There are usually multiple 

subcomponents under a higher-level controller i.e. there 

are multiple agents under a node and multiple nodes under 

a top controller. Agents and a node controller have direct 

communication connection by which they share 

information. The information sharing is restricted between 

the components, which are connected in the hierarchy. In 

this case, an agent reports its workload and performance 

(see 5.2) and the node controller announces control (the 

visit order and frequency) or state information such as 

estimates about environment parameters, which could be 

more effectively observed by the node rather than agents. 

We could also consider similar information exchanges 

between nodes and the top controller. Here, nodes report 

their node level workload trend to the top controller. On the 

other hand, the top controller could inform the system level 

environment parameters and order to move agents from 

one node to another. 

We assume the control frequency to be different at 

different levels. It is higher in lower levels compared to the 

higher levels. Control decisions on service rate are more 

frequent than the changes on the CPU time assignment 

policy in the node level. For a given arrival rate and 

configuration, CPU time assignment policy in node level 

will not be changed until the arrival rate and configuration 

change. At higher level, the frequency of events is less and 

the time intervals between events become longer. This 

could be said to have multi-time scale depending on the 

level [16]. The difference of our problem from other multitime 

scale problems is that there are multiple components 

in the lower levels. In addition, it is reasonable to assume 

that the environment does not change very frequently, in 

such a manner that the system may not be able to 

estimate environment parameters and control it. 

A Higher-level controller has coarser information than the 

lower level subsystems. A higher-level controller could 

have better global information over its territory because it 

collects information from its subcomponents. However, it 

will not use the information gathered as a system state

directly. It has coarser scale. It will neglect a certain range 

of fluctuations in the measurements from subcomponent. 

In this framework, controls in a level do not affect the 

higher level. However, higher-level controls constrain the 

lower level components’ working conditions and thus 

decreasing the degree of freedom on the control problem 

of the lower level. 

Now we want to show our ideas on the optimal control 

problem in which the features of hierarchical control 

structure are reflected. 

5.2 Optimal Control Problem 

The load control problem is to find optimal control policies 

for each component such that it minimizes the long run 

cost of the overall system, while it is subjected to time 

varying computational workload. The cost function we want 

to optimize through load control is a multi-objective 

function of holding cost and penalty cost for service quality. 

The performance is measured for each agent 

independently. The performance of the overall system is 

assumed to be the sum of the individual agent’s 

performance. 

Each agent could have different algorithms for a service 

depending on the service type. For simplicity, we assume 

that every agent has two algorithms, called level 6 and 

level 2 respectively. The quality of solutions for Level 6 

algorithm is higher than Level 2 on an average while the 

computation time for Level 2 is less than that of Level 6. 

The default algorithm is the Level 6 algorithm. We will 

impose a penalty cost for using level 2 algorithm. Thus, 

when the system is congested, using level 2 algorithm 

would be helpful, even though it incurs the penalty cost. 

There could be various stresses such as (1) the 

unexpected increase in service request, and (2) loss of 

CPU time due to other applications. Stress (1) could be 

seen as an arrival rate change in queueing model. In agent 

systems, arrival of service requests could be time varying 

and bursty. Stress (2) could be seen as a server in a 

polling system serving an imaginary additional queue (an 

agent) at random time and for random duration. The 

sources of these two stresses in (1) and (2) are 

environmental factors, which we cannot control. Thus, we 

have to model them as random events. The controller will 

choose control action considering estimations about these 

events. 

Increasing arrival rate could be modeled using a Markov 

Modulated Poisson Process (MMPP). MMPP describes an 

arrival rate, which changes depending on the source state 

where if the source state is i, the arrival process is Poisson 

with arrival rate λ i . The source state is modeled as 

continuous time markov chain. [16] 

Depending on the level, the problem definitions are 

different. In the perspective of the agent, the problem is to 

find optimal service rate in a situation where there are 

random server vacations. The server vacation means that 

other agents get a run-right and process their work. The 

frequency to visit the agent is controlled by the node. The 

agent should find an optimal service control policy for the 

given expected vacation time. Taking into account the 

service rate in server vacation problem has not been 

studied so far to our best knowledge. Usually in server 

vacation control problems, the optimal service beginning 

time or the optimal time to add additional server 

(removable server case) have been studied [18]. 

A node has a problem of finding polling policy. The runright 

assignment frequency will not change frequently 

because we assumed that the changes in the 

circumstances around agents are not frequent. Thus for a 

given node state, the problem is to find static polling table 

[19]. Whenever the node state changes, the node will pick 

another appropriate polling table. 

The problem is fairly complex because the self-regulating 

agents share a single resource CPU. They are 

interdependent. One agent’s decision could affect other 

agents’ waiting time for CPU. This feature makes our 

problem significantly different from other queueing models. 

6 HIERARCHICAL CONTROL FOR COUGAAR 

In this section, we show how to apply our hierarchical 

control ideas to a multiagent system, Cougaar [5]. We use 

Cougaar version 10.2.1. 

6.1 The Cougaar Infrastructure 

Distinguishing features of Cougaar are blackboard 

communication and plugins [5]. An agent consists of 

plugins that implement agents’ functions. In Cougaar, it is 

recommended that the functions of agents should be well 

divided into the sufficiently small units of program modules, 

plugins. They communicate by posting and reading 

messages on the blackboard that exists in every agent. 

Communications between agents are also conducted 

through the blackboard. 

Cougaar runs its code – plugins or some infrastructural 

level modules via shared thread. The total number of 

shared threads is limited in a node. This means that the 

number of simultaneously running plugins could not be 

greater than the upper limit. The number of available 

threads for the node is predetermined at the initial loading 

stage. 

Cougaar provides mobile agent functions. Agents can 

move from one machine to another. This function could 

also be used for load control by moving agents from highloaded 

machine to less loaded machines. 

6.2 Queueing Model in Cougaar 

Usually an agent has many plugins in cougaar applications 

[5]. From the perspective of infrastructure, the plugin could 

be seen as a workload or a job. If the agent does not get a 

thread for this plugin, the plugin should be put in the queue 

until the agent obtains a thread. 

A service request is processed in an agent through a 

sequence of plugins. The service request is represented as 

an object, called Task. While Tasks go through those 

plugins, they are expanded or aggregated and finally 

allocated to assets, which represent actual physical 

resources. Each plugin repeats a series of processes – 

retrieving Tasks, processing Tasks and publishing another 

Task. For example, if the application is a planning system 

and re-planning is triggered whenever there are 

discrepancies between planning and real world, we could 

see continuous arrival of plugins to the queue in the 

agents. This phenomenon could be naturally described as 

queueing models. 

6.3 Control Using Thread Services 

Even though the shared threads are managed by thread 

service, they are not used for control purpose. Current 

infrastructure just assigns threads to agents in a roundrobin 

fashion. We build a control structure utilizing the 

thread services so that agent and node level controls are 

feasible in the hierarchical control structure as we 

mentioned before. Through the control structure, agents 

(nodes) assign threads at their will through a 

predetermined scheme to plugins (agents) through thread 

service. 

After the plugin finishes its work, it will release the runright. 

Thus, the run right could be reassigned to other 

plugins. On the other hand, a node can dynamically

change the limitation on the number of run rights of each 

agent. If a certain agent has high workload, the node could 

reduce the number of run-rights on other agents so that the 

agent can get more opportunity to run its work. 

(Agent) 

Scheduler 

Sensor 

(ThreadListener) 

DynamicSortedQueue 

TreeNode 

(Node) 

Scheduler 

Resource Allocator 

(RightsSelector) 

(Agent) 

Scheduler 

Sensor 

(ThreadListener) 

Figure 1: infrastructure level 

Figure 1 shows a schematic representation of hierarchical 

control structure within a node. In Cougaar, nodes and 

agents have their own schedulers. They assume a tree 

data structure. Cougaar does not have a direct 

communication channel between agents and nodes. We 

modified them to exchange feedback report and control 

message through the scheduler. Each agent could monitor 

every plugins’ arrival, service start and service end through 

the Thread Listener. Plugins have algorithms to process 

Tasks. Agent could choose an algorithm of a plugin for 

service rate control through Dynamic Sorted Queue. Every 

agent has the Dynamic Sorted Queue. A java interface to 

Plugin is added to let the agent set the algorithm in a 

plugin. Node could assign the run-right (thread) to specific 

agent using right selector. We let the scheduler have a set 

of control policies. We could see that the modified 

infrastructure could effectively control the plugins and runright 

assignment through experiments on a small example 

agent society. 

7 CONCLUDING REMARKS 

Load balancing in multiagent systems is different from 

other load balancing problems because of agent’s selfregulation 

and highly dynamic communication load 

between agents [7]. This paper discussed a hierarchical 

control structure, which would help agents or nodes make 

control decisions based on local information and obtain an 

overall optimal system’s performance. In addition, the 

higher-level controller’s estimation on changes in system 

parameters could help the agent adapt to changing 

environment. 

Agent’s self-regulation makes load-balancing problem 

significantly difficult. Each agent wants to use more CPU 

time. However, if every agent uses CPU time greedily, 

overall system performance may not be optimal. This could 

be seen as a Social dilemma. Use of Game theory is a 

promising approach in finding equilibrium among agents. 

8 ACKNOWLEDGMENTS 

The work described here was performed under the DARPA 

UltraLog Grant#: MDA972-1-1-0038. The authors wish to 

acknowledge DARPA for their generous support. 

[2] Meyer, B., 1997, Object-Oriented Software 

Construction, Second Edition, Upper Saddle River, 

N.J., Prentice Hall. 

[3] Csete, M.E. and Doyle, J.C., 2002, Reverse 

Engineering of Biological Complexity, Science, 

295:1664-1669. 

[4] Huhns, M.N. and Holderfield, V.T., 2002, Robust 

Software, IEEE Internet Computing, March/April:80- 

82. 

[5] Cougaar Open Source Site. http://www.cougaar.org 

[6] Java Agent DEvelopment Framework (JADE). 

http://sharon.cselt.it/projects/jade/ 

[7] Chow, K. and Kwok, Y., 2002, On Load Balancing for 

Distributed Multiagent Computing, IEEE Transactions 

On Parallel and Distributed Systems, 13/8:787-801. 

[8] Lee, L.C., Nwana, H.S., Ndumu, D.T. and De Wilde, 

P., 1998, The stability, scalability and performance of 

multiagent systems, BT Technology Journal, 16/3: 

94-103. 

[9] Schaerf, A., Shoham, Y., and Tennenholtz, M. , 1995, 

Adaptive Load Balancing: A Study in Multiagent 

Learning. Journal of Artificial Intelligence Research, 

2:475-500. 

[10] Zilberstein, S. and Russell, S., 1996, Optimal 

composition of real-time systems, Artificial 

Intelligence, 82:181-213. 

[11] George, J. M. and Harrison, J. M., 2001, Dynamic 

control of a queue with adjustable service rate, 

Operations Research, 49/5:720-731. 

[12] Ooi, J.M., Verbout, S.M., Ludwig, J.T., and Wornell, 

G.W., 1997, A Separation Theorem for Periodic 

Sharing Information Patterns in Decentralized 

Control, IEEE Transactions On Automatic Control, 

32/2:1546-1550. 

[13] Yao, D.D. and Schechner, Z., 1989, Decentralized 

Control of Service Rates in a Closed Jackson 

Network, IEEE Transactions On Automatic Control, 

42/11:236-240. 

[14] Lygeros, J., Godbole, D. N., and Sastry, S., 1997, A 

Design Framework for Hierarchical, Hybrid Control., 

California PATH Research Report, UCB-ITS-PRR-97- 

24. 

[15] Gershwin, S.B., 1989, Hierarchical Flow Control: A 

Framework for Scheduling and Planning Discrete 

Events in Manufacturing Systems, Proceedings of the 

IEEE, 77/1:195-209. 

[16] Chang, H.S., Fard, P.J., Marcus, S.I., and Shayman, 

M., 2003, Multitime Scale Markov Decision 

Processes, IEEE Transactions On Automatic Control, 

48/6:976-987. 

[17] Gusella, R., 1991, Characterizing the variability of 

arrival processes with indexes of dispersion, IEEE 

Journal on Selected Areas in Communications, 

9/2:203-211. 

[18] Zhang, R., Phillis, Y.A., and Zhu, X., 1998, Fuzzy 

Control of Queueing Systems with Removable 

Servers, IEEE International Conference on Systems, 

Man, and Cybernetics, 3:2160-2165. 

[19] Levy, H. and Sidi, M., 1990, Polling Systems: 

Applications, Modeling, and Optimization, IEEE 

Transactions on Communications, 38/10:1750-1760. 

9 REFERENCES 

[1] Jennings, N. R., 2001, An agent-based approach for 

building complex software systems, Communications 

of the ACM, 44/4:35-41.

Understanding Agent Societies Using Distributed Monitoring and Profiling 

† Wilbur Peng, † Vikram Manikonda, and ‡ Soundar Kumara 

† Intelligent Automation Incorporated 

7519 Standish Place, Suite 200, Rockville, MD 20855 

{wpeng ,vikram}@i-a-i.com 

‡ 

Industrial and Manufacturing Engineering 

310 Leonhard Building, The Pennsylvania State University, University Park, PA 16802 

{skumara}@psu.edu 

Abstract 

In this paper, we describe methodologies for 

understanding large-scale agent societies using the 

Castellan, a distributed profiling and logging system 

developed for Cougaar. Castellan enables the detailed 

efficient logging of blackboard plan activity. We describe 

the design, functionality, use and a number of 

applications of the Castellan tool, including a 

visualization and data mining tool based on a flexible 

algorithm for finding subgraph isomorphisms. By 

mapping “equivalent” meaningful graph nodes and edges 

to representative subgraph elements, the graph reduction 

approach reduces large plan graphs of hundreds of 

thousands to millions of nodes to meaningful and 

understandable clusters and graph nodes. This algorithm 

is demonstrated through its application to event traces 

obtained from running Castellan within a military 

logistics planning society. In addition to providing data 

for static analysis after planning and execution, the 

Castellan approach is also useful for on-line analysis of 

active, running agent systems. We also describe a 

number of other potential applications of distributed 

monitoring for modeling, control, load balancing and 

analysis. 


Distributed agent systems provide significant challenges 

for debugging, testing, profiling and tuning. Agent 

societies consist of distributed, state encapsulated entities 

that can run concurrently. Additionally, they have the 

additional constraint of state encapsulation, i.e. each 

agent does not have direct access to the state of other 

agents. Instead, they interact solely through message 

passing. Within an agent, different functionscan interact 

through sharing state. 

The Cougaar agent infrastructure supports an approach to 

distributed planning in which tasks are created and 

expanded into subtasks by agents which can in turn be 

forwarded to other agents. The planning process creates a 

plan graph that spans multiple agents that can potentially 

be very large, growing to hundreds thousands to millions 

of elements. Adding to the complexity of understanding 

system function, the plan graph generated by the agents 

can be dynamically modified during the planning and 

execution phases of the society. As Cougaar agent 

societies increase in size and scope, understanding the 

distributed execution of the system becomes increasingly 

difficult. Being able to trace the time-evolving, eventdriven 

behavior across agents running societies becomes 

increasingly important. 

In this paper, we discuss methods for understanding, 

analyzing and controlling Cougaar agent societies 

through distributed profiling. Section 1.1 covers 

background concepts in distributed planning used by 

Cougaar. In Section 2, the Castellan profiling and logging 

system is introduced and its implementation and design 

described. Section 3 presents in detail an application of 

Castellan to data mining and visualization application 

using a plan graph reduction algorithm. Finally, Section 

4 discusses potential applications of Castellan. 

1.1 Distributed plan graphs in Cougaar societies 

In this section, we review some basic concepts of 

planning in the Cougaar context. Additional details about 

plan representation can be found in [3]. 

In Cougaar applications such as logistics planning and 

execution, agents generate plans by decomposing tasks 

into subtasks, aggregating tasks, and forwarding tasks to 

other organization entities which are in turn are 

represented by other agents.

In the Cougaar planning model, the basic element is a 

task. Each task has a unique identifier (UID) and a set of 

fields including the task verb (e.g. “Supply”, “Project”, 

“Transport”) and the direct object (e.g. the UID of the 

Asset which the task acts on). 

Each task element must be allocated to a plan element 

during the distributed planning process. These include: 

• Allocation elements. An allocation is a 

assignments of tasks to particular assets. The 

assets can be locally represented (e.g. an 

inventory) or an organizational asset (e.g. a 

customer organization allocates a task T to a 

Supplier asset. Here, the Supplier asset 

represents an actual agent to which T will be 

forwarded.) 

• Expansions. Decomposition of tasks into 

subtasks. 

• Aggregations. These collect multiple tasks into 

a single task. 

Each agent can therefore be modeled as taking inputs as a 

set of tasks, generating local blackboard tasks and plan 

elements, and generating outputs as a set of tasks to be 

forwarded to the representative agent(s). (In Cougaar, 

tasks are forwarded to another agent by the logic provider 

if they are allocated to the (local) organization asset 

which represents the target agent.) All the blackboards 

and task elements are assumed to be persistent, unique 

objects. 

The result of a single planning run is logically a 

connected, distributed plan graph that spans multiple 

agents and multiple nodes. In addition, the plan graph 

may evolve and change during replanning as tasks are 

rescinded, modified and replanned. 

2. Castellan System Design and 

Implementation 

The primary distinguishing aspect of Castellan is that 

provides the ability to observe the time evolving state of 

the distributed agent blackboards rather than the final 

state after planning has been reached. 

The Castellan system has two aspects: the client 

implementation which monitors planning at agents, and 

the server implementation that collects the logs 

accumulated by the client side. Figure 1 shows an 

example of the concept of operations. It shows a set of 

agents which are being monitored and sending events 

traces to a server application. In turn, the server 

application can log them to a database or feed them 

directly to monitoring and analysis applications. In the 

current implementation of Castellan, the server 

application is itself implemented as a plugin which can be 

embedded in a Cougaar agent. 

Castellan has evolved to support the following modes of 

operation on the client implementation: 

• Plugin based execution. A monitoring client 

plugin loaded in each agent subscribes to all 

modifications to the blackboard. 

• Logic provider based execution. A Castellan 

logic provider is attached to each agent and 

monitors changes to the blackboard through the 

logic provider interface. 

The primary difference between these two approaches is 

that the latter allows monitoring the source of each 

change to the blackboard as well as the number of 

execute cycles associated with each change. The latter 

approach is useful in debugging since it can observe 

which plugins execute and the number of execute cycles 

of each plugin loaded for an agent. These features are 

useful for debugging and detailed performance analysis of 

agents. 

Agent 1 Agent 2 Agent 3 

Event 

Database 

Castellan Server 

Plan Analysis 

Applications 

Sensors 

Event Protocol 

Figure 1. Castellan System Concept 

As agent execution proceeds, the client implementation 

generates a stream of events for each task and plan 

element added, changed, or removed to the system 

blackboard. The event trace and logging protocol extracts 

a subset of the data encapsulated by the tasks, assets, and 

plan elements sufficient to reconstitute the entire plan 

graph. These include: 

• The unique identifier, encoded as a symbol id 

rather than a string. 

• The timestamps associated with the blackboard 

action (both simulation and wall clock time.) 

• For tasks, the verb for tasks encoded as a 

symbol id.

• For tasks, the UID of the direct object also 

encoded as a symbol. 

• For allocations, the allocation results observed. 

A design objective of Castellan was to reduce the amount 

of bandwidth consumed by the event traces while 

retaining key data and minimizing CPU consumption. 

This was accomplished through a variety of approaches, 

including: 

• Compression of UID and other symbols using a 

space efficient symbol id based protocol. 

• Detecting and transmitting only changes in the 

allocation results for each task. 

• Batching mechanisms, e.g. serializing batches of 

messages rather than individual messages. 

The generated stream of events can be delivered to the 

Castellan server implement during planning. The message 

transport between the client implementation and the 

server implementation can be varied depending on the 

application. Currently, a buffered relay is used to transmit 

events through batches of serialized events. (A relay is an 

point-to-point communications mechanism between 

Cougaar agents.) This form of “in-band” communications 

shares the communications channel with other Cougaar 

message traffic and hence is more suitable for distributed 

control applications in which agents need to be aware of 

the detailed planning status of other agents within the 

society. Alternative message transport implementations 

(e.g. using a separate communications backplane) can 

provide non-intrusive analysis for debugging and testing. 

The scalability of Castellan as a real-time monitoring tool 

is limited primary by the following factors: 

• Bandwidth of the links between the monitoring 

agents. For example, a 160,000 element event 

trace from a 30 agent, 60 min. long planning run 

in total takes approximately a total of 8 MB of 

bandwidth. This is not significant for a LAN test 

environment but may be an issue for real world 

operating environments. 

• The impact on CPU consumption for each agent 

being monitored. In a small Cougaar society of 

~30 agents, this was measured to impact total 

planning time by approximately 5%. 

• The processing bottleneck at the server agent. 

While receiving the data is not very expensive, 

inserting the results at run time into a SQL 

database tends to overwhelm most typical 

processors. 

In large agent societies, we do not expect that all agents 

can be monitored by a single server due to the volume of 

data generated. Instead, Castellan can be configured to 

monitor any subset of agents as desired, e.g. a community 

or an enclave. For debugging and profiling purposes, the 

data can then be merged after the run is complete. 

3. Graph Reduction Using Subgraph 

Isomorphism 

This section describes a significant application of the 

Castellan system that enables real-time and off-line 

analysis of the planning functions of a distributed agent 

system. Existing graph “clustering” and reduction 

approaches have been used in data mining applications to 

find meaningful repeated subgraphs within a larger graph. 

[1][2] 

The plan graphs generated during a single planning run of 

a Cougaar agent society can be extremely large. For 

example, the event traces generated from a relatively 

small test planning society engaged in logistics planning 

consisting of more than thirty agents resulted in over one 

hundred sixty thousand events and a very large 

corresponding plan graph with thousands of individual 

tasks, plan elements and assets. In this section, a general 

graph reduction algorithm is described that can be used to 

reduce the size of the plan graph in a manner tailored to 

specific applications. 

We define a plan graph P={N,E} as a set of nodes 

N={T,A} and a set of directed, attributed edges E={L, 

X,G}. Here, the nodes N consist of a set of tasks T and a 

set of assets A. The attributes associated with each task t 

∈ T are expressed as a tuple (V,Uid,Agent) and can 

include other properties depending on the amount of 

detail collected within the event trace. The plan graph P 

is a directed acyclic graph (DAG) since no cycles can 

exist in the current Cougaar task grammar. The output of 

the algorithm is a reduced graph R={N’,E’}. 

The set of edges E consist of a set of allocations L, a set 

of expansions X, and a set of aggregations G. Also, the 

nature of the Cougaar task grammar dictates a number of 

additional constraints on the plan graph. These include: 

• Each task node t 1 ∈ T can be connected to either 

an asset node s∈S through an allocation edge l 

or another task node t 2 through an expansion or 

allocation edges e ∈{X, G}. 

• Each task node t ∈ T has exactly one edge (and 

hence one parent node) except for those task 

nodes which are connected by a set of edges G’ 

⊂G. 

• The set of aggregation edges G is subdivided 

into a set of disjoint subsets which have the same 

destination node.

• The set of expansion edges X is subdivided into 

a set of disjoint subsets which have the same 

source node. 

• A task t is connected to exactly one task by an 

edge e unless e is a member of the set of 

expansion edges X. 

The basic principle behind the graph reduction approach 

is as follows. For each type of reduced graph mapping R, 

we define equivalence criteria between nodes in the graph 

to find abstract nodes. An example of criteria C 1 would 

be “All tasks at the same agent with parent tasks 

originating from another agent.” In applying the graph 

reduction algorithm, all nodes which satisfy this criteria 

are aggregated into a single node. 

Also, for every graph reduction mapping R, we define a 

equivalence criteria for abstract edges. Abstract edges 

are aggregates of equivalent subgraphs into a single 

representative edge. 

Continuing the example, consider a criteria C 2 that states 

“Subgraphs that connect all aggregate nodes satisfying C 1 

and connect to organizational assets associated with the 

same agent as C 1 .” Also, we define a second node 

equivalence criterion C 3 as “All tasks at the same agent 

which are allocated to assets representing external 

organizations.” 

We apply the criteria C 1 , C 2 and C 3 to a subgraph 

t 1 (→x 1 )t 2 (→x 2 ) t 3 (→l 2 )a 1 associated with agent A. This 

subgraph represents a task t 1 expanded to a task t 2 which 

is in turn expanded to a task t 3 and allocated to an asset 

a 1 . Here, we assume that t 1 has a parent external to A. We 

further assume that the asset a 1 is an organizational asset 

representing agent B. The task node t 1 satisfies C 1 and 

hence is aggregated into an abstract node n 1 . Similarly, 

the task node t 3 satisfies C 3 and is aggregated into a node 

n 2 . The subgraph (→x 2 )t 2 (→x 2 ) therefore matches C 2 and 

is associated with a single edge e 1 ∈E’ which connects 

the two abstract nodes n 1, , n 2, ∈ N’. 

Together, the equivalence criteria for abstract nodes and 

edges leads to the identification of equivalent subgraphs. 

By changing the equivalence criteria for aggregated nodes 

and edges, a variety of different graph reduction 

mappings can be achieved. 

The computational complexity of the approach described 

above varies depending on the complexity of finding and 

matching isomorphic subgraphs. Cougaar task graphs are 

generally well structured, and for most of the equivalence 

criteria that are described in the following, subgraphs can 

be matched using a (worst case) O(n) graph traversal, 

where n is the size of the subgraph. In the equivalence 

criteria described in the next section, all subgraphs fall 

within a single agent’s plan graph, thus bounding the size 

of the matched subgraphs. If m is the total number of 

abstract nodes discovered, then the total computational 

complexity is O(m * n). 

3.1 Algorithm implementation and applications 

The implementation of the graph reduction algorithm 

within Castellan allows generation of the task graph from 

an arbitrary stream of events. Except for asset and 

organizational information, it is not necessary to have a 

complete plan graph to use this approach to devise 

reduced graphs. 

The following types of reduced graphs were found to be 

useful for understanding Cougaar societies and are 

implemented within the Castellan system. 

Aggregate task graphs defined using an equivalence 

criterion that maps tasks of the same type with the 

identical verb to single nodes. Also, theis equivalence 

criterion requires a strict ordering in depth between tasks 

which are aggregated. Specifically, in order to map a 

node n to an abstract node n’, the node n’s parents (and 

all of its ancestors by implication) must map to an 

abstract node n 2 ’ which is an ancestor of n’. This 

requirement is imposed to prevent cycles from appearing. 

Figure 2 shows a conceptual representation of task 

aggregation in which multiple “similar” subgraphs are 

collapsed into a single aggregate subgraph. An example 

of a task aggregate plan graph is shown in Figure 3. 

Asset dependency graphs consider the assets (both 

organizational and physical) as abstract nodes. In this 

case, no aggregation of assets is performed as all assets 

are considered unique. The criteria for abstract edges are 

as follows: 

• All assets are mapped to abstract nodes. 

(Optionally, additional asset matching criteria 

can be introduced to aggregate assets.) 

• In addition, all agents that generate tasks are 

designed as “Source” abstract nodes. (These 

serve as the roots of the reduced DAG.) 

• All tasks and allocations that form plan graph 

dependencies between different assets are 

mapped to a single abstract edge. 

Asset dependency graphs are useful for finding both 

organization and physical dependencies within a 

distributed plan. 

Workflow graphs characterize the input/output 

relationships between agents and are particularly useful

for tracking the dependencies between agents incurred 

during distributed planning under a particular society 

configuration. The equivalence criteria for abstract nodes 

and edges are defined as follows: 

• Task nodes that are on the boundary (e.g. have a 

parent from another agent) and have identical 

verbs are considered equivalent. 

• All task nodes with the identical verb allocated 

to another agent are equivalent. 

• All subgraphs linking boundary nodes of the two 

types described above are mapped to the same 

abstract edges 

CLEAR- 

PAYMENT 

ORDERBOOKS 

BOOKS- 

FROM 

WAREHOUSE 

PACK 

SHIP 

CLEAR- 

PAYMENT 

ORDERBOOKS 

CLEARPAYMENT 

BOOKS- 

FROM 

WAREHOUSE 

PACK 

ORDERBOOKS 

SHIP 

ROUTE 

BOOKS- 

FROM 

WAREHOUSE 

0.6 

PACK 

SHIP 

CLEAR- 

PAYMENT 

0.3 

PUBLISH 

ORDERBOOKS 

BOOKS- 

FROM 

WAREHOUSE 

PUBLISH 

AND ( EXPANSION) 

AGGREGATION 

OR 

An example of a workflow graph is shown in Figure 4, 

with a detailed blowup in Figure 5. This particular graph 

was extracted from an event trace consisting of more than 

160,000 events and more thirty agents; however, it clearly 

shows the input/output relationships between agents and 

the number of each type of task transmitted between 

agents. Each box contains abstract nodes belonging a 

single agent. Here, the hexagonal nodes depict a set of 

tasks with identical verbs which are generated in an 

specific agent and subsequently forwarded to other agents 

for planning, the light colored boxes depict abstract nodes 

represented tasks which are inputs to the agent, and the 

ovals represent tasks allocated to assets within the agent. 

This type of graph is useful for deriving the dependencies 

between agents by finding the types of tasks which are 

inputs and outputs from agents and filtering out the 

internal details of the plan graph within each agent. A 

potential application of this system would be a smart load 

balancer which anticipates the generation and allocation 

of tasks and allocated higher priority accordingly. 

ROUTE 

Figure 2. Example of Task Aggregation 

In summary, each of these provides a different logical 

view of the overall task graph that is useful for different 

purposes. 

Figure 3. Example Task Aggregate Graph 

Reduction

1-6-INFBN 

Transport(2) 

Supply(338) 

ProjectSupply(96) 

ProjectWithdraw(33) 

1-35-ARBN 

Supply(354) 


Transport(2) 


Supply(692) 


Transport(2) 


Supply(375) 

47-FSB 

2-BDE-1-AD 

Transport(2) 

DISCOM-1-AD 

Transport(2) 

Withdraw(692) 


Supply(82) 


Transport(2) 


Supply(168) 


Supply(464) 


Supply(628) 

123-MSB 


Supply(45) 

Supply(131) 


Transport(2) 

1-AD 

Transport(12) 

227-SUPPLYCO 


Supply(337) 

Transport(2) 



Supply(532) 

Supply(54) 


Supply(160) 

Transport(2) 

485-CSB 

Transport(2) 

71-MAINTBN 

Transport(2) 

18-MAINTBN 

Transport(2) 

565-RPRPTCO 

Transport(2) 

106-TCBN 

Transport(2) 

16-CSG 

Transport(2) 

7-CSG 

Transport(2) 


Withdraw(383) 

592-ORDCO 

102-POL-SUPPLYCO 

Supply(29) 



Supply(67) 

Transport(2) 

51-MAINTBN 

Transport(2) 

37-TRANSGP 

Transport(2) 

6-TCBN 

Transport(2) 

28-TCBN 

Transport(2) 

29-SPTGP 

Transport(2) 

Withdraw(108) 


343-SUPPLYCO 

Transport(2) 


Supply(337) 


Withdraw(487) 

Transport(2) 



Supply(54) 

Supply(89) 


Supply(199) 

3-SUPCOM-HQ 

Transport(2) 

Transport(20) 

191-ORDBN 

110-POL-SUPPLYCO 

OSC 

Supply(44) 



Withdraw(67) 

21-TSC-HQ 

Transport(16) 

Transport(2) 

DLAHQ 


Supply(337) 

HNS 


Supply(161) 


Withdraw(199) 

Transport(52) 

TRANSCOM 

Transport(1706) 




GlobalAir 

GlobalSea 











PlanePacker 

TheaterGround 

CONUSGround 

ShipPacker 

Transit(448) 



Transit(892) 


Transit(312) 

Transit(32) 

Transport(16) 

capture the detailed interactions that span multiple nodes, 

applications and plugins, nor can they track the dynamic 

evolution of system state during planning and execution. 

Moreover, agent systems may be non-deterministic, 

resulting in different results for each run. In the absence 

of such tools, understanding and debugging agent systems 

becomes exceeding difficult. 

Castellan can be used to analyze global agent society 

behavior. The plan graph reduction algorithms can be 

used to evaluate the completeness of the plan and to 

confirm whether or not the patterns of plan generation are 

correct. The approach can also identify groups of tasks, 

which are not complete, i.e. which have not been 

associated with any plan element. 

Profiling tools are also often useful to increase and 

optimize performance. Although distributed agent 

systems can benefit from parallelism, often serial 

bottlenecks may be present, e.g. planning/execution may 

be depending on single agents within the system that 

constrain the rest of the planning process. 

Figure 4. Example Workflow Graph 

1-6-INFBN 

Transport(2) 

47-FSB 

Supply(338) 

Supply(692) 

Withdraw(692) 





Transport(2) 

1-35-ARBN 

The Castellan event traces can provide useful information 

by capturing the time dependent evolution of the plan 

rather than capturing a single snapshot at the end of 

planning. Moreover, the event traces measure the time to 

perform planning actions that may be time consuming, 

enabling the identification of hotspots and bottlenecks. 

4.2 On-Line Control and Monitoring 

Applications 

Supply(354) 


Transport(2) 



Supply(375) 

In the current version of Castellan, applications such as 

workflow analysis, visualization and data mining that 

used static event trace databases have been supported. 

However, the concepts and approaches used in Castellan 

can be applied to on-line analysis as well. 

Figure 5 Example Workflow Graph (Detail) 

4. Discussion 

Applications of distributed logging and monitoring 

applications within large-scale agent societies include 

both offline static analysis and on-line sensors and 

monitoring. 

4.1 Profiling and Debugging Applications 

Conventional debugging tools are inadequate to handle 

large-scale agent based systems. They cannot easily 

Sensors and control strategies that require prediction of 

system performance can benefit from distributed 

monitoring. These include: 

• Falling behind sensors. We have used to 

Castellan to extract data streams to build falling 

behind sensors that can predict whether the 

society as a whole is falling behind due to 

excessive CPU load. In this case, the data from 

Castellan was used to train various neural 

network systems that would inform agents 

within the system when the society was in 

danger of falling behind. 

• Load balancing sensors. Based on the workflow 

analysis, it is possible to dynamically at runtime 

find the flow of tasks between multiple agents

within the monitored enclave. With such a 

model present, it becomes possible to identify 

the processing requirements of tasks as they flow 

through the agent society and hence allocate 

resources accordingly as planning/execution 

progresses. 


Tools for monitoring agent systems have been noticeably 

missing from many agent infrastructures. As Cougaar has 

evolved and been applied to increasingly large societies 

and complex applications, the need for systems such as 

Castellan that provide detailed run-time event information 

will increase for both analysis of static event traces for 

and on-line monitoring applications. It also provides a 

general purpose graph reduction algorithm than enables a 

wide variety of approaches to analyzing and 

understanding large distributed plan graphs. 

6. Acknowledgements 

This research was performed under the DARPA Ultralog 

effort and was supported by DARPA grant MDA972-1-1- 

0038 and Contract 2087-IAI-ARPA-0038. We would like 

to thank Dr. Mark Greaves, Marshall Brinn and Beth 

DePass for their support, comments and insightful 

discussions. 

7. References 

[1] Emden R. Gansner and Stephen C. North. “An open 

graph visualization system and its applications to 

software engineering”, Software Practice and 

Experience, pp. 1–5, 1999. 

[2] Jonyer, L. B. Holder, and D. J. Cook. ``Graph-Based 

Hierarchical Conceptual Clustering in Structural 

Databases'', In the Proceedings of the Seventeenth 

National Conference on Artificial Intelligence, 2000 

[3] Cougaar Developers Guide, Version 11.0 

http://www.cougaar.org.

Reliable MAS Performance Prediction Using Queueing Models 

Nathan Gnanasambandam, Seokcheon Lee, Natarajan Gautam, Soundar R.T. Kumara 


State College, PA 16801 

{gsnathan, stonesky, ngautam, skumara}@psu.edu 

Wilbur Peng, Vikram Manikonda 

Intelligent Automation Inc. 

Rockville, MD, 20855 

{wpeng, vikram}@i-a-i.com 

Marshall Brinn 

BBN Technologies 

10 Moulton Street, Cambridge, MA 02138 

mbrinn@bbn.com 

Mark Greaves 

DARPA IXO 

3701 North Fairfax Drive, Arlington, VA 22203-1714 

mgreaves@darpa.mil 

Abstract 

In this paper, we model a multi-agent system (MAS) 

in military logistics based on the systemic specifications 

of the capabilities and attributes of individual agents 

(TechSpecs). Assuring the survivability of the MAS that implements 

distributed planning and execution is a significant 

design-time and run-time challenge. Dynamic battlefield 

stresses in military logistics range from heavy computational 

loads (information warfare) to being destructive 

to infrastructure. In order to sustain and recover from 

damages to continuously deliver performance, a mechanism 

that distributes knowledge about the capabilities and 

strategies of the system is crucial. Using a queueing model 

to represent the network of distributed agents, strategies 

are developed for a prototype military logistics system. 

The TechSpecs contain the capabilities of the agents, playbooks 

or rules, quantities to monitor, types of information 

flow (input/output), measures of performance (Quality of 

Service) and their computation methods, measurement 

points, defenses against stresses and configuration details 

(to reflect command and control structure as well as task 

flow). With these details, models could be dynamically 

developed and analyzed in real-time for fine-tuning the 

system. Using a Cougaar (DARPA Agent Framework) 

based model for initial parameter estimation and analysis, 

we obtain an analytical and a simulation model and 

extract generic results. Results indicate strong correlation 

between experimental and actual events in the agent society. 

0-7803-8799-6/04/$20.00 ©2004 IEEE. 

Keywords: Multi-agent systems, Survivability, Queueing 

network models, Technical specifications 


Multi-agent systems that implement distributed planning 

and execution are highly complex systems to design and 

model. In this research, we model a survivable multiagent 

system (MAS) based on the systemic specifications 

(TechSpecs) of the capabilities and attributes of individual 

agents. The MAS under consideration is exposed to significant 

stresses because it operates in highly unpredictable 

battlefield-like environments. Even under such hostile conditions, 

the stated goal of this survivable MAS based logistics 

system is to deliver robustness, security and performance. 

Hence, performance prediction using suitable models 

is vital to being able to tune the actual performance delivered 

by the MAS. 

Within the research domain of military logistics, we are 

conducting our studies using a continuous planning and execution 

(CPE) agent society. The CPE society is constructed 

using the Cougaar MAS development platform developed 

under DARPA’s leadership [2]. From the modeling perspective, 

the CPE society (or otherwise) is nothing but a collection 

of distributed agents that lend themselves to be represented 

by a network of queues. With this motivation, we analytically 

modeled the CPE society using queueing theory. 

In doing so, we realized that if the TechSpecs were suitably 

specified, the generation of the queueing model could be 

55

Figure 1. Agent Hierarchy in CPE Society 

accomplished with lesser human intervention. The primary 

function of the model is to help evaluate the performance of 

the MAS and provide alternatives to steer the agent society 

towards optimal regions of operation boosting performance 

in a distributed environment. Therefore the main focus of 

this research lies in specifying the MAS in a systematic 

fashion so that queueing models can be derived from the 

specification. 

1.1 Continuous Planning and Execution Society 

Overview 

The CPE society comprises of agents and a world model. 

Agents in the CPE society assume a combination of command 

and control, and customer-supplier roles as required 

in a military logistics scenario. The world model is an artificial 

source that provides the agents with external stimuli. 

Figure 1 represents the superior-subordinate and the 

customer-supplier relations between the brigade (BDE), 

battalion (BN), company (CPY) and supplier (SUPP) agents 

as modeled in this research. Each agent in the society constantly 

performs one or more of the following tasks: 1) 

Evaluates its own perception of the world state through local 

sensors and remote inputs; 2) Performs planning, replanning, 

plan reconciliation and plan refinement; 3) Executing 

plans, either through local actuators or through sending 

messages to other agents; 4) Adapting to the environment, 

e.g. centralizing or decentralizing planning as computational 

resources permit. 

1.2 Definitions 

The following definitions are in order when relating to 

the system under consideration. 

Stresses occur due to the operation of the MAS in battlefield 

environments where events such as permanent infrastructure 

damage and information attacks adversely affect 

overall system performance. 

Based on the planning activity in CPE, we simply base 

our measures of performance (MOPs) on timeliness or 

freshness of a plan at the point of usage and on the quality 

of the plan. Based on the requirements of Ultra*log 

[3], a broad series of performance measures categorized according 

to timeliness, completeness, correctness, accountability 

and confidentiality is available but is outside the requirements 

of CPE. Some insights about these MOPs can 

be gained from [6]. The MOPs are the components of the 

quality of service (QoS) expected from the system. 

Survivability of a distributed agent based system (or otherwise) 

is the extent to which the quality of service (QoS) 

of the system is maintained under stress [6]. 

Although we consider a survivable MAS, we only concern 

ourselves with performance analysis in this work. We 

assume that a global controller exists that coordinates between 

threads relating to performance, robustness and security. 

The contents of this paper are organized in the following 

way. In Section 2, we introduce the concept of Tech- 

Specs based design and some of the benefits associated with 

this approach. We then discuss the components of the CPE 

society in detail and organize the TechSpecs for CPE into 

various categories in Section 3. The discussion on Tech- 

Specs leads us further in the direction of how to utilize them 

to form models. We dicuss some models we created in Section 

4. We provide two analytical methods using queueing 

networks to model a small example in CPE and verify our 

models using a simulation. Finally, in Section 5 we discuss 

our conclusions and some possible directions for future research. 

2. The Concept of TechSpecs Based Design 

Technical Specifications (or TechSpecs) refer to 

component-wise, static information relating to agent 

input/output behavior, operating requirements, control 

actions and their consequences for adaptivity [7]. In 

addition to outlining a comprehensive set of functionalities, 

the TechSpecs are responsible for the definition of domain 

MOPs, their respective computational methodologies and 

QoS measurement points. The construction of TechSpecs 

helps us proceed in the following direction: 

1. Use the specs to ensure a close mapping between MAS 

functionality and an abstracted model. An apparent 

choice here is a queueing model because of similarities 

between multi-class traffic in queueing networks and 

the different types of flows in CPE. 

2. Establish the parameters of the queueing model - from 

TechSpecs directly (eg. update rate at a node) as well 

as by collecting empirical data from sample runs (eg. 

processing times). 

56

Benefits of TechSpecs 

The advantage of establishing comprehensive TechSpecs 

is that it leads to the codification of requirements, functionalities, 

measurements and responses to situations. Further, 

it enhances the potential to aid the MAS configuration (what 

nodes to put agents on) both statically and dynamically. An 

incomplete list of potential benefits of using a TechSpecs 

based approach to MAS design is provided below: 

• Enhancement of the MAS Design: Since TechSpecs 

impose the requirement of predictability, the MAS 

components must be built with fidelity 

Figure 2. TechSpecs based MAS Design 

3. As the queueing model provides an indication of system 

performance for a given configuration, use it to 

quickly explore options for control (choices resulting 

from adjusting (queueing) parameters or configurations). 

Once a suitable candidate is obtained, this 

choice is translated back into the application level knob 

settings (for control) to result in better QoS for the 

MAS. 

The direction that TechSpecs motivates us to take is illustrated 

in Figure 2. Figure 2 indicates that we could use 

the specs in an online or offline fashion. Because the functionality 

is clearly defined using TechSpecs, offline analysis 

can be independently carried out to remove instabilities 

from the MAS design. Assuming automatic conversion 

from TechSpec to a model is feasible, TechSpecs have a 

real-time use as well - i.e. use the specs as a template to 

derive the model. As noted above, the candidate parameters 

from the queueing model (parameters that may lead to 

performance improvement) cannot be used directly. Reconverting 

these choices to actual control knob settings may be 

handled by a seperate global controller. We allude to this in 

Section 3.2. 

It can be noted that the idea of TechSpecs bears analogy 

to the conventional control problems in electronic or 

hardware realms where the technical specification or rating 

could be leveraged to effect better design and control. This 

was one of the motivating factors for TechSpecs based design 

for MAS. 

• Distribution of Knowledge: TechSpecs carries with it 

the idea of being composable. By using the TechSpecs 

of smaller components as building blocks we can build 

the TechSpecs of larger systems when the system expands. 

• Concurrent Analysis: Model building can be concurrent 

with actual MAS design. Provides a look-ahead 

capability to avoid regions of instability or bottlenecks 

(especially from queueing analysis). 

3. CPE Society TechSpecs 

In this section we discuss the formulation of TechSpecs. 

In order to build TechSpecs the functionalities of the components 

of the CPE society are defined as described in Section 

3.1. We then categorize the capabilities of CPE components 

in a manner that would lend itself to easy translation 

into the queueing models. We then show through examples 

how the mapping process between a TechSpec and a queueing 

model could be interpreted. This would enable us to 

analyze the MAS using the models we develop in Section 

4. 

3.1 Description of CPE Society Components 

The World Model: The world model refers to the 

conceptual set-up that provides the agents with external 

stimuli. It captures a military engagement scenario using a 

2-dimensional model of the world. As shown in Figure 3, 

CPY agents moving along the x-axis engage an unlimited 

supply of targets that move along the y-axis. The targets 

move at a fixed rate but engagement slows them down. 

While a probabilistic model is chosen to create targets 

and engaging them, a deterministic model is chosen for 

fuel consumption (which is dependent on the distance 

moved). A logistics model for resupplying the units with 

fuel or ammunition is based on the demand generation 

from maneuver plans. Currently, the world model is also 

implemented as an agent. 

57

Figure 3. The World Model 

CPY Agent: Each CPY unit is designated a target 

area for engaging in combat actions. These action require a 

superior agent (BN) to supply a maneuver plan to each of 

the CPY agents. This plan enables the CPY agent to move 

along the x-axis and engage the enemy by firing. Each 

of these agents simulate sensors and actuators. The CPY 

agents consume resources and subsequently forward the 

demand to SUPP agents. The current status is reported to 

superior agents to enable replanning. 

BN Agent: The BN agent maintains situational awareness 

of all the agents under its direct command and performs 

(re)planning for them using a consistent set of observations 

that is collected continously. The BN agent has to execute 

a branch and bound algorithm of a specified planning depth 

and breadth to generate a maneuver plan for its subordinates. 

The BN agent serves as a medium for transferring 

orders from superiors to subordinates. 

BDE Agent: The BDE agent is responsible for generating 

maneuver plans for the BN and CPY agents although 

this implementation does not empower the BDE with that 

functionality. 

SUPP Agent: SUPP agents represent an abstracted 

set of supply and inventory and sustainment services. 

These agents take maneuver plans from the CPY agents 

and supply them with fuel or ammunition. It is currently 

assumed that the SUPP units have infinite inventory. Projected 

and actual consumption depend on the sustainment 

plan generated from orders and the presence of enemy 

targets. 

3.1.1 TechSpec Organization 

Right at the outset, our goal is to embed enough transperancy 

in the TechSpecs to allow the generation of models 

(queueing models). Hence, we extract the input/output behavior, 

state, actions and QoS for each entity within CPE 

and form the following categories within the TechSpecs : 

• Internal State of an Agent: Corresponds to continously 

updated variables or data structures corresponding to 

the actual working of the agent. 

• Inputs: Relates to distinct classes of information received 

or sent to or from an agent respectively. 

• Outputs: Information provided to other agents. 

58

• Actions: Determines the actions that need to be taken 

as a result of state changes or the dependencies introduced 

by input/output operations. 

• Operating Modes: The fidelity or the rate at which outputs 

are sent may relate to the operating mode of an 

agent. Switching operating modes may be necessary 

to alter QoS requirements or as counter-measure for 

stress. 

• QoS Measurement (QoS Measurement Points): Indicates 

the measure of performance that needs to be 

monitored or measured in order to compute the QoS 

at the designated measurement point. For example, 

when we consider queueing models, we would be interested 

in measuring the average waiting times at different 

agents to compute a quantity such as the freshness 

of the maneuver plan. 

Table 1. TechSpec Categories: 

Perspective 

Application 

• Tradeoffs: While these may not pertain to every agent, 

some agents have the capability to trade-off a certain 

measure of performance to gain another. These are 

specified explicitly in TechSpecs. 

This categorization facilitates the delineation of specific 

flows of jobs between agents. For example, consider the 

following flow: External stimuli at CPY gets converted to 

update tasks at CPY, delivered to BN as updates, converted 

to a manueuver plan at BN, delivered to CPY and then forwarded 

to SUPP for sustainment. From a queueing theory 

perspective, the update tasks that originates at CPY and end 

up at BN for the purpose of planning could constitute a 

class of traffic with CPY and BN acting as servers to process 

these tasks. Similarly, consider the flow where external 

stimuli received at CPY end up as updates at BDE through 

BN. This could be regarded as another class of traffic. At 

this point it is important to notice that classes of traffic could 

be derived form the input/output details embedded within 

TechSpecs. We decribe how we handle these flows in the 

queueing network formulation in Section 4. 

Another example of how we could describe something in 

the application domain (say a QoS metric) with the queueing 

model is as follows. If one is interested in how fresh a 

maneuver plan is at its usage point (i.e. CPY), the model 

could describe it in terms of the queueing delays for a particular 

class of traffic. In our application, this very quantity 

happens to be a QoS metric called manuever plan freshness. 

In the actual MAS, this metric is calculated directly from the 

timestamps that are tagged to the tasks. 

3.1.2 TechSpec Representation 

Although an elaborate discussion of the format of TechSpec 

representation is outside the scope of this paper, we present 

some aspects of the specification directly relating to the application 

and some infrastructural requirements that need to 

be part of the specification. 

Table 1 represents some TechSpecs categories specific 

to this application. Simply speaking, this is a tabular representation 

of the information contained in Section 3.1 organized 

using the aforementioned categories. From Table 1 

one can understand that an output called update originates 

from CPY agent and travels up at BN because BN is CPY’s 

superior. Similarly, an output called maneuver plan would 

reach CPY from BN. One assumption that is being made 

here is that updates travel up the hierarchy and plans downward. 

These outputs form part of the different classes of 

traffic if observed from a queueing perspective. Another example 

would be that the plan action in the BN agent relates 

to a functionality in the MAS domain and would simply be 

abstracted by a processing time in the queueing domain. 

In addition to the above specification, static requirements 

of the agents in terms of infrastructure are also embedded 

into TechSpecs. Some of these requirements for BDE, BN, 

CPY and BDE agents shown in Table 2. 

3.2 Translating TechSpecs to the Queueing Domain 

In order to translate the specs into queueing models we 

first use the following rules: 

1. Inputs and outputs are regarded as tasks; 

2. The rate at which external stimuli are received is captured 

by the arrival rate(λ); 

3. Actions take time to perform so they get abstracted by 

processing times(µ i ); 

59

Table 2. TechSpecs: Infrastructure Perspective 

an open system because tasks constantly enter and exit 

the system. 

• Does any parameter of the model require empirical 

data from the actual society? 

Although some aspects in this research are currently being 

resolved, the following observations can be made. 

4. QoS Metrics such as freshness are in terms of average 

waiting times at several nodes ( ∑ W ij , i is the node, j 

is the class of traffic); 

5. If tasks follow a particular route (or flow as described 

in Section 3.1.1), then that route gets associated to a 

class of traffic; 

6. If a particular task goes into the node and gets converted 

to another task, we say class-switching has occured. 

For example, in our application update tasks go 

to BN and get converted to plan tasks; 

7. If a connection exists between two nodes, this is converted 

to a transition probabilty p ij , where i is the 

source and j is the target node. 

Using the above rules as well as the aforementioned representations 

of TechSpecs we develop a mapping between 

the TechSpecs and a queueing model. Although the current 

procedure is manual, in thoery this procedure could 

be automated. Such an automatic capabilty of translating 

TechSpecs would prove very beneficial for predicting performance 

of the MAS in real-time. Table 3 captures the 

queueing model abstraction from TechSpecs for the CPY 

agents. Similarly, we can establish the mapping for other 

agents as well. Some useful guidelines that were followed 

in order to translate the TechSpecs into models are as follows: 

• Identify flows of traffic: Trace the route followed 

by each type of packet completely within the system 

boundary i.e. from the entry into the system until it exits 

the system. These would subsequently form classes 

of traffic in the queueing model. Care has to be taken 

to note any class switching. 

• Identify the network type: The network could be 

closed (fixed number of tasks) or open. The CPE is 

• Who does the TechSpecs translation? Where does the 

model run? In our case the translation is done manually 

at present. The model would run at a place visible 

to the controller (possibly as a seperate agent at the 

highest level). The controller we refer to here is the 

actual effector of control actions throughout the CPE 

society and is seperate from all we have discussed so 

far. The role of the controller is also to balance between 

other threads such as robustness and security. 

• The identification of control alternatives is currently 

centralized. However, we visualize a decentralized, hierarchical 

controller for effecting the changes. 

4. Queueing Network Models (QNMs) 

A complex logistics system such as the CPE society has 

numerous interactions. Yet, if the functionalities are abstracted 

to capture some application level specifics in terms 

of queueing model elements (example as shown in Table 3), 

analytical predictions on the behavior of the MAS can be 

made. Analytical models are good candidates for enforcing 

adaptive control quickly and in real-time. Each agent behaves 

like a server that process jobs waiting in line. Hence, 

the mapping between an agent and a server with a queue 

is easily established. Because of the task flow structure 

and the superior-subordinate relationships in the TechSpecs, 

queues can be connected in tandem with jobs entering and 

exiting the system. This results in the formation of an open 

queuing network. 

We conducted initial experiments using an actual 

Cougaar based MAS, an analytical formulation and an 

Arena simulation. We used this experiment to bootstrap 

our modeling process in terms of parameter estimation and 

calibration. However, working with the MAS was timeconsuming 

as our goal was to identify modeling alternatives 

and control ramifications. Hence we continued our experimentation 

with a scaled up queueing model and simulation 

with the insight gained from working with the actual society. 

Thus the open queueing network’s parameters were carefully 

chosen and tasks sub-divided into mutiple-classes to 

denote a particular task within the MAS. The TechSpecs 

clearly delineate the input and output tasks facilitating the 

60

Table 3. Queuing Model Abstraction from 

TechSpecs for CPY Agent 

Figure 4. Task Flow in the MAS 

mapping to arrivals and services in a queueing network. Application 

level QoS measures of the MAS are calcuated in 

terms of the waiting times (or other equivalent perfromance 

measures) at the individual nodes of the QNM. 

Figure 4 is a representation of the CPE society from a 

queueing perspective. We show two types of tasks flowing 

in the network namely the plan (denoting maneuver and 

sustainment) and the update tasks. These tasks can be divided 

further into three classes of traffic. The first class 

refers to update packets entering at the CPY nodes and proceeding 

further as updates to BDE through BN. Class 2 

relates to those update packets that are converted to plan 

tasks. There is class-switching at nodes 2 and 3 and we introduce 

approximations to deal with this later in the paper. 

The third class relates to the maneuver plan tasks that reach 

SUPP nodes through CPY. Although we know multiple task 

types exist in the MAS, by making the simplifying assumption 

and treating all job classes alike we analyze the MAS 

using Jackson networks [5] in Section 4.1. We further analyze 

the system taking into accout multiple classes of traffic 

as discussed in Section 4.2. We compare the two analytical 

approaches with a simulation model. 

4.1 Jackson Network Model 

We apply a single class Jackson network [5] formulation 

for open queuing networks to our example by choosing 

a weighted average service time for nodes with multiple 

classes. The nine agents of the MAS considered here can 

then be assumed to be M/M/1 systems. The arrival rates 

of the open network can be computed by solving the traffic 

equations. Assuming the load is balanced to start-with, the 

routing probabilities are also known. If each node of the 

system is ergodic, we can calculate the steady state probabilities 

and performance measures of the entire network by 

computing these measures for every agent exactly as in an 

M/M/1 system. 

We consider a simple example. For this queueing 

model, we assume all tasks are of a single type and do not 

distinguish between classes as shown in Figure 4. Let λ 0i 

and λ i0 be the rate of arrival and exit into and from the i th 

node respectively. Since the routing probabilties are known 

we can calculate the arrival rates λ i of each of the nodes of 

the open network by solving the following traffic equations: 

λ i = λ 0i + 

9∑ 

λ j p ji , i =1, ..., 9 . 

j=1 

The routing probabilties (p ji : probability from i (column 

index) to j (row index)) for the balanced case are as follows: 

0 1/5 1/5 0 0 0 0 0 0 

0 0 0 1/4 1/4 1/4 1/4 0 0 

0 0 0 1/4 1/4 1/4 1/4 0 0 

0 1/5 1/5 0 0 0 0 0 0 

0 1/5 1/5 0 0 0 0 0 0 

0 1/5 1/5 0 0 0 0 0 0 

0 1/5 1/5 0 0 0 0 0 0 

0 0 0 1/4 1/4 1/4 1/4 0 0 

0 0 0 1/4 1/4 1/4 1/4 0 0 

Note that the customer exits from a node i with probability 

1 − ∑ j p ji. Once the arrival rates are known, we can 

calculate the average waiting times at the nodes by using 

the following formula: 

1/µ i 

W i = 

, i =1, ..., 9 . 

1 − (λ i /µ i ) 

The QoS metrics namely maneuver plan freshness (MPF) 

and sustainment plan freshness (SPF) are calculated in 

61

terms of the average waiting times of the nodes at each 

level (W CPY ,W BN ,W SUPP ) as follows: 

MPF =2W CPY + W BN , 

SPF =2W CPY + W BN + W SUPP . 

If the load is not balanced and the waiting times are different 

for the different branches, the QoS measures are accordingly 

calculated. It can be observed that two methods 

of control are straightaway obvious: 1) Adjust the µ i so 

that we could process faster if possible, 2) Alter the transition 

probabilties p ji to divert traffic to nodes that are less 

loaded. Although we allude to some control methods, these 

are outside the scope of this paper. 

4.2 BCMP Network Model 

We apply the Baskett, Chandy, Muntz and Palacios 

(BCMP) algorithm [5] with a small modification to the 

above example. The network considered here consists of 

nine nodes and three class of traffic. The first class correponds 

to the stream that enter the CPY nodes and get sent to 

BDE through BN as updates. The second class corresponds 

to the tasks that enter the CPY nodes and get sent to the 

BN nodes for planning. The second class is converted to a 

plan and fed back to the CPY nodes. As class-switching occurs 

here we make a first order approximation and feed this 

as an independent class back at CPY nodes as tasks of the 

third class. Since most tasks are of the update type it makes 

sense to serve the latest update first and hence we follow the 

LCFS-PR (last come first served with preemptive resume) 

scheme whereever there are multiple classes. This allows us 

to assume the service rates to be exponential. Since all tasks 

arrive from the environment we assume the arrival process 

to be a Poisson Process. 

If λ ir is the arrival rate of the r th class at the i th node, 

λ 0,ir is the arrival rate of the arrival rate of the r th class 

at the i th node, and p js,ir is the probability that a task 

of class s at the j th node is transferred to a task of class 

r at the i th node, then the arrival rates for each class at 

the individual nodes can be calculated using the following 

traffic equations: 

λ ir = λ 0,ir + 

9∑ 

j=1 s=1 

3∑ 

λ js p js,ir , i =1, ..., 9 . 

The routing probabilties (p ji : probability from i to j) for 

the class 1 tasks (portion of update tasks that go to BDE) 

are as follows: 

0 1 1 0 0 0 0 0 0 

0 0 0 1/2 1/2 1/2 1/2 0 0 

0 0 0 1/2 1/2 1/2 1/2 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 


index) to j (row index)) for the class 2 tasks (portion of 

update tasks that leave at 2 or 3) are as follows: 

0 0 0 0 0 0 0 0 0 

0 0 0 1/2 1/2 1/2 1/2 0 0 

0 0 0 1/2 1/2 1/2 1/2 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 


index) to j (row index)) for the class 3 tasks (portion of 

update tasks that enter node 4,5,6 or 7 and proceed to node 

8 or 9) are as follows: 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 0 0 0 0 0 0 

0 0 0 1/2 1/2 1/2 1/2 0 0 

0 0 0 1/2 1/2 1/2 1/2 0 0 

Once the arrival rates for the different classes at all nodes 

are known, the waiting time (W ir or W i,r ) at node i for 

class r was calculated as follows: 

W ir = 

λ ir /µ ir 

(1 − ∑ 3 

r=1 λ ir/µ ir )µ ir 

. 

The application level QoS measures were calculated in 

terms of the node level average waiting times of the different 

classes of the BCMP network as follows: 

MPF = W CPY,2 + W BN,2 + W CPY,3 , 

SPF = W CPY,2 + W BN,2 + W CPY,3 + W SUPP,3 . 

62

Figure 5. Maneuver Plan Freshness using 

Jackson Network 


BCMP Network 

4.3 Discussion 

We assume that the load is initially balanced. Yet in the 

unbalanced case, waiting times for the different branches 

can be calculated seperately. 

We studied the impact of changing the processing rates at 

the nodes to illustrate the benefit of deriving a online queueing 

model that could form an integral part of a controller. 

Three methods were followed: 1) Jackson network model, 

2) BCMP network model, 3) A Discrete-Event Simulation 

Model in Arena [1]. We compute the maneuver plan and 

sustainment plan freshness from the average waiting times 

of the individual nodes. We assume the processing rate for 

class 1 tasks, µ update_tasks =10Mb/s at all the nodes. We 

assume that the overall arrival rate from the environment 

is according to a Poisson Process with λ =2Mb/s. We 

vary the processing rates for the class 2 tasks at BN and 

CPY and observe the impact on maneuver plan freshness as 

shown in Figure 5 and Figure 6. The low value of processing 

rates at the BN agent for class 2 tasks are in line with 

reality, wherein the BN agent implements a search procedure 

that is more time-consuming than to process class 1 

tasks which are updates meant for superiors in the chain 

of command. We found that the Jackson network matched 

reasonably well with the simulation results. The multi-class 

BCMP method performed better than the Jackson network 

because it was able to capture more of the MAS’s characteristics 

using different classes of traffic. This can be observed 

by comparing Figure 5 and Figure 6 with Figure 7. 

We consider only two parameters (processing rates for 

the class 2 tasks at BN and CPY) for variation and nine 

experiments for each method. We do this to keep the calculations 

simple. It can be observed from Figure 5 and Figure 

6 that adjusting the processing rate in BN impacts the QoS 

significantly as opposed to altering processing rates at CPY. 

Hence, to increase performance, the controller may have to 

adjust the application level knobs to provide a greater processing 

rate for the planning tasks. Similarly, other trends 

can be observed by adjusting other parameters. 

With these models, we believe it is be possible to identify 

unstable regions and steer the MAS towards regions 

providing better QoS. The running time of these models in 

Matlab is less than one second per iteration. If embedded 

within the system, several alternate and feasible system configurations 

can be simulated to identify candidate choices 

for performance improvement. 

5. Results and Future Directions 

The hierarchy within the MAS, the specification of static 

attributes and the similarity between a distributed MAS 

based planning procedure and queueing network with multiple 

classes facilitate the performance modeling of the MAS 

using QNMs. TechSpecs are a structured method to encapsulate 

static data and distribute them because agent based 

planning applications are inherently distributed. From 

TechSpecs, queueing models (offline and online) can be developed 

for a cluster of nodes. The QNM will serve as an 

performance analysis tool for that cluster of nodes. 

63


The work described here was performed under the 

DARPA UltraLog Grant#: MDA972-1-1-0038. The authors 

wish to acknowledge DARPA for their generous support. 

References 

[1] Arena. www.arenasimulation.com. Rockwell Automation. 

[2] Cougaar open source site. http://www.cougaar.org. 


[3] Ultralog program site. http://www.ultralog.net. 



Simulation 

[4] Web-ontology (webont) working group. 

http://www.w3.org/2001/sw/WebOnt/. 

[5] G. Bolch, S. G. H. de Meter, and K. S.Trivedi. Queuing 

Networks and Markov Chains: Modeling and Performance 

Evaluation with Computer Science Applications. 

John Wiley and Sons, Inc., 1998. 

The main contributions of this work is that we have identified 

that TechSpecs could serve as good template that can 

guide MAS design and model development in a concurrent 

fashion. We have codified the static attributes of MAS 

in such a way that QNMs may be constituted from distributed 

information, especially in realtime. This technique 

for adaptivity by using a model on demand to predict trends 

in QoS may be helpful in building survivable systems. 


to assure survivability of distributed multi-agent systems. 

Proceedings of the Second Joint Conference on 

Autonomous Agents and Multi-Agent Systems (Poster 

Session), 2003. 

[7] A. Cassandra, D. Wells, M. Nodine, and P. Pazandak. 

Techspecs: Content, issues and nomenclature. Technical 

Report, Telcordia Inc. and OBJS Inc., 2003. 

Currently, work is ongoing to identify an appropriate 

method of representation of TechSpecs that would have 

some reasoning and deduction capabilties such as OWL 

[4]. A module that could convert this representation of 

TechSpecs into queueing models automatically would be 

useful in this endeavor. An approach that would identify 

alternate choices for performance improvement is also 

necessary. Finally, a controller that actually uses the 

analysis from the QNMs to optimize the global utility is 

also being pursued. 

64

Supply Chain Network: A Complex Adaptive Systems Perspective 

AMIT SURANA † , SOUNDAR KUMARA ‡* , MARK GREAVES**, 

USHA NANDINI RAGHAVAN 

In this era where on one hand, information technology is revolutionizing almost every 

domain of technology and society, on the other hand the “complexity revolution” is 

occurring in science at a silent pace. In this paper we look at the impact of the two, in the 

context of supply chain networks. With the advent of information technology, supply 

chains have acquired complexity almost equivalent to that of biological systems. 

However, one of the major challenges that we are facing in supply chain management is 

the deployment of coordination strategies that lead to adaptive, flexible and coherent 

collective behavior in supply chains. The main hurdle has been the lack of the principles 

that govern how supply chains with complex organizational structure and function arise 

and develop, and what organizations and functionality is attainable, given specific kinds 

of lower-level constituent entities. The study of Complex Adaptive Systems (CAS), has 

been a research effort attempting to find common characteristics and/or formal 

distinctions among complex systems arising in diverse domains (like biology, social 

systems, ecology and technology) that might lead to better understanding of how 

complexity occurs, whether it follows any general scientific laws of nature, and how it 

might be related to simplicity. In this paper we argue that supply chains should be treated 

as a CAS. With this recognition, we propose how various concepts, tools and techniques 

used in the study of CAS can be exploited to characterize and model supply chain 

networks. These tools and techniques are based on the fields of nonlinear dynamics, 

statistical physics and information theory. 


Supply chain is a complex network with an overwhelming number of interactions and interdependencies 

among different entities, processes and resources. The network is highly nonlinear, 

shows complex multi-scale behavior, has structure spanning several scales, and evolves and selforganizes 

through a complex interplay of its structure and function. This sheer complexity of 

supply chain networks, with inevitable lack of prediction makes it difficult to manage and control 

them. Furthermore, the changing organizational and market trends requires the supply chains to 

be highly dynamic, scalable, reconfigurable, agile and adaptive: the network should sense and 

respond effectively and efficiently to satisfy customer demand. Supply chain management 

necessitates that decisions made by business entities take more global factors into considerations. 

The successful integration of the entire supply chain process now depends heavily on the 

availability of accurate and timely information that can be shared by all members of the supply 

chain. Information Technology with its capability of setting up dynamic information exchange 

network has been a key enabling factor in shaping supply chains to meet such requirements. A 

major obstacle remains, however in the deployment of coordination and decision technologies to 

achieve complex, adaptive, and flexible collective behavior in the network. This is due to the lack 

of our understanding of organizational, functional and evolutionary aspects in supply chains. A 

† Department of Mechanical Engineering, The Massachusetts Institute of Technology, Cambridge, MA, 02139, email: 

surana@mit.edu 

‡ The Harold and Inge Marcus Department of Industrial & Manufacturing Engineering, University Park, PA 16802, 

email: skumara@psu.edu. * Corresponding Author 

**IXO, DARPA, 3701 North Fairfax Drive, Arlingon, VA 22203, 1714, mgreaves@darpa.mil

key realization to tackle this problem is that supply chain networks should not just be treated as a 

“system”, but as a “Complex Adaptive System” (CAS). The study of CAS augments the systems 

theory and provides a rich set of tools and techniques to model and analyze the complexity arising 

in systems encompassing science and technology. In this paper we take this perspective in 

dealing with supply chains and show how various advances in the realm of CAS, provide novel 

and effective ways to characterize, understand and manage their emergent dynamics. 

A similar viewpoint has been emphasized in (Choi et al. 2001). The focus of Choi et al. was to 

demonstrate how supply networks should be managed if we recognize them as CAS. The concept 

of CAS allows one to understand how supply networks as living systems co-evolve with the 

rugged and dynamic environment in which they exist and identify patterns that arise in such an 

evolution. The authors conjecture various propositions stating how the patterns of behavior of 

individual agents in a supply network can be related to the emergent dynamics of the network. 

One of the important deductions made is that when managing supply networks, managers must 

appropriately balance how much to control and how much to let emerge. However, no concrete 

framework has been suggested under which such conjectures can be verified and generalized. It is 

the onus of this paper to show how the theoretical advances made in the realm of CAS can be 

used to study such issues systematically and formally in the context of supply chain networks. 

This paper is divided into eight sections, including the introduction. In Section 2, we give a 

brief introduction to complex adaptive systems in which we discuss the architecture and 

characteristics of complex systems in diverse areas encompassing biology, social systems, 

ecology and technology. In Section 3 we discuss characteristics of supply chain network and 

argue that they should be understood in terms of a CAS. We also present some emerging trends in 

supply chains and the increasing critical role of information technology in supply chain 

management in the light of these trends. In Section 4 we give a brief overview of the main 

techniques that have been used for modeling and analysis of supply chains and then discuss how 

the science of complexity provides a genuine extension and reformulation of these approaches. 

Like any CAS, the study of supply chains, should involve a proper balance of simulation and 

theory. System dynamics based and recently agent based simulation models (inspired from 

complexity theory) have been extensively used to make theoretical investigations of supply 

chains feasible and to support decision-making in real world supply chains. System dynamics 

approach often leads to models of supply chains, which can be described in the form of a 

dynamical system. Dynamical systems theory provides a powerful framework for rigorous 

analysis of such models and thus can be used to supplement the system dynamics simulation 

approach. We illustrate this in Section 5, using some nonlinear models, which consider the effect 

of priority, heterogeneity, feedback, delays and resource sharing on the performance of supply 

chain. Furthermore, the large volumes of data, generated from simulations can be used to 

understand and comprehend the emergent dynamics of supply chains. Even though an exact 

understanding of the dynamics is difficult in complex systems, archetypal behavior patterns can 

often be recognized, using techniques from complexity theory like Nonlinear Time Series 

Analysis and Computational Mechanics, which are discussed in Section 6. The benefits of 

integrated supply chain concepts are widely recognized, but the analytical tools that can exploit 

those benefits are scarce. In order to study supply chains as a whole it is critical to understand the 

interplay of organizational structure and functioning of supply chains. Network dynamics an 

extension of nonlinear dynamics to networks, provides a systematic framework to deal with such 

issues and is discussed in Section 7. We conclude in Section 8, with the recommendations for 

future research. 

2. Complex Adaptive Systems 

Many natural systems and increasingly many artificial (man-made) systems as well, are 

characterized by apparently complex behaviors that arise as the result of nonlinear spatiotemporal 

interactions among a large number of components or subsystems. We would use the

term agent and node interchangeably to refer to the component or subsystems. Examples of such 

natural systems include immune systems, nervous systems, multi-cellular organisms, ecologies, 

insect societies and social organizations. However, such systems are not just confined to biology 

and society. Engineering theories of controls, communications and computing have matured in 

recent decades, facilitating the creation of various large–scale systems, which have turned out to 

possess bewildering complexity, almost equivalent to that of biological systems. Systems sharing 

this property include parallel and distributed computing systems, communication networks, 

artificial neural networks, evolutionary algorithms, large-scale software systems, and economies. 

Such systems have been commonly referred to as Complex Systems (Baranger , Flake 1998, 

Adami 1998, Bar-Yam 1997). However, at the present time, the notion of complex system is not 

precisely delineated. 

The most remarkable phenomena exhibited by the complex systems, is the emergence of highly 

structured collective behavior over time from the interaction of simple subsystems without any 

centralized control. Their typical characteristics include: dynamics involving interrelated spatial 

and temporal effects, correlations over long length and time scales, strongly coupled degrees of 

freedom, non-interchangeable system elements, exist in quasi equilibrium and show a 

combination of regularity and randomness (i.e. interplay of chaos and non-chaos). Such systems 

have structures spanning several scales and show emergent behavior. Emergence is generally 

understood to be a process that leads to the appearance of structure not directly described by the 

defining constraints and instantaneous forces that control a system. The combination of structure 

and emergence leads to self-organization, which is what happens when an emerging behavior has 

an effect of changing the structure or creating a new structure. Complex Adaptive System is a 

special category of complex systems to accommodate living beings. As the name suggests they 

are capable of changing themselves to adapt to changing environment. In this regard many 

artificial systems like those stated earlier can be considered as CAS, due to their capability of 

evolving. Coexistence of competition and cooperation is another dichotomy exhibited by CAS. 

A CAS can be considered as a network of dynamical elements where the states of both the 

nodes and the edges can change, and the topology of the network itself often evolves in time in a 

nonlinear and heterogeneous fashion. A dynamical system can be considered as simply behaving: 

“obeying the laws of physics”. From another perspective, it can be viewed as processing 

information: how systems get information, how they incorporate that information in the models of 

their surroundings, and how they make decisions on the basis of these models determine how they 

behave (Llyod and Slotine 1996). This leads to one of the more heuristic definitions of a complex 

system: one that “stores, processes and transmits, information” (Sawhil 1995). From a 

thermodynamic viewpoint such systems have the total energy (or its analogy) unknown, yet 

something is known about the internal state structure. In these large open systems (do not possess 

well defined boundaries) energy enters at low entropy and is dissipated. Open systems organize 

largely due to the reduction in the number of active degrees of freedom caused by dissipation. 

Not all behaviors or spatial configurations can be supported. The result is a limitation of the 

collective modes, cooperative behaviors, and coherent structures that an open system can express. 

A central goal of the sciences of complex systems is to understand the laws and mechanisms by 

which complicated, coherent global behavior can emerge from the collective activities of 

relatively simple, locally interacting components. 

Complexity arises in natural system thorough evolution, while design plays an analogous role 

for the complex engineering systems. Convergent evolution/design leads to remarkable 

similarities at higher level of organization, though at the molecular or device level natural and 

man-made systems differ significantly. Complexity in both cases is driven far more by the need 

for robustness to uncertainty in the environment and component parts than by basic functionality. 

Through design/evolution, such systems develop highly structured, elaborate internal 

configurations, with layers of feedback and signaling. It is the protocols that organize highly 

structured and complex modular hierarchies to achieve robustness, but also create fragilities to

are or ignored perturbations. The evolution of protocols can lead to a 

robustness/complexity/fragility spiral where complexity added for robustness also adds new 

fragilities, which in turn leads to new and thus spiraling complexities (Csete and Doyle 2002). 

However all this complexity remains largely hidden in normal operation becoming conspicuous 

acutely when contributing to rare cascading failures or chronically through fragility/complexity 

evolutionary spirals. Highly Optimized Tolerance (HOT) (Carlson and Doyle 1999) has been 

introduced recently to focus on the "robust, yet fragile" nature of complexity. It is also becoming 

increasingly clear that robustness and complexity in biology, ecology, technology, and social 

systems are so intertwined that they must be treated in a unified way. Given the diversity of 

systems falling into this broad class, the discovery of any commonalities or “universal” laws 

underlying such systems requires very general theoretical framework. 

The scientific study of CAS has been attempting to find common characteristics and/or formal 

distinctions among complex systems that might lead to better understanding of how complexity 

develops, whether it follows any general scientific laws of nature, and how it might be related to 

simplicity. The attractiveness of the methods developed in this research effort for generalpurpose 

modeling, design and analysis, lies in their ability to produce complex emergent 

phenomena out of a small set of relatively simple rules, constraints and the relationships couched 

in either quantitative or qualitative terms. We believe, that the tools and techniques developed in 

the study of CAS, offers a rich potential for design, modeling and analysis of large-scale systems 

in general and supply chains in particular. 

3. Supply Chain Networks as Complex Adaptive Systems 

A supply chain network is where information, products and finances are transferred between 

various suppliers, manufacturers, distributors, retailers and customers. A supply chain is 

characterized by a forward flow of goods and a backward flow of information. Typically a supply 

chain is comprised of two main business processes: material management and physical 

distribution (Min and Zhou 2002). The material management supports the complete cycle of 

material flow from the purchase and internal control of production material to the planning and 

control of work-in-process, to the warehousing, shipping, and distribution of finished products. 

On the other hand, physical distribution encompasses all the outbound logistics activities related 

to providing customer services. Combining the activities of material management and physical 

distribution, a supply chain does not merely represent a linear chain of one-on-one business 

relationships, but a web of multiple business networks and relationships. 

Supply chain network is an emergent phenomenon. From the view of each individual entity, the 

supply chain is self-organizing. Although the totality may be unknown individual entities partake 

in the grand establishment of the network by engaging in their localized decision-making i.e. in 

doing their best to select capable suppliers and ensure on-time delivery of products to their 

buyers. The network is characterized by nonlinear interactions and strong interdependencies 

between the entities. In most circumstances, order and control in the network is emergent, as 

opposed to predetermined. Control is generated through nonlinear though simple behavioral rules 

that operate based on local information. We argue that a supply chain network forms a complex 

adaptive system: 

• Structures spanning several scales: The supply chain network is a bi-level hierarchical 

and heterogeneous network where at the higher level each node represents an individual 

supplier, manufacturer, distributor, retailer or customer. However at the lower level the 

nodes represent the physical entities that exist inside each node in the upper level. The 

heterogeneity of most networks is a function of various technologies being provided by 

whatever vendor could supply them at the time their need was recognized. 

• Strongly coupled degrees of freedom and correlations over long length and time 

scales: Different entities in a supply chain typically operate autonomously with different 

objectives and subject to different set of constraints. However when it comes to

improving due date performance, increasing quality or reducing costs they become highly 

inter-dependent. It is the flow of material, resources, information and finances that 

provides the binding force. The well fare of any entity in the system directly depends on 

the performance of the others and their willingness and ability to coordinate. This leads to 

correlations between entities over long length and time scales. 

Figure 1. Supply Chain Network 

• Coexistence of Competition and Cooperation: The entities in a supply chain often have 

conflicting objectives. Competition abounds in the form of sharing and contention of 

resources. Global control over nodes is an exception rather than a rule; more likely is a 

localized cooperation out of which a global order emerges, which is itself unpredictable. 

• Nonlinear dynamics involving interrelated spatial and temporal effects: Supply 

chains have wide geographic distribution. Customers can initiate transactions at any time 

with little or no regard for existing load, thus contributing to a dynamic and noisy 

network character. The characteristics of a network tend to drift as workloads and 

configuration change, producing a non-stationary behavior. The coordination protocols 

attempt to arbitrate among entities with resource conflicts. Arbitration is not perfect 

however; hence over and under corrections contribute to the nonlinear character of the 

network. 

• Quasi Equilibrium and combination of regularity and randomness (i.e. interplay of 

chaos and non-chaos) The general tendency of a supply chain is to maintain a stable and 

prevalent configuration in response to external disturbances. However they can undergo a 

radical structural change when they are stretched from equilibrium. At such a point a 

small event can trigger a cascade of changes that eventually can lead to system wide 

reconfiguration. In some situations unstable phenomena can arise, due to feedback 

structure, inherent adjustment delays and nonlinear decision-making processes that go in 

the nodes. One of the causes of unstable phenomena is that the information feedback in 

the system is slow relative to the rate of changes that occur in the system. The first mode 

of unstable behavior to arise in nonlinear systems is usually the simple one-cycle self-

sustained oscillations. If the instability drives the system further into the nonlinear 

regime, more complicated temporal behavior may be generated. The route to chaos 

through subsequent period-doubling bifurcations, as certain parameters of the system are 

varied, is generic to large class of systems in physics, chemistry, biology, economics and 

other fields. Functioning in chaotic regime deprives the ability for long-term predictions 

about the behavior of the system, while short-term predictions may be possible 

sometimes. As a result, control and stabilization of such a system becomes very difficult. 

• Emergent behavior and Self-Organization: With the individual entities obeying a 

deterministic selection process, the organization of the overall supply chain emerges 

through a natural process of order and spontaneity. This emergence of highly structured 

collective behavior, over time from the interaction of the simple entities leads to 

fulfillment of customer orders. Demand amplification, inventory swing are some other 

but undesirable emergent phenomena that can also arise. For instance, the decisions and 

delays downstream in a supply chain often leads to amplifying non-desirable effect 

upstream, a phenomena commonly known as “Bull Whip” effect. 

• Adaptation and Evolution: Supply chain both reacts to and creates it environment. 

Generally speaking a supply chain interacts with almost every other conceivable network. 

Operationally, the environment depends on the chosen scale of analysis, for e.g. it can be 

taken as the customer market. Typically, significant dynamism exists in the environment 

which necessitates a constant adaptation of the supply network. However the 

environment is highly rugged making the co evolution difficult. The individual entities 

constantly observe what emerges from a supply network and make adjustments to 

organizational goals and supporting infrastructure. Another common way of adaptation is 

through altering boundaries of the network. The boundaries can change as a result of 

including or excluding particular entity and by adding or eliminating connections among 

entities, thereby changing the underlying pattern of interaction. As we discuss next, 

Supply chain management plays a critical role in making the network evolve in a 

coherent manner. 

3.1 Supply Chain Management 

Supply chain management is defined as the integration of key business processes from endusers 

through original suppliers that provide products, services, and information and add value for 

customers and other stakeholders (Cooper et. al. 1997). It involves balancing reliable customer 

delivery with manufacturing and inventory costs. It is evolved around a customer-focused 

corporate vision, which drives changes throughout a firm’s internal and external linkages and 

then captures the synergy of inter-functional, inter-organizational integration and coordination. 

Due to the inherent complexity it is a challenge to coordinate the actions of entities across 

organizational boundaries so that they perform in a coherent manner. 

An important element in managing SCN is to control the ripple effect of lead-time so that the 

variability in supply chain can be minimized. Demand forecasting is used to estimate demand for 

each stage, and the inventory between stages for the network is used for protecting against 

fluctuations in supply and demand across the network. Due to the decentralized control properties 

of the SCN, control of ripple effect requires coordination between entities in performing their 

tasks. The problem of coordination has reached another dimension due to some other trends in the 

current supply chains. 

Two important organizational and market trends that are on their way have been the 

atomization of markets as well as that of organizational entities (Balakrishnan et al. 1999). In 

such a scenario product realization process has a continuous customer involvement in all phases - 

from design to delivery. Customization is not only limited to selecting from pre-determined 

model variants; rather, product design, process plans, and even the supply chain configuration 

have to be tailored for each customer. The product realization organization has to be formed on

the fly- as a consortium of widely dispersed organizations to cater to the needs of a single 

customer. Thus organizations consist of series of opportunistic alliances among several focused 

organizational entities to address particular market opportunities. For manufacturing 

organizations to operate effectively in this environment of dynamic, virtual alliances, products 

must have modular architectures, processes must be well characterized and standardized, 

documentation must be digitized and widely accessible, and systems must be interoperable. 

Automation and intelligent information processing is vital for diagnosing problems during 

product realization and usage, coordination, design and production schedules, searching for 

relevant information in multi-media databases. These trends exacerbate the challenges of 

coordination and collaboration as the number of product realization networks increase, and so 

does the number of partners in each network. 

Inventory is unwise approach to dealing with highly changing market demand and short life 

cycle products. Information is an appropriate substitute for inventory. Information about the 

material lead-time from different suppliers can be used for planning the material arrival, instead 

of building up inventory. The demand information can be transmitted to the manufactures on a 

timely basis, so that the orders can be fulfilled with less inventory costs. In fact it is widely 

realized, that the successful integration of the entire supply chain process depends heavily on the 

availability of accurate and timely information that can be shared by all members of the supply 

chain. Supply chain management now increasingly relies on Information Technology as discussed 

below. 

3.2 Information Technology in Supply Chain Management 

Information technology with its capability of providing global reach and wide range of 

connectivity, enterprise integration, micro autonomy and intelligence, object and networked 

oriented computing paradigms and rich media support; has been key enabler for the management 

of future manufacturing enterprises. It is vital for eliminating collaboration and coordination 

costs, and to permit rapid setup of dynamic information exchange networks. Connectivity 

permits involvement of customers and other stakeholders in all aspects of manufacturing. 

Enterprise integration facilitates seamless interaction among global partners. Micro autonomy and 

intelligence permit atomic tracking and remote control. New software paradigms enable 

distributed, intelligent and autonomous operations. Distributed computing facilitates quick 

localized decisions without loosing vast data gathering potential and powerful computing 

capabilities. Rich media support, which includes capabilities like digitization, visualization tools 

and virtual reality, facilitate collaboration and immersion. 

Many improvements have occurred in supply chain management because IT enables changes to 

be made in inventory management and production, dynamically. It assists the managers in coping 

up with uncertainty and lead-time through improved collection and sharing of information 

between supply chain nodes. The success of an enterprise is now largely dependent on how its 

information resources are designed, operated and managed, especially with the Information 

technology emerging as a critical input to be leveraged for significant organizational productivity. 

However, the difficulty arises when trying to design an information system that can handle the 

information needs of supply chain nodes to allow efficient, flexible and decentralized supply 

chain management. The main hurdle in efficiently using information technology is the lack of our 

understanding of the organizational, functional and evolutionary principles of supply chains. 

Recognizing supply chains as CAS, can however lead to novel and effective ways to 

understand their emergent dynamics. It has been found that many of the diverse looking CAS 

share similar characteristics and problems and thus can be tackled through similar approaches. 

While at present networks are largely controlled by humans; the complexity, diversity and 

geographic distribution of the networks, makes it necessary that the networks maintain 

themselves in a sort of evolutionary sense, just as biological organisms do (Maxon 1990). 

Similarly, the problem of coordination, which is a challenge in supply chains, has been routinely

solved by biological systems for literally billions of years. We believe that the complexity, 

flexibility and adaptability in the collective behavior of the supply chains can be accomplished 

only by importing the mechanisms that govern these features in nature. Along with these robust 

design principles, we require equally sound techniques for modeling and analysis of supply 

chains. This would form the focus of this paper. We first give a brief overview of the main 

techniques that have been used for modeling and analysis of supply chains and then discuss how 

the science of complexity provides a genuine extension and reformulation of these approaches. 

4. Modeling and Analysis of Supply Chain Networks 

As pointed out the key challenge in designing supply chain networks or for that matter any 

large-scale systems is the difficulty of reverse engineering, i.e., determining what individual agent 

strategies lead to the desired collective behavior. Due to this difficulty in understanding the effect 

of individual characteristics on the collective behavior of the system, simulation have been the 

primary tools for designing and optimizing such systems. Simulation makes investigations 

possible and useful, when in the real world situation experimentation would be too costly or for 

ethical reasons not feasible, or where the decisions and their consequences are well separated in 

space and time. It seems at present that large-scale simulations of future complex processes may 

be the most logical, and perhaps, an important vehicle to study them objectively (Ghosh, 2002). 

Simulation in general helps one to detect design errors, prior to developing a prototype in a cost 

effective manner. Secondly, simulation of system operations may identify potential problems that 

might occur during actual operation. Thirdly, extensive simulation may potentially detect 

problems that are rare and otherwise elusive. Fourthly, hypothetical concepts that do not exist in 

nature, even those that defy natural laws, may be studied. The increased speed and precision of 

today’s computers promise the development of high fidelity models of physical and natural 

processes, ones that yield reasonably accurate results, quickly. This in turn would permit system 

architects to study the performance impact of wide variation of key parameters, quickly and in 

some cases, even in real time. Thus a qualitative improvement in system design may be achieved. 

In many cases, unexpected variations in external stress can be simulated quickly to yield 

appropriate system parameters values, which are then adopted into the system to enable it to 

successfully counteract the external stress. 

Mathematical analysis on the other hand has to a play a critical role because it alone can enable 

us to formulate rigorous generalizations or principle. Neither physical experiments nor computerbased 

experiments on their own can provide such generalizations. Physical experiments usually 

are limited to supplying inputs and constraints for rigorous models, because experiments 

themselves are rarely described in a language that permits deductive exploration. Computer based 

experiments or simulations have rigorous descriptions, but they deal only in specifics. A welldesigned 

mathematical model on the other hand generalizes the particulars revealed by the 

physical experiments, computer based models and any interdisciplinary comparisons. Using 

mathematical analysis we can study the dynamics, predict long term behavior, gain insights into 

system design: e.g., what parameters determine group behavior, how individual agent 

characteristics affect the system and that the proposed agent strategy leads to the desired group 

behavior. In addition, mathematical analysis may be used to select parameters that optimize 

system’s collective behavior, prevent instabilities, etc. 

It seems that successful modeling efforts of large-scale systems like supply chain network, 

large-scale software systems, communication networks, biological ecosystems, food webs, social 

organizations, etc. would require a solid empirical base. Pure abstract mathematical 

contemplation would unlikely lead to useful models. The discipline of physics provides an 

appropriate parallel; advances in theoretical physics are more often than not inspired by 

experimental findings. The study of supply chain networks should therefore involve an amalgam 

of both simulation and analytical techniques.

Considering the broad spectrum of a supply chain, no model can capture all the aspects of 

supply chain processes. The modeling proceeds at three levels: 

• Competitive Strategic analysis, which includes location-allocation decision, demand 

planning, distribution channel planning, strategic alliances, new product development, 

outsourcing, IT selection, pricing, and network structuring. 

• Tactical problems like inventory control, production/distribution coordination, material 

handling, layout design. 

• Operational level problems, which includes routing/scheduling, workforce scheduling 

and packaging. 

The models in supply chains can be categorized into four classes (Min and Zhou 2002): 

• Deterministic: single objective and multiple objective models. 

• Stochastic: optimal control theoretic and dynamic programming models. 

• Hybrid: with elements of both deterministic and stochastic models and includes inventory 

theoretic and simulations models. 

• IT driven: models that aim to integrate and coordinate various phases of supply chain 

planning on a real-time bases using application software, like ERP. 

Mathematical programming techniques and simulation have been primarily two approaches for 

the analysis and study of the supply chains models. The mathematical programming mainly takes 

into consideration static aspects of supply chain. The simulation on the other hand studies 

dynamics in supply chains and generally proceeds based on “system dynamics” and “agent 

based” methodologies. System dynamics is a continuous simulation methodology that uses 

concepts from engineering feedback control to model and analyze dynamic socioeconomic 

systems (Forrester, 1961). The mathematical description is realized with the help of ordinary 

differential equation. An important advantage of system dynamics is the possibility to deduce the 

occurrence of a specific behavior mode because the structure that leads to the system dynamics is 

made transparent. We present some nonlinear models in Section 5 which are useful for 

understanding the complex interdependencies, effects of priority, nonlinearities, delays, 

uncertainties and competition/cooperation for resource sharing in supply chains. The drawback of 

system dynamics model is that the structure has to be determined before starting the simulation. 

Agent-based modeling (a technique from complexity theory) on the other hand is a “bottom up 

approach” which simulates the underlying processes believed responsible for the global pattern, 

and allows us to evaluate what mechanisms are most influential in producing that emergent 

pattern. In (Schieritz and Grobler, 2003) a hybrid modeling approach has been presented that 

intends to make the system dynamics approach more flexible by combining it with the discrete 

agent-based modeling approach. Such large-scale simulations with their many degrees of freedom 

raise serious technical problems about the design of experiments and the sequence in which they 

should be carried out in order to obtain the maximum relevant information. Furthermore, in order 

to analyze data from such large-scale simulations we require systematic analytical and statistical 

methods. In Section 8, we describe two such techniques: Nonlinear Time Series Analyses and 

Computational Mechanics. 

A useful paradigm for modeling a supply chain, taking into consideration the detailed pattern of 

interaction is to view it as a network. A network is essentially anything that can be represented by 

a graph: a set of points (also generically called nodes or vertices), connected by links (edges, ties) 

representing some relationship. Networks are inherently difficult to understand due to their 

structural complexity, evolving structure, connection diversity, dynamical complexity of nodes, 

node diversity and meta–complication where all these factors influence each other. Queuing 

theory has primarily been used to address the steady-state operation of a typical network. On the 

other hand techniques from mathematical programming have been used to solve the problem of 

resource allocation in networks. This is meaningful when dynamic transients can be disregarded. 

However, present day supply chain networks are highly dynamic, reconfigurable, intrinsically

non-linear and non-stationarity. New tools and techniques are required for their analysis such that 

the structure, function and growth of networks can be considered simultaneously. In this regard 

we discuss “Network Dynamics” in Section 9, which deals with such issues and can be used to 

study the structure of supply chain and its implication for its functionality. Understanding the 

behavior of large complex networks is the next logical step for the field of nonlinear dynamics, 

because they are so pervasive in the real world. We begin with a brief introduction to dynamical 

systems theory, in particular nonlinear dynamics in next section. 

5 Dynamical Systems Theory 

Many physical systems that produce continuous-time response can be modeled by a set of 

differential equations of the form: 

dy = f ( y, 

a) 

, (I) 

dt 

where, y = y ( t), 

y ( t),...... 

y ( )) represents the state of the system and may be thought of as a 

( 

1 2 

n 

t 

point in a suitably defined space S-which is known as phase space and 

a = a ( t), 

a ( t) 

L, 

a ( )) is a parameter vector. The dimensionality of S is the number of 

( 

1 2 

m 

t 

apriori degrees of freedom in the system. The vector field f(y,a) is in general a non-linear operator 

acting on points in S. If f(y,a) is locally Lipschtiz, above equation defines an initial value problem 

in the sense that a unique solution curve passes through each point y in the phase space. Formally 

we may write the solution at time t given an initial value y0 as y( t) 

= ϕ 

t 

y0 

. ϕ 

t 

represents a oneparameter 

family of maps of the phase space into itself. We can perceive the solutions to all 

possible initial value problems for the system by writing them collectively as ϕ . This may be 

thought of as a flow of points in the phase space. Initially the dimension of the set ϕ t 

S will be 

that of S itself. As the system evolves, however, it is generally the case for the so-called 

dissipative system that the flow contracts onto a set of lower dimension known as attractor. The 

attractors can vary from simple stationary, limit cycle, quasi-periodic to complicated chaotic ones 

(Strogatz 1994, Ott 1996). The nature of attractor changes as parameters (a) are varied, a 

phenomena studied in bifurcation analysis. Typically a nonlinear system is always chaotic for 

some range of parameters. Chaotic attractors have a structure that is not simple; they are often not 

smooth manifolds, and frequently have a highly fractured structure, which is popularly referred to 

as Fractals (self–similar geometrical objects having structure at every scale). On this attractor, 

stretching and folding characterize the dynamics; the former phenomenon causes the divergence 

of nearby trajectories and latter constraints the dynamics to finite region of the state space. This 

accounts for fractal structure of attractors and the extreme sensitivity to changes in initial 

conditions, which is hallmark of chaotic behavior. System under chaos is unstable everywhere 

never settling down, producing irregular and aperiodic behavior which leads to a continuous 

broadband spectrum. While this feature can be used to distinguish chaotic behavior from 

stationary, limit cycle, quasi-periodic motions using standard Fourier Analysis it makes it 

difficult to separate it from noise which also has a broadband spectrum. It is this “deterministic 

randomness” of chaotic behavior, which makes standard linear modeling and prediction 

techniques unsuitable for analysis. 

5.1 Nonlinear Models for Supply Chain 

Understanding the complex interdependencies, effects of priority, nonlinearities, delays, 

uncertainties and competition/cooperation for resource sharing is fundamental for prediction and 

control of supply chains. System dynamics approach often leads to models of supply chains, 

which can be described in the form of equation (I). Dynamical systems theory provides a 

powerful framework for rigorous analysis of such models and thus can be used to supplement the 

S t

system dynamics approach. We next describe some nonlinear models and their detailed analysis. 

These models can be either used to represent entities in a supply chain or as macroscopic models, 

which capture collective behavior. The models reiterate the fact that simple rules can lead to 

complex behavior, which in general are difficult to predict and control. 

5.1.1 Preemptive Queuing Model with delays 

Priority and heterogeneity are fundamental to any logistic planning and scheduling. Tasks have 

to be prioritized in order to do the most important things first. This comes naturally as we try to 

optimize an objective and assign the tasks their “importance.” Priorities may also arise due to the 

non-homogeneity of the system where “knowledge” level of one agent is different from the other. 

In addition in all logistics systems, resources are limited, both in time and space. Temporal 

dependence plays an important role in logistic planning (interdependency). Sometime they can 

also arise from the physical facts when different stages of processing have certain temporal 

constraint. 

The considerations regarding the generality of assumptions and the clear one-to-one 

correspondence between the physical logistics tasks and the model parameters described in 

(Erramilli, and Forys 1991) made us apply the queuing model in context of supply chains 

(Kumara et al. 2003). The Queuing system considered here has two queues (A and B) and a 

single server with following characteristics: 

• Once served, the class A customer returns as a class B customer after a constant interval of 

time 

• Class B has non-preemptive priority over class A, i.e., the class A queue does not get served 

until the class B queue is emptied. 

• The schedules are organized every T units of time, i.e., if the low priority queue is emptied 

within time T, the server remains idle for the reminder of the interval. 

• Finally, the higher priority class B has a lower service rate than the low priority class A. 

Figure 2. Preemptive Queuing Model 

Suppose the system is sampled at the end of every schedule cycle, and the following 

quantities are observed at the beginning of the kth interval: 

A 

k 

: Queue length of low priority queue 

B : Queue length of high priority queue 

k 

C : Outflow from low priority queue in the kth interval 

k

D 

k 

: Outflow from high priority queue in the kth interval 

λ : Inflow to low priority queue from the outside in the kth interval 

k 

The system is characterized by the following parameters: 

µ 

a 

: Rate per unit of the schedule cycle at which the low priority queue can be served 

µ : Rate per unit of the schedule cycle at which the high priority queue can be served 

b 

l: The feedback interval in units of the schedule cycle 

The following four equations then completely describe the evolution of the system: 

A 

C 

B 

k + 1 

= Ak 

+ λk 

− Ck 

(1) 

Dk 

= min( Ak 

+ λk 

, µ 

a 

(1 − )) 

(2) 

µ 

k 

b 

k +1 

= Bk 

+ Ck 

−l 

− Dk 

(3) 

D = min( B + C , µ ) 

(4) 

k 

k 

k −l 

b 

Equations (1) and (3) are merely conservation rules, while equations (2) and (4) model the 

constraints on the outflows and the interaction between the queues. This model while 

conceptually simple, exhibits surprisingly complex behaviors. 

Dynamical Behavior 

The analytic approach to solve for the flow model under constant arrivals (i.e. λ 

k 

= λ for all k) 

shows several classes of solutions. The system is found to batch its workload even for perfectly 

smooth arrival patterns. Following are the characteristics of behavior of the system: 

1) Above a threshold arrival rate ( λ ≥ / 2 ), a momentary overload can send the system 

µ b 

into a number of stable modes of oscillations. 

2) Each mode of oscillations is characterized by distinct average queuing delays. 

3) The extreme sensitivity to parameters, and the existence of chaos, implies the system at a 

given time may be any one of a number of distinct steady-state modes. 

The batching of the workload can cause significant queuing delays even at moderate occupancies. 

Also such oscillatory behavior significantly lowers the real-time capacity of the system. For 

details of application of this model in supply chain context, refer to (Kumara et al. 2003). 

5.1.2 Managerial Systems 

Decision-making is another typical characteristic in which the entities in a supply chain are 

continuously engaged in. Entities make decisions to optimize their self-interests, often based on 

local, delayed and imperfect information. 

To illustrate the effects of decisions on the dynamics of supply chain as a whole, we consider a 

managerial system, which allocates resources to its production and marketing departments in 

accordance with shifts in inventory and/or backlog (Rasmussen and Moseklide 1988). It has four 

level variables: resources in production, resources in sales, inventory of finished products and 

number of customers. In order to represent the time required to adjust production, a third order 

delay is introduced between production rate and inventory. The sum of the two resource variables 

is kept constant. The rate of production is determined from resources in production through a 

nonlinear function, which expresses a decreasing productivity of additional resources as the

company approaches maximum capacity. The sales rate, on the other hand, is determined by the 

number of customers and by the average sales per customer-year. Customers are mainly recruited 

through visits of the company salesman. The rate of recruitment depends upon the resources 

allocated to marketing and sales, and again it is assumed that there is a diminishing return to 

increasing sales activity: Once recruited, customers are assumed to remain with the company for 

an average period AT, the association time. 

A difference between production and sales causes the inventory to change. The Company is 

assumed to respond to such changes by adjusting its resource allocation. When the inventory is 

lower than desired, on the other hand, resources are redirected from sales to production. A certain 

minimum of resources is always maintained in both production and sales. In the model, this is 

secured by means of two limiting factors, which reduce the transfer rate when a resource floor is 

approached. Finally the model assumes that there is a feedback from inventory to customer 

defection rate. If the inventory of finished products becomes very low, the delivery time is 

assumed to become unacceptable to many customers. As a consequence, the defection rate is 

enhanced by a factor 1+H. 

Figure 3. Managerial System 


The managerial system described is controlled by two interacting negative feedback. Combined 

with the delays involved in adjusting production and sales, these loops create the potential for 

oscillatory behavior. If the transfer of resources is fast enough, this behavior is destabilized and

the system starts to perform self-sustained oscillations. The amplitude of these oscillations is 

finally limited by the various nonlinear restrictions in the model, particularly by the reduction of 

resource transfer rate as lower limits to resources in production or resources in sales are 

approached. 

A series of abrupt changes in the system behavior is observed as competition between the basic 

growth tendency and nonlinear limiting factors is shifted. The simple one-cycle attractor 

corresponding to H=10, becomes unstable for H=13 and a new stable attractor with twice the 

original period arises. If H is increased to 28 the stable attractor attains a period of 4. As H is 

further increased, the period-doubling bifurcations continue until H=30 the threshold to chaos is 

exceeded. The system now starts to behave in an aperiodic and apparently random behavior. 

Hence the system shows chaotic behavior through a series of period doubling bifurcations. 

5.1.3 Deterministic Queuing Model 

In this section we consider an alternate discrete-time deterministic queuing model, for studying 

decision making at an entity level in supply chains. The model consists of one server and two 

queuing lines (X and Y) representing some activity (Feichtinger et al. 1994). The input rates of 

both queues are constant and their sum equals the server-capacity. In each time period the server 

has to decide how much time to spend on each of the two activities. 

The following quantities can be defined: 

α : Constant input rate for activity X 

β : Constant input rate for activity Y 

Φ 

X 

: Time spent on activity X 

Φ 

Y 

: Time spent on activity Y 

x : Queue length of X 

k 

y : Queue length of Y 

k 

The amount of time 

Figure 4. Deterministic Queuing Model 

Φ 

X 

and Φ that will be spent on activities X and Y in period k+1 are 

Y 

determined by an adaptive feedback rule depending on the difference of the queue lengths 

x k 

and

y 

k 

.The decision rule or policy function says that longer queues are served with higher priority. 

Two possibilities considered are: 

1) All-or nothing decision: the server decides to spend all its time on the activity corresponding to 

the longer queue. Hence Φ is a Heaviside function given by 

Φ( x − y) 

= 1 if x ≥ y 

=0 if x < y . 

2) Mixed Solutions: the server decides to spend most of its time to the activity corresponding to 

the longer queue. For this decision function a S-shaped logistic function is used as given by: 

1 

Φ ( x − y) 

= . 

k ( x− 

) 

1+ 

e 

y 

The parameter k tunes the “steepness” of the S-shape. 

With these decision functions the new queue lengths and are given equations 

xk+1 

y k+ 1 

xk+ 1 

= xk 

+ α − Φ( 

xk 

− yk 

) , 

yk+ 1 

= yk 

+ β − Φ( 

xk 

− yk 

) . 

Using the constraints α + β = 1 and Φ 

X 

+ ΦY 

= 1, it is sufficient to consider the dynamics of 

the map in order to study the behavior of the system 

f ( x) 

= x + α − Φ(2x 

− 2) . 


For 0

sustained in a supply chain. Resources can be of various types: physical resources, manpower, 

information and monetary. With the IT architectures being developed to realize supply chains, 

sharing of computational resources (like CPU, Memory, Bandwidth, databases etc.) is also 

becoming a critical issue. It is through resource sharing that interdependencies arise between 

different entities. This leads to a complex web of interactions in supply chains just like for e.g. in 

a food web or an ecology. As a result such systems can be referred to as “Computational 

Ecosystems” (Hogg and Huberman 1988) in analogy with biological ecosystems. 

“Computational Ecosystems” is a generic model of the dynamics of resource allocation among 

agents trying to solve a problem collectively. The model captures following features: distributed 

control, asynchrony in execution, resource contention and cooperation among agents and 

concomitant problem of incomplete knowledge and delayed information. The behavior of each 

agent is modeled using a payoff function whose nature determines whether an agent is 

cooperative or competitive. The agent here can be any entity in a supply chain like a distributor, 

retailer etc. or a software agent in e-commerce scenario. The state of the system is represented as 

an average number of entities using different resources and follows a delay differential equation 

under mean field approximation. The resources can be physical or computational as discussed 

before. For example in case of two resources with n identical agents, law governing the rate of 

change of occupation of a resource is given by: 

d n1 

( t) 

= α ( n ρ − n1 

( t) 

) 

dt 

where, 

n1 ( t) 

= Expected no. of agents using resource 1 at given instant of time t. 

α : Expected no. of choices made by an agent per unit time 

ρ : A random variable that denotes that resource1 will be perceived to have a higher payoff than 

res ource 2 and ρ gives its expected value. 

Figure 5. Computational Ecosystems

(τ = Time delay and σ : Standard deviation of ρ ) 

The global performance of the ecosystems can be obtained from the above equation. Under 

different conditions of delay, uncertainty, cooperation/competition the system shows a rich 

panoply of behaviors ranging from stable, sustained oscillations to intermittent chaos and finally 

to fully developed chaos. Furthermore, following generic deductions can be made from this 

model (Kephart et al. 1989): While information delay has adverse impact on the system 

performance, uncertainty has a profound effect on the stability of the system. One can 

deliberately increase uncertainty in agents’ evaluation of the merits of choices to make it stable 

but at the expense of performance degradation. Second possibility is very slow reevaluation rate 

of the agents, which however makes them non-adaptive. Heterogeneity in the nature of agents 

can however lead to more stability in the system compared to homogenous case but the system 

loses its ability to cope up with unexpected changes in the system such as new task requirements. 

On the other hand poor performance can be traced to the fact that the non-predictive agents do not 

take into account the information delay. 

If the agents are able to make accurate predictions of its current state, the information delay 

could be overcome, and the system would perform well. This results in a “co-evolutionary” 

system in which all of the individual are simultaneously trying to adapt to one another. In such a 

situation agents can act like Technical Analysts and System Analysts (Kephart et al. 1990). Agents 

as technical analysts (like those in market behavior) use either linear extrapolation or cyclic trend 

analysis to estimate the current state of the system. On the other hand, agents as system analysts 

have knowledge about both the individual characteristics of the other agents in the system and 

how those characteristics are related to the overall system dynamics. Technical Analysts are 

responsive to the behavior of the system, but suffer from an inability to take into account the 

strategies of other agents. Moreover good predictive strategy for a single agent may be disastrous 

if applied on a global scale. System Analysts perform extremely well when they have very 

accurate information about other agents in the system, but can perform very poorly when their 

information is even slightly inaccurate. They take into account the strategies of other agents, but 

pay no heed to the actual behavior of the system. This suggests combining the strengths of both 

methods to form a hybrid- adaptive system analyst-, which modifies its assumptions about other 

to feedback about success of its own predictions. The resultant hybrid is able 

agents in response 

to perform well. 

In order to avoid chaos while maintaining high performance and adaptability to unforeseen 

changes more sophisticated techniques are required. One such way is by reward mechanism 

(Hogg and Huberman 1991) whereby the relative number of computational agents following 

effective strategies is increased at the expense of the others. This procedure, which generates a 

right mix of diverse population out of essentially homogenous ones, is able to control chaos by a 

series of bifurcations into a stable fixed point. 

In the above description each agent chooses amongst different resources according to its 

perceived payoff, which depends on the number of agents already using it. Even the agent with 

predictive ability is myopic in its view, as it considers only its current estimate of the system 

state, without regard to the future. Expectations come into play if agents use past and present 

global behavior in estimating the expected future payoff for each resource. A dynamical model of 

collective action that includes expectations can be found in (Glance 1993). 

6. Models from Observed Data 

One of the central problems in a supply chain, closely related to modeling, is that of demand 

forecasting: given the past, how can we predict the future demand? The classic approach to 

forecasting is to build an explanatory model from the first principle and measure the initial 

conditions. Unfortunately this has not been possible for two reasons in systems like supply 

chains: Firstly, we still lack the general “first principles” for demand variation in supply chains,

which are necessary to make good models. Secondly, due to the distributed nature of the supply 

chains, the initial data or the conditions are often difficult to obtain. 

Due to these factors, the modern theory of forecasting that has been used in supply chains, 

views a time series x(t) as a realization of a random process. This is appropriate when effective 

randomness arises from complicated motion involving many independent, irreducible degrees of 

freedom. An alternative cause of randomness is chaos, which can occur even in very simple 

deterministic systems as we discussed in the earlier sections. While chaos places a fundamental 

limit on long-term prediction, it suggests possibilities for short-term prediction. Random-looking 

data may contain only few irreducible degrees of freedom. Time traces of the state variable of 

such chaotic systems display a behavior, which is intermediate between regular periodic or 

quasiperiodic motions, and unpredictable, truly stochastic behavior. It has long been seen as a 

form of “noise” because the tools for its analysis were couched in language tuned to linear 

process. The main such tool is Fourier analysis which is precisely designed to extract the 

composition of sines and cosines found in an observation x(t). Similarly, the standard linear 

modeling and prediction techniques, such as autoregressive moving average (ARMA) models are 

not suitable for nonlinear systems. 

With the advances in IT and science of complexity both the challenges for forecasting can be 

revived. Large-scale simulation and micro autonomy (Section 2) enable tracking of the detailed 

interaction between different entities in a supply chain. The large volumes of data, so generated 

can be used to understand demand patterns in specific and comprehend the emergence of other 

characteristics in general. Even though an exact prediction of future behavior is difficult, often 

archetypal behavior patterns can be recognized using this data. Techniques from the complexity 

theory like Nonlinear Time Series Analysis and Computational Mechanics are appropriate for this 

purpose. 

6.1 Nonlinear Time Series Analysis 

The need to extract interesting physical information about the dynamics of observed systems 

when they are operating in a chaotic regime has led to development of nonlinear time series 

analysis techniques. Systematically, the study of potentially, chaotic systems may be divided into 

three areas: identification of chaotic behavior, modeling and prediction and control. The first area 

shows how chaotic systems may be separated from stochastic ones and, at the same time, 

provides estimates of the degrees of freedom and the complexity of the underlying chaotic 

system. Based on such results, identification of a state space representation allowing for 

subsequent predictions may be carried out. The last stage, if desirable involves control of a 

chaotic system. 

Given the observed behavior of a dynamical system as a one-dimensional time series x(n) we 

want to build models for prediction. The most important task in this process is phase space 

reconstruction, which involves building topologically and geometrically equivalent attractor. In 

general steps in nonlinear time series analysis can be summarized as (Abarbanel 1996): 

• Signal Separation (Finding the signal): Separation of broadband signal form broadband 

“noise” using deterministic nature of signal. 

• Phase Space reconstruction (Finding the space): Using the method of delays one can 

construct series of vectors, which is diffeomorphically equivalent to the attractor of the 

original dynamical system and at the same time distinguish it from the being stochastic. 

The basis for this is Taken’s Embedding theorem (Takens 1981). Time lagged variables 

are used to construct vectors for a phase space in d E dimension: 

y( n) 

= [ x( 

n), 

x( 

n + T),...... 

x( 

n + ( d − E 

1) T)] 

The time lag T can be determined using mutual information (Fraser and Swinney 1983) 

and d E using false nearest neighbors test (Kennel et al. 1992).

• Classification of the signal: System identification in nonlinear chaotic systems means 

establishing a set of invariants for each system of interest and then comparing 

observations to that library of invariants. The invariants are properties of attractor and are 

independent of any particular trajectory of the attractor. Invariants can be divided into 

two classes: fractal dimensions (Farmer et. al. 1983) and Lyapunov exponents (Sano and 

Sawada 1985). Fractal dimensions characterize geometrical complexity of dynamics i.e. 

how the sample of points along a system orbit are distributed spatially. Lyapunov 

exponents on the other hand describe the dynamical complexity i.e. “stretching and 

folding” in the dynamical process. 

• Making models and Prediction: This step involves determination of the parameters of 

the assumed model of the dynamics: 

y( 

n) 

→ y( 

n + 1) 

y( 

n + 1) = F( 

y( 

n), 

a , a 

,..... a 

1 2 p 

which is consistent with invariant classifiers (Lyapunov exponents, dimensions). The functional 

form F (⋅) often used, includes polynomials, radial basis functions etc. Local False Nearest 

Neighbor (Abarbanel and Kennel 1993) test is used to determine how many dimensions are 

locally required to describe the dynamics generating the time series, without knowing the 

equations of motion and hence gives the dimension for the assumed model. The methods for 

building nonlinear models can be classified as Global and Local (Farmer and Sidorowich 1987; 

Casdalgi 1989). By definition Local methods vary from point to point in the phase space while 

Global Models are constructed once and for all in the whole phase space. Models based on 

Machine Learning techniques such as radial basis functions or Neural Networks (Powell 1987) 

and Support Vector Machines (Mukherjee et al. 1997) carry features of both. They are usually 

used as global functional forms, but they clearly demonstrate localized behavior too. 

The techniques from nonlinear time series analysis are well suited for modeling the 

nonlinearities in the supply chains. For an application of nonlinear time series analysis in supply 

chains, the reader is referred to Lee et al., 2002. Using it one can deduce that the time series is 

deterministic, so that it should be possible in principle to build predictive models. The invariants 

can be used to effectively characterize the complex behavior. For e.g., the largest Lyapunov 

exponent gives an indication of how far into the future, reliable predictions can be made while the 

fractal dimensions gives an indication of how complex a model should be chosen to represent the 

data. These models then provide the basis for systematically developing the control strategies. It 

should be noted the functional forms used for modeling in the step (4) above, are continuous in 

their argument. This approach builds models viewing a dynamical system as obeying laws of 

physics. From another perspective a dynamical system can be considered as processing 

information. So an alternative class of discrete “computational” models inspired from the theory 

of automata and formal languages can also be used for modeling the dynamics (Marcus 1996). 

“Computational Mechanics”, considers this viewpoint and describes the system behavior in terms 

of its intrinsic computational architecture i.e. how it stores and processes information. 

6.2 Computational Mechanics 

Computational mechanics is a method for inferring the causal structure of stochastic processes 

from empirical data or arbitrary probabilistic representations. It combines ideas and techniques 

from nonlinear dynamics, information theory and automata theory, and is, as it were, an “inverse” 

to statistical mechanics. Instead of starting with a microscopic description of particles and their 

interactions, and deriving macroscopic phenomena, it starts with observed macroscopic data, and 

infers the simplest causal structure: the “ε -machine” capable of generating the observations. The 

ε -machine in turn describes the system's intrinsic computation, i.e., how it stores and processes 

information. This is developed using the statistical mechanics of orbit ensembles, rather than 

) 

a j

focusing on the computational complexity of individual orbits. By not requiring a Hamiltonian, 

computational mechanics can be applied in a wide range of contexts, including those where an 

energy function for the system may not manifest like for the supply chains. Notions of 

Complexity, Emergence and Self-Organization have also been formalized and quantified in terms 

of various information measures (Shalizi 2000). 

Given a time series, the (unknowable) exact states of an observed system are translated into 

sequence of symbols via a measurement channel (Crutchfield 1992). Two histories (i.e., two 

series of past data) carry equivalent information if they lead to the same (conditional) probability 

distribution in the future (i.e., if it makes no difference whether one or the other data-series is 

observed). Under these circumstances, i.e., the effects of the two series being indistinguishable, 

they can be lumped together. This procedure identifies causal states, and also identifies the 

structure of connections or succession in causal states, and creates what is known as an “epsilonmachine”. 

The ε -machines form a special class of Deterministic Finite State Automata (DFSA) 

with transitions labeled with conditional probabilities and hence can also be viewed as Markov 

chains. However, finite-memory machines likeε -machines may fail to admit a finite size model 

implying that the number of casual states could turn out to be infinite. In this case, a more 

powerful model than DFSA needs to be used. One proceeds by trying to use the next most 

powerful model in the hierarchy of machines known as the casual hierarchy (Crutchfield 1994), 

in analogy with the Chomsky hierarchy of formal languages. While “ε -machine reconstruction” 

refers to the process of constructing the machine given an assumed model class, “hierarchical 

machine reconstruction” describes a process of innovation to create a new model class. It detects 

regularities in a series of increasingly accurate models. The inductive jump to a higher 

computational level occurs by taking those regularities as the new representation. 

ε -machines reflect a balanced utilization of deterministic and random information processing 

and this is discovered automatically during ε -machine reconstruction. These machines are 

unique and optimal in the sense that they have maximal predictive power and minimum model 

size (hence satisfy Principle of Occam Razor i.e. causes should not be multiplied beyond 

necessity). ε -machine provides a minimal description of the pattern or regularities in a system in 

the sense that the pattern is the algebraic structure determined by the causal states and their 

transitions. ε -machines are also minimally stochastic. Hence computational mechanics acts as a 

method for automatic pattern discovery. 

ε -machine is the organization of the process, or at least of the part of it which is relevant to 

our measurements. The ε -machine being a model of the observed time series from a system can 

be used to define and calculate macroscopic or global properties that reflect the characteristic 

average information processing capabilities of the system. Some of these include Entropy rate, 

Excess entropy and Statistical Complexity (Feldman and Crutchfield 1998) and (Crutchfield and 

Feldman 2001). The entropy density indicates how predictable the system is. Excess entropy on 

other hand provides a measure of the apparent memory stored in a spatial configuration and 

represents how hard it is the prediction.ε -machine reconstruction leads to a natural measure of 

the statistical complexity of a process, namely the amount of information needed to specify the 

state of the ε -machine i.e. the Shannon Entropy. Statistical Complexity is distinct and dual from 

information theoretic entropies and dimension (Crutchfield and Young 1989). The existence of 

chaos shows that there is rich variety of unpredictability that spans the two extremes: periodic and 

random behavior. This behavior between two extremes while of intermediate information content 

is more complex in that the most concise description (modeling) is an amalgam of regular and 

stochastic processes. Information theoretic description of this spectrum in terms of dynamical 

entropies measures raw diversity of temporal patterns. The dynamical entropies however do not 

measure directly the computational effort required in modeling the complex behavior, which is 

what statistical complexity captures.

Computational mechanics sets limits on how well processes can be predicted and shows how at 

least in principle, those limits can be attained. ε -machines are what any prediction method would 

build, if only they could. Similar to ε -machine reconstruction, techniques exists which can be 

used to discover casual architecture in memory less transducers, transducers with memory and 

spatially extended systems (Shalizi 2000). Computational mechanics can be used for modeling 

and prediction in supply chains in the following way: 

• In systems like supply chain, it is difficult to define analogs of various thermodynamic 

quantities like energy, temperature, pressure etc as we can do for physical systems. Each 

component in the network has a cognition, which is absent in physical systems; say a 

molecule of a gas. Due to such difficulties statistical mechanics cannot be applied directly 

to build prediction models for supply chains. As discussed previously by not requiring a 

Hamiltonian (the energy like function), computational mechanics is still applicable in 

case of supply chains. 

• ε -machines can be built to discover patterns in behavior of various quantities in supply 

chains like the inventory levels, demand fluctuations, etc. 

• ε -machines can be used for prediction through a process known as “synchronization” 

(Crutchfield and Feldman 2003). 

• ε -machines can be used to calculate various global properties like entropy rate, excess 

entropy and statistical complexity, that reflect how the system stores and processes 

information. The significance of these quantities has been discussed earlier. 

• We can also quantify notions of Complexity, Emergence and Self-Organization in terms 

of various information measures derived from ε -machines. By evaluating such quantities 

we can compare complexity of different supply chains and quantify the extent to which 

the network is showing emergence. We can also infer when a supply chain is undergoing 

self-organization and to what extent. Such quantification can help us to compare 

precisely what policies or cognitive capabilities possessed by individual agents can lead 

to different degrees of emergence and self-organization. Hence we can decide to what 

extent we desire to enforce the control and to what extent we want to let the network 

emerge. 

7. Network Dynamics 

The ubiquity of networks in the social, biological and physical sciences and in technology leads 

naturally to an important set of common problems, which are being currently studied under the 

rubric of “Network Dynamics” (Strogatz 2001). Structure always affects function and it is 

important to consider dynamical and structural complexity together in the study of networks. For 

instance, the topology of social networks affects the spread of information and disease, and the 

topology of the power grid affects the robustness and stability of power transmission. The 

different problem areas in network dynamics are discussed below. 

One area of research in this field has been primarily concerned with the dynamical complexity 

in regular networks without regard to other network topologies. While the collective behavior 

depends on the details of the network, some generalization can still be drawn (Strogatz 2001). For 

instance, if the dynamical system at each node has stable fixed points and no other attractor, the 

network tends to lock into a static fixed pattern. If the nodes have competing interactions, 

network may become frustrated and display enormous number of locally stable equilibria. In the 

intermediate case where each node has a stable limit cycle, synchronization and patterns like 

traveling waves can be observed. For non-identical oscillators temporal analogue of phase 

transition can be seen with the control parameter as the coupling coefficient. At the opposite 

extreme if each node has identical chaotic attractor, the network can synchronize their erratic 

fluctuations. For a wide range of network topologies, synchronized chaos requires that the 

coupling be neither too weak nor too strong; otherwise spatial instabilities are triggered. Related

line of research that deals with networks of identical chaotic map is coupled map lattices 

(Kaneko and Tsuda 1996) and cellular automata (Wolfram 1994). However these systems have 

been used mainly as test-beds for exploring spatio-temporal chaos and pattern formation in the 

simplest mathematical settings, rather than as models of real systems. 

The second area in network dynamics is concerned about characterizing the network structure. 

Network structure or topologies in general can vary from completely regular like chains, grids, 

lattices and fully connected to completely random. Moreover the graphs can be directed or 

undirected and cyclic or acyclic. In order to characterize topological properties of the graphs, 

various statistical quantities have been defined. Most important of them include average path 

length, clustering coefficient, degree distributions, size of giant component and various spectral 

properties. A review of the main models and analytical tools, covering regular graphs, random 

graphs, generalized random graphs, small-world and scale-free networks, as well as the interplay 

between topology and the network's robustness against failures and attacks can be found in 

(Albert and Barabasi 2002, Dorogovtsev and Mendes 2002). 

The classic random graphs were introduced by Erdos and Renyi (Bollobas 1985) and have been 

the most thoroughly studied models of networks. Such graphs have Poisson degree distribution 

and statistically uncorrelated vertices. At large N (total number of nodes in the graph) and large 

enough p (probability that two arbitrary vertices are connected), a giant connected component 

appears in the network, a process known as percolation. The random graphs exhibit low average 

path length, and low clustering coefficient. The regular networks on other hand show high 

clustering coefficient and also a greater average path length compared to the random graphs of 

similar size. The networks found in real world, however are neither completely regular nor 

completely random. This has been recently discovered in the form of “small world” and “scale 

free” characteristics, for many real networks like: social networks, internet, WWW, power grids, 

collaboration networks, ecological and metabolic networks to name a few. 

In order to describe the transition from a regular network to a random network, Watts and 

Strogatz introduced the so-called small-world graphs as models of social networks (Watts and 

Strogatz 1998) and (Newman 2000). This model exhibits a high degree of clustering as in the 

regular network and a small average distance between vertices as in the classic random graphs. A 

common feature of this model with random graph model is that the connectivity distribution of 

the network peaks at an average value and decays exponentially. Such an exponential network is 

homogeneous in nature: each node has roughly the same number of connections. Due to high 

degree of clustering the models of dynamical systems with small-world coupling display 

enhanced signal-propagation speed, rapid disease propagation, and synchronizability (Watts and 

Strogatz 1998). 

Another significant recent discovery in the field of complex networks is the observation that 

the connectivity distributions of a number of large-scale and complex networks, including the 

−γ 

WWW, Internet, and metabolic network, have the power law form P ( k) 

≈ k , where P(k) 

is 

the probability that a node in the network is connected to k other nodes, and γ is a positive real 

number (Barabasi et al. 2000, Barabasi 2001). Since power-laws are free of characteristic scale, 

such networks are called “scale-free network”. A scale-free network is inhomogeneous in nature: 

most nodes have few connections but small number (but statistically significant) have many 

connections. The average path length is smaller in the scale free network than in a random graph, 

indicating that the heterogeneous scale-free topology is more efficient in bringing the nodes 

closer than homogenous topology of the random graphs. The clustering coefficient of the scalefree 

network is about 5 times higher than that of the random graph, and this factor slowly 

increases with the number of nodes. It has been shown that it is practically impossible to achieve 

synchronization in a nearest-neighbor coupled network (regular connectivity) if the network is 

sufficiently large. However, it is quite easy to achieve synchronization in a scale-free dynamical 

network no matter how large the network is (Weng and Chen, 2002). Moreover, the

synchronizability of a scale-free dynamical network is robust against random removal of nodes, 

but is fragile to specific removal of the most highly connected nodes. 

The scale free property and high degree of clustering (the small world effect) however are not 

exclusive for a large number of real networks. Yet most models proposed to describe the topology 

of complex networks have the difficulty capturing simultaneously these two features. It has been 

shown in (Ravasz and Barabasi, 2003) that these two features are the consequence of a 

hierarchical organization present in the networks. This argument also agrees with that proposed 

by Herbert Simon (Simon 1997) who argues: “we could expect complex systems to be hierarchies 

in a world in which complexity has to evolve from simplicity. In their dynamics, hierarchies have 

a property, near decomposability, that greatly simplifies their behavior. Near decomposability 

also simplifies the description of complex systems and makes it easier to understand how the 

information needed for the development of the system can be stored in reasonable compass”. 

Indeed many networks are fundamentally modular: one can easily identify groups of nodes that 

are highly interconnected with each other, but have only a few or no links to nodes outside of the 

group to which they belong. This clearly identifiable modular organization is at the origin of high 

degree of clustering coefficient. On the other hand these modules can be organized in a 

hierarchical fashion into increasingly large groups, giving rise to “hierarchical networks”, while 

still maintaining the scale-free topology. Thus modularity, scale-free character and high degree of 

clustering can be achieved under a common roof. Moreover, in hierarchical networks the degree 

of clustering characterizing the different groups follows a strict scaling law, which can be used to 

identify the presence of hierarchical structure in real networks. 

The mathematical theory of graphs with arbitrary degree distributions known as “generalized 

random graphs” can be found in (Newman et al. 2001) and (Newman 2003). Using the 

“generating function formulation”, the authors have been able to solve the percolation problem 

(i.e. have found conditions for predicting the appearance of a giant component), have obtained 

formulae for calculating clustering coefficient and average path length for generalized random 

graphs. The authors have proposed and studied models of propagation of diseases, failures, fads 

and synchronization on such graphs and have extended their results for bipartite and directed 

graphs. 

Network dynamics though in its infancy promises a formal framework to characterize the 

organizational and functional aspects in supply chains. With the changing trends in supply chains, 

many new issues have become critical like: organizational resistance to change, inter-functional 

or inter-organizational conflicts, relationship management, and consumer and market behavior. 

Such problems are ill structured and behavioral and cannot be commonly addressed by analytical 

tools such as mathematical programming. Successful supply chain integration depends on the 

supply chain partners’ ability to synchronize and share real-time information. The establishment 

of collaborative relationship among supply chain partners is a pre-requisite to information 

sharing. As a result successful supply chain management relies on systematically studying 

questions like 1) what are the robust architectures for collaboration and what are the coordination 

strategies that lead to such architectures, 2) if different entities make decisions on whether or not 

to cooperate on the basis of imperfect information about the group activity, and incorporate 

expectations on how their decision will affect other entities, can overall cooperation be sustained 

for long periods of time 3) how do the expectations, group size, and diversity affect coordination 

and cooperation and 4) which kinds of organizations are most able to sustain ongoing collective 

action, and how might such organizations evolve over time. Network dynamics addresses many 

of such questions and should be explored in context of supply chains. 

8. Conclusions and Future Work 

The idea of managing the whole supply chain and transform them into a highly autonomous, 

dynamic, agile, adaptive and reconfigurable network certainly provides an appealing vision for 

managers. The infrastructure provided by Information technology has made this vision partially

ealizable. But the inherent complexity of supply chains makes the efficient utilization of 

information technology an elusive endeavor. Tackling this complexity has been beyond the 

existing tools and techniques and requires revival and extensions. 

As a result we emphasized in this paper, that in order to effectively understand a supply chain 

network, it should be treated as a CAS. We laid down some initial ideas for the extension of 

modeling and analysis of supply chains using the concepts, tools and techniques arising in the 

study of CAS. As a future work we need to verify the feasibility and usefulness of the proposed 

techniques in the context of large scale supply chains. 


The authors wish to acknowledge DARPA (Grant#: MDA972-1-1-0038 under UltraLog Program) 

for their generous support for this research. In additions the partial support provided by NSF 

(Grant#:DMII-0075584) for Professor Kumara is greatly appreciated. 

References 

Abarbanel, H.D.I, 1996, The Analysis of Observed Chaotic Data, Springer-Verlag, New York. 

Abarbanel, H. D. I. and Kennel, M. B., 1993, Local False Nearest Neighbors and Dynamical 

Dimensions from Observed Chaotic Data, Phys. Rev. E, 47, 3057-3068. 

Adami, C., 1998, Introduction to Artificial Life, Springer-Verlag. 

Albert, R. and Barabasi, A. L., 2002, Statistical Mechanics of Complex Networks, Reviews of 

Modern Physics, 74, 47. 

Albert, R., Barabási, A. L., Jeong, H. and Bianconi, G., 2000, Power-law distribution of the 

World Wide Web, Science, 287, 2115. 

Albert R., Jeong, H., Barabasi, A. L.,2000, Error and attack tolerance of complex networks, 

Nature, 406, 378-382. 

Balakrishnan, A., Kumara, S. and Sundaresan, S., 1999, Exploiting Information Technologies for 

Product Realization, Information Systems Frontiers, A Journal of Research and Innovation, 

1(1), 25-50. 

Barabasi, A.L., July 2000, The Physics of Web, Physics Web. 

Barabasi, A. L., Albert, R., and Jeong, H., 2000, Scale-free characteristics of random networks: 

The topology of the World Wide Web, Physica A, 281, 69-77. 

Baranger, M., Chaos, Complexity, and Entropy: A physics talk for non-physicists, 

http://necsi.org/projects/baranger/cce.pdf. 

Bar-Yam, Y., 1997, Dynamics of complex systems, Reading, Mass, Addison-Wesley. 

Bollobas, B., 1985, Random Graphs, Academic Press, London. 

Callaway, D. S., Newman, M. E. J., Strogatz, S. H. and Watts, D. J., 2000, Network robustness 

and fragility: Percolation on random graphs, Phys. Rev. Lett. 85, 5468-5471. 

Carlson, J. M., Doyle, J., 1999, Highly optimized tolerance: a mechanism for power laws in 

designed systems, Physics Review E, 60(2), 1412-1427. 

Casdalgi, M., 1989, Nonlinear prediction of chaotic time series, Physica D, 35, 335-356. 

Choi, T. Y., Dooley, K. J., Ruangtusanathan, M., 2001, Supply networks and complex adaptive 

systems: control versus emergence, Journal of Operations Management 19(3), 351-366. 

Cooper, M. C., Lambert, D. M., and Pagh, J. D., 1997, Supply chain management: More than a 

new name for logistics, The International Journal of Logistics Management, 8(1), 1-13. 

Crutchfield, J. P., 1992, Knowledge and Meaning … Chaos and Complexity, in Modeling 

Complex Systems, L. Lam and H. C. Morris, editors, Springer-Verlag, Berlin, 66 -101. 

Crutchfield, J. P., 1994, The Calculi of Emergence: Computation, Dynamics and Induction, 

Physica D, 75, 11-54. 

Crutchfield, J. P. and Young, K., 1989, Inferring Statistical Complexity, Physical Review Letters, 

63, 105-108.

Crutchfield, J. P. and Feldman, D. P., 2001, Synchronizing to the Environment: Information 

Theoretic Constraints on Agent Learning, Advances in Complex Systems, 4, 251-264. 

Crutchfield, J. P. and Feldman, D. P., 2003, Regularities Unseen, Randomness Observed: Levels 

of Entropy Convergence, Chaos (submitted). 

Csete, M. E. and Doyle, J., 2002, Reverse Engineering of Biological Complexity, Science, 295, 

1664. 

Dorogovtsev, S. N. and Mendes, J. F. F., 2002, Evolution of networks, Advances in Physics, 51, 

1079-1187. 

Erramilli, A. and Forys, L. J., 1991, Oscillations and Chaos in a Flow Model of a Switching 

System, IEEE Journal on selected areas in communications, 9(2), 171-178. 

Farmer, J. D., Ott, E. and Yorke, J. A., 1983, The dimension of chaotic attractors, Physica D, 7, 

153-180. 

Farmer, J. D. and Sidorowich, J. J., 1987, Predicting chaotic time-series, Physics Review Letters, 

59(8), 845-848. 

Feichtinger, G., Hommes, C. H. and Herold, W., 1994, Chaos in a simple Deterministic Queuing 

System, ZOR- Mathematical Methods of Operations Research, 40, 109-119. 

Feldman, D. P. and Crutchfield, J. P., Discovering Noncritical Organization: Statistical 

Mechanical, Information Theoretic, and Computational Views of Patterns in One-Dimensional 

Spin Systems, Santa Fe Institute Working Paper 98-04-026. 

Flake, G. W., 1998, The Computational Beauty of Nature, MIT Press. 

Forrester, J. W., 1961, Industrial Dynamics. Cambridge: MIT press. 

Fraser, A. M. and Swinney, H. L., 1983, Independent coordinates for strange attractors from 

mutual information, Phys. Rev. A, 33(2), 1134-1140. 

Ghosh, S., 2002, The role of Modleing and Asynchronous Distributed Simulation in Analyzing 

Complex systems of the Future, Information System Frontiers, A Journal of Research and 

Innovation, 4(2), 166-171. 

Glance, N. S., 1993, Dynamics with Expectations, PhD Thesis Physics Department Stanford 

University. 

Hogg, T. and Huberman, B. A., 1988, The Behavior of Computational Ecologies, in The Ecology 

of Computation North-Holland, 77-116. 

Hogg, T. and Huberman, B. A., 1991, Controlling Chaos in Distributed Systems, IEEE Trans. on 

Systems, Man and Cybernetics, 21, 1325-1332. 

Kaneko, K. and Tsuda, I., 1996, Complex Systems: Chaos and Beyond, Springer-Verlag. 

Kennel, M., Brown, R. and Abarbanel, H. D. I., 1992, Determining embedding dimension for 

phase-space reconstruction using a geometrical construction, Phys. Rev. A, 45(6), 3403-3068. 

Kephart, J. O., Hogg, T. and Huberman, B. A., 1989, Dynamics of Computational Ecosystems, 

Physical Review A, 40 (1), 404-421. 

Kephart, J. O., Hogg, T. and Huberman, BA, 1990, Collective Behavior of Predictive Agents, 

Physica D, 42, 48-65. 

Kumara, S., Ranjan, P., Surana, A. and Narayanan, V., Decision Making in Logistics: A Chaos 

Theory Based Analysis, Annals of the International Institution for Production Engineering 

Research (Annals of CIRP) (accepted to appear). 

Lee, S., Gautam, N., Kumara, S., Hong, Y., Gupta, H., Surana, A., Narayanan, V., Thadakamalla, 

H., Brinn, M. and Greaves, M., 2002, Situation Identification Using Dynamic Parameters in 

Complex Agent-Based Planning Systems, Intelligent Engineering Systems Through Artificial 

Neural Networks, 12, 555-560. 

Llyod, S. and Slotine, J. J. E., 1996, Information theoretic tools for stable adaptation and learning, 

Int. Journal of Adaptive Control and Signal Processing, 10, 499-530. 

Maxion, R. A., Toward Diagnosis as an Emergent Behavior in a Network Ecosystem, Physica D, 

42, 66-84.

Min, H. and Zhou, G., 2002, Supply chain modeling: past, present and future, Computers and 

Industrial Engineering, 43, 231-249. 

Mukherjee, S., Osuna, E. and Girosi, F., 1997, Nonlinear Prediction of Chaotic Time Series 

Using Support Vector Machines, IEEE Workshop on Neural Networks for Signal Processing 

VII, 511-519. 

Newman, M. E. J., 2000, Models of the small world, J. Stat. Phys., 101, 819-841. 

Newman, M. E. J., 2002, The spread of epidemic disease on networks, Phys. Rev. E, 66, 

Newman, M. E. J., 2003, Random graphs as models of networks, in Handbook of Graphs and 

Networks, S. Bornholdt and H. G. Schuster (eds.), Wiley-VCH, Berlin. 

Newman, M. E. J., Strogatz, S. H. and Watts, D. J., 2001, Random graphs with arbitrary degree 

distribution and their applications, Phys. Rev. E, 64. 

Ott, E., 1996, Chaos in Dynamical Systems, Cambridge University Press. 

Powell, M. J. D., 1987, Radial basis function approximation to polynomials, preprint University 

of Cambridge. 

Rasmussen, D. R. and Moseklide, M., 1988, Bifurcations and chaos in generic management 

model, European Journal of Operations Research, 35, 80-88. 

Ravasz, E. and Barabasi A. L., 2003, Hierarchical organization in complex networks, Physical 

Review E, 67. 

Sano, M. and Sawada, Y., Measurement of the Lyapunov Spectrum form a Chaotic Time Series, 

1985, Phys. Rev. Lett., 55, 1082-1084. 

Sawhill, B. K., 1993, Self-Organized Criticality and Complexity Theory, Lectures in Complex 

Systems, edited by Nadel L. and Stein DL, Addison Wesley Longman, 143-170. 

Schieritz, N. and Grobler, A., 2003, Emergent Structures in Supply Chains- A study Integrating 

Agent-Based and System Dynamics Modeling, Paper presented at the 36th Annual Hawaii 

Internation Conference on System Sciences, Big Island. 

Shalizi, C. R. and Crutchfield, J. P., Computational Mechanics: Pattern and Prediction, Structure 

and Simplicity, SFI Working Paper 99-07-044. 

Shalizi, C. R., 2001, Causal Architecture, Complexity and Self-Organization in Time Series and 

Cellular Automata, http://www.santafe.edu/~shalizi/thesis. 

Simon, H. A., 1997, The Sciences of the Artificial, Cambridge, MA: The MIT Press, 3rd Edition. 

Strogatz, S. H., 1994, Nonlinear Dynamics and Chaos, Addison-Wesley, Reading, MA. 

Strogatz, S. H., 2001, Exploring complex networks, Nature, 410, 268-276. 

Takens, F., 1981, in Dynamical Systems and Turbulence, Warwick, 1980, edited by D. Rand and 

L.S. Young, Lecture Notes in Mathematical No. 898 (Springer, Berlin), 366. 

Wang, X. F. and Chen, G., 2002, Synchronization in Scale-free Dynamical networks: Robustness 

and Fragility, IEEE Transactions on Circuits and Systems I-Fundamental Theory And 

Applications, 49(1), 54-62. 

Watts, D. J. and Strogatz, S. H., 1998, Collective dynamics of ‘small-world’ networks, Nature, 

393, 440-442. 

Wolfram, S., 1994, Cellular Automata and Complexity: Collected Papers, Reading, Mass: 

Addison-Wesley Pub. Co.

Decision Making in Logistics: A Chaos Theory Based Analysis 

S. R. T. Kumara 1 , P. Ranjan, A. Surana , V. Narayanan 


310 Leonhard Building, University Park, PA 16802 

Abstract 

Logistics in general is a complex system. In this paper we investigate the existence of chaos in logistics 

systems. Such an investigation is necessary to use appropriate and correct methods for further analysis, 

as linear systems techniques will not be useful. If a system exhibits chaos, decision-making should 

consider the system characterization parameters from a chaos theory perspective. In this paper, we 

consider a non-preemptive queuing model and its extensions to the logistics domain. A prototypical supply 

chain example is used and the resulting behavior is characterized. At certain input values the behavior of 

the logistics system exhibits chaos. This information is useful for further analysis for prediction and control. 

The working prototype is implemented in the DARPA Couggar agent architecture. 

Keywords: 

Non-linear Dynamics, Production, Distributed 

1 INTRODUCTION 

In a logistics system one of the most fundamental 

questions is the analysis of the system behavior. We 

define the system as the entities (software and hardware) 

along with their interconnections (network). A typical 

logistics system is characterized by a supply chain. Our 

hypothesis is that these systems are nonlinear, dynamic 

and in specific, chaotic. That is, the time evolution of the 

system behavior (measured by certain behavioral 

parameters of the system) is chaotic. The question now is 

how do we really characterize the time evolution and how 

can we use the insights obtained from such an analysis? 

This paper deals with these questions. We first give a brief 

explanation of the notion of nonlinear dynamics and 

continue further discussion. 

2 NONLINEAR DYNAMICS, CHAOS AND FRACTALS 

In this section, we present a concise description of 

nonlinear dynamics, chaos and fractals. During the past 

decade, chaos theory has elicited a lot of interest among 

scientists and researchers. As a result, its ideas are 

beginning to be applied to many scientific and engineering 

disciplines, especially where nonlinear models are relevant 

[1]. 

Many physical systems that produce continuous-time 

response may be modeled by a set of differential 

equations of the form 

_ 

_ 

d x( t) 

F( 

x( 

t)) 

dt 

() ⋅ 

= (1) 

F is generally a nonlinear vector field. The solution to 

this results in a trajectory 

x () t = f ( x() 

0 , t) 

(2) 

where f : M → M represents the flow that determines 

the evolution of x(t) for a particular initial condition x(0). 

If the system is dissipative, as the system evolves from 

different initial conditions, the solutions usually shrink 

asymptotically to a compact subset of the whole state 

space M. This compact subset is called an attracting set. 

Every attracting disjoint subset of an attracting set is called 

an attractor [1]. 

In dissipative systems, the overall volume of the state 

space shrinks with time. However, there may be some 

directions along which the state space actually expands. 

That is, the system trajectories tend to move apart along 

certain directions and shrink along the others. However, as 

the attractors usually remain bounded, the flow exhibits a 

horseshoe-type pattern [2]. Because of this, trajectories 

starting from near-by points within an attractor may get 

separated exponentially as the system evolves. This 

condition is known as the sensitive dependence on initial 

condition (SDIC), and the attractor exhibiting SDIC is 

called a strange attractor. 

A flow f, for a particular initial condition, is said to be 

chaotic if the trajectories in an attractor exhibit: sensitive 

dependence on initial conditions, but are bounded, 

irregular and aperiodic behavior, and continuous broad 

band spectrum. 

The irregular and aperiodic response of chaotic systems, 

usually betrays a special property of self-similarity or scale 

invariance. That is, the response appears similar over 

multiple scales of observation. Scale invariant 

mathematical entities are commonly known as fractals. 

Analytical techniques used to deduce the characteristics of 

nonlinear systems collectively constitute fractal analysis. 

The main objectives of fractal analysis can be broadly 

categorized depending on the end-purpose as follows: 

identification of the presence of chaos from the system 

response, establishing the invariants of the system 

dynamics for system identification or indirect state 

estimation, and chaos modeling, when the end-purpose is 

to capture and later reproduce the system dynamics. A 

more detailed description of these concepts may be found 

in [3]. For applications of Nonlinear Dynamics in the 

modeling and control of complex production systems, refer 

to [4][ 5].

In the rest of this paper we report a queuing model that is 

useful in supply chain analysis. We explain our rationale 

for selecting this model for adaptation to the logistics 

domain. For some of the other models existing in literature, 

refer to [6][7]. We extend the queuing model and apply it to 

the logistics scenario in the Cougaar architecture and 

discuss the results. We raise the fundamental question of 

how we can use these results for further analysis and 

control of a logistics system. 

3 SUPPLY CHAIN AND NONLINEARITY 

The notion of evolution over time falls into the realm of 

what physicists call dynamics. Logistics systems are 

dynamic. Their behavior can be nonlinear. Therefore we 

can model a logistics system using the principles of 

nonlinear dynamics. A supply chain is an example of a 

logistics system. A typical supply chain exhibits stable 

behavior with damped oscillations in response to external 

disturbances. Unstable phenomena however can arise, 

due to feedback structure, inherent adjustment delays [6], 

nonlinear decision-making [7] and interactions that go in a 

supply chain. One of the causes of unstable phenomena is 

that the information feedback in the system is slow relative 

to rate of changes that occur in the system. Nonlinearity is 

inherent in a supply chain. The first mode of unstable 

behavior to arise in nonlinear systems is usually the simple 

one-cycle self-sustained oscillations. If the instability drives 

the system further into the nonlinear regime, more 

complicated temporal behavior may be generated. The 

route to chaos through subsequent period-doubling 

bifurcations, as certain parameters of the system are 

varied, is generic to large class of systems in physics, 

chemistry, biology, economics and other fields. 

Functioning in chaotic regime deprives us the ability for 

long-term predictions about the behavior of the system, 

while short-term predictions may be possible sometimes. 

As a result, control and stabilization of such a system 

becomes almost impossible. Here we investigate such 

dynamical behaviors that can arise in models that 

represent some of the components in a supply chain. 

3.1 Preemptive Queuing Model with Delays 

The Queuing system [8] considered here has two queues 

(A and B) and a single server with following characteristics: 

•Once served, the class A customer returns as a class B 

customer after a constant interval of time 

•Class B has non-preemptive priority over class A, i.e., the 

class A queue does not get served until the class B queue 

is emptied. 

•Schedules are organized every T units of time, i.e., if the 

low priority queue is emptied within time T, the server 

remains idle for the remaining time interval. 

•Finally, the higher priority class B has a lower service rate 

than the low priority class A 

Suppose the system is sampled at the end of every 

schedule cycle, and the following quantities are observed 

at the beginning of the kth interval: Ak 

the queue length of 

low priority queue, Bk 

the queue length of high priority 

queue, C 

k the outflow from low priority queue in the kth 

interval and Dk 

the outflow from high priority queue in the 

kth interval. In the model, λ 

k denotes the arrival rate, µ 

a 

is the service rate for the lower priority queue, µ 

b is the 

service rate for the higher priority queue and l the 

feedback interval in units of the schedule cycle. 

The following four equations then completely describe the 

evolution of the system: 

Ak 

+ 1 

= Ak 

+ λk 

− Ck 

(3) 

C 

B 

k 

min( A 

k 

Dk 

+ λk 

, µ 

a 

(1 − )) 

µ 

= (4) 

k +1 

= Bk 

+ Ck 

−l 

− Dk 

(5) 

D min( B + C , µ ) 

k 

= 

k k −l 

b 

(6) 

Equations (3) and (5) are merely conservation rules, while 

equations (4) and (6) model the constraints on the outflows 

and the interaction between the queues. This model while 

conceptually simple, exhibits surprisingly complex 

behaviors. The dynamical behavior reported in [8] is 

summarized in the following. 

Figure 1: Non-preemptive Queuing Model 

Dynamical Behavior: The analytic approach to solve for 

the flow model under constant arrivals (i.e. 

b 

λ = 

k 

λ 

for all 

k) shows several classes of solutions. The system is found 

to batch its workload even for such perfectly smooth arrival 

patterns. Following are the characteristics of behavior of 

the system: 

•Above a threshold arrival rate ( λ ≥ µ b 

/ 2 ), a 

momentary overload can send the system into a number of 

stable modes of oscillations. 

•Each mode of oscillations is characterized by distinct 

average queuing delays. 

•Extreme sensitivity to parameters, and the existence of 

chaos, implies that the system at a given time may be in 

any one of a number of distinct steady-state modes. 

The batching of the workload can cause significant 

queuing delays even at moderate occupancies. Also such 

oscillatory behavior significantly lowers the real-time 

capacity of the system. 

4 APPLICATION OF QUEUING MODEL TO 

LOGISTICS SCENARIO 

The assumptions in the model proposed by [8] are generic 

in the sense that priorities are widely observed in large 

systems due to economic and administrative compulsions. 

Sometime they can also arise from the physical facts when 

two different stages of processing have certain temporal 

constraint. Priorities may also arise due to the nonhomogeneity 

of the system where “knowledge” level of one 

agent is different from the other. 

Varying service time again follows from physical 

constraints on the task. For example in a simple logistics 

scenario tasks like unpacking, shipping, logging and 

dispatching may take different times. These times scales 

can vary widely depending on the nature and physical 

characteristics of the tasks. 

The considerations regarding the generality of 

assumptions and the clear one-to-one correspondence

etween the physical logistics tasks and the model 

parameters described in [8] made us apply the queuing 

model to a simple, yet, realistic logistics scenario. 

4.1 Example Logistics Scenario 

The example scenario consists of two stages modeled by 

the non-preemptive queuing formalism. We take a simple 

battle front scenario (this can be any context of supply of 

materials, not necessarily battle front). During the first 

stage, supplies are processed by the node (agent) This 

involves two tasks: Unpacking (Task A) and Shipping 

(Task B). Our assumptions are that shipping takes more 

resources than packing, shipping gets a non preemptive 

priority and resources are common to both the tasks 

The second stage consists of disbursement of supplies. 

The output of first stage feeds into the second stage (as 

arrival). The two associated tasks are: Maintaining an 

inventory (Task A) and Disbursing the supply to the troops 

(Task B). The assumptions at stage two are that 

disbursing takes more resources than maintaining 

inventory, disbursing has a non pre-emptive priority and 

resources are common to both the tasks. 

Figure 1 shows the queuing model. This is figure is 

reproduced from [8]. It must be noted that that rules are 

very simple and generic. Priority and heterogeneity are 

fundamental to any logistic planning and scheduling. 

Tasks have to be prioritized in order to do the most 

important thing first. This comes naturally as we try to 

optimize an objective and assign the tasks their 

"importance.” In addition in all logistics systems, resources 

are limited, both in time and space. Temporal constraints 

considered in the example are realistic, in the sense that 

you cannot disburse supplies without unpacking them. 

Temporal dependence plays an important role in logistic 

planning (interdependency). This simple example also 

simulates the effect of arbitrary but bounded initial 

conditions 

Cougaar (Cognitive Agent Architecture) is developed 

under DARPA Advanced Logistics Program (ALP). 

Survivability of Cougaar is addressed in the UltraLog 

program of DARPA. In the above example each stage is 

modeled as an agent. The activities are modeled as agent 

processes. We do not discuss Cougaar architecture in this 

paper. Details can be found at the URL: 

http://www.couggar.org. 

4.2 Analysis 

One of the hallmarks of chaos is sensitive dependency to 

initial conditions (SDIC). External environment (the world 

in which the logistics scenario resides) changes and hence 

changing the initial conditions and the parameters. The 

following affects the initial conditions and parameters of 

the agents ( thereby affecting the initial conditions of the 

queuing model): change in arrival rate of supplies (inputs 

to the agents), change in resources (assets) available in 

each agent, and delay in processing of Tasks. 

The internal states of the two agents are characterized by: 

supplies waiting to be shipped (X 1 ), supplies waiting to be 

unpacked (X 2 ), supplies actually shipped, supplies waiting 

to be inventoried (X 3 ), supplies waiting to be disbursed (X 4 ) 

to the troops and supplies actually shipped. We have 

considered these variables and observed their behavior. 

Characterization of these behaviors leads to some 

interesting inferences. 

We simulated the queuing models in each agent with the 

following model parameters. There are 162 personnel in 

each of the agents, who can be allocated to either task. 

We assume that it takes 1 unit of time and one person to 

do task A and one unit of time with 2 people to do task B. 

This defines the capacity/arrival rate as 54 items/unit time. 

Hence arrival rate can be 0-54 per unit time. We assume 

that the initial conditions are given by: X10=131, X20=201, 

X30=151 and X40=29. 

S tate X 1andX2-> 

M agnitude(dB) 

P o w er S pectru m 

120 

100 

80 

60 

40 

20 

50 

40 

30 

20 

10 

0 

-10 

-20 

-30 

Evolution of system states 

50 55 60 65 70 75 80 85 90 95 100 

Time-> 

(a): Time evolution of system state 

Power Spectrum of state X1 

-40 

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 

Frequency 

50 

(b) Power spectrum 

State space trajectories for center 2(blue) and for center 1(red) 

150 

100 

2 ,x4-> x 

0 

0 50 100 150 

x1,x3-> 

(c) State-space plot 

d) Multi-stability:Bifurcation diagram for the system 

Figure 2: Plots for arrival rate =53 

We have used Matlab for computations. We have 

experimented with several arrival rates and delays. We 

observe the state-space structure (time evolution) of the 

following: arrival rates at all the queues, time series of 

various parameters and the Power-spectrum. 

At arrival rates of 40 the system has a period of 1, at 

arrival rate of 50 a period 2, at 52 a period of 4 and at 53 

the system shows a seemingly random behavior. This

shows relatively irregular behavior with several different 

peaks in the power spectrum. The bifurcation diagram 

shows that at the arrival rate of 53 the system is chaotic. 

We show illustrative plots in figure 2. Time evolution (2a) 

clearly shows the existence of many periods showing the 

possible existence of chaotic behavior. 

4.3 Discussion 

We could successfully show with certain initial conditions 

the existence of chaos in the simple yet realistic logistic 

system. The underlying queuing model at arrival rates of 

53 leads to the chaotic behavior of the number of jobs 

waiting to be processed. The bifurcation diagram points to 

the fact that X j ’s (for j=1,2,3,4) exhibit aperioidic behavior. 

The physical implication is that the resources needed vary 

from time to time and the logistics system will exhibit 

nervousness, which is an undesirable property. We have 

also observed a cascading effect (when one agent enters 

the chaotic behavior, the connected agent also tends to 

exhibit chaos). This leads to the problem of planning of 

later stages facing much more uncertainty compared to 

the first stage even for simple fixed deterministic arrivals. 

We have also observed increased average delay. There is 

an increase in delay by 25% if the system starts batching 

the load From our analysis we can conclude that if the two 

agents start load batching then inventory requirement may 

go to 200% as evident from the plots. 

It is necessary to make sure in this case to keep the arrival 

rates to less than 53, there by enforcing control policies 

which will keep the system stable or quasi-stable. If the 

system ends up being chaotic then we could perform 

further analysis to study the characteristics and use them 

to control the behavior in the short term. We also compute 

: Average mutual information, Global dimension, Local 

dimension, Correlation dimension and Largest Lyapunov 

Exponent.These computed values also indicate the 

existence of Chaos in this logistics system. 

5 SUMMARY 

Chaotic behavior in deterministic dynamical systems is an 

intrinsically non-linear phenomenon. We could successfully 

show that a simple example logistics system is chaotic. A 

characteristic feature of a chaotic systems is an extreme 

sensitivity to changes in initial conditions while the 

dynamics, at least for the so-called dissipative systems, is 

still constrained to a finite region of the state space called 

an attractor. In such instances, Fourier analysis and 

ARMA models may not be useful to study the time traces 

of supply chain systems. The need to extract interesting 

physical information about the dynamics of observed 

systems when they are operating in a chaotic regime has 

led to development of nonlinear time series analysis 

techniques. Systematically, the study of potentially, chaotic 

systems may be divided into three areas: identification of 

chaotic behavior, modeling and prediction and control. The 

first area shows how chaotic systems may be separated 

form stochastic ones and, at the same time, provides 

estimates of the degrees of freedom and the complexity of 

the underlying chaotic system. Based on such results, 

identification of a state space representation allowing for 

subsequent predictions may be carried out. The last stage, 

if desirable, involves control of a chaotic system. In this 

short paper we have concentrated on the first area i.e. 

identification of chaotic behavior. In general if we consider 

this step in spatio-temporal regime, the following tasks are 

needed to be accomplished [9]: 

1.Signal Separation (Finding the signal): Separation of 

broadband signal form broadband “noise” using 

deterministic nature of signal. 

2.Phase Space reconstruction (Finding the space): Time 

lagged variables are used to form coordinates for a phase 

space in the embedding dimension. The embedding 

dimension can be determined using false nearest 

neighbors test and time lag using mutual information. 

3.Classification of the signal: Determination of invariants of 

system such as Lyapunov exponents and various fractal 

dimensions. 

4.Making models and Prediction: Determination of the 

parameters of the assumed model, which are consistent 

with the invariant classifiers (like Lyapunov exponents and 

dimensions). 

In this paper the non-preemptive queuing model is used 

for detailed application to a part of a supply chain, two 

agents interacting in a military logistics scenario. The 

queuing model forms the processing component of the 

logistics agents implemented in the Cougaar architecture. 

One of the manifestations of complexity is through the 

onset of chaos. Our analysis shows the cascading effect of 

chaos. This points to the conjecture that the supply chain 

may exhibit chaotic behavior. The underlying motivation of 

our study is to build control models. Our next step in this 

research is to build adaptive predictive and control models 

for larger supply chain networks from the insights we have 

derived from the current analysis. 

6 ACKNOWLEDGMENTS 

The authors acknowledge DARPA for its support (Grant#: 

MDA 972-01-1-0563) under the UltraLog program for this 

research. The help of Seokcheon Lee, Yunho Hong and 

Hariprasad, T. is greatly appreciated. 

7 REFERENCES 

[1] Isham, V., 1993, Statistical Aspects of chaos: A 

review, Networks and Chaos - Statistical and 

Probabilistic Aspects, Barndorff-Nielsen et al. 

(editors). 

[2] Wiggins, S., 1990, Introduction to Applied Nonlinear 

Dynamical Systems and Chaos, Springer-Verlag, New 

York, Inc. 

[3] Bukkapatnam, S.T.S., Kumara, S., and Lakhtakia, 

A., 2000, Fractal estimation of flank wear in turning 

,ASME Journal of Dynamic Systems, Measurements 

and Control, 122:89-94. 

[4] Reiter, S. R., Freitag, M. and Schmieder, A., 2002, 

Modeling and Control of Production Systems Based 

on Nonlinear Dynamics Theory, Annals of the CIRP, 

51/1 :375-378. 

[5] Wiendahl, H.P. and Scheffczyk, H., 1999, Simulation 

Based Analysis of Complex Production Systems with 

methods of Nonlinear Dynamics, Annals of the CIRP, 

48/1 :357-360. 

[6] Rasmussen, R. D. and Moseklide, E., 1988, 

Bifurcations and Chaos in Generic Management 

Model, European Journal of Operations Research 

Science, 35:80-88. 

[7] Feichtinger, G., Hommes, C. H. and Herold W., 1994, 

Chaos in a simple Deterministic Queuing System, 

ZOR- Mathematical Methods of Operations Research, 

40:109-119. 

[8] Erramilli, A. and Forys, L. J., 1991. Oscillations and 

Chaos in a Flow Model of a Switching System. IEEE 

Journal on Selected Areas in Communications, Vol.9, 

No 2:171-178. 

[9] Abarbanel, H.D.I., 1996, Analysis of Observed 

Chaotic Data, Springer-Verlag, New York Inc.

DARPA ULTRALOG Final Report - Industrial and Manufacturing ...

Create successful ePaper yourself

Delete template?

Save as template?