This PDF file contains the front matter associated with SPIE Proceedings Volume 10653 including the Title Page, Copyright information, Table of Contents, Introduction, and Conference Committee listing.

Human-Agent teaming requires a fundamental understanding of humans’ interaction with information. The current dynamics of Human Information Interaction (HII) are not fully understood or formalized. The dynamics of current and increasingly, future operations will mandate seamless coupling of humans and automated capabilities. Faster decision making and asymmetric views will critically depend on the performance of these human-agent teams. There are various interactions within the HII field of study including how and why humans find, consume, and use information in order to solve problems, make decisions, and carry out other tasks. There are several parallel between HII and biological interactions; one is the concept of energy. No matter the interaction, energy is acquired and expended. We will focus on one interaction, information consumption. The parallel in biology is consumption to the cellular rate of free energy from the Laws of Thermodynamics in a system at chemical equilibrium. Gibbs Standard Free Energy (ΔG° = − RT ln K) represents the maximum amount of work obtained from a process under conditions of fixed temperature and pressure. This equation can represent the idea of level of work within HII. We mapped variables in the equation to concepts within HII, for example the equilibrium constant (K) links to the balance of information units before and after interaction task. For this research, we are developing an Agent Based Model where complex interaction can be constructed and evaluated. We are using Netlogo, an integrated environment for model development, visualization, and analysis as a tool for developing this model. In this paper, we will present details of the current implementation of our model with the Gibbs Standard Free Energy equation and initial results from the Netlogo simulations of our model.

Modern Soldiers increasingly rely on computational and autonomous systems for the completion of their missions^1,2. Challenges arise around the use of such systems that depend largely on the relationship between agents and human actors³ . Improving agents in the field requires the development of adaptive, human-aware systems that learn behaviors based on the needs of their human counterparts, acting effectively as teammates rather than tools⁴ . The development of such agent teammates is non-trivial, but recent advances in machine-learning and artificial-intelligence are promising. We identify deep reinforcement learning (RL)⁵ , multi-agent RL⁶ , and human-guided RL⁷ as powerful tools for the creation of adaptive agent teammates. We propose a three-armed approach to the development of agent teammates that leverages these advances in RL. First, multi-agent deep learning can be used to solve increasingly complex problems. Second, human-guided reinforcement can be used to constrain agent behavior and speed up the discovery of optimal strategies. Third, human behavioral profiles derived from surveys of work-interest variables for specific military occupation specialty (MOS) codes can be used to tailor agent behavior to the needs of Soldiers. This approach addresses the necessary computational framework, the learning paradigm needed to discover behavior, and the human dimension that contextualizes behavior.

The U.S. Army uses a standardized operation order (OPORD) for planning military operations. In this paper the U.S. Army Research Laboratory (ARL) considers using the OPORD as a basis for prioritizing information from the plethora of intelligence overwhelming an intelligence analyst. The OPORD would provide the input from which to calculate relevancy. To support this effort we review current approaches for calculating relevancy to improve existing information prioritization models, specifically value of information (VoI).

Many novel DoD missions, from disaster relief to cyber reconnaissance, require teams of humans and machines with diverse capabilities and intelligence. To succeed, DoD planners organize available personnel and technologies into mission-based teams and organizations. Enabled by next generation of sensors, new ways to access information, increasing capabilities of robotic platforms, and advances in machine learning and artificial intelligence for distributed inference and control applications, the new types of teams are emerging that include autonomous collaborating human and machine agents. Developing models to extract highest potential from human-machine teaming is the defense technology of the future. While many empirical studies have demonstrated the benefits of alternative organizations, such as adaptive networks command and control structures, traditional computational team design solutions have mostly focused on teams of homogeneous agents (such as swarms or social networks), and simple problems (such as cooperative task allocation, geospatial movement, and collaborative decision making). Because machines and humans often have distinct and complementary skills, team members could perform different roles and have changing relations over time. To improve team performance, new solutions are needed to dynamically adapt team structure to better fit the tasks that a team executes. In this paper, we present a continuation of our work on adaptive self-organizing teams. Our model is based on team active inference, the model that describes the approximate inference as an iterative minimization of the free variational energy encoding the task performance and team process complexity. Our model provides the methodology for adapting the structure of heterogeneous organization in distributed manner, where the agents on the team make local decisions to change their roles and relations which are synchronized through explicit collaborative messages. The roles of agents are defined through decomposition of the generalized task types into groups, and assignment of these groups to agents. We obtain decomposition groups using variational clustering on the factor graph, which defines the contribution of the tasks and their dependencies on the team’s objective function. This clustering constructs regions in the factor graph that trade-off independence, work balancing, and the overlap to help optimized organization obtain globally-optimal solutions in distributed manner under communication uncertainties.

Personalized search provides a potentially powerful tool, however, it is limited due to the large number of roles that a person has: parent, employee, consumer, etc. We present the role-relevance algorithm: a search technique that favors search results relevant to the user’s current role. The role-relevance algorithm uses three factors to score documents: (1) the number of keywords each document contains; (2) each document’s geographic relevance to the user’s role (if applicable); and (3) each document’s topical relevance to the user’s role (if applicable). Results on a pre-labeled corpus show an average improvement in search precision of approximately 20% compared to keyword search alone. We further consider several extensions to this algorithm.

Publicly Available Information (PAI), also known as Open Source Intelligence (OSINT), is an increasingly important to the work done by intelligence analysts. Because OSINT can be used to identify emerging trends, tips, and cues, it is well suited to aid analysts in generating the breadth of hypotheses needed to maintain analytic rigor.  However, managing an evolving set of (potentially) interdependent hypotheses comprised of the vast OSINT data landscape is both unwieldy and challenging.  Our research team, under the sponsorship of the Air Force Research Laboratory (AFRL), has developed the Sensemaking for OSINT eXploitation (SOX) tool to assist analysts in creating, branching, and managing OSINT-based hypotheses using a unique visual model of hypothesis and evidence management.  SOX integrates directly with web-based OSINT sources, and includes a custom suite of capabilities that analyze social network trends, patterns of life, and geospatial information to collect, filter, analyze, and aggregate OSINT intelligence.  The result is a web-based tool that helps analysts “follow the data,” manage and corroborate evidence, and collaborate with peers to reduce workload in the OSINT big-data environment.  In this paper we will describe the SOX approach to OSINT hypothesis management and human/autonomy collaboration and detail feedback gathered from USAF intelligence analysts in a series of evaluation events hosted by AFRL.

Intelligent agents are devices, software, and simulations that perceive the environment and take actions to achieve a goal through the use of artificial intelligence. These AI agents are increasingly incorporated into every aspect of our lives. This is particularly true for soldiers and analysts as they must increasingly perform tasks in varied, dynamic, and fast paced operational environments. There is a common idea that, in the future, the pace of operations will increasingly far exceed soldiers’ or analysts’ ability to react to extreme, complex activities. Accelerated decision making in Army operations will relying on AI agents and enabling technologies such as autonomous systems and simulations. However, what happens when the decisions from these AI agents are wrong, produce results contrary to expectations, or simply in disagreement with a person? Explanations can help resolve these issues. Any errors or uncertainty from the AI agent in an accelerated environment will present unique and unforeseen challenges that may potentially inhibit analysts’ or soldiers’ ability to make decisions effectively and efficiently. Providing explanations for AI outputs, predictions, or behaviors is challenging. Algorithms or techniques frequently obfuscate features and how actions are decided. In addition, results from these systems do not always include uncertainty information related to the factors that influenced the actions or decisions. Therefore, information on the uncertainty explicitly in the explanation is necessary. We explore the use of abductive reasoning to provide explanations for situations where an agents answers are not in line with human assessment nor provide uncertainty information needed for human interpretation of the answers. The primary goal of this work is to strengthen the communication of information and increase the effectiveness of interactions between humans and non-human agents.

In this paper, we report on research underpinning CISpaces.org, a tool to support the process of sense-making, complementing human expertise in the generation of intelligence products. The model combines a structured, graphical representation of the analyst’s reasoning process with efficient artificial intelligence algorithms to automatically identify plausible hypotheses. Information extracted from open sources can be exploited in the sense-making process, and analysts may collaborate to bring different perspectives to the problem concerned. The provenance of both evidence used and analyses (co-)produced are recorded and may be used for further investigation, reporting and audit. The methodology provides a rigorous means to record and support the process of forming hypotheses from the relationships among information. We use natural language processing algorithms to extract factual claims from open information sources. The core process of reasoning is made explicit in the structuring of evidence. Given this, we do not rely on the analyst exhaustively enumerating all possible hypotheses; we automate the identification of what evidence and claims together constitute a plausible interpretation of an analysis, enabling the analyst may explore all possibilities. As a further means to mitigate biases in human reasoning, we highlight critical questions that may undermine inferential assumptions of various kinds: Is there an alternative cause? Do other experts disagree? These structured models may then be used to automatically generate tailored reports to key decision makers as required, or as the situational understanding shifts.

Today’s analysts must process increasing amounts of information, including “Twitter-INT” ¹ (social information such as Facebook, You-Tube videos, blogs, Twitter), as well as discern threat signatures in “gray zone” or hybrid conflicts distinguished by both aggression and ambiguity. The information environment is characterized by continuous change: the growing volume of data and the speed at which data is created and new influence tactics developed. Rebecca Goolsby wrote, “…the creation of hoaxes, hate speech, and other attempts at crowd manipulation and exploitation reveal the darker side of the social media phenomenon; the targeted “social-cyberattack” is rapidly coming of age.” The goal is to be able to describe, diagnose, and predict actions/behaviors/events based on an environment in which both humans and bots are attempting to influence, in which disinformation is common and curation and fact checking are rare The information environment is changing much faster than the ability of analysts to process and make meaning about actors, events and influence. Learning how to “surf” the information environment, rather than drown in the proverbial big data will require a new approach. Leveraging the fundamentals and “tricks of the trade” used by data scientists/analysts can serve to close the gap. This paper will highlight exemplar methods and tools and provide contextualized examples of how they improve the ability to describe and diagnose, and, ultimately, make meaning.

Social computing blends computational techniques such as statistics, machine learning, text mining, and graph theory, with psychological and organizational theories of process and structure, and social science theories of membership, engagement and communication. The application of social computing approaches for assessing and shaping the sociocultural landscape within an area of operations is of growing interest to the defense and intelligence community. These approaches can enable the understanding of how patterns of relations among actors, their environment, and resources influence behavior, and how interventions might change those patterns so altering that behavior. Recently the Army conducted a Workshop on Social Computing Research at the U.S. Army Research Laboratory (ARL). The purpose of the workshop was to understand the strengths and limits of current computational research applied to socially-created data while identifying critical research needs and opportunities of interest to the Army. We discuss several social computing strategies resulting from the workshop and propose a set of recommendations for integrating social computing in key ARL basic research science and technology objectives.

Currently, obtaining reliable situational awareness of the social landscape is an arduous, lengthy process involving manual analyses by social scientists. These traditional methods do not scale to the speed and diversity required by DoD operations or the high-speed, international business model in today’s corporate environment. Conversely, “big data” easily scales to meet these challenges but lacks the rigor of social science theory. We present Big Open-Source Social Science (BOSSS), a research and development project that leverages the strengths of social- and computer-science technology to address the operational need for rapid and reliable human-landscape situational-awareness. BOSSS iteratively filters, navigates, and summarizes diverse open-source data to characterize a local population’s social structure, conflicts, cleavages, affinities, and animosities. BOSSS automatically scrapes open-access data from the web and performs natural language processing to populate a knowledge graph with a custom schema. BOSSS then mines the graph to extract key, theory-agnostic socialscience principles of human inter-relations and dynamics: homophily, stratification, sentiment, and conflict. Automated quantitative social-network analysis provides up-to-date indicators of trends or anomalies within the local population’s social landscape. BOSSS’s emerging technology will provide a dramatic reduction in the cognitive workload for the next generation of analysts and will facilitate more rapid situational awareness both for deployed soldiers and private companies conducting operations abroad.

To be effective in complex operations, the U.S. military requires understanding about populations in the physical and information environments. Operations executed without sufficient understanding lead to unintended consequences with potentially far-reaching implications. We present Apropos, a platform that aims to improve mission outcomes through socioculturally informed course of action analyses. Apropos uses deep learning over multiple data modalities to efficiently derive information on operational and civil factors. Research efforts focus on deep learning approaches and model fusion techniques centered around knowledge graph embeddings enabling semantic search, predictive surfaces, and other analytics (e.g. route planning and site selection).

This research effort is developing a computational framework to support federated models of complex urban systems and enable information support for planning and response in emergency management. Systems analysis has been advocated to support emergency management activities, and there are a number of individual domain models designed to represent various system elements. However, effective implementation of this approach has its challenges. Traditional system analysis is often performed at regional or country scales. Further, information collection tends to be reductionist in process focusing on mission before the operating environment. Thus, there is limited data available to support high resolution urban systems modeling beyond localized areas. However, dense urban environment complexity requires the ability to capture and integrate the interrelationships between subpopulations and infrastructural systems. This system of systems modeling approach supports the analysis of cascading effects through interdependent infrastructure networks and the anticipated impacts on the subpopulations it supports, such as ethnicity, social class, access to transportation, or previously available services. The results are expected to reduce analyst workload by generating geospatial products and systems perspectives of demographic and infrastructure characteristics. We will be presenting an integrated infrastructure system demonstrating the cascading effects of component failure(s) combined with the effects on neighborhood-scale populations. The results are delivered to end-users using a geospatial visualization tool that includes information about the quality of the data products and the ability of the data to support information critical to emergency planning and response.

Psychological theories of inter-group behaviour offer justified representations for interaction, influence, and motivation for coalescence. Agent-based modelling of this behaviour, using evolutionary approaches, further provides a powerful tool to examine the implications of these theories in a dynamic context. In particular, this can enhance our understanding of the escalation of hostility and warfare, and its mitigation, contributing to policy and interventions. In this paper we propose a framework through which social psychology can be embedded in computation for the examination of inter-group behaviour. We examine how various social-psychological theories can be embedded in evolutionary models, and identify ways in which visualisation can support the objective assessment of emergent behaviour. We also discuss how real-world data can be used to parameterise scenarios on which modelling is conducted.

The US and nations of the NATO Alliance are increasingly threatened by the global spread of terrorism, humanitarian crises/disaster response, and public health emergencies. These threats are influenced by the unprecedented rise of information sharing technologies and practices, where mobile access to social networking sites is ubiquitous. In this new information environment, agile data algorithms, machine learning software, and threat alert mechanisms must be developed to automatically create alerts and drive quick response. US science and technology investments in Artificial Intelligence and Machine Learning (AI/ML) and Human Agent Teaming (HAT) are increasingly focused on developing capabilities toward that end. A critical foundation of these technologies is the awareness of the underlying context to accurately interpret machine-processed warnings and recommendations. In this sense, context can be a dynamic characteristic of the operating environment and demands a multi-analytic approach. In this paper, we describe US doctrine that formulates capability requirements for operations in the information environment. We then describe a promising social computing approach that brings together information retrieval strategies using multimedia sources that include text, video, and imagery. Social computing is used in this case to increase awareness of societal dynamics at various scales that influence and impact military operations in both the physical and information domains. Our focus, content based information retrieval and multimedia analytics, involves the exploitation of multiple data sources to deliver timely and accurate synopses of data that can be combined with human intuition and understanding to develop a comprehensive worldview.

This project employs data extracted from unstructured text and quantitative behavioral models to understand, forecast, and mitigate US adversaries' aggressive actions against the US and our allies. We use a combination of quasi-experimental causal modeling and counterfactual assessment techniques to assess the effectiveness of US courses of action (COAs) to quell aggressive states’ hostile activities. Results illustrate actions may yield unintended consequences through their impacts on other contextual factors. Additional analyses employ forecasting and ensemble techniques to examine the likely anticipated consequences of various US COAs in future scenarios and cases. Ultimately, the data, methods, and results provide a useful decision-support tool for planners and analysts faced with how best to mitigate against unfavorable outcomes.

This paper discusses applying automated social media analytics to address challenges facing the OSINT analyst. The growing use and trust of social media presents significant potential for planning, executing, and assessing military information support operations (MISO). The enabling technology for this capability is called SURF, which is a GOTS tool developed to perform what is referred to as ‘group search’ by classifying individuals based on their network features and interactions. This language-agnostic tool has been validated at very high levels of accuracy for classifying users. This same technology can be used to support other critical MISO and commercial sector activities. In this paper, we present the background research, motivation for algorithm development, validation and example usage.

A major task in information extraction is to extract relations between named entities. Relation extraction not only builds and extends knowledge bases and ontologies but also supports downstream application processing such as graph mining. In this paper, we report a relation extraction framework based on the natural language theory of link grammar. Our methodology uses and extends Akbik and Broß’s Wanderlust approach, where linguistic paths that are defined over the dependency grammar of sentences guide the relation extraction process. In particular, our framework splits a document into sentences, creates a dependency tree of each sentence, tags and categorizes entities, and extract relations between these entities. The accuracy of our framework is parametrized with the choice of linguistic paths, and accuracy scores as high as 95% precision, 36% recall, and 44% f-score are obtained. We also envision natural extensions of our work, where cross-sentence references are resolved and/or the context and content of the sentence constrains the linguistic paths.

Visual analytics is a field of study which imparts knowledge through visual representations. The use of these visual representations provide a common method for analysts to sift through vast amounts of information and make informed decisions on critical matters. However, assisting the analyst in making connections with visual tools can be challenging if the information is not presented in an intuitive manner. This study aims to build upon our previous work and further investigate whether line thickness can be used as a valid visualization tool to improve situational awareness. In this paper, we follow-up on previous work to discuss research results exploring the impact that information complexity, measured as graph density, has on situational awareness. Our results indicate an increase in situational awareness, compared to non-enhanced visualizations for select graph densities. Furthermore, the results obtained in this study validate previous pilot study findings. The enhancement identified and validated with this research confirms that the line thickness visual cue represents a perceived information value tied to situational awareness. We conclude that this improved situational awareness and time savings occur from the decreased mental burden placed on the analyst.

Creating network graphs is a manual, time consuming process for an intelligence analyst. Beyond the traditional big data problem, individuals are often referred to by shifting titles and multiple names as they advance in their organizations over time; this reality makes simple string or phonetic comparison methods to search for entities insufficient. Conversely, automated methods for relationship extraction and entity disambiguation typically produce questionable results as ground truth with no way for users to vet results, correct mistakes or influence the algorithm’s future results. We present an Entity Disambiguation tool, DAC Resolution and DISambiguation (DRADIS), which aims to bridge this gap between human-centric and machine-centric methods. DRADIS automatically extracts entities from multi-source datasets and models them as a complex set of attributes and relationships. Entities are disambiguated across the corpus using a hierarchical model executed in Spark allowing it to scale to operational data volumes. Resolution results are presented to the analyst complete with sourcing information for each mention and relationship allowing analysts to quickly vet the correctness of results as well as correct resolution mistakes by splitting and merging clusters. Vetted results are used by the system to refine the underlying model for future runs allowing analysts to course correct the general model to better deal with their operational data. Providing analysts with the ability to validate and correct the model to produce a system they can trust enables them to better focus their time on producing higher quality analysis products.

Collaborative decision-making remains a significant research challenge that is made even more complicated in real-time or tactical problem-contexts. Advances in technology have dramatically assisted the ability for computers and networks to improve the decision-making process (i.e. intelligence, design, and choice). In the intelligence phase of decision making, mixed reality (MxR) has shown a great deal of promise through implementations of simulation and training. However little research has focused on an implementation of MxR to support the entire scope of the decision cycle, let alone collaboratively and in a tactical context. This paper presents a description of the design and initial implementation for the Defense Integrated Collaborative Environment (DICE), an experimental framework for supporting theoretical and empirical research on MxR for tactical decision-making support.

Mixed Reality (MxR) represents an emerging technological paradigm, involving the merger of real and virtual environments to enable physical and digital objects to co-exist and interact in real time. Prior research efforts demonstrate the potential of MxR systems as decision support aids for Soldiers, capable of providing enhanced battlefield Situational Awareness (SA). Towards providing needed content ingest for MxR decision support, Internet of Things (IoT) based services appear to show growing promise. Ongoing advances in IoT technology have resulted in steady growth in both quantity of IoT assets deployed and data generated globally. However, towards supporting IoTMxR systems integration in battlefield environments, novel methods are presently required to support ingest and management of digital content for MxR device users. This paper provides discussion on supporting tools and techniques to support MxR-IoT integration as a means of decision support. Here, focus will be placed on both leveraging of existing IoT middleware solutions as well as methods for Value of Information (VoI) assessment according to consumer needs.

In today’s increasingly divided political climate there is a need for a tool that can compare news articles and organizations so that a user can receive a wider range of views and philosophies. NewsAnalyticalToolkit allows a user to compare news sites and their political articles by coverage, mood, sentiment, and objectivity. The user can sort through the news by topic, which was determined using Natural Language Processing (NLP) and Latent Dirichlet Allocation (LDA). LDA is a probabilistic method used to discover latent topics within a series of documents and cluster them accordingly. Each news article can be considered a mix of multiple topics and LDA assigns a set of topics to each with a probability of it pertaining to that topic. For each topic, a user can then discover the coverage, mood, sentiment and objectivity expressed by each author and site. The mood was determined using IBM Watsons ToneAnalyzerV3, which uses linguistic analysis to detect emotional, social and language tones in written text. The analyzer is based on the theory of psycholinguistics, a field of research that explores the relationship between linguistic behavior and psychological theories. The sentiment and objectivity scores were determined using SentiWordNet, which is a lexical database that groups English words into sets of synonyms and assigns sentiment scores to them. The features were combined to plot an interactive graph of how opinionated versus how analytical an article is, so that the user can click through them to get a better understanding of the topic in question.

With the advent of commercial-off-the-shelf sensors for use in a variety of applications, integration with analytical software tools, and expansion of available archived datasets, there is a critical need to address the problem of transforming resultant data into comprehensible, actionable information for decision-makers through rigorous analysis. In previous research the participating authors have emphasized that users are often faced with the situation in which they are “drowning in a sea of data” but still “thirsting for knowledge”. The availability of analysis software, tools, and techniques provide opportunities for information collection of ever increasing complexity, but the need for the training of analysts to employ appropriate tools and processes to ensure accurate and applicable results has not been addressed. The purpose of this paper is to discuss the challenges and opportunities facing the training of effective analysts capable of handling a wide-range of data types in this era of dynamic tools and techniques.

This basic research project focused on the specification of a top-level functional design of a new mixed-initiative approach providing effective and innovative computational support strategies that efficiently exploit human cognition while minimizing cognitive workload for achieving new intelligence analysis and decision-support capabilities. Toward improving analysis capabilities, we build in part on the similarly-oriented works of researchers that have argumentation methods at the core of their strategies for providing computationally-based support for analysis. However, in our approach, a central theme combines the story- and argumentation-based methods following suggestions in the literature into a hybrid scheme. The argumentation-based foundation provides the advantages of: 1) a basis on simple principles of reasoning, 2) explication of the generalizations and the evidence in the arguments, and 3) allowing the reasoning from the evidence to a conclusion to be easy to follow. In framing our overall functional analysis and decision-support architecture, we also leverage our own research in topic modeling for computational support to narrative development, and in methods for hard and soft data association, fusion, and inferencing. Our approach also takes an Open-World approach and as well addresses the issue of uncertainty in a mathematically rigorous way using a technique called the Transferable Belief Model (TBM). This paper focuses on the highlights of this overall approach; extended details are provided in our citations.

Many important decisions historically made by humans are now being made by algorithms - often learnt from data - whose accountability measures and legal standards are far from satisfactory. While model transparency is important, it is neither necessary nor sufficient. Accountability is arguably more important. However, accountability needs to carefully take into consideration the weaknesses of the original data, as well as the weaknesses of the model itself: indeed, robust datasets enable model robustness, and vice versa. In this paper we will focus on unfair datasets, as an example of the weaknesses in datasets. Fairness directly involves privacy problems, since learning without fairness can emphasize certain features or directions that generate private information leakage. For instance, a model may inadvertently reveal a persons age if age is a discriminating feature in a models decision making. Moreover, we will investigate the robustness of model in presence of adversarial activities. Indeed, we should strengthen our models by estimating what an adversary will do based on continuous dynamic learning, mindful of concealment and deception, and with a clear, explainable, insightful summary for the final decision makers. In this paper we will discuss how models based on unfair datasets can hardly be robust; and datasets used by weak models can hardly be fair.

The proliferation of real-time information on social media opens up unprecedented opportunities for situation awareness that arise from extracting unfolding physical events from their social media footprints. The paper describes experiences with a new social media analysis toolkit for detecting and tracking such physical events. A key advantage of the explored analysis algorithms is that they require no prior training, and as such can operate out-of-the-box on new languages, dialects, jargon, and application domains (where by "new", we mean new to the machine), including detection of protests, natural disasters, acts of terror, accidents, and other disruptions. By running the toolkit over a period of time, patterns and anomalies are also detected that offer additional insights and understanding. Through analysis of contemporary political, military, and natural disaster events, the work explores the limits of the training-free approach and demonstrates promise and applicability.

Within networks one can identify motifs that are significant recurring patterns of interaction between nodes. Here motifs are sub-graphs that occur more frequently than would be explained by random connections. Graphs can be used to model internal network structures of human groups, or links between groups, with group dynamics being governed by these structures. Graphs can also model behavior in engineered systems, and internal network structures can significantly affect dynamic behavior. A graph may only be partially visible (such as in hostile or coalition environments), however detectable network motifs may in some cases be reflective of the entire graph. We outline a research plan and describe basic network motifs and their properties, along with current analytic techniques for static and dynamic settings. We offer suggestions as to how network motif techniques can be applied to intra- or inter- group behavior, for example to detect whether multiple groups behave as a co-operative alliance, or whether coalition networks inter-operate in positive ways. As an example, we examine a complex time-series graph dataset relevant to coalition focused aspects of the class of networks under study, specifically related to the social network resulting from the authorship of academic papers within a coalition. We provide details of the basic analysis of this network over time and outline how this can be used as one of the datasets for our planned network motif research activities, especially with regards to the temporal and evolutionary aspects.

Ingest and maintenance of disparate data is a difficult problem to solve. As an example, we describe the fusion of violent events across multiple, disparate data sources. The Information Consumption, Exploitation, and Dissemination (ICED) framework provides an end to end system that solves this problem. It offers a set of ontologies and a mapping tool, OSCAR, to align and ingest the disparate data sources. These alignments are registered with a tool that performs management of the ingestion process and monitors how many records are translated and ingested successfully. A critical aspect of ingestion is the fusion step and we offer a novel, attribute-based, cloud scale fusion engine to match, in our example, mentions of the same event across the sources. The fusion engine offers the capability for multiple resolution runs and the exploration of results to assess the impact and results of custom scoring models. Next, ICED indexes a subset of objects of interest, for example the violent events themselves and the actors involved, for fast query and access. ICED offers a flexible object-definition interface to allow users to develop object views for index and retrieval. This allows end-users access to succinct information ‘baseball card’ views through interaction with the search or summary user interfaces. Such ‘baseball card’ views are developed for entities and events. For example, an individual violent event instance will have a baseball card view associated with it and, importantly, connections to the baseball card views of supporting information like the actors involved or the location where the event occurred. The results can also be viewed geospatially on a map for intuitive exploration.