When writing scientific modeling and simulation software, frequent regression tests can expose bugs that would otherwise create future obstacles. For this reason, regression testing should be a fundamental part of any development process in medium to large-sized projects. In order to implement a flexible solution to this problem, a software testing framework that is based on simple one-to-one comparisons was designed. The comparisons are performed between two different representations of a simulation with one representation considered valid and the other unknown. Using a simple framework has proven to be advantageous in several ways. One of the biggest advantages is that of portability for testing other software. Implementing standardized design patterns allows a degree of flexibility which keeps it from being bound to specific software. For output, the framework is designed to use the eXtensible Markup Language (XML). This results in the ability to publish results in several different formats, archive into a database, and maintain compatibility with other simulation outputs. The preliminary results of implementing this framework have proven promising. Using object-oriented design has not only simplified development but has allowed for a more user friendly approach to testing. Future improvements include user-customized test cases, ad hoc queries for archived results, and automatic test result publication.
As simulations and the machines they run on become larger and
more complex the inputs and outputs become more unwieldy. Increased
complexity makes the setup of simulation problems difficult. It also
contributes to the burden of handling and analyzing large
amounts of output results. Another problem is that among a class
of simulation codes (such as those for physical system simulation)
there is often no single standard format or resource for input
data. To run the same problem on different simulations
requires a different setup for each simulation code.
The eXtensible Markup Language (XML) is used to represent
a general set of data resources including physical system
problems, materials, and test results. These resources provide
a "plug and play" approach to simulation setup. For example,
a particular material for a physical system can be selected from
a material database. The XML-based representation of the selected
material is then converted to the native format of the simulation
being run and plugged into the simulation input file. In this manner
a user can quickly and more easily put together a simulation setup.
In the case of output data, an XML approach to regression testing
includes tests and test results with XML-based representations.
This facilitates the ability to query for specific tests and make
comparisons between results. Also, output results can easily be
converted to other formats for publishing online or on paper.
A scientist who sets up and runs experiments typically keeps notes of this process in a lab notebook. A scientist who runs computer simulations should be no different. Experiments and simulations both require a set-up process which should be documented along with the results of the experiment or simulation. The documentation is important for knowing and understanding what was attempted, what took place, and how to reproduce it in the future. Modern simulations of physical systems have become more complex due in part to larger computational resources and increased understanding of physical systems. These simulations may be performed by combining the results from multiple computer codes. The machines that these simulations are
executed on are often massively parallel/distributed systems. The output result of one of these simulations can be a terabyte of data and can require months of computing. All of these things contribute to the difficulty of keeping a useful record of the process of setting up and executing a simulation for a physical system. An electronic notebook for physical system simulations has been designed to help document the set up and execution process. Much of the documenting is done automatically by the simulation rather than the scientist running the simulation. The simulation knows what codes, data, software libraries, and versions thereof it is drawing together. All of these pieces of information become documented in the electronic notebook. The electronic notebook is designed with and uses
the eXtensible Markup Language (XML). XML facilitates the representation, storage, interchange, and further use of the documented information.
A language has been created to facilitate the automatic execution of simulations for purposes of enabling parametric study and test and evaluation. Its function is similar in nature to a job-control language, but more capability is provided in that the language extends the notion of literate programming to job control. Interwoven markup tags self document and define the job control process. The language works in tandem with another language used to describe physical systems. Both languages are implemented in the Extensible Markup Language (XML). A user describes a physical system for simulation and then creates a set of instructions for automatic execution of the simulation. Support routines merge the instructions with the physical-system description, execute the simulation the specified number of times, gather the output data, and document the process and output for the user. The language enables the guided exploration of a parameter space and can be used for simulations that must determine optimal solutions to particular problems. It is generalized enough that it can be used with any simulation input files that are described using XML. XML is shown to be useful as a description language, an interchange language, and a self-documented language.
The analysis of use cases describing construction of simulation configuration files in a data/information management system can lead to the acquisition of new information and knowledge. In this application, a user creates a use case with an eXtensible Markup Language (XML) description representing a configuration file for simulation of a physical system. INtelligent agents analyze separate versions of the XML descriptions of a user and additionally, make comparisons of the descriptions with examples form a library of use cases. The agents can then make recommendations to a user on how to proceed or if tutoring is necessary. In a proof-of-concept test, new information is acquired and a user learns from the agent-facilitated tutoring.
The notion of perspective, when supported in an object-based knowledge representation, can facilitate better abstractions of reality for modeling and simulation. The object modeling of complex physical and chemical processes is made more difficult in part due to the poor abstractions of state and plasma changes available in these models. The notion of perspective can be used to create different views to represent the different states of matter in a process. These techniques can lead to a more understandable model.
The notion of perspective when supported in knowledge representation can allow the representation of multiple and varying points of view, some of which may even be inconsistent with one another. In an object-based knowledge representation methodology created and used by the authors, a perspective is defined by consolidating a number of objects and a number of those objects' associated attributes and method into a view. This view can help partition a knowledge domain into separate portions. A separate portion represents an individual's view of the knowledge domain. Representation of multiple and varying perspectives can add detail and context to the knowledge in a knowledge domain. The ability to create new perspectives may add to the existing knowledge as well as reveal paths to additional knowledge. A simple example is presented where perspectives are used to represent game playing strategies and levels of expertise in those strategies. Players' perspectives are adapted and changed to provide additional knowledge and insight into further game playing strategies. Results show improvement in the playing of the games. Additionally, a more complex problem for applying these techniques is introduced.
Dissemination of battlespace information involves getting information to particular warfighters that is both useful and in a form that facilitates the tasks of those particular warfighters. There are two issues which motivate this problem of dissemination. The first issue deals with disseminating pertinent information to a particular warfighter. This can be thought of as information suppression. The second issue deals with facilitating the use of the information by tailoring the computer interface to the specific tasks of an individual warfighter. This can be thought of as interface complexity suppression. This paper presents a framework for suppressing information using an object-based knowledge representation methodology. This methodology has the ability to represent knowledge and information in multiple perspectives. Information can be suppressed by creating a perspective specific to an individual warfighter. In this way, only the information pertinent and useful to a warfighter is made available to that warfighter. Information is not removed, lost, or changed, but spread among multiple perspectives. Interface complexity is managed in a similar manner. Rather than have one generalized computer interface to access all information, the computer interface can be divided into interface elements. Interface elements can then be selected and arranged into a perspective-specific interface. This is done in a manner to facilitate completion of tasks contained in that perspective. A basic battlespace domain containing ground and air elements and associated warfighters is used to exercise the methodology.
Repertory grids and other matrix-like structures can be used to represent knowledge and elicit knowledge from experts. A grid or matrix is a representation of a knowledge domain where the elements in the domain appear along the horizontal axis and constructs or attributes of the elements appear along the vertical axis. Each construct is rated for its presence in a given element or how much a construct applies to an element. Analysis of these ratings can determine similarities and differences between the elements. Traditionally, constructs are bipolar entities where a rating falls on a range from one pole to the other. For example, temperature may be represented by the bipolar construct hot-cold and a range of 1 to 5 in which 1 represents hot and 5 represents cold. Ratings of 2, 3, and 4 lie in-between hot and cold. Additionally, all constructs in a grid have the same range of values and the range is arbitrarily chosen. This paper presents a method for translating grid ratings into fuzzy membership values. The fuzzy membership values become the values for describing and analyzing the associations between elements. Thus, constructs no longer need to use the same scaling range and no longer need to be bipolar. A construct of an element now becomes a true attribute of an element. An attribute can be rated in its own range and with its own unit of measurement. In the previous example, the bipolar construct hot-cold becomes simply, temperature measured in degrees. Experts or users need no longer translate to an artificial rating range.