Sortie: Re-engineering a modeling and simulation tool to support gradual program evolution and increased data visualization.
Primary researcher: Sachen Gendron
We are currently soliciting developers of state of the art tools to join existing tool developers from research and industry to participate in a collaborative tool demonstration. Tools are being used in combination to solve a significant reverse engineering task on a legacy software system, called SORTIE. Initial results from a preliminary collaborative effort were presented at a working session at WCRE’2001, which was held in Stuttgart from Oct 2-5, 2001. Initial results have been posted on the Results page of this site. See the Case Study page for more information on SORTIE.
There are three main motivating factors for this demonstration.
Evidence to Encourage Tool Adoption
Research has produced many significant advances in compilers, web-based interfaces, visualizations, impact analysis, clone detection, object identification and slicing algorithms, yet many of the mainstream tools in use today are not radically different from tools that have been available for the past two decades. Indeed, many reverse engineering and reengineering projects still rely on fully manual methods for documenting and redesigning legacy code. One reason for this reticence is the lack of empirical evidence to demonstrate the benefits of the innovations and the suitability of the tools for various problems and work environments. Open and public demonstrations of tools, such as this one, are needed to provide such evidence.
Exploration of Tool Interactions
Although tools are designed to solve related tasks in a software maintenance project, they are usually
researched, developed and evaluated in isolation from one another. There has been little work examining how the tools work in tandem and how they should interoperate from not only a technical perspective, but also
from an end user’s perspective. This collaborative tools demonstration also seeks to remedy this shortfall. Tool designers and potential end users will be able to observe how the best tools and technologies available can be combined to solve reengineering tasks.
Promoting Tool Interoperability
This tool demonstration promotes tool interoperability in two ways: (1) by providing a forum for collaboration; and (2) by promoting the use of GXL, the emerging standard exchange format.
Due to the nature of this evaluation, tool designers will need to share data and results with one another, which in turn will encourage them to critically examine interoperability with other tools. It is hoped that by participating, tool designers will move towards a common understanding on how complementary tools can be used to solve a reengineering task. We also expect that this demonstration will also lead to suggestions for advancing interoperability, such as component-based design, mechanisms for sharing control between tools, the improvements to the standard exchange format and so on.
To improve tools and develop better ones by comparing and evaluating existing tools.
To provide an opportunity for collaboration and community-building.
To learn more about the complementary nature of various tools.
To acquire experience with tool interoperability necessary to design an infrastructure for community-wide sharing of tools.
To improve tool evaluation techniques.
To encourage the use of GXL.
SORTIE is an established research tool for modeling forest succession. It is actively used in British Columbia, Canada and in the northeastern U.S. to manage forests. The program consists of approximately 28 KLOC of C++ code with sparse documentation. The system has evolved over a long period of time leading to a very brittle and complex architecture.
During the past six months, tool teams have been collaborating to analyze the SORTIE system. The first task was to recover the existing architecture of SORTIE.
Preliminary results from this part of the project are described on the Results page. Many of the teams are still working on this first phase. The next phase will be to propose a new architecture that is better suited to meeting requirements for future changes.
As part of a separate parallel project, a graduate student from the University of Victoria has been reengineering and re-architecting the SORTIE system using predominantly manual methods. She presented her findings and compared her results at the WCRE’2001 workshop. She will be continuing in this role and is currently proposing a new architecture for SHriMP.
More details on the reverse engineering tasks to be considered are also available.
The collaborative demonstration is a part of a series of structured tool demonstrations. This phase of the demonstration will take place over the next six or so months. The results of this phase of the tools demonstration may be presented in the Spring at the IWPC’2001 workshop in a working session (if our proposal for a working session is accepted). We will compare results from the collaborative effort and the parallel manual project.
University of Victoria
Susan Elliott Sim
University of Toronto
University of Alberta
A Summary of the User Requirements for the SORTIE Reengineering Project
Supplied by Sachen Gendron, our domain expert
There are two potential uses of the SORTIE program: it can be used as a tool for research exploration in understanding forest dynamics; and it can also be used as a test environment for forest management decisions. In order to support research exploration, the forestry researchers would like to incorporate new relationships and formulas without the assistance of professional programmers. Currently this is not possible, and therefore the code needs to be re-engineered to more easily allow these changes. The re-engineered program should be based on an object-oriented, extensible and well documented framework.
More background on the tool and re-engineering project
The subject system to be re-engineered in the collaborative demonstration is a legacy research tool called SORTIE. SORTIE is a resource mediated spatially explicit mixed-species forest dynamics model that allows direct input of data from rigorous field studies. SORTIE was originally developed in eastern US deciduous forests.
During the past four years, the model has been extensively modified for the temperate mixed-species forests of northwestern British Columbia by the BC Ministry of Forests (SORTIE/BC). The model simulates and visualizes forest succession and can be used to help develop alternative logging prescription to clearcutting. The SORTIE software system has evolved over a number of years as the underlying model was developed, refined and extended. Like most aging software systems, the architecture of the SORTIE tool has become brittle and monolithic making it difficult to change and maintain.
SORTIE is less than 28KLOC including comments and documentation (of which there is very little). It was originally written in C, but was recently ported to C++. It runs only on a PC and relies heavily on Borland C++ compiler and libraries. It has been developed mainly by one programmer, so there is little documentation available. It is possible to consult this person as a system expert. Some attempts were made to analyse it with the Rigi and Imagix program comprehension tools, but it wasn’t possible to parse the code, due to the tight integration with the UI generated code by Borland.
SORTIE/BC is currently being used to model the dynamics of the interior cedar-hemlock forests of Northwestern B.C. In addition, work is also underway to use SORTIE for southern hardwood and boreal mixed wood forests of Quebec, spruce/aspen boreal forests of Canada, temperate forests of New Zealand and tropical forests of Puerto Rico. The SORTIE developers are finding it difficult to easily calibrate the model for different forest types and ideally, the tool could be used by more researchers (other than the original developers) to model different kinds of forests in other locations. The SORTIE model consists of several submodels. The existing submodels are “hard coded” into the software and are
fragmented throughout the system making it difficult to either change the existing submodels or to add new submodels. The current architecture of SORTIE does not easily allow the reuse of components for other applications. Unfortunately, the current architecture of SORTIE does not easily support changes and any that are made, invariably have unintended side effects. Adding new variables, new visualisation features, and making changes of any kind are currently very difficult.
The collaborative demonstration will be held in parallel with a shadow project using predominantly manual and traditional methods. A Master’s student at the University of Victoria has already documented and analyzed the existing architecture. She is currently gathering requirements from users at the Ministry of Forests and from other researchers interested in using the tool. The gathered requirements will be used to propose a new architecture.
The first task to be completed is an analysis of the existing architecture. This analysis along with any existing information (such as code and any documentation) will then be used by other tools to re-document the existing system. The re-documentation from the parallel shadowed project will not be made available to the collaborative demonstration participants.
The next phase will be to propose a new architecture based on user requirements and the existing architecture and code. The Forestry client would prefer, if possible, to reuse as much as possible of the existing code in a reimplementation. The user requirements gathered during the manual reverse engineering project will also be made available to the demonstration tools responsible for generating and proposing a new architecture for SORTIE.
Comparison of tools
Identifying the strengths and weaknesses of a number of tools will serve to codify knowledge about reverse engineering tools. We also expect to learn about which tools work well together, and by extension the steps necessary to achieve greater tool interoperability.
These lessons can be used to improve existing tools and to develop better ones, as well as acquire the experience necessary to design infrastructure for community-wide sharing of tools.
In this collaborative demonstration, participants will necessarily work together to produce results as well as interacting during the presentations and discussion. Furthermore, the collaboration will also promote learning and exploration of interoperability between reverse engineering tools.
The results from the first phase of this collaborative exercise were presented at the WCRE’2001 working conference in Stuttgart, October 2-5, 2001. Details on expected deliverables were outlined in the handbook.
The main reverse engineering task for this collaborative demonstration is to propose a new architecture for the SORTIE system that facilitates expected future development. In proposing the new architecture, participants will need to perform a number of sub-tasks, such as recovering the existing architecture.
Reengineering of the SORTIE system is expected (but not required) to follow the tasks outlined in Figure 1. Source code from the legacy system will be parsed to extract key information. This information will then be analysed to produce documentation and/or diagrams.
These two steps will be iterative. The reverse engineering phase will mostly involve rearchitecting the system and may be followed by a reengineering phase with some implementation to apply the changes to the source code. Teams will be expected to use the results from tools in adjacent or complementary phases.
Since we are also interested in comparing peer technologies, the actual process may be similar to that shown in Figure 2.
The collaborative demonstration will be a longitudinal tool evaluation
involving both peer and complementary technologies. Comparisons can be made between tools in the same phase (peer technologies) and between tools in different phases (complementary
technologies). Teams will be expected to share data and information with each other (ideally using GXL, the emerging standard exchange format).
See the case study description for a description of the user requirements for this project.
Submitted reports from participants (updated Sept 24/2001):
|Rigi tool||Rigi groupUniversity of Victoria||Holger Kienle||Report (text)|
(Software Architecture Group)University of Waterloo, Canada
|Andrew Malton||Report (HTML)|
|TkSee tool||KBRE Group
University of Ottawa,Canada
|Sergey Marchenko||Early results
|SCG P.U.R.E.||Software Composition Group
University of Berne, Switzerland
|Michele Lanza||Report (HTML)|
|COLUMBUS/CAN tool||Research group on Artificial IntelligenceHungarian Academy of Sciences, University of Szeged||Rudolf Ferenc||Report (HTML)|
|KLOCwork Suite||KLOCwork group
KLOCwork Solutions Corporation
|VIBRO (VIsualisation BROker Framework)||Visualisation Research GroupUniversity of Durham, UK||Claire Knight||Report
University of Waterloo
|Davor Svetinovic||Incorporated in cppX report|
Note: This handbook was written for the first phase of this demonstration which was presented at WCRE’2001 from 2-5 October, 2001 in Stuttgart. The handbook will be updated when the next meeting milestone has been set (we are currently considering IWPC’2001 which will be held in Paris at the end of June 2002.
Participant Handbook – for a Collective Demonstration of Reverse Engineering Tools
Handbook Version: July 26, 2001.
Dr. Margaret-Anne Storey
University of Victoria
Susan Elliott Sim
University of Toronto
University of Alberta
In this collaborative demonstration of reverse engineering tools, teams of tool designers will be applying their tools to a common subject system and task, to re-architect a forestry management tool, SORTIE. This handbook describes the expected participation and contributions of the teams. Teams will be required to:
- perform the reverse engineering and re-engineering tasks on SORTIE
- participate in a discussion forum during the process (to share results and promote further collaboration)
- share data or results (ideally using GXL) with at least one other team
- make your results and reports publicly available (on this or another web site)
- submit deliverables and reports to the organizers by Monday, September 17, 2001
- participate in a lively panel and discussion at WCRE’2001
- have a display in the Tools Fair at WCRE2001
These requirements will be described in detail in the remainder of this handbook. The most current version of this handbook can be found on the collaborative demonstration web
The following deliverables must be submitted to the organisers by Monday, September 17, 2001. We need to have these materials in advance of the conference, so we can prepare an analysis of the process and the results for the panel presentations.
- Written report
- Work products
Since we have a wide variety of tools participating, we are expecting a wide variety of results. Examples of work products are visualizations, metrics, file clustering, factbase or data files to be used with a query engine or analyzer, customized scripts for your tool, converters between file formats, and modified source code. These work products should illustrate your solutions to the assigned task and substantiate your written report.
- Results of analysis
What are your solutions to the assigned tasks of assessing and proposing a new architecture for the SORTIE system?
Section 1. Introduction
- Brief description of tool and its main functionality
- Team — background of each team member, including education, level of experience as software developer, level of experience with tool
Section 2. Experience Report
- What steps were taken to analyze the software system?
- How much time was spent on each step?
- What difficulties were encountered?
- Were there any surprises?
In this section of the report, you should describe what happened when you tried analyzing
SORTIE with your tool. You should talk about the specific steps you took and the abstract reverse engineering or re-engineering approach you followed. One way to help describe your abstract process is to look at the assumptions you make, for example, about the types of
subject systems that your tool is suitable for, heuristics that you used when generating your output or doing your analysis.
Section 3. Collaboration Partners
- Identify peer tools
- Identify complementary tools
- What other tool teams did you work with?
- What did you have to do to make your tools work together?
- What lessons were learn about tool interoperability?
Peer tools are ones that offer similar functionality to yours, while complementary tools offer different functionality that may be used in tandem with your tool. When you work with another tool team, both partners should ideally write up their experiences. Describe also other tools you used that were not directly part of the collaboration tool demonstration.
Section 4. Solution to tasks
Based on the analysis that you performed using your tool with or without the collaboration
of anyone else’s tool, what can you say about the SORTIE system? These are your discoveries, recommendations, and remediations of the SORTIE system. hat is the existing architecture of the SORTIE system? What should the architecture of the new system be?
Format of Report
Please use the formatting found in this page (i.e. in HTML) so that we can easily post the results on this website. If this is really not possible, please provide PDF or
Participation at WCRE 2001
- Demo at Tools Fair
We do not yet have details regarding the organisation of the Tools Fair at WCRE. We will make these available once they are known.
- Presentation and Panel
The format of the presentation has yet to be determined because it will largely depend what results are produced by the study as a whole. Each team will have an opportunity to present, however, they may be required to co-ordinate their presentation with other teams.
Details about both the Tools Fair and Presentations will be disseminated on the discussion forum as they become available