This page documents remaining things to do which are not a part of the

todo list generated specific to each IR node by Doxygen. The source for this web page is in ROSE/docs/testDoxygen/ProjectToDoList.docs (which is copied into ROSE/src/frontend/SageIII/docs before internal releases).

TODO List for Next Checkin

What is left is:

Add Gary's expect scripts and documentation (to ROSE User Manual).
Make tutorial example for Peter's changes to AstCopy.
Need to document the EDG upc mode (–edg:upc); supported internally by ROSE (data in in the IR but no runtime library is currently available with ROSE, so the support for UPC is incomplete depending on your point of view).
Dominator trees are not built into librose.so, this was a mistake. Also Brian White suggests fixing "~SimpleDirectedGraphNode();" to be "~SimpleDirectedGraphNode() { }".
Define a mechanism to pass options to the backend (is this even needed)
Andreas says that we can remove astQuery files: astQueryInheritedAttribute.* nodeQueryInheritedAttribute.* numberQueryInheritedAttribute.* nameQueryInheritedAttribute.*
Add option to permit system header files to be installed separately (to be run before "make check"), perhaps "make install-system-headers".
Gary Yuan's bug: specification of multiple source files on the command line fails if we are linking, because the backend is invoked separately for each ROSE-generated source file (instead of generating all the source files first and then calling the backend compiler specifiying the same multiple (ROSE-generated) source files).
Move ROSE/src/wave-merge to ROSE/src/3rdPartyLibraries/wave-merge, check with Andreas. Also the subdirectory in wave, "NEW_ROSE" matches a common name used for the root of the ROSE directory tree by one of the developers, and should be changed to be a better name.
Remove DESTDIR from Makefile.am, it is not set, it is always empty, and it is confusing. In many cases it should likely be $distdir instead.
Sg_File_Info objects for the pointer dereference and array dereference (SgPointerDerefExp and SgPntrArrRefExp) (e.g. x = *Deref; x= badDeref2[3];).
Change #define IA64_ABI FALSE to TRUE in EDG (src/host_envir.h).
Add Gallery of figures from the ROSE project (include Thomas' figures at ~panas2/screens/.

report the position of the declaration of the variable referenced

Work that will be done within a later release:

Use of "--edg:old_c" to support K&R C does not work because the prototyped versions of functions are used within rose_edg_macros_and_functions_required_for_gnu.h and these are not allowed in EDG's { –old_c} mode (translated from the ROSE { –edg:old_c}.
Need to add -FPIC flags to compile line by default (currently specified explicitly)
Overloaded operators need to have their precedence computed so that we can avoid the numerous special cases that we currently handle in the code generation (unparsing).
Liao reports a bug in the LowLevelRewrite::insert() (as I recall). This will have to be fixed later.
Strings of spaces can be more easily built using "string (count,' ');" (suggestion from Jeremiah). Might be useful in ROSETTA and/or the code generation (unparser).
Operator precedence needs to be reviewed, "==" precedence has been changed from 2 –> 9. Cast operator precedence different between C style and C++ style casts (problem for Rama?).
Need to handle case of:
"if (parentExpr != NULL && parentExpr->variantT() == V_SgAssignInitializer)" in modified_sage.C in ROSE/src/backend/unparse, currently commented out as it has been for a long time but perhaps it should be there. Test code test2006_115.C demonstrates some problems specific to SgExprListExp in return statements, initializers, and SgAggregateInitializer objects. This is part of revisiting precedence handling for other than just primative type operators within ROSE.
Remove the SgScopeOp (or at least depricate it for now!) It is part of the older CC++ support within ROSE and is no longer used or associated with C or C++.
The performance of the compilation using ROSE for template code is dominated by the ast postprocessing resetTemplateNames. The performance of this function could be significantly improved and would improve compilation of Kull and other template rich applications.
Add example (such as in "bool SageInterface::is_C_language()") to demonstrate traversals over a subset of the AST (in this case SgFile IR nodes).
Debug name mangling (existing system can cause some names to be very long currently some statistical information is output to monitor this problem).
QualifiedNameList handling is done by value and should be handled via pointers to be consistant with the rest of SageIII and avoid possible memory leaks where they are used! There are places where set_qualifiedNameList() is called redundently on the same named entity.
EDG registers a Control-C signal handler and so Control-C causes a non-intuative error message for users. This could be unregistered after the EDG phase, and a more ROSE specific signal handler could be built (or none at all).
1. SgAsmStmt should not be derived from a SgDeclaration (where the asm option is used as a declaration it is really just a SgVariableDeclaration (using the asm option). It should be just a regular SgStatement.
Option "-rose:verbose n" requires the parameter and needs to be fixed so that it is not a failure to provide it. e.g. "sampleRoseTranslator -rose:verbose test.C" should work but currently segfaults. Note from Lingxiao: Certain command line options are not handled rightly. For example, "./sampleRoseTranslator -rose:verbose test.C" causes a seg fault.I figured it may due to "CommandlineProcessing::isOptionWithParameter". It seems no check whether there is a real parameter provided for an option. I see this because I use CommandlineProcessing::isOptionWithParameter(argc, argv, "--uc:", "polymorphic-unit", uc_polymorphic_unit, true); in my code, and "./UC test.C --uc:polymorphic-unit" seg faults but "./UC test.C --uc:polymorphic-unit 1" works fine.
Checking the file info object before unparsing statements has been common for a long time in ROSE, but a number of test codes demonstrate that this is required on a statement level since include file can operate on expressions. So this should be handled on a statement AND expression level instead of just a statement level.
Need to fix K&R mode; currently function prototypes in rose_edg_required_macros_and_functions.h.in fail to compile when using K&R C mode in EDG (usign: –edg:old_c).
Certain command line options are not handled rightly. For example, "./sampleRoseTranslator -rose:verbose test.C" causes a seg fault.I figured it may due to "CommandlineProcessing::isOptionWithParameter". It seems no check whether there is a real parameter provided for an option. I see this because I use CommandlineProcessing:: isOptionWithParameter(argc, argv, "--uc:", "polymorphic-unit", uc_polymorphic_unit, true); in my code, and "./UC test.C --uc:polymorphic-unit" seg faults but "./UC test.C --uc:polymorphic-unit 1" works fine.

Consider tutorial example to convert for loop to while loop, specific to C language.

I've done some more testing on object-oriented codes and implemented all tests that might be interesting: It appears that (according to all my tests):
- that the SgSpecialFunctionModifier is always correct (for a declaration AND the first_nondefining declaration)
- but the SgFunctionModifier is ONLY correct at the first_nondefining declaration For example: whether a function is virtual or not is only correct at the first_nondefining declaration, but whether a function is a constructor/destructor etc. is always correct. (See email from Markus, 9/1/2006: 4:21pm)

Jeremiah suggests adding a test to make sure that the if the parent is set then the current node is a child of the parent. This might be an issue since it might not be traversed, yet still be a child.

Unclear what to do here, but the static keyword in a variable declaration only sets the SgStorageModifier in the SgDeclarationModifier and not the SgStorageModifier held in the SgIntializedName. This is confusing to users. Also, isStatic() is always false in the SgStorageModifier held in the SgInitializedName! I think that we should just verify these properties, they are already documented in the *.docs files.
Reimplement how we handle mangled names. Use a global state to record when any transformation has taken place and record what IR nodes have been transformed in a list included as part of the global state.
Refactor code used in get_type to strip modifiers etc. to recover the base type.
Remove compilerName where used as function input since it is now ignored and we use BACKEND_C_COMPILER_NAME_WITH_PATH and BACKEND_CXX_COMPILER_NAME_WITH_PATH instead. These are set at configure time and may be overridden with explicit paths to the target compiler. The name of the C++ compiler (without path) is used internally for some testing. Same for linkerName in ::link() functions.

Rejected modifications for ROSE:

OpenMP "opm parallel" pragma can be considered to reference a statement, however when used in ROSE it is just a SgPragmaDeclaration followed by a SgStatement. Neither preorder or postorder tree traversal will give us the desired bottomup processing for OpenMP pragmas. For example: the code
#pragma omp parallel
{
#pragma omp single
statement1;
}
Its AST looks like:
–SgBasicBlock1
----— /---—\
---— /-----—\
—SgPragma1–SgBasicBlock2
------------------—/---—\
--------------—SgPragma2–SgStatement1
For preorder, the order of visit would be: (bb1, pragma1, bb2, pragma2, stmt1). For postorder, it is: (pragma1, pragma2, stmt1,bb2, bb1) In both cases, pragma1(omp parallel) is always visited before pragma2(omp single). Alternative way: use the default preorder in AST query and reverse it: We get what we want then: (stmt1, pragma2, bb2,pragma1, bb1). Comments from Dan: the problem here is that it is only OpenMP that expects that semantics of pragma to work this way and since it is not uniform for how #pragma can be used this is application specific. More discussion on this subject is welcome.

Currently tested ROSE

excludes CompilerTests (commented out to speed up the tests)

Hall C/C++ Compiler Test suite

A number of negative tests pass, and ROSE should be fixed so that there fail properly.

Nonexistant header file in #include statement. This fails in EDG, but is recorded as a pass in ROSE when -rose:negative_test is used.
An source file with no definition (n09003.c), should be an error. It is however not an error in EDG or GNU (and it should be under ANSI C mode). We could enforce it in ROSE, but we have not defined a place in ROSE to semantic analysis not handled by the frontend, but this could be done.

General Documentation of ROSE

Suggestions of things to do to improve the documentation of ROSE:

Switch documentation to PDF, instead of Postscript.
ROSE Overview (proposed chapter for Rich to think about)
1. ROSE documentation
  - ROSE features
  - Language front-ends
  - Code generation
  - ROSE directory organization
2. SAGE III Overview
  - Rules for IR nodes (which contain variable length lists of children)
  - Figure to describe IR node hierarchy
  - description of categories of IR nodes
  - relationship to C++ grammar
  - Integrate with current "Sage III Intermediate Representation" (current Chapter 10).
  - Properties of IR nodes (No null pointers, what few pointers are NULL, what IR nodes are shared,defining/non-defining declarations)
  - AST Merge Mechanism
  - File I/O
  - How modifiers are handled
Query Library (Chapter 5)
1. Documentation should be shorter and reference the tutorial where there should be a chapter on the Query Library. The HTML documentation should also document the API.
2. Show use of queries: IR nodes, names, numbers, IR types
3. Show composition of queries.
4. Show use of extensible features in Query Library (user-defined functions, etc.)
AST Rewrite Mechanism (Chapter 7)
1. function interfaces in table need to be updated, location is now scope and position in scope (two parameters).
2. Remove examples and reference the tutorial.
3. Note that High-Level Interface is under construction.
4. After the newer proposed Sage Interface API is done, document it here.
5. Reference the tutorial (build chapter in ROSE Tutorial specific to AST Rewrite Mechanism)
6. Include overview of what is in the tutorial.
Program Analysis (Chapter 8)
1. Figure out from Michelle and Brian what from OpenAnalysis can be referenced. Try to include tutorial examples and see if we can include Open Analysis in ROSE distribution.
2. Remove the data base discussion.
3. reference tutorial and add chapter with sections on CFG, Call-graph, Dependence Analysis, Slicing, etc.
The Class Hierarchy page generated by Doxygen has a large number of classes which I think whould not be documented.
AstProcessingChapter
1. AstTopDownProcessing example in ROSE Manual specifies the order of traversal and this is incorrect.
2. AstProcessing examples use Sg prefix name and the ROSE manual does not.

Additional Comments from Rich on ROSE manual 0.8.7a:

Eliminate Section 2.3.1, as it's only a little bit shorter than 2.3.2 (and so really does not summarize or add new information).
Chapter 3: Rename to 'Writing Your First Source-to-Source Translator in ROSE'
Chapter 3: subsection on testing. Explain line 15, when it may be called (i.e., after transformations even though none shown in this example). This helps to establish a sense of 'ROSE best-practices.'
Chapter 4 (Overview of ROSE): Rename to the slightly more descriptive, 'Overview of the ROSE Infrastructure'. I interpreted this section as highlighting the major components of the infrastructure that users will use to build their translators. That is, this chapter is something of a preview of all the chapters to come.
Chapter 10 (Sage III): This chapter should appear right after the current Chapter 4.
Chapter 11 (Appendix): The appendix is quite unwieldy. How can we better organize it? What about the following?

Adminstrative

Getting a version of the EDG license (11.6)
- This section includes the e-mail addresses of people outside the group and its immediate collaborators, and so we might not want to publish this on the web.
- Rather than having this information here (aside from a mention), we could alternatively keep form letter with this information on an internal (or unlinked) ROSE webpage that we can send out when people ask.

Properties of the AST

Correctness (11.10)
Normalization (11.11)
Declarations (forward vs. defining) (11.15)
Name mangling (11.17)
Comments handling (11.2)
cpp directive handling (11.3)
Join these into a section called 'Comments and Preprocessor Directives'
Pragma handling (11.4)
Template handling (11.7)
Prelinking (11.8)
Language and translator issues
- Compiling ROSE-generated code using ROSE (11.9)
- Non-standard features (Section 11.13)
- ROSE-specific headers (11.14)
- Passing options to EDG and ROSE (11.18)
- C and C++ modes (11.19)
Stuff to move?
Separation of EDG source (11.16) –> MOVE to Chapter 12
Adding new Sage III IR nodes (11.5) –> MOVE to Chapter 12

Stuff to eliminate?

Easy mistakes to make (11.12) –> Says never to compile in the source tree, which contradicts text of Chapter 2 (which says it is merely not well-tested). Could probably eliminate this section unless we come up with other material.
Error messages (11.1) Do we need this section? It's one sentence.

General Configuration and Build Issues

 - Fix up configuration to permit building in the source tree
    Should work currently, but make clean will remove *.tex 
    files in ROSE/docs/Rose directory.

 - Remove many old configure options (see configure --help)

ROSE Tutorial Todo List

 - Put PDF example into ROSE Tutorial (since it will be removed from the ROSE User Manual).

 - Michelle Strout reports that the ROSE/tutorial/codeCoverage.C
   example code does not compile properly with 64bit architectures using the 4.0.2
   compiler on the Intel Opteron processor.  Her solution was to comment out this
   tutorial example.

 - Add Class Hierarchy graph example to tutorial.

 - Add example showing how to figure out if a variable declaration is part of a class.

 - Add example showing how to test code for patterns of "=" used instead of "==" in
   conditionals of branchs.  Simple code pattern recognition.

 - Each tutorial example should explain what the reader should learn.

 - Comments from Rich:
    - All sample translator source code examples: The comments at the top
      of most examples need to be updated to be a short (1-sentence)
      summary of the example. (These can usually be copied from the Figure
      caption, but should also appear at the top of the source for people
      who copy/edit the examples directly.)

 - Add an example bug report so that it is clear how to submit a bug report.

 - Develop and demonstrate a mechanism to add attributes to IR nodes so that they
   will be colored in the dot output.  Helpful for debugging the new AST merge
   (based on the new name mangling and the new memory pool traversal).

ROSETTA Todo List

 - Add mechanism to build static functions for each IR node to identify it as
   being an IR node supporting:
      - C
      - C++
      - FORTRAN
   If we return an std::vector<bool> type then we can support combinations of them
   else we could use an "int:3" or a "short int" type.  Since it would be static
   the space is not an issue.  However since it is likely to be called often
   there might be a performance advantage of one over the other.

 - Need to define array new operators for each IR node so that allocation of an array
   of IR nodes will either be properly handled or will be disabled.  Several
   mechanisms within ROSE require IR nodes be build within the memory pool.  This
   makes it a problem for any IR nodes not built with the IR node's new operator.
   We can backup this requirement by testing if all referenced IR nodes are in the
   memory pools.  This test catches IR nodes that are build in global scope, on the 
   stack, or in the heap.  This constraint is required for the memory pool traversal
   and visitor pattern support and the AST File I/O.

 - ROSETTA uses char* type variables in many places and we need to move it to using
   C++ strings instead.

ROSE Translator Construction Todo List

 - Need a policy on output from ROSE translators of that commandline options
   such as "-E" and "-H" don't have output in them.  This can cause problems
   for autoconf macros within large projects.

Namespace Support

   Need a policy on what namespaces we want to have, current suggestion is:
   - Frontend
   - Backend
   - Analysis
      - PRE (Currently have this)
      - DominatorTreesAndDominanceFrontiers (Currently have this)
   - Transformation
   - ROSE BASE
      - SageIII IR
      - Rewrite Mechanism (high level would depend on front-end)
      - Traversals
      - astDiagnostics
      - SageInterface

SageIII Details

Need to test mangled names for uniqueness, and for equality when they define the same construct. This should be a test that can be run on any input code, as apposed to hand crafted inputs. A suggested test would be:
- For each non-defining declaration, verify that the mangled name matches the defining declaration.
- For each defining declaration save the name in a list and report any name collisions in the list (ODR, One-time Definition Rule, would be used to verify that there would be only one definition and so any collision would be an error.
- Templates would have to be resolved back to their template definition and be unique in a modified sense.
- Need to nail down any other missing details required for this test...

ROSE Configuration

This section documents things to do that are specific to the ROSE configuration.

There are a number of Automake configuration options and we might not need all of these:
- AM_CONDITIONAL(ROSE_USE_BOOST_WAVE,test "$with_boost_wave" = yes)
- AM_CONDITIONAL(ROSE_USE_SQLITE_DATABASE,test ! "$with_SQLite" = no)
- AM_CONDITIONAL(ROSE_USE_MAPLE,test ! "$with_maple" = no)
- AM_CONDITIONAL(ROSE_USE_VISUALIZATION,(test ! "$with_FLTK_include" = no) || ...
- AM_CONDITIONAL(AXXPXX_SPECIFIC_TESTS,test ! "$with_AxxPxx" = no)
- AM_CONDITIONAL(DQ_DEVELOPER_TESTS,test "$enable_dq_developer_tests" = yes)
- AM_CONDITIONAL(ROSE_USE_DATABASE,test ! "$with_MySQL" = no)
- AM_CONDITIONAL(ROSE_PERFORMANCE_TESTS,test ! "$with_PERFORMANCE_TESTS" = no)

ROSE Quick Reference Guide

It has been proposed that we build a ROSE Quick Reference Guide. If we do this it should include:

Significant configuration option for ROSE
Location of documentation
Top level graph of ROSE directory file structure
Top level graph of AST structure
Common Traversal base class names
???

and testing of C++ covariant return types for virtual functions

Covariant return type handling is presented in test2006_81.C, and the support for it is in place within ROSE in that we store the return type and the original return type for all function types.

I think that the original return type might only make sense for member function types.
This whole subject needs to be reviewed and tested within ROSE.
At the moment the original return type is generally a reference or a copy of the return type.
This work may be tied to the support for both name mangling (or function types) and the construction of the global function type table (recently completed, but not checked for proper handling of covariant return types).

Normalization of Implicit C++ functionality

The AST Rewrite mechanism provides a simple way to specify transformations using strings. However, it is not very robust and will require more work before it is useful in a production environment. This is a start at a bug list:

Isaac reports that the MiddleLevelRewrite works for him, but not when 2 files are specified on the commandline (Instressing!)

Normalization of Implicit C++ functionality

This is a suite of transformations designed to transform C++ code to make those things that are expressed implicitly in the language be expressed explicitly in the AST. The goal of these transformations is to make the AST more amenable to analyses such as control-flow graph analysis. It currently consists of the following transformations:

defaultFunctionGenerator Generates default constructors, destructors, operator=(), and copy constructors.

destructorCallAnnotator Annotates the AST with calls to class destructors whenever objects go out of scope.

shortCircuitingTransformation Transforms the evaluation of short-circuited expressions to explicitly evaluate each step. A prerequisite of destructorCallAnnotator.

Credits

Brian White Chunhua Liao Peter Collingbourne

List of remaining work to do:

Brian White has written about changing SgAssignmentInitializers to SgConstructorInitializers (need to find that email and do this work).
Function calls to the operator=() are not implemented in the AST.
Need to look at casts, which might appear differently in the AST if the constrctors or conversion operators are defined explicitly.
Conversion operators are not implicit, and so not an issue.