Follow this link to skip to the main content

Draft Proposal for Improving Development Process

Proposal for a New Development Process

(Updated March 18, 2004) - notes

Notes from meeting with (Wright, Kunz, Sargent, Nesnas, Nov 24, 2003)

  • Make module release no longer requires releases of compiled binaries and libraries. Release code with binaries and libraries only at major milestones in the development process (e.g. yearly or bi-yearly)
  • Limit the use of Link Modules to 3rd party libraries (ext_ modules) that CLARAty no modify
  • Disable direct commits on the main branch. To commit, the system will ask the user whether he/she wants to make a release or move the changes onto a branch
  • Enable direct commits to main head of CVS repository. Allow branching for a non-release point in main by adding appropriate tag (details TBD)
  • Major developments on new features or major bug fixes are implemented on a separate topic branch (e.g. sync stereo, flattening device hierarchy, estimation, etc).
  • Usage and minor bug fixes are all done on the main branch by making releases directly from the main branch. People can use "cvs" update to get the latest software.
  • Distinguish and tag code appropriately between these two concepts: released, and tested
    • Release modules with source code only. Release process should include at least:
      • Tagging CVS repository
      • Capturing module dependency and updating the database
    • For releases, only require sufficient build and test to reasonably prove that the changes introduced are operational
    • Reduce the number of actively supported targets
    • Introduce an automated build and test process for all targets; once this succeeds for a module, it will be tagged as tested (results from the nightly build after passing automated test. Additional tag hw-tested after tested on relevant platform.
  • With the exception of 3rd party (ext_) modules, all sandboxes will include the necessary source code.
  • System prevents you from releasing a module in your sandbox that depends on other modules that you have modified. To release a module, you have to start releasing from the top of the dependency tree. System produces an ordered list of modules that need to be released.
  • Open issue - how to get people to realize that they are on a branch and how to easily move them to the main branch when appropriate
  • Improve current recursive makefile: speed up the compile time, and abort on errors
  • Explore having a build manager / same person doing tools and build
  • Keep two builds of the system: last night and tonight and ping pong.
  • When automated test fails to catch a bug, write a test case to capture such future occurrences.
  • Future addition: allow release of a module without requiring the release of modules that depend on it. The system will build against the proper modules from orig checkout
  • Speed needs improvement on all yam latest
  • Fix makefile to behave Makefile -k . Abort on errors unless specifically using a -k option but report error at end
  • Modules that are independent of each other can be easily checked out with a test (2004-13-1)

Problems with AFS

  • Centers and universities experience severe speed performance problems with AFS file access
  • AFS does not work behind a NAT
  • Requires individual accounts for external users and a psuedo badge for each
  • Explore use of pserver and non-AFS dependency (or have a dual option: AFS and pserver)

Problems with Current Development Process

  • The system is very complex and large
  • People are struggling to get up to speed on the development process in a timely manner
  • The YaM model appears to not match the current CLARAty development model. Some members feel that CVS is better and more efficient than the current system (Sargent, Bajracharya)
  • Maintaining the process is an overwhelming responsibility - need to allocate more resources and raise the FTE level
  • Progress is not made rapidly
  • Lack of commonality of general and test data formats - need more standards in file/serialization formats and support for PDS
  • Problem inherently complex and made more so with distributed hardware. Changes are difficult to make
  • Current makefile system does not behave properly. If the build fails, the makefile continues its work.
  • Too many passwords for website, database, code access, etc.
  • Access to large disk space at reasonable cost proved very difficult with AFS
  • Release process too complex (only few people have been able to release modules)
  • Need to staff the position for tool development maintenance (0.5 FTE)
  • Lack of a build process manager
  • What we do is very different: flexibility and modularity is very expensive
  • Difficult to see what is in CLARAty to find out what exactly is in modules
  • Modules are at too fine granularity
  • Broken release process
    • People are on a branch for too long. Need a line in the sand that work together.
    • Building and testing on all platforms is very difficult

Other Problems

  • Change to navigator on K9 get checked in, how does this affect R8, R7, etc.?
  • IP issue (SLOG tracker), mesh tracking
  • Time constraints for properly investigating 3rd party libraries
  • Impact of changes
  • Capture things into FAQ
  • How to transition (merge divergence).
  • HTrans and DH parameters. Transforms
  • Frame needs work