|
|
This is the original email announcing the findunref tool. It needs a bit of updating, but still explains the rationale behind the tool. Date: Wed, 21 Mar 2001 22:25:19 -0500 (EST) (19:25:19 PST) From: Peter Memishian <meem at sun dot com> Subject: heads-up: findunref and unreferenced files in ON Gatelings, My recent putback adds a new tool to ON called `findunref'. Simply put, findunref allows both developers and gatekeepers to track down files that are never used during the nightly build process. I hope that each of you will take a minute to get familiar with it and make it part of your development environment. You may wonder why we care about unreferenced files. Simply put, unused files in the build are like unused variables in a program: they are signs of brokenness. While there are some legitimate reasons why files are not used during the build process (see "Permitted Unreferenced Files", below), the vast majority of these files are just crud taking up space in our tree. Others are less benign: unused packaging scripts are signs that something is amiss in a package; unused .xcl files are evidence of programs that have not been correctly internationalized. There are currently a staggering ~1100 accidentally unreferenced files in ON -- that is, roughly 1 out of every 28 files is accidentally unused. This is an unacceptably high number which we will be working to reduce in the coming months (and years). That said, we understand that most of you are hard at work on your own projects and do not have time to help fight a war against the unreferenced file backlog. However, starting as of now, the gatekeepers will be tracking differences in the unused file list as part of the nightly build. Additionally, any putbacks which introduce new unreferenced files into ON will be required to remedy the problem or be subject to backout. Since you will be required to fix up any unreferenced files that result from your putbacks, you may want to run findunref yourself prior to putback. Using Findunref --------------- The findunref tool is quite simple: it merely walks through the files in a source tree and compares the access times of each file against a timestamp file. All files under SCCS control which have not been accessed more recently than the timestamp and are not in a special exception list are considered unreferenced. At the start of each nightly build, nightly touches $SRC/.build.tstamp and then proceeds with the build. Thus, once the build is finished, all files which have not been accessed more recently than $SRC/.build.tstamp were never used by the build. Since findunref's default timestamp file is .build.tstamp at the root of the specified source tree, running findunref on the $SRC of a completed nightly build will list the files that were unused by the build. Unfortunately, a considerable number of files are only used on SPARC or Intel nightly builds (but not both). As a result, findunref must be run on both SPARC and Intel nightly workspaces and the results must be merged to get the correct list of unreferenced files. For instance, if $INTELSRC and $SPARCSRC are set to the usr/src directories of your Intel and SPARC nightly workspaces, then you must merge the results like: $ findunref $INTELSRC exception_list | sort > ~/unref-i386.out $ findunref $SPARCSRC exception_list | sort > ~/unref-sparc.out $ comm -12 ~/unref-i386.out ~/unref-sparc.out > ~/unref.out In the example above, exception_list can be any file (or /dev/null if there are no exceptions), but should be the same exception list that the gate uses: $SRC/tools/findunref/exception_list. You can then diff your results against the gate's current list at /ws/on81-gate/usr/src/unrefmaster.out to see any changes that are due to your work (as an aside, if you get odd results, make sure that the `noatime' option (see mount_ufs(1M)) is turned off on the filesystem that your workspace is on). Since that process is admittedly cumbersome, another approach is to include the new `-f' flag in NIGHTLY_OPTIONS. This option, which is ignored for incremental builds, tracks any differences in the unreferenced file list from the previous nightly and lists those differences in the mail message. Note of course that this may produce false positives: a new file which is only used on Intel will show up in as a new unreferenced file when a SPARC nightly is next run on the workspace. This option also builds $SRC/unref-$(MACH).out in your nightly workspace, which may be compared at any time against the unref-$(MACH).out files in /ws/on81-gate/usr/src to see changes that are due to your work. Project gatekeepers and enterprising developers may also wish to make use of the enhanced `-u' option, which when used with `-f' not only copies the unref-$(MACH).out files to the parent workspace, but also merges all of the parent's unref-$(MACH).out files into a master file (usr/src/unrefmaster.out in the parent) which may be compared against the gate's unrefmaster.out to see any changes that are due to your project's workspace. Permitted Unreferenced Files ---------------------------- As mentioned previously, we understand that not all unreferenced files indicate problems. For instance, projects are encouraged to include READMEs and other textfiles that aid understanding along with their putback. Other examples of permitted unreferenced files include the source for test programs, goop for internal packages, and files which are part of programs which are maintained externally (such as perl and apache). To handle these cases without continually adding exceptions to the exception list, we are asking developers to follow some common naming conventions for these files. For instance, the current exception list causes findunref to ignore any file ending in ".txt" or starting with "README", so as long as you use these naming conventions with your new textfiles, they will automatically be ignored. There are similar conventions for the source for test programs and other files; please see $(SRC)/tools/findunref/exception_list for the current list of permitted exception globs. Please let me know if you have questions, comments, or issues... -- meem |