Source Code Views

You can view a file's source code in five different forms:

1. The plain source code, will only provide you the file's code text
2. The source code with unprocessed regions marked, will enable you to see which parts of the file was not processed due to conditional compilation instructions. You may want to use the marked parts as a guide to construct a more inclusive workspace definition (perhaps by processing the project multiple times, with different preprocessor options).

   360 #if defined(__GNUC__) && defined(__STDC__)   361 static __inline int __sputc(int _c, FILE *_p) {   362         if (--_p->_w >= 0 || (_p->_w >= _p->_lbfsize && (char)_c != '\n'))   363                 return (*_p->_p++ = _c);   364         else   365                 return (__swbuf(_c, _p));   366 }   367 #else   368 /*   369  * This has been tuned to generate reasonable code on the vax using pcc.   370  */   371 #define __sputc(c, p) \   372         (--(p)->_w < 0 ? \   373                 (p)->_w >= (p)->_lbfsize ? \   374                         (*(p)->_p = (c)), *(p)->_p != '\n' ? \   375                                 (int)*(p)->_p++ : \   376                                 __swbuf('\n', p) : \   377                         __swbuf((int)(c), p) : \   378                 (*(p)->_p = (c), (int)*(p)->_p++))   379 #endif   380

3. Source code with identifier hyperlinks, will provide you with a page of the file's code text where each identifier is represented as a hyperlink leading to the identifier's page. The following is a representative example.

   int copy_fifo(from_stat, exists)         struct stat *from_stat;         int exists; {         if (exists && unlink(to.p_path)) {                 warn("unlink: %s", to.p_path);                 return (1);         }         if (mkfifo(to.p_path, from_stat->st_mode)) {                 warn("mkfifo: %s", to.p_path);                 return (1);         }         return (pflag ? setfile(from_stat, 0) : 0); }

4. As the above display can be overwhelming, you may prefer to browse the source code with hyperlinks only to project-global writable identifiers, which are typically the most important identifiers. Consider again how the above example would be displayed:

   int copy_fifo(from_stat, exists)         struct stat *from_stat;         int exists; {         if (exists && unlink(to.p_path)) {                 warn("unlink: %s", to.p_path);                 return (1);         }         if (mkfifo(to.p_path, from_stat->st_mode)) {                 warn("mkfifo: %s", to.p_path);                 return (1);         }         return (pflag ? setfile(from_stat, 0) : 0); }

5. Source code with hyperlinks to function and macro declarations provides you hyperlinks to the function pages for each function declaration (implicit or explict) and macro definition. Again, here is an example:

   #if !defined(_ANSI_SOURCE) && !defined(_POSIX_SOURCE) int     digittoint __P((int)); int     isascii __P((int)); int     isblank __P((int)); int     ishexnumber __P((int)); int     isideogram __P((int)); int     isnumber __P((int)); int     isphonogram __P((int)); int     isrune __P((int)); int     isspecial __P((int)); int     toascii __P((int)); #endif __END_DECLS #define __istype(c,f)    (!!__maskrune((c),(f))) #define isalnum(c)       __istype((c), _CTYPE_A|_CTYPE_D) #define isalpha(c)       __istype((c), _CTYPE_A) #define iscntrl(c)       __istype((c), _CTYPE_C) #define isdigit(c)       __isctype((c), _CTYPE_D) /* ANSI -- locale independent */ #define isgraph(c)       __istype((c), _CTYPE_G) #define islower(c)       __istype((c), _CTYPE_L) #define isprint(c)       __istype((c), _CTYPE_R) #define ispunct(c)       __istype((c), _CTYPE_P) #define isspace(c)       __istype((c), _CTYPE_S) #define isupper(c)       __istype((c), _CTYPE_U) #define isxdigit(c)      __isctype((c), _CTYPE_X) /* ANSI -- locale independent */ #define tolower(c)       __tolower(c) #define toupper(c)       __toupper(c)

File Metrics

All files

Read-only files

Writable files

Files containing unused project-scoped writable identifiers

In our example, the following list is generated:

Files containing unused file-scoped writable identifiers

In our example, the following list is generated:

Writable .c files without any statements

In our example, the result set only contains the processing script (the compiled workspace definition file).

Writable files containing unprocessed lines

In our case the results are:

Writable files containing strings

In our case the results are:

Writable .h files with #include directives

In our example, the result is:

Generic File Queries

You specify the query through the following form:

You start by specifying whether the file should be writable (i.e. typically part of your application) and/or readable (i.e. typically part of the compiler or system). Next come a series of metrics CScout collects for each file. For each metric (e.g. the number of comments) you can specify an operator ==, !=, < or > and a number to match that metric against. Thus to locate files without any comments you would specify
Number of block comments == 0.

On the left of each metric you can specify whether that metric will be used to sort the resulting file list. In that case, the corresponding number will appear together with each file listed. A separate option allows you to specify that files should be sorted in the reverse order.

You can request to see files matching any of your specifications (Match any of the above) or to see files matching all your specifications (Match all of the above).

Sometimes you may only want to search in a subset of files; you can then specify a regular expression that filenames should match (or not match) against: "File names should (not) match RE".

Finally, you can also specify a title for your query. The title will then appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

Include Graphs

Two global options specify the format of the include graph and the content on each graph's node. Through these options you can obtain graphs in

• plain text form: suitable for processing with other tools,
• HTML: suitable for browsing via CScout,
• dot: suitable for generating high-quality graphics files,
• SVG: suitable for graphical browsing, if your browser supports this format, and
• GIF: suitable for viewing on SVG-challenged browsers.

including file -> included file

Two links on the main page (file include graph - writable files and file include graph - all files) can give you the include graphs of the complete program. For programs larger than a hundred thousand lines, these graphs are only useful in their textual form. In their graphical form, even with node information disabled, they can only serve to give you a rough idea of how the program is structured. The following image depicts how writable (non-system) files are included in the awk source code.

and the following is a part of the include file structure of the Windows Research Kernel

More useful are typically the include graphs that can be generated for individual files. These can allow you to see what paths can possibly lead to the inclusion of a given file (include graph of all including files) or what files a given file includes (include graph of all included files). (call graph of all callers), which functions can be reached starting from a given function, and how functions in a given file relate to each other.

As an example, the following diagram depicts all files that main.c includes

while the following diagrams shows all the files including (directly or indirectly) proto.h.

C Namespaces

int i = 3;

foo()
{
	int i = 7;
}

main()
{
	foo();
	printf("%d\n", i);
}

In addition, C also partitions a program's identifiers into four namespaces. Identifiers in one namespace, are also considered different from identifiers in another. The four namespaces are:

1. Tags for a struct/union/enum
2. Members of struct/union (actually a separate namespace is assigned to each struct/union)
3. Labels
4. Ordinary identifiers (termed objects in the C standard)

/* structure tag */
struct id {
	int id;		/* structure member */
};

/* Different structure */
struct id2 {
	char id;	/* structure member */
};

/* ordinary identifier */
id()
{
id:	/* label */
}

Normally when you want to locate or change an identifier name, you only consider identifiers in the same scope and namespace. Sometimes however, a C preprocessor macro can semantically unite identifiers living in different namespaces, so that changes in one of them should be propagated to the others. The most common case involves macros that access structure members.

struct s1 {
	int id;
} a;

struct s2 {
	char id;
} b;

#define getid(x) ((x)->id)

main()
{
	printf("%d %c", getid(a), getid(b));
}

Finally, the C preprocessor's token concatenation feature can result in identifiers that should be treated for substitution purposes in separate parts. Consider the following example:

int xleft, xright;
int ytop, ybottom;

#define coord(a, b) (a ## b)

main()
{
	printf("%d %d %d %d\n",
		coord(x, left),
		coord(x, right),
		coord(y, top),
		coord(y, bottom));
}

Identifier Elements

• If the identifier is read-only (i.e. it appears in at least one read-only file)
• The C namespace(s) it appears in (the same identifier can be a member of multiple namespaces)
• Whether the identifier is visible at file or project scope
• Whether the identifier is a typedef (typedef's belong to the ``ordinary identifier'' namespace, but are obviously important, so CScout will tag them as such).
• Whether the identifier crosses a file boundary, i.e. it appears in more than one file
• Whether the identifier is unused i.e. it appears in exactly one location
• The identifier's number of occurences in all the workspace's files
• The projects the identifier appears in
• The function names the identifier forms; the link "function page" will provide you more details regarding the function
• An option to substitute the identifier's name with a different name

Finally, the identifier's page will list the writable and all files the specific identifier appears in. Clicking on the ``marked source'' hyperlink will display the respective file's source code with only the given identifier marked as a hyperlink. By pressing your browser's tab key you can then see where the given identifier is used. In our example the cp.c source code with the copy_file identifier marked would appear as follows:

Identifier Metrics

You can use these metrics to compare characteristics of different projects, adherance to coding standards, or to identify identifier classes with abnormally short or long names. The ratio between the distinct number of identifiers and the total number of identifiers is the number of times each identifier is used. Notice the difference in our case between the read-only identifiers (which are mostly declarations) and the writable identifiers (which are actually used).

All identifiers

Read-only identifiers

Writable identifiers

File-spanning writable identifiers

Unused project-scoped writable identifiers

Unused file-scoped writable identifiers

Unused writable macros

Generic Identifier Queries

• The properties (namespace, scope, instances) of the identifier
• Whether the specified properties should be treated
  • as a disjunction (match any marked),
  • as a conjunction (match all marked),
  • as a negation excluding all identifiers matching any property (exclude marked), or
  • as an exact match specification matching only identifiers that match exactly the properties specified (exact match)
• A regular expression against which identifier names should match (or not match)
• A regular expression that filenames in which identifiers occur should match (or not match)
• A query title to be used for naming the result page. The title will appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

• the identifiers that match the specific query,
• the files containing identifiers that match the query, or
• the functions containing identifiers that match the query.

As an example, the following query could be used to identify unused file-scoped writable identifiers, but excluding the copyright and rcsid identifiers:

Function Elements

From this page you can refactor the function's arguments (more on this in the next section) and obtain the following data.

• The identifier or identifiers composing the function name. These can be modified (from the corresponding identifier page) to change the function's name.
• The function's declaration. This may be an implicit declaration (the location of its first use). CScout only maintains the location of one declaration for each function. You can locate additional points of declaration by looking at the places where the corresponding identifier is used. The "marked source" link allows you to see the declaration as a hyperlink in the file where it occurs. In many browsers pressing the tab key on that page will lead you directly to the function's declaration.
• The function's definition (if a definition was found). Library functions obviously will not have a definition associated with them.
• The number of functions this function directly calls. These are the functions (C functions and function-like macros) that appear inside the function's body.
• A list allowing you to explore interactively the tree of called functions. The tree will appear in the following form:
  
  Each plus or minus box will open or close the list of called functions. Each function name is a hyperlink to the corresponding function page.
• A list of all called functions. This list includes all functions that can be called, starting from the function we are examining. On the right of each function is a hyperlink to a call graph of the path(s) leading from the function being examined to the function listed.
• A call graph of all called functions (explained in a following section).
• A page allowing you to explore interactively all callers. These are the functions that directly call the function we are examining. The functionality of this page is the same as that of the one for exploring the called functions.
• A list of all callers. These are all functions that can directly or indirectly call the function we are examining.
• A call graph of all callers (explained in a following section).
• A call graph of all the function's callers and called functions showing the function in context (explained in a following section).
• A comprehensive set of metrics regarding the function (only for defined functions and macros).

All Functions

Project-scoped writable functions

File-scoped writable functions

Writable functions that are not directly called

Writable functions that are called exactly once

Refactoring Function Arguments

To refactor the function's arguments, one simply enters in the text box a template describing the argument replacement pattern. The template consists of text, which is copied verbatim as a function's argument, and elements starting with the operator @, which have a special meaning. The combined effect of this template mechanism allows you to

• Introduce new arguments
• Remove existing arguments
• Change the arguments' order

@N

pastes the original Nth argument passed to the function. Thus, @1 will get replaced with the function's first argument. Specifying in a template for a function taking two arguments "@2, @1" will swap their order, while specifying "@1, sizeof(@1), stdin" as the arguments for gets will refactor them in a form suitable for calling fgets (if the original argument refers to a fixed-size character array).

@.N

pastes the Nth argument and all subsequent ones, separated by commas. This is useful for handling functions with a variable number of arguments, like printf. Specifying in a template for the printf function "stdout, @1, @.2" will introduce an extra first parameter, named stdout. (Presumably the function will also be renamed to fprintf.)

@+N{...}

pastes the text in the braces, if the specific function being replaced has an Nth argument. The text in the braces can include arbitrary text, including nested @ operators.

@-N{...}

pastes the text in the braces, if the specific function being replaced does not have an Nth argument.

Note that the refactorings will take place on all instances where the identifier is found to match the function or macro. This includes declarations and definitions (which might require some hand-editing if arguments are introduced), and the appearance of the name in the replacement text of a macro, when that macro is used in a way that makes the function match the one being refactored. The replacements will not be performed to function calls that are executed through a function pointer.

Call Graphs

Two global options specify the format of the call graph and the content on each graph's node. Through these options you can obtain graphs in

• plain text form: suitable for processing with other tools,
• HTML: suitable for browsing via CScout,
• dot: suitable for generating high-quality graphics files,
• SVG: suitable for graphical browsing, if your browser supports this format, and
• GIF: suitable for viewing on SVG-challenged browsers.

calling function -> called function

Two links on the main page (function and macro call graph, and non-static function call graph) can give you the call graphs of the complete program. For any program larger than a few thousand lines, these graphs are only useful in their textual form. In their graphical form, even with node information disabled, they can only serve to give you a rough idea of how the program is structured. The following image depicts how the three different programs we analyzed in the bin example relate to each other.

More useful are the call graphs that can be generated for individual functions or files. These can allow you to see what paths can possibly lead to a given function (call graph of all callers), which functions can be reached starting from a given function, the function in context, and how functions in a given file relate to each other.

As an example, the following diagram depicts all paths leading to the setfile function.

Correspondingly, the functions that can be reached starting from the copy_file function appears in the following diagram.

while the following shows the function setsymtab in context, depicting all the paths leading to it (callers) and leaving from it (called functions).

Finally, the following is an example of how the functions in a single file (parse.c) relate to each other.

Generic Function Queries

On the top you can specify whether each function you want listed:

• is a C function
• is a function-like macro
• has a writable declaration
• has a read-only declaration
• is visible in the whole project scope
• is visible only in a file scope
• has a definition body.

As is the case in file queries, next comes a series of metrics CScout collects for each defined function. For each metric (e.g. the number of comments) you can specify an operator ==, !=, < or > and a number to match that metric against. Thus to locate functions containing goto statement you would specify
Number of goto statements != 0.

On the left of each metric you can specify whether that metric will be used to sort the resulting file list. In that case, the corresponding number will appear together with each file listed. A separate option allows you to specify that files should be sorted in the reverse order.

Similarly to the identifier query, you can also specify whether the specified properties should be treated

• as a disjunction (match any marked),
• as a conjunction (match all marked),
• as a negation excluding all identifiers matching any property (exclude marked), or
• as an exact match specification matching only identifiers that match exactly the properties specified (exact match)

In addition you can specify:

• That the function should have a specified number of direct callers.
• A regular expression against which function names should match (or not match)
• A regular expression against which the names of calling functions should match (or not match)
• A regular expression against which the names of any called functions should match (or not match)
• A regular expression that filenames in which functions are declared occur should match (or not match)
• A query title to be used for naming the result page. The title will appear on the result document annotating the results, and will also provide you with a sensible name when creating a bookmark to it.

Global Options

File and Identifier Pages

Show Only True Identifier Classes

Show Associated Projects

Show Lists of Identical Files

Source Listings

Show Line Numbers

Tab Width

Refactoring

Allow the renaming of read-only identifiers

Allow the refactoring of function arguments of read-only functions

Check for renamed identifier clashes when saving refactored code

Queries

Case-insensitive File Name Regular Expression Match

Query Result Lists

Number of Entries on a Page

Show File Lists With File Name in Context

Sort Identifiers Starting from their Last character

Call and File Dependency Graphs

Call Graph Links Should Lead to Pages of

• Plain text: suitable for processing with other text tools.
• HTML: suitable for interactive browsing
• dot: suitable for processing with GraphViz dot into different graphics formats, like PNG, MIF, VRML, and EPS. Dot files can also be processed as graphs using the AT&T gpr program
• SVG: suitable for interactively browsing the graphical representation of the call graph. This option requires your browser to support the rendering of SVG (directly or via a plugin, such as Adobe's (http://www.adobe.com/svg/)), and the existence of the AT&T GraphViz (http://www.graphviz.org) dot program in your executable file path.
• GIF: suitable for directly viewing relatively small images.

Call Graphs Should Contain

• Only edges, will not display anything on the node. This option can be used in the graphics formats (dot, SVG, GIF) to provide an overall picture of the program's call structure.
• Function names: only include the function names. Functions with the same name will still be separately listed, but you will have to follow their hyperlinks to see where they are defined.
• File and function names: the name of the file where a function is declared will precede the name of the function.
• Path and function names: the complete file path of the file where a function is declared will precede the name of the function.

File Graphs Should Contain

• Only edges, will not display anything on the node. This option can be used in the graphics formats (dot, SVG, GIF) to provide an overall picture of the program's file dependency structure.
• File names: only include the file names. Files with the same name will still be separately listed, but you will have to follow their hyperlinks to see where they are defined.
• Path and file names: the complete path of each path will be show.

Maximum number of call levels in a graph

Maximum dependency depth in a file graph

Include URLs in dot output

Graph options

Node options

Edge options

The graph, node, and edge options can be used to fine tune the graph's look. See the GraphViz documentation (http://www.graphviz.org/doc/info/attrs.html) for more details. For instance, the following diagram

was created using

Saved Files

When Saving Modified Files Replace

• continue operating CScout, even after the changes have been saved, and
• easilly back out changes your are not satisfied with.

Note than when this option is specified the existing and new locations of the file must reside on the same drive and partition (under Windows) or file system (under Unix).

Editing

xterm -c "$VISUAL +/'%s' '%s'"

echo Ignoring search for "%s" & start notepad "%s"

start  C:\Progra~1\Vim\vim70\gvim.exe +/"%s" "%s"

Option Files

When CScout starts-up it will attempt to load the options file by searching in $CSCOUT_HOME, $HOME/.cscout, or .cscout in the current directory.

The options file is text based and contains key-value pairs. The order of the entries is not significant. This is an example of an options file.

show_true: 1
show_projects: 1
show_identical_files: 1
show_line_number: 0
tab_width: 8
rename_override_ro: 1
refactor_fun_arg_override_ro: 1
file_icase: 0
entries_per_page: 20
fname_in_context: 1
sort_rev: 0
cgraph_type: s
cgraph_show: n
fgraph_show: n
cgraph_depth: 5
fgraph_depth: 5
cgraph_dot_url: 0
sfile_re_string: sfile_repl_string:
sfile_repl_string: entries_per_page:
start_editor_cmd: start  C:\Progra~1\Vim\vim71\gvim.exe +/"%s" "%s"

Operations

Identifier Replacements

The following is an example of the identifier replacements page. You see all identifiers for which replacements have been specified. All specified replacements are originally active. If a particular replacement appears to be causing problems you can deactivate it from this page. In addition, you can change the replaced name of any of the replaced identifiers. Finally, clicking on an identifier name will lead you to the corresponding identifier page.

Select Active Project

Save Changes and Continue

Exit - Saving Changes

Exit - Ignore Changes

Hand-Editing

The automatic global identifier replacement and the hand-editing of files are mutualy exclusive operations. Once either of the two is performed the other ceases to be available. This is done to protect the integrity of the underlying source code. Furthermore, all CScout's operations, such as queries and source code listings, are always performed on a snapshot of the source code taken just before a file is edited by hand.

Interfacing with Version Management Systems

As an example, for a system managed with Perforce (https://www.perforce.com/) the following commands could be used:

cscout_checkout

#!/bin/sh
p4 edit $1

cscout_checkin

#!/bin/sh
p4 submit -d 'CScout identifier name refactoring' $1

Language Extensions

1. Preprocessor directives can appear within a call to a function-like macro (gcc)
2. Initializers and compound literals can be empty (gcc)
3. The alignof operator can be used on types (gcc)
4. A declaration expression as the first element of a for statement (C99)
5. The restrict qualifier and the inline specifier (C99)
6. The _Bool type (C99)
7. The _Thread_local storage class specifier (C11)
8. The _Generic generic selection expression (C11)
9. Support static and other qualifiers in array type declarations (C99)
10. Initialization designators (C99)
11. Array initialization designators can include ranges (gcc)
12. ANSI-style function definitions can be nested (gcc) (gcc also allows nested K&R-style function definitions)
13. The equals sign following an initializer designator is optional (gcc)
14. Array and structure initialization (gcc)
15. Compound literals (C99)
16. Declarations can be intermixed with statements (C99).
17. Recognise __atribute__(__unused__) for determining which identifiers should not be reported as unused (gcc).
18. // line comments (common extension)
19. __asm__ blocks (gcc)
20. enum lists ending with a comma (common extension)
21. Anonymous struct/union members (gcc, Microsoft C)
22. Allow case expression ranges (gcc).
23. An enumeration list can be empty (Microsoft C)
24. Allow braces around scalar initializers (common extension).
25. Indirect goto targets and the label address-of operator (gcc).
26. __typeof keyword (gcc)
27. A compound statement in brackets can be an expression (gcc)
28. Macros expanding from /##/ into // are then treated as a line comment (Microsoft C)
29. Exception handling using the __try __except __finally __leave keywords (Microsoft C)
30. #include_next preprocessor directive (gcc)
31. #warning preprocessor directive (gcc)
32. Variable number of arguments preprocessor macros (support for both the gcc and the C99 syntax)
33. Allow empty member declarations in aggregates (gcc)
34. long long type (common extension)
35. A semicolon can appear as a declatation (common extension)
36. An aggregate declaration body can be empty (gcc)
37. Aggregate member initialization using the member: value syntax (gcc)
38. Statement labels do not require a statement following them (gcc)
39. #ident preprocessor directive (gcc)
40. Allow assignment to case expressions (common extension)
41. Accept an empty translation unit (common extension).
42. Support locally declared labels (__label__) (gcc).
43. The second argument of a conditional expression can be omitted (gcc).
44. Dereferencing a function yields a function (common extension).

Processing Yacc Files

CScout processes yacc files as follows:

• All terminal and non-terminal names are considered to be defined as file-scoped identifiers in a special yacc-only namespace.
• All terminal symbols are also marked as an undefined macro. Thus, processing the yacc-generated file y.tab.h immediately after the grammar (i.e. in the same scope) will unify the terminal symbols with the corresponding macro definitions throughout the program.
• Members of the %union construct are defined as members of the YYSTYPE union typedef. These are then considered to be accessed in all %type and the precedence specification constructs, as well as the explicit type specification through the $<tag># construct.
• No yacc-specific macros (yyerrok, YYABORT, YYACCEPT, etc) are defined. Feel free to define anything required to silence CScout as a macro in the workspace definition file.

CScout is designed to process well-formed modern-style yacc files. All rules must be terminated with a semicolon (apparently this is optional in the original yacc version). The accepted grammar appears below.

body:
	defs '%%' rules tail
	;

tail:
	/* Empty */
	| '%%' c_code
	;

defs:
	/* Empty */
	| defs def
	;

def:
	'%start' IDENTIFIER
	| '%union' '{' member_declaration_list  '}' 
	| '%{' c_code '%}'
	| rword name_list_declaration
	;

rword:
	'%token'
	| '%left'
	| '%right'
	| '%nonassoc'
	| '%type'
	;

tag:
	/* Empty: union tag is optional */
	| '<' IDENTIFIER '>'
	;

name_list_declaration:
	tag name_number
	| name_list_declaration opt_comma name_number
	;

opt_comma:
	/* Empty */
	| ','
	;

name_number:
	name
	| name INT_CONST
	;

name:
	IDENTIFIER
	| CHAR_LITERAL
	;

/* rules section */

rules:
	rule
	| rules rule
	;

rule:
	IDENTIFIER ':'  rule_body_list ';'
	;

rule_body_list:
	rule_body
	| rule_body_list '|' rule_body
	;

rule_body:
	id_action_list prec
	;

id_action_list:
	/* Empty */
	| id_action_list name
        | id_action_list '{' c_code '}' 
	;

prec:
	/* Empty */
	| '%prec' name
	| '%prec' name  '{' c_code '}'
	;

variable:
	'$$'
	| '$' INT_CONST
	| '$-' INT_CONST
		{ $$ = basic(b_int); }
	| '$<' IDENTIFIER '>' variable_suffix
		{ $$ = $3; }
	;

variable_suffix:
	'$'
	| INT_CONST
	| '-' INT_CONST
	;

Regular Expression Syntax

Regular expressions (``REs''), as defined in IEEE Std 1003.2 (``POSIX.2''), come in two forms: modern REs (roughly those of egrep(1); 1003.2 calls these ``extended'' REs) and obsolete REs (roughly those of ed(1); 1003.2 ``basic'' REs). CScout has adopted the use of modern (extended) REs.

A (modern) RE is one= or more non-empty= branches, separated by `|'. It matches anything that matches one of the branches.

A branch is one= or more pieces, concatenated. It matches a match for the first, followed by a match for the second, etc.

A piece is an atom possibly followed by a single= `*', `+', `?', or bound. An atom followed by `*' matches a sequence of 0 or more matches of the atom. An atom followed by `+' matches a sequence of 1 or more matches of the atom. An atom followed by `?' matches a sequence of 0 or 1 matches of the atom.

A bound is `{' followed by an unsigned decimal integer, possibly followed by `,' possibly followed by another unsigned decimal integer, always fol- lowed by `}'. The integers must lie between 0 and RE_DUP_MAX (255=) inclusive, and if there are two of them, the first may not exceed the second. An atom followed by a bound containing one integer i and no comma matches a sequence of exactly i matches of the atom. An atom fol- lowed by a bound containing one integer i and a comma matches a sequence of i or more matches of the atom. An atom followed by a bound containing two integers i and j matches a sequence of i through j (inclusive) matches of the atom.

An atom is a regular expression enclosed in `()' (matching a match for the regular expression), an empty set of `()' (matching the null string)=, a bracket expression (see below), `.' (matching any single character), `^' (matching the null string at the beginning of a line), `$' (matching the null string at the end of a line), a `\' followed by one of the characters `^.[$()|*+?{\' (matching that character taken as an ordinary character), a `\' followed by any other character= (matching that character taken as an ordinary character, as if the `\' had not been present=), or a single character with no other significance (matching that character). A `{' followed by a character other than a digit is an ordinary character, not the beginning of a bound=. It is illegal to end an RE with `\'.

A bracket expression is a list of characters enclosed in `[]'. It nor- mally matches any single character from the list (but see below). If the list begins with `^', it matches any single character (but see below) not from the rest of the list. If two characters in the list are separated by `-', this is shorthand for the full range of characters between those two (inclusive) in the collating sequence, e.g. `[0-9]' in ASCII matches any decimal digit. It is illegal= for two ranges to share an endpoint, e.g. `a-c-e'. Ranges are very collating-sequence-dependent, and portable programs should avoid relying on them.

To include a literal `]' in the list, make it the first character (fol- lowing a possible `^'). To include a literal `-', make it the first or last character, or the second endpoint of a range. To use a literal `-' as the first endpoint of a range, enclose it in `[.' and `.]' to make it a collating element (see below). With the exception of these and some combinations using `[' (see next paragraphs), all other special charac- ters, including `\', lose their special significance within a bracket expression.

Within a bracket expression, a collating element (a character, a multi- character sequence that collates as if it were a single character, or a collating-sequence name for either) enclosed in `[.' and `.]' stands for the sequence of characters of that collating element. The sequence is a single element of the bracket expression's list. A bracket expression containing a multi-character collating element can thus match more than one character, e.g. if the collating sequence includes a `ch' collating element, then the RE `[[.ch.]]*c' matches the first five characters of `chchcc'.

Within a bracket expression, a collating element enclosed in `[=' and `=]' is an equivalence class, standing for the sequences of characters of all collating elements equivalent to that one, including itself. (If there are no other equivalent collating elements, the treatment is as if the enclosing delimiters were `[.' and `.]'.) For example, if `x' and `y' are the members of an equivalence class, then `[[=x=]]', `[[=y=]]', and `[xy]' are all synonymous. An equivalence class may not= be an end- point of a range.

Within a bracket expression, the name of a character class enclosed in `[:' and `:]' stands for the list of all characters belonging to that class. Standard character class names are:

   alnum    digit    punct
   alpha    graph    space
   blank    lower    upper
   cntrl    print    xdigit

There are two special cases= of bracket expressions: the bracket expres- sions `[[:<:]]' and `[[:>:]]' match the null string at the beginning and end of a word respectively. A word is defined as a sequence of word characters which is neither preceded nor followed by word characters. A word character is an alnum character (as defined by ctype(3)) or an underscore. This is an extension, compatible with but not specified by IEEE Std 1003.2 (``POSIX.2''), and should be used with caution in soft- ware intended to be portable to other systems.

In the event that an RE could match more than one substring of a given string, the RE matches the one starting earliest in the string. If the RE could match more than one substring starting at that point, it matches the longest. Subexpressions also match the longest possible substrings, subject to the constraint that the whole match be as long as possible, with subexpressions starting earlier in the RE taking priority over ones starting later. Note that higher-level subexpressions thus take priority over their lower-level component subexpressions.

Match lengths are measured in characters, not collating elements. A null string is considered longer than no match at all. For example, `bb*' matches the three middle characters of `abbbc', `(wee|week)(knights|nights)' matches all ten characters of `weeknights', when `(.*).*' is matched against `abc' the parenthesized subexpression matches all three characters, and when `(a*)*' is matched against `bc' both the whole RE and the parenthesized subexpression match the null string.

If case-independent matching is specified, the effect is much as if all case distinctions had vanished from the alphabet. When an alphabetic that exists in multiple cases appears as an ordinary character outside a bracket expression, it is effectively transformed into a bracket expres- sion containing both cases, e.g. `x' becomes `[xX]'. When it appears inside a bracket expression, all case counterparts of it are added to the bracket expression, so that (e.g.) `[x]' becomes `[xX]' and `[^x]' becomes `[^xX]'.

Access Control

To allow other hosts to connect CScout features an access control list. The list is specified in a file called acl which should be located in $CSCOUT_HOME, $HOME/.cscout, or .cscout in the current directory. The list contains lines with IP numeric addresses prefixed by an A (allow) or D (deny) prefix and a space. Matching is performed by comparing a substring of a machine's IP address against the specified access rule. Thus an entry such as

A 128.135.11.

• If the address matches an allowed entry the machine will be allowed access.
• If no allowed entries have been specified, the machine will be allowed access unless it has been denied access. (i.e. you can not use a deny entry to exclude a machine from an allowed group)

Obfuscation Back-end

Each source code file is obfuscated by

• Giving meaningless names to all identifiers: macros, arguments, ordinary identifiers, structure tags, structure members, and labels.
• Removing comments.
• Removing extraneous whitespace.

Before running CScout to obfuscate, make a complete backup copy of your source code files, and store them in a secure place; preferably off-line. Once the source code files are obfuscated and overwritten, there is no way to get back their original contents.

To obfuscate the workspace, first ensure that CScout can correctly process the complete set of its source code files. Use the "unprocessed lines" metric of each file to verify that no parts of a file are left unprocessed; unprocessed regions will not be obfuscated. You can easily increase the coverage of CScout's processing by including in the workspace multiple projects with different defined directives.

Also ensure that all your project's files are considered writable, and no files outside your project (for example system headers) are considered writable. This will allow CScout to rename your identifier names, but keep the names of library-defined identifiers (for example printf) unchanged.

Finally, run CScout with the switch -o. For each writable workspace file CScout will create a file ending in .obf that will contain the obfuscated version of its contents. The files are not overwritten, providing you with another countermeasure against accidentally destroying them. To overwrite the original source with the obfuscated one, use the following Unix command:

find . -name '*.obf' |
sed 's/\\/\//g;s/\(.*\)\.obf$/mv "\1.obf" "\1"/' |
sh

SQL Back-end

To generate the SQL script simply run CScout with the switch -s dialect, where the argument specifies the SQL dialect (for example, mysql, or postgresql). The SQL script will appear in CScout's standard output, allowing you to directly pipe the results into the database's client. For example, say the database you would want to create for your project was called myproj.
For SQLite you would write:

rm -f myproj.db
cscout -s sqlite myproj.cs | sqlite3 myproj.db

(
	echo "create database myproj; use myproj ;"
	cscout -s mysql myproj.cs
) | mysql

createdb -U username myproj
cscout -s postgres myproj.cs | psql -U username myproj

cscout -s hsqldb myproj.cs |
java -classpath $HSQLDBHOME/lib/hsqldb/hsqldb.jar org.hsqldb.util.SqlTool --rcfile $HSQLDBHOME/lib/hsqldb/sqltool.rc mem -

Schema of the Generated Database

-- Details of interdependant identifiers appearing in the workspace
CREATE TABLE IDS(
  EID BIGINT PRIMARY KEY, -- Unique identifier key
  NAME CHARACTER VARYING, -- Identifier name
  READONLY BOOLEAN, -- True if it appears in at least one read-only file
  UNDEFMACRO BOOLEAN, -- True if it is apparantly an undefined macro
  MACRO BOOLEAN, -- True if it a preprocessor macro
  MACROARG BOOLEAN, -- True if it a preprocessor macro argument
  ORDINARY BOOLEAN, -- True if it is an ordinary identifier (variable or function)
  SUETAG BOOLEAN, -- True if it is a structure, union, or enumeration tag
  SUMEMBER BOOLEAN, -- True if it is a structure or union member
  LABEL BOOLEAN, -- True if it is a label
  TYPEDEF BOOLEAN, -- True if it is a typedef
  ENUM BOOLEAN, -- True if it is an enumeration member
  YACC BOOLEAN, -- True if it is a yacc identifier
  FUN BOOLEAN, -- True if it is a function name
  CSCOPE BOOLEAN, -- True if its scope is a compilation unit
  LSCOPE BOOLEAN, -- True if it has linkage scope
  UNUSED BOOLEAN -- True if it is not used
);


-- File details
CREATE TABLE FILES(
  FID INTEGER PRIMARY KEY, -- Unique file key
  NAME CHARACTER VARYING, -- File name
  RO BOOLEAN -- True if the file is read-only
);


-- File metrics
CREATE TABLE FILEMETRICS(
  FID INTEGER, -- File key
  PRECPP BOOLEAN, -- True for values before the cpp false for values after it
  NCHAR INTEGER, -- Number of characters
  NCCOMMENT INTEGER, -- Number of comment characters
  NSPACE INTEGER, -- Number of space characters
  NLCOMMENT INTEGER, -- Number of line comments
  NBCOMMENT INTEGER, -- Number of block comments
  NLINE INTEGER, -- Number of lines
  MAXLINELEN INTEGER, -- Maximum number of characters in a line
  MAXSTMTLEN INTEGER, -- Maximum number of tokens in a statement
  MAXSTMTNEST INTEGER, -- Maximum level of statement nesting
  MAXBRACENEST INTEGER, -- Maximum level of brace nesting
  MAXBRACKNEST INTEGER, -- Maximum level of bracket nesting
  BRACENEST INTEGER, -- Dangling brace nesting
  BRACKNEST INTEGER, -- Dangling bracket nesting
  NULINE INTEGER, -- Number of unprocessed lines
  NPPDIRECTIVE INTEGER, -- Number of C preprocessor directives
  NPPCOND INTEGER, -- Number of processed C preprocessor conditionals (ifdef, if, elif)
  NPPFMACRO INTEGER, -- Number of defined C preprocessor function-like macros
  NPPOMACRO INTEGER, -- Number of defined C preprocessor object-like macros
  NTOKEN INTEGER, -- Number of tokens
  NSTMT INTEGER, -- Number of statements or declarations
  NOP INTEGER, -- Number of operators
  NUOP INTEGER, -- Number of unique operators
  NNCONST INTEGER, -- Number of numeric constants
  NCLIT INTEGER, -- Number of character literals
  NSTRING INTEGER, -- Number of character strings
  NPPCONCATOP INTEGER, -- Number of token concatenation operators (##)
  NPPSTRINGOP INTEGER, -- Number of token stringification operators (#)
  NIF INTEGER, -- Number of if statements
  NELSE INTEGER, -- Number of else clauses
  NSWITCH INTEGER, -- Number of switch statements
  NCASE INTEGER, -- Number of case labels
  NDEFAULT INTEGER, -- Number of default labels
  NBREAK INTEGER, -- Number of break statements
  NFOR INTEGER, -- Number of for statements
  NWHILE INTEGER, -- Number of while statements
  NDO INTEGER, -- Number of do statements
  NCONTINUE INTEGER, -- Number of continue statements
  NGOTO INTEGER, -- Number of goto statements
  NRETURN INTEGER, -- Number of return statements
  NPID INTEGER, -- Number of project-scope identifiers
  NFID INTEGER, -- Number of file-scope (static) identifiers
  NMID INTEGER, -- Number of macro identifiers
  NID INTEGER, -- Total number of object and object-like identifiers
  NUPID INTEGER, -- Number of unique project-scope identifiers
  NUFID INTEGER, -- Number of unique file-scope (static) identifiers
  NUMID INTEGER, -- Number of unique macro identifiers
  NUID INTEGER, -- Number of unique object and object-like identifiers
  NLABEL INTEGER, -- Number of goto labels
  NMACROEXPANDTOKEN INTEGER, -- Tokens added by macro expansion
  NCOPIES INTEGER, -- Number of copies of the file
  NINCFILE INTEGER, -- Number of directly included files
  NPFUNCTION INTEGER, -- Number of defined project-scope functions
  NFFUNCTION INTEGER, -- Number of defined file-scope (static) functions
  NPVAR INTEGER, -- Number of defined project-scope variables
  NFVAR INTEGER, -- Number of defined file-scope (static) variables
  NAGGREGATE INTEGER, -- Number of complete aggregate (struct/union) declarations
  NAMEMBER INTEGER, -- Number of declared aggregate (struct/union) members
  NENUM INTEGER, -- Number of complete enumeration declarations
  NEMEMBER INTEGER, -- Number of declared enumeration elements
  PRIMARY KEY(FID, PRECPP),
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);


-- Instances of identifier tokens within the source code
CREATE TABLE TOKENS(
  FID INTEGER, -- File key
  FOFFSET INTEGER, -- Offset within the file
  EID BIGINT, -- Identifier key
  PRIMARY KEY(FID, FOFFSET),
  FOREIGN KEY(FID) REFERENCES FILES(FID),
  FOREIGN KEY(EID) REFERENCES IDS(EID)
);


-- Comments in the code
CREATE TABLE COMMENTS(
  FID INTEGER, -- File key
  FOFFSET INTEGER, -- Offset within the file
  COMMENT CHARACTER VARYING, -- The comment, including its delimiters
  PRIMARY KEY(FID, FOFFSET),
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);


-- Strings in the code
CREATE TABLE STRINGS(
  FID INTEGER, -- File key
  FOFFSET INTEGER, -- Offset within the file
  STRING CHARACTER VARYING, -- The string, including its delimiters
  PRIMARY KEY(FID, FOFFSET),
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);


-- Remaining, non-identifier source code
CREATE TABLE REST(
  FID INTEGER, -- File key
  FOFFSET INTEGER, -- Offset within the file
  CODE CHARACTER VARYING, -- The actual code
  PRIMARY KEY(FID, FOFFSET),
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);


-- Line number offsets within each file
CREATE TABLE LINEPOS(
  FID INTEGER, -- File key
  FOFFSET INTEGER, -- Offset within the file
  LNUM INTEGER, -- Line number (starts at 1)
  PRIMARY KEY(FID, FOFFSET),
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);


-- Project details
CREATE TABLE PROJECTS(
  PID INTEGER PRIMARY KEY, -- Unique project key
  NAME CHARACTER VARYING -- Project name
);


-- Identifiers appearing in projects
CREATE TABLE IDPROJ(
  EID BIGINT, -- Identifier key
  PID INTEGER, -- Project key
  FOREIGN KEY(EID) REFERENCES IDS(EID),
  FOREIGN KEY(PID) REFERENCES PROJECTS(PID)
);


-- Files used in projects
CREATE TABLE FILEPROJ(
  FID INTEGER, -- File key
  PID INTEGER, -- Project key
  FOREIGN KEY(FID) REFERENCES FILES(FID),
  FOREIGN KEY(PID) REFERENCES PROJECTS(PID)
);


-- Foreign keys for the following four tables are not specified, because it is
-- difficult to satisfy integrity constraints: files (esp. their metrics,
-- esp. ncopies) can't be written until the end of processing, while
-- to conserve space, these tables are written after each file is processed.
-- Alternatively, inserts to these tables could be wrapped into
-- SET REFERENTIAL_INTEGRITY { TRUE | FALSE } calls.


-- Included files defining required elements for a given compilation unit and project
CREATE TABLE DEFINERS(
  PID INTEGER, -- Project key
  CUID INTEGER, -- Compilation unit key
  BASEFILEID INTEGER, -- File (often .c) requiring (using) a definition
  DEFINERID INTEGER -- File (often .h) providing a definition
  -- FOREIGN KEY(PID) REFERENCES PROJECTS(PID),
  -- FOREIGN KEY(CUID) REFERENCES FILES(FID),
  -- FOREIGN KEY(BASEFILEID) REFERENCES FILES(FID),
  -- FOREIGN KEY(DEFINERID) REFERENCES FILES(FID)
);


-- Included files including files for a given compilation unit and project
CREATE TABLE INCLUDERS(
  PID INTEGER, -- Project key
  CUID INTEGER, -- Compilation unit key
  BASEFILEID INTEGER, -- File included in the compilation
  INCLUDERID INTEGER -- Files that include it
  -- FOREIGN KEY(PID) REFERENCES PROJECTS(PID),
  -- FOREIGN KEY(CUID) REFERENCES FILES(FID),
  -- FOREIGN KEY(BASEFILEID) REFERENCES FILES(FID),
  -- FOREIGN KEY(INCLUDERID) REFERENCES FILES(FID)
);


-- Included files providing code or data for a given compilation unit and project
CREATE TABLE PROVIDERS(
  PID INTEGER, -- Project key
  CUID INTEGER, -- Compilation unit key
  PROVIDERID INTEGER -- Included file
  -- FOREIGN KEY(PID) REFERENCES PROJECTS(PID),
  -- FOREIGN KEY(CUID) REFERENCES FILES(FID),
  -- FOREIGN KEY(PROVIDERID) REFERENCES FILES(FID)
);


-- Tokens requiring file inclusion for a given compilation unit and project
CREATE TABLE INCTRIGGERS(
  PID INTEGER, -- Project key
  CUID INTEGER, -- Compilation unit key
  BASEFILEID INTEGER, -- File requiring a definition
  DEFINERID INTEGER, -- File providing a definition
  FOFFSET INTEGER, -- Definition's offset within the providing file
  LEN INTEGER -- Token's length
  -- FOREIGN KEY(PID) REFERENCES PROJECTS(PID),
  -- FOREIGN KEY(CUID) REFERENCES FILES(FID),
  -- FOREIGN KEY(BASEFILEID) REFERENCES FILES(FID),
  -- FOREIGN KEY(DEFINERID) REFERENCES FILES(FID)
);


-- C functions and function-like macros
CREATE TABLE FUNCTIONS(
  ID BIGINT PRIMARY KEY, -- Unique function identifier
  NAME CHARACTER VARYING, -- Function name (redundant; see FUNCTIONID)
  ISMACRO BOOLEAN, -- True if a function-like macro (otherwise a C function)
  DEFINED BOOLEAN, -- True if the function is defined within the workspace
  DECLARED BOOLEAN, -- True if the function is declared within the workspace
  FILESCOPED BOOLEAN, -- True if the function's scope is a single compilation unit (static or macro)
  FID INTEGER, -- File key of the function's definition, declaration, or use
  FOFFSET INTEGER, -- Offset of definition, declaration, or use within the file
  FANIN INTEGER, -- Fan-in (number of callers)
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);


-- Details of defined functions and macros
CREATE TABLE FUNCTIONDEFS(
  FUNCTIONID BIGINT PRIMARY KEY, -- Function identifier key
  FIDBEGIN INTEGER, -- File key of the function's definition begin
  FOFFSETBEGIN INTEGER, -- Offset of definition begin within the file
  FIDEND INTEGER, -- File key of the function's definition end
  FOFFSETEND INTEGER, -- Offset of definition end within the file
  FOREIGN KEY(FUNCTIONID) REFERENCES FUNCTIONS(ID)
);


-- Metrics of defined functions and macros
CREATE TABLE FUNCTIONMETRICS(
  FUNCTIONID BIGINT, -- Function identifier key
  PRECPP BOOLEAN, -- True for values before the cpp false for values after it
  NCHAR INTEGER, -- Number of characters
  NCCOMMENT INTEGER, -- Number of comment characters
  NSPACE INTEGER, -- Number of space characters
  NLCOMMENT INTEGER, -- Number of line comments
  NBCOMMENT INTEGER, -- Number of block comments
  NLINE INTEGER, -- Number of lines
  MAXLINELEN INTEGER, -- Maximum number of characters in a line
  MAXSTMTLEN INTEGER, -- Maximum number of tokens in a statement
  MAXSTMTNEST INTEGER, -- Maximum level of statement nesting
  MAXBRACENEST INTEGER, -- Maximum level of brace nesting
  MAXBRACKNEST INTEGER, -- Maximum level of bracket nesting
  BRACENEST INTEGER, -- Dangling brace nesting
  BRACKNEST INTEGER, -- Dangling bracket nesting
  NULINE INTEGER, -- Number of unprocessed lines
  NPPDIRECTIVE INTEGER, -- Number of C preprocessor directives
  NPPCOND INTEGER, -- Number of processed C preprocessor conditionals (ifdef, if, elif)
  NPPFMACRO INTEGER, -- Number of defined C preprocessor function-like macros
  NPPOMACRO INTEGER, -- Number of defined C preprocessor object-like macros
  NTOKEN INTEGER, -- Number of tokens
  NSTMT INTEGER, -- Number of statements or declarations
  NOP INTEGER, -- Number of operators
  NUOP INTEGER, -- Number of unique operators
  NNCONST INTEGER, -- Number of numeric constants
  NCLIT INTEGER, -- Number of character literals
  NSTRING INTEGER, -- Number of character strings
  NPPCONCATOP INTEGER, -- Number of token concatenation operators (##)
  NPPSTRINGOP INTEGER, -- Number of token stringification operators (#)
  NIF INTEGER, -- Number of if statements
  NELSE INTEGER, -- Number of else clauses
  NSWITCH INTEGER, -- Number of switch statements
  NCASE INTEGER, -- Number of case labels
  NDEFAULT INTEGER, -- Number of default labels
  NBREAK INTEGER, -- Number of break statements
  NFOR INTEGER, -- Number of for statements
  NWHILE INTEGER, -- Number of while statements
  NDO INTEGER, -- Number of do statements
  NCONTINUE INTEGER, -- Number of continue statements
  NGOTO INTEGER, -- Number of goto statements
  NRETURN INTEGER, -- Number of return statements
  NPID INTEGER, -- Number of project-scope identifiers
  NFID INTEGER, -- Number of file-scope (static) identifiers
  NMID INTEGER, -- Number of macro identifiers
  NID INTEGER, -- Total number of object and object-like identifiers
  NUPID INTEGER, -- Number of unique project-scope identifiers
  NUFID INTEGER, -- Number of unique file-scope (static) identifiers
  NUMID INTEGER, -- Number of unique macro identifiers
  NUID INTEGER, -- Number of unique object and object-like identifiers
  NLABEL INTEGER, -- Number of goto labels
  NMACROEXPANDTOKEN INTEGER, -- Tokens added by macro expansion
  NGNSOC INTEGER, -- Number of global namespace occupants at function's top
  NMPARAM INTEGER, -- Number of parameters (for macros)
  NFPARAM INTEGER, -- Number of parameters (for functions)
  NEPARAM INTEGER, -- Number of passed non-expression macro parameters
  FANIN INTEGER, -- Fan-in (number of calling functions)
  FANOUT INTEGER, -- Fan-out (number of called functions)
  CCYCL1 INTEGER, -- Cyclomatic complexity (control statements)
  CCYCL2 INTEGER, -- Extended cyclomatic complexity (includes branching operators)
  CCYCL3 INTEGER, -- Maximum cyclomatic complexity (includes branching operators and all switch branches)
  CSTRUC REAL, -- Structure complexity (Henry and Kafura)
  CHAL REAL, -- Halstead complexity
  IFLOW REAL, -- Information flow metric (Henry and Selig)
  PRIMARY KEY(FUNCTIONID, PRECPP),
  FOREIGN KEY(FUNCTIONID) REFERENCES FUNCTIONS(ID)
);


-- Identifiers comprising a function's name
CREATE TABLE FUNCTIONID(
  FUNCTIONID BIGINT, -- Function identifier key
  ORDINAL INTEGER, -- Position of the identifier within the function name (0-based)
  EID BIGINT, -- Identifier key
  PRIMARY KEY(FUNCTIONID, ORDINAL),
  FOREIGN KEY(FUNCTIONID) REFERENCES FUNCTIONS(ID),
  FOREIGN KEY(EID) REFERENCES IDS(EID)
);


-- Function calls
CREATE TABLE FCALLS(
  SOURCEID BIGINT, -- Calling function identifier key
  DESTID BIGINT, -- Called function identifier key
  FOREIGN KEY(SOURCEID) REFERENCES FUNCTIONS(ID),
  FOREIGN KEY(DESTID) REFERENCES FUNCTIONS(ID)
);


-- Files occuring in more than one copy
CREATE TABLE FILECOPIES(
  GROUPID INTEGER, -- File group identifier
  FID INTEGER, -- Key of file belonging to a group of identical files
  PRIMARY KEY(GROUPID, FID),
  FOREIGN KEY(FID) REFERENCES FILES(FID)
);

Examples of SQL Queries

Find identifiers of a given type (typedefs, in this case):

select name from
ids left join tokens on ids.eid = tokens.eid
where ids.typedef = true

Number of different files that use a given identifier:

select name, count(*) as cf from (
 select fid, tokens.eid, count(*) as c from
 tokens
 group by
 eid, fid) as cl inner join ids on
cl.eid = ids.eid
group by ids.eid, ids.name
order by cf desc;

Number of times an identifier occurs in a single file:

SELECT IDS.NAME AS INAME, FILES.NAME AS FNAME, COUNT(*) AS C FROM TOKENS
INNER JOIN IDS ON
IDS.EID = TOKENS.EID
INNER JOIN FILES ON
TOKENS.FID = FILES.FID
GROUP BY IDS.EID, TOKENS.FID
ORDER BY C DESC;

Number of times an identifier occurs in the workspace:

select name, count(*) as c from tokens
inner join ids on
ids.eid = tokens.eid
group by eid
order by c desc

Reconstitute the file with fid = 4:

select s from
(select name as s, foffset  from ids inner join tokens on
ids.eid = tokens.eid where fid = 4
union select code as s, foffset from rest where fid = 4
union select comment as s, foffset from comments where fid = 4
union select string as s, foffset from strings where fid = 4
)
order by foffset

sed -e '/^[0-9][0-9]* rows/d' |
tr -d '\n' |
sed 's/\\u0000d/\
/g'

sed -e '/^[0-9][0-9]* rows/d' |
tr -d '\n\r' |
sed 's/\\u0000d\\u0000a/\
/g'

Show the projects each identifier belongs to:

select IDS.NAME, PROJECTS.NAME from IDS
INNER JOIN IDPROJ ON IDS.EID = IDPROJ.EID
INNER JOIN PROJECTS ON IDPROJ.PID = PROJECTS.PID
ORDER BY IDS.NAME;

Show the included files required by other files for each compilation unit and project.

select
	projects.name as projname,
	cufiles.name as cuname,
	basefiles.name as basename,
	definefiles.name as defname
from
	definers inner join projects on definers.pid = projects.pid
	inner join files as cufiles on definers.cuid=cufiles.fid
	inner join  files as basefiles on definers.basefileid=basefiles.fid
	inner join files as definefiles on definers.definerid = definefiles.fid;

Speed-up processing:

create index teid on tokens(eid)
create index tfid on tokens(fid)

Obtain identifiers common between files participating in a define/use relationship:

SELECT
    tokensa.eid,
    min(ids.name) as identifier,
    min(filesb.name) as defined,
    min(filesa.name) as used
FROM definers
INNER JOIN tokens AS tokensa ON definers.basefileid = tokensa.fid
INNER JOIN tokens AS tokensb ON definers.definerid = tokensb.fid
INNER JOIN ids ON ids.eid = tokensa.eid
INNER JOIN files as filesa ON tokensa.fid = filesa.fid
INNER JOIN files as filesb ON tokensb.fid = filesb.fid
WHERE tokensa.eid = tokensb.eid
GROUP BY tokensa.eid, definerid, basefileid
ORDER BY defined, identifier

Create a function and macro call graph:

SELECT source.name AS CallingFunction, dest.name AS CalledFunction
FROM fcalls
INNER JOIN functions AS source ON fcalls.sourceid = source.id
INNER JOIN functions AS dest ON fcalls.destid = dest.id

Details of the Collected Metrics

Metrics Common to Files and Functions

Number of C preprocessor directives

See note 1.

Number of processed C preprocessor conditionals (ifdef, if, elif)

See note 1.

Number of defined C preprocessor function-like macros

See note 1.

Number of defined C preprocessor object-like macros

See note 1.

Number of preprocessed tokens

Although during preprocessing whitespace is considered a valid token, this metric does not take whitespace tokens into account. This makes it easy to compare the number of preprocessed tokens with the number of compiled tokens. The two metrics are equal if no macro expansion takes place.

Number of compiled tokens

See note 1.

File-Specific Metrics

Number of statements

This metric measures number of statements parsed while processing the file, including statements generated by macro expansion. See note 1.

Number of defined project-scope functions

See note 1.

Number of defined file-scope (static) functions

See note 1.

Number of defined project-scope variables

See note 1.

Number of defined file-scope (static) variables

See note 1.

Number of complete aggregate (struct/union) declarations

Also includes complete declarations without a tag. See note 1.

Number of declared aggregate (struct/union) members

See note 1.

Number of complete enumeration declarations

See note 1.

Number of declared enumeration elements

See note 1.

Number of directly included files

This counts the number of header files that were directly included while processing the project's source files. If each file is processed exactly once, the metric is roughly similar to the number of #include directives in the project's files. See also note 1.

Function-Specific Metrics

Number of statements or declarations

Nested statements are counted recursively. Thus

while (a)
	if (b)
		c();

counts as three statements.

Number of operators

See note 2.

Number of unique operators

See note 2.

Number of if statements

See note 3.

Number of else clauses

See note 3.

Number of switch statements

See note 3.

Number of case labels

See note 3.

Number of default labels

See note 3.

Number of break statements

See note 3.

Number of for statements

See note 3.

Number of while statements

This metric does not include the do .. while form. See note 3.

Number of do statements

See note 3.

Number of continue statements

See note 3.

Number of goto statements

See note 3.

Number of return statements

See note 3.

Total number of object and object-like identifiers

Also includes macros.

Number of unique object and object-like identifiers

Also includes macros.

Number of global namespace occupants at function's top

This metric measures the namespace pollution in the object namespace at the point before entering a function. Its value is the count of all macros as well as objects with file and project-scope visibility that are declared at the point it is measured. See note 1. See note 4.

Number of parameters

See note 1.

Maximum level of statement nesting

In order to avoid excessively inflating this metric when measuring sequences of the form

if (a) {
	...
} else if (b) {
	...
} else if (c) {
	...
} else
	...
}

this metric does not take into account the nesting of else clauses. Thus the above code will be given a nesting level of 1, rather than 3, which is implied by the following (actual) reading of the code.

if (a) {
	...
} else
	if (b) {
		...
	} else
		if (c) {
			...
		} else
			...
		}

See note 1. See note 4.

Fan-in (number of calling functions)

This is also listed under a function's details for functions that are not defined (and have not metrics associated with them).

Cyclomatic complexity (control statements)

This metric, CC₁ measures the number of branch points in the function. In order to avoid misleadingly high values that occur from even trivial switch statements, this metric measures the complexity of a switch statement as 1.

Extended cyclomatic complexity (includes branching operators)

This metric, CC₂, takes into account the nodes introduced by the Boolean-AND, boolean-OR, and conditional evaluation operators.

Maximum cyclomatic complexity (includes branching operators and all switch branches)

This metric, CC₃, considers each case label as a separate node.

Structure complexity (Henry and Kafura)

This metric is calculcated as follows.
C_p = (fan_in * fan_out)²

Halstead complexity

This metric is calculcated as follows.
HC = (number_of_operators + number_of_operands) * log₂( unique_number_of_operators + unique_number_of_operands)
Where operands are object identifiers, macros, numeric and character constants. For the purpose of determining unique operands, each numeric or character constant is considered a separate operand.

Information flow metric (Henry and Selig)

This metric is calculcated as follows.
HC_p = CC₁ * C_p

Notes

#define x(if, while, else) (if + while + else)

#define WHILE(x) while(x) {
#define WEND	 }

WHILE (x)
	foo();
WEND

Shortcomings

1. Conditional compilation

   Some programs have parts of them compiled under conditional preprocessor directives. Consider the following example:

   
   #ifdef unix
   #include <unistd.h>
   #define erase_file(x) unlink(x)
   #endif
   
   #ifdef WIN32
   #include <windows.h>
   #define erase_file(x) DeleteFile(x)
   #endif
   
   main(int argc, char *argv[])
   {
   	erase_file(argv[1]);
   }
   

   As humans we can understand that erase_file occurs three times within the file. However, because CScout preprocesses the file following the C preprocessor semantics, it will typically match only two instances. In some cases you can get around this problem by defining macros that will ensure that all code inside conditional directives gets processed. In other cases this will result in errors (e.g. a duplicate macro definition in the above example). In such cases you can include in your workspace the same project multiple times, each time with a different set of defined macros.

   workspace example {
   	project idtest {
   		define DEBUG 1
   		define TEST 1
   		file idtest.c util.c
   	}
   	project idtest2 {
   		define NDEBUG 1
   		define PRODUCTION
   		file idtest.c util.c
   }
   

2. Partial coverage of macro use

   Consider the following example:

   
   struct s1 {
   	int id;
   } a;
   
   struct s2 {
   	char id;
   } b;
   
   struct s3 {
   	double id;
   } c;
   
   #define getid(x) ((x)->id)
   
   main()
   {
   	printf("%d %c", getid(a), getid(b));
   }
   

   In the above example, changing an id instance should also change the other three instances. However, CScout will not associate the member of s3 with the identifier appearing in the getid macro or the s1 or s2 structures, because there is no getid macro invocation to link them together. If e.g. id is replaced with val the program will compile and function correctly, but when one tries to access the c struture's member in the future using getid an error will result.

   
   struct s1 {
   	int val;
   } a;
   
   struct s2 {
   	char val;
   } b;
   
   struct s3 {
   	double id;
   } c;
   
   #define getid(x) ((x)->val)
   
   main()
   {
   	printf("%d %c", getid(a), getid(b));	/* OK */
   	printf(" %g", getid(c));		/* New statement: error */
   }
   

   To avoid this (rare) problem you can introduce dummy macro invocations of the form:

   
   #ifdef CSCOUT
   	(void)getid(d)
   #endif
   

3. Undefined macros

   We employ a heuristic classifying all instances of an undefined macro as being the same identifier. Thus in the following sequence foo will match all three macro instances:

   
   #undef foo
   
   #ifdef foo
   #endif
   
   #ifdef foo
   #endif
   
   #define foo 1
   

   In most cases this is what you want, but there may be cases where the macro appears in different files and with a different meaning. In such cases the undefined instances of the macro will erroneously match the defined instance.

In addition, the analysis of functions can be confused by the following situations.

1. Functions getting called through function pointers will not appear in the call graphs. This is a common limitation of static call analysis.
2. Function-like macros called from inside function bodies that were generated by macro expansion will not be registered as calls.
3. Non-function like macros that expand into function calls will not appear in the call graph; the corresponding functions will appear to be called by the function containing the macro.

Finally, because function argument refactoring works at a higher level thann simple identifiers, the following limitations hold.

1. When a function call's arguments macro-expand into unballanced brackets or into multiple function arguments the replacement can misbehave.
2. When there is not a one-to-one correspondence between a function's name and its associated identifier (i.e. when the function's name is generated through macro-token concatenation) the function argument refactoring is not offered as an option.

Error Messages

Warnings

• #ifdef argument is not an identifier
  The token following a #ifdef directive is not a legal identifier.

• Application of macro " ... ": operator # only valid before macro parameters
  The stringizing operator # was not followed by a macro parameter.

• Assuming declaration int ... (...)
  An undeclared identifier is used as a function.

• Duplicate (different) macro definition of macro ...
  A defined macro can be redefined only if the two definitions are exactly the same.

• Empty character literal
  Character lirerals must include a character.

• Illegal combination of sign specifiers
  The signedness specifiers used can not be combined (e.g. unsigned signed).

• Processing automatically generated file; #line directive ignored.
  A #line directive was found. This signifies that the source file is automatically generated and refactoring changes on that file may be lost in the future. References to the original source file are ignored.

• Sign specification on non-integral type - ignored
  A signedness specification was given on a non-integral type.

• Undeclared identifier in typeof expression: ...
  The identifier appearing within typeof has not been declared.

• Undeclared identifier: ...
  An undeclared identifier was used in a primary expression.

Fatal Errors

• #pragma pushd: unable to get current directory: ...
  The call to getcwd failed while processing the #pragma pushd CScout-specific directive.

• EOF in comment
  The end of file was reached while processing a block comment.

• Error count exceeds 100; exiting ...
  To avoid cascading errors only the first 100 errors are reported.

• Invalid C token: '$'
  A '$' token was encountered in C code. Values starting with a '$' token are only allowed inside yacc rules.

• Unable to get path of file ...
  The Win32 GetfullPathName system call used to retrieve the unique path file name failed (Windows-specific).

• Unable to stat file ...
  The POSIX stat system call used to retrieve the unique file identifier failed (Unix-specific).

Errors

• #pragma echo: string expected
  The #pragma echo CScout-specific directive was not followed by a string.

• #pragma includepath: string expected
  The #pragma includepath CScout-specific directive was not followed by a string.

• #pragma process: string expected
  The #pragma process CScout-specific directive was not followed by a string.

• #pragma project: string expected
  The #pragma project CScout-specific directive was not followed by a string.

• #pragma pushd: string expected
  The #pragma pushd CScout-specific directive was not followed by a string.

• #pragma readonly: string expected
  The #pragma readonly CScout-specific directive was not followed by a string.

• #pragma ro_prefix: string expected
  The #pragma ro_prefix CScout-specific directive was not followed by a string.

• % not followed by yacc keyword
  In the definitions section of a yacc file the % symbol was not followed by a legal yacc keyword.

• %union does not have a member ...
  The member used in a $<name>X yacc construct was not defined as a %union member.

• Application of macro " ... ": operator # at end of macro pattern
  No argument was supplied to the right of the stringizing operator #.

• Array not an abstract type
  The underlying array object for which a type is specified is not an abstract type.

• At most one storage class can be specified
  More than one storage class was given for the same object.

• Conflicting declarations for identifier ...
  An identifier is declared twice with compilation or linkage unit scope with conflicting declarations.

• Declared parameter does not appear in old-style function parameter list: ...
  While processing an old-style (K&R) parameter declaration, a declared parameter did not match any of the parameters appearing in the function's arguments definition.

• Division by zero in #if expression
  A #if expression divided by zero.

• Duplicate definition of identifier ...
  An identifier is declared twice within the same block.

• Duplicate definition of tag ...
  A structure, union, or enumeration tag was defined twice for the same entity.

• EOF while processing #if directive
  The processing of code within a #if block reached the end of file, without a corresponding #endif / #else / #elif directive.

• Empty #include directive
  A #include directive was not followed by a filename specification.

• End of file in character literal
  The end of file was reached while processing a character literal: a single quote was never closed.

• End of file in string literal
  The end of file was reached while processing a string.

• Explicit element tag without no %union in effect
  The yacc $n syntax was used to specify an element of the %union but no union was defined.

• Illegal characters in hex escape sequence
  A hexadecimal character escape sequence \x continued with a non-hexadecimal character.

• Illegal combination of type specifiers
  The type specifiers used can not be combined (e.g. double char).

• Illegal pointer dereference
  An attempt was made to dereference an element that is not a pointer.

• Invalid #include syntax
  A #include directive was not followed by a legal filename specification.

• Invalid application of basic type, storage class, or type specifier
  A basic type, storage class or type specified was specified in an invalid underlying object.

• Invalid character escape sequence
  A character escape sequence \c can not be recognised.

• Invalid macro name
  The macro name specified in a #define or #undef directive is not a valid identifier.

• Invalid macro parameter name
  A macro parameter name specified in a #define directive is not a valid identifier.

• Invalid macro parameter punctuation
  The formal parameters in a #define macro definition are not separated by commas.

• Invalid preprocessor directive
  An invalid preprocessor directive was found. The directive did not match an identifier.

• Invalid type specification
  An attempt was made to specify a type on an object that did not allow this specification.

• Label ... already defined
  The same goto label is defined more than once in a given function.

• Macro [ ... ]: EOF while reading function macro arguments
  The end of file was reached while gathering a macro's arguments.

• Macro [ ... ]: close bracket expected for function-like macro
  The arguments to a function-like macro did not terminate with a closing bracket.

• Member access in incomplete struct/union: ...
  The member access for a structure or union is applied on an object with an incomplete definition.

• Missing close bracket in defined operator
  The identifier of a defined operator was not followed by a closing bracket.

• Modulo division by zero in #if expression
  A #if expression divided by zero in a modulo expression.

• Multiple storage classes in type declaration
  Incompatible storage classes were specified in a single type declaration.

• No identifier following defined operator
  The defined operator was not followed by an identifier.

• Object is not a function
  The object used as a function to be called is not a function or a pointer to a function.

• Only struct/union anonymous elements allowed
  Anonymous members within a member declaring list (e.g. struct {int x, y;}) can only be structures or unions. (GCC/Microsoft C extension).

• Pointer not an abstract type
  The underlying pointer object for which a type is specified is not an abstract type.

• Popd: directory stack empty
  The #pragma popd CScout-specific directive was performed on an empty directory stack.

• Structure or union does not have a member ...
  The structure or union on the left of the . or -> operator does not have as a member the identifier appearing on the operator's right.

• Subscript not on array or pointer
  The object being subscripted using the [] operator is not an array or a pointer.

• Syntax error in preprocessor expression
  A #if or #elif expression was syntactically incorrect.

• Unable to open include file ...
  The specified include file could not be opened.

• Unbalanced #elif
  A #elif directive was found without a corresponding #if.

• Unbalanced #else
  A #else directive was found without a corresponding #if.

• Unbalanced #endif
  A #endif directive was found without a corresponding #if.

• Undefined label ...
  A goto label used within a function was never defined.

• Unexpected end in character escape sequence
  A character escape sequence (\c) was not completed; no character follows the backslash.

• Unknown preprocessor directive: ...
  An unkown preprocessor directive was found.

• Unkown %union element tag ...
  The yacc %union does not have as a member the identifier appearing on the element's tag.

• Yacc $value out of range
  The number used in a $n yacc variable was greater than the number of identifiers and actions on the rule's left side.

License

CScout is distributed as open source software under the GNU General Public License, which is reproduced below. Other licensing options and professional support are available on request.

GNU GENERAL PUBLIC LICENSE

Version 3, 29 June 2007

Copyright © 2007 Free Software Foundation, Inc. <http://fsf.org/ (http://fsf.org/)>

Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.

Preamble

The GNU General Public License is a free, copyleft license for software and other kinds of works.

The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too.

When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.

To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others.

For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.

Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it.

For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions.

Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users.

Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free.

The precise terms and conditions for copying, distribution and modification follow.

TERMS AND CONDITIONS

0. Definitions.

“This License” refers to version 3 of the GNU General Public License.

“Copyright” also means copyright-like laws that apply to other kinds of works, such as semiconductor masks.

“The Program” refers to any copyrightable work licensed under this License. Each licensee is addressed as “you”. “Licensees” and “recipients” may be individuals or organizations.

To “modify” a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a “modified version” of the earlier work or a work “based on” the earlier work.