Flexc++(1) was designed after flex(1) and flex++(1). Like these two programs flexc++ generates code performing pattern-matching on text, possibly executing actions when certain regular expressions are recognized.
Refer to flexc++(1) for a general overview. This manual page describes how flexc++'s input s should be organized. It contains the following sections:
The complete list of affected names is:
ActionType_, Leave_, StartConditon_, PostEnum_;
actionType_, continue_, echoCh_, echoFirst_, executeAction_, getRange_, get_, istreamName_, lex_, lop1_, lop2_, lop3_, lop4_, lopf_, matched_, noReturn_, print_, pushFront_, reset_, return_;
d_in_ d_token_ s_finIdx_, s_interactive_, s_maxSizeofStreamStack_, s_nRules_, s_rangeOfEOF_, s_ranges_, s_rf_.
Flexc++ expects an input file containing directives and the regular expressions that should be recognized by objects of the scanner class generated by flexc++. In this man page the elements and organization of flexc++'s input file is described.
Flexc++'s input file consists of two sections, separated from each other by a line merely containing two consecutive percent characters:
%%
The section before this separator contains directives; the section
following this separator contains regular expressions and possibly actions to
perform when these regular expressions are matched by the object of the
scanner class generated by flexc++. If a second line is encountered immediately
beginning with two consecutive percent characters then this ends flexc++'s
input file processing. See also section 6 (%% SEPARATOR) below.
White space is usually ignored, as is comment, which may be of the traditional C form (i.e., /*, followed by (possibly multi-line) comment text, followed by */, and it may be C++ end-of-line comment: two consecutive slashes (//) start the comment, which continues up to the next newline character.
Flexc++'s input file may be split into multiple files. This allows for the definition of logically separate elements of the specifications in different files. Include directives must be specified on a line of their own. To switch to another specification file the following stanza is used:
//include file-location
The //include directive starts in the line's first column. File
locations can be absolute or relative to the location of the file containing
the //include directive. White space characters following //include
and before the end of the line are ignored. The file specification may be
surrounded by double quotes, but these double quotes are not required and are
ignored (removed) if present. All remaining characters are expected to define
the name of the file where flexc++'s rules specifications continue. Once end of
file of a sub-file has been reached, processing continues at the line beyond
the //include directive of the previously scanned file. The end-of-file of
the file that was initially specified when flexc++ was called indicates the end
of flexc++'s rules specification.
The first section of flexc++'s input file consists of directives. In addition it may associate regular expressions with symbolic names, allowing you to use these identifiers in the rules section. Each directive is defined on a line of its own. When available, directives are overridden by flexc++ command line options.
Some directives require arguments, which are usually provided following separating (but optional) = characters. Arguments of directives are text, surrounded by double quotes (strings), or embedded in raw string literals (rawstrings). Double quotes or backslashes inside strings must themselves be preceded by backslashes; these backslashes are not required when rawstrings are used.
The %s and %x directives are immediately followed by name lists, consisting of identifiers separated by blanks. Here is an example of the definition of a directive:
%class-name = "MyScanner"
Directives accepting a `filename' do not accept path names, i.e., they cannot contain directory separators (/); options accepting a 'pathname' may contain directory separators. A 'pathname' using blank characters should be surrounded by double quotes.
Some directives may generate errors. This happens when a directive conflicts with the contents of an existing file which flexc++ cannot modify (e.g., a scanner class header file exists, but doesn't define a name space, but a %namespace directive was provided). To solve the error the offending directive could be omitted, the existing file could be removed, or the existing file could be hand-edited according to the directive's specification. Note that flexc++ currently does not handle the opposite error condition: if a previously used directive is omitted, then flexc++ does not detect the inconsistency. In those cases you may encounter compilation errors.
It is an error if this directive is used and an already existing scanner-class header file does not include `filename'.
Corresponding command-line option: --cases-insensitive.
When this directive is specified the resulting scanner does not distinguish between the following rules:
First // initial F is transformed to f
first
FIRST // all capitals are transformed to lower case chars
With a case-insensitive scanner only the first rule can be matched,
and flexc++ will issue warnings for the second and third rule about
rules that cannot be matched.
Input processed by a case-insensitive scanner is also handled case insensitively. The above mentioned First rule is matched for all of the following input words: first First FIRST firST.
Although the matching process proceeds case insensitively, the matched text (as returned by the scanner's matched() member) always contains the original, unmodified text. So, with the above input matched() returns, respectively first, First, FIRST and firST, while matching the rule First.
It is an error if this directive is used and an already existing scanner-class header file does not define class `className'.
%filenames = "scanner"
the names of the generated files are, respectively, scanner.h,
scanner.ih, and scannerbase.h. Corresponding command-line
option: --filenames. The name of the source file (by default
lex.cc) is controlled by the %lex-source directive.
It is an error if this directive is used and an already 'filename' file does not include the scanner class header file.
It is an error if this directive is used and an already scanner-class header file does not define namespace identifier.
The name of the startcondition-enum may be modified, and the directive can also be omitted again after it has been specified before. When changing the name of the startcondition-enum or when reverting to the default name newly generated lex.cc and ScannerBase.h files will use the currently defined startcondition-enum name. Be advised, though, that the startcondition-enum name may also be used in user-defined members of the scanner-class, or in the scanner's header and internal header files. If so, the user is responsible for updating those files to the currently defined name of the startcondition-enum.
Mini scanners come in two flavors: inclusive mini scanners and exclusive mini scanners. The rules that apply to an inclusive mini scanner are the mini scanner's own rules as well as the rules which apply to no mini scanners in particular (i.e., the rules that apply to the default (or INITIAL) mini scanner). Exclusive mini scanners only use the rules that were defined for them.
To define an inclusive mini scanner use %s, followed by one or more identifiers specifying the name(s) of the mini-scanner(s). To define an exclusive mini scanner use %x, followed by or more identifiers specifying the name(s) of the mini-scanner(s). The following example defines the names of two mini scanners: string and comment:
%x string comment
Following this, rules defined in the context of the string mini
scanner (see below) will only be used when that mini scanner is active.
A flexc++ input file may contain multiple %s and %x specifications.
Definitions are of the form
identifier regular-expression
Each definition must be entered on a line of its own. Definitions
associate identifiers with regular expressions, allowing the use of
${identifier} as synonym for its regular expression in the rules section
of flexc++'s input file. One defined, the identifiers representing regular
expressions can also be used in subsequent definitions.
Example:
FIRST [A-Za-z_]
NAME {FIRST}[-A-Za-z0-9_]*
Following directives and definitions a line merely containing two consecutive % characters is expected. Following this line the rules are defined. Rules consist of regular expressions which should be recognized, possibly followed by actions to be executed once a rule's regular expression has been matched.
If the rule section contains a line starting with two consecutive % characters, then any remaining input is ignored. Note that this second %% separator does not have to be specified. It is purely optional. To specify a regular expression starting with %% surround the %% with double quotes ("%%") or prefix the %% with a blank space: the %%-characters are only considered a separator if they are encountered at the very beginning of a line.
The regular expressions defined in flexc++'s rules files are matched against the information passed to the scanner's lex function.
Regular expressions begin as the first non-blank character on a line. Comment is interpreted as comment as long as it isn't part of the regular expresssion. To define a regular expression starting with two slashes (at least) the first slash can be escaped or double quoted. (E.g., "//".* defines C++ comment to end-of-line).
Regular expressions end at the first blank character (to add a blank character, e.g., a space character, to a regular expression, prefix it by a backslash or put it in a double-quoted string).
Actions may be associated with regular expressions. At a match the action that is associated with the regular expression is executed, after which scanning continues when the lexical scanning function (e.g., lex) is called again. Actions are not required, and regular expressions can be defined without any actions at all. If such action-less regular expressions are matched then the match is performed silently, after which processing continues.
Flexc++ tries to match as many characters of the input file as possible (i.e., it uses `greedy matching'). Non-greedy matching is accomplished by a combination of a scanner and parser and/or by using the `lookahead' operator (/).
The following regular expression `building blocks' are available. More complex regular expressions are created by combining them:
If flexc++ detects patterns potentially not matching any text it generates warnings like this:
[Warning] input, line 7: null-matching regular expression
By placing the comment
//%nowarn
on the line just before a regular expression that potentially does not
match any text, the warning for that regular expression is suppressed;
Character classes
Inside a character class all regular expression operators lose their special meanings, except for the escape character (\), the character range operator -, the end of character class operator ], and, at the beginning of the class, ^. All ordinary escape sequences are supported, all other escaped characters are interpreted as literal characters (e.g., \c is a literal c).
To add a closing bracket to a character class use [] or \]. To add a closing bracket to a negated character class use [^] (or use [^ followed by \] somewhere within the character class). Minus characters are used to define character ranges (e.g., [a-d], defining [abcd]) except in the following cases, where flexc++ recognizes a literal minus character: [-, or [^- (a minus at the very beginning of a character class); -] (a minus at the very end of a character class); or \- (an escaped minus character)) Once a character class has started, all subsequent character (ranges) are added to the set, until the final closing bracket (]) has been reached.
Operator precedence
The regular expressions listed above are grouped according to precedence, from highest precedence at the top to lowest at the bottom. From lowest to highest precedence, the operators are:
The lex standard defines concatenation as having a higher precedence than the interval expression. This is different from many other regular expression engines, and flexc++ follows these latter engines, giving all `multiplication operators' equal priority.
Name expansion has the same precedence as grouping (using parentheses to influence the precedence of the other operators in the regular expression). Since the name expansion is treated as a group in flexc++, it is not allowed to use the lookahead operator in a name definition (a named pattern, defined in the definition section).
Predefined sets of characters
Character classes can also contain character class expressions. These are expressions enclosed inside [: and :] delimiters (which themselves must appear between the [ and ] of the character class. Other elements may occur inside the character class as well). The character class expressions are:
[:alnum:] [:alpha:] [:blank:]
[:cntrl:] [:digit:] [:graph:]
[:lower:] [:print:] [:punct:]
[:space:] [:upper:] [:xdigit:]
Character class expressions designate a set of characters equivalent to the corresponding standard C isXXX function. For example, [:alnum:] designates those characters for which isalnum returns true - i.e., any alphabetic or numeric character. For example, the following character classes are all equivalent:
[[:alnum:]]
[[:alpha:][:digit:]]
[[:alpha:][0-9]]
[a-zA-Z0-9]
A negated character class such as the example [^A-Z] above will match a newline unless \n (or an equivalent escape sequence) is one of the characters explicitly present in the negated character class (e.g., [^A-Z\n]). This differs from the way many other regular expression tools treat negated character classes, but unfortunately the inconsistency is historically entrenched. Matching newlines means that a pattern like [^"]* can match the entire input unless there's another quote in the input.
Flexc++ allows negation of character class expressions by prepending ^ to the POSIX character class name.
[:^alnum:] [:^alpha:] [:^blank:]
[:^cntrl:] [:^digit:] [:^graph:]
[:^lower:] [:^print:] [:^punct:]
[:^space:] [:^upper:] [:^xdigit:]
Combining character sets
The {-} operator computes the difference of two character classes. For example, [a-c]{-}[b-z] represents all the characters in the class [a-c] that are not in the class [b-z] (which in this case, is just the single character a). The {-} operator is left associative, so [abc]{-}[b]{-}[c] is the same as [a].
The {+} operator computes the union of two character classes. For example, [a-z]{+}[0-9] is the same as [a-z0-9]. This operator is useful when preceded by the result of a difference operation, as in, [[:alpha:]]{-}[[:lower:]]{+}[q], which is equivalent to [A-Zq] in the C locale.
Trailing context
A rule can have at most one instance of trailing context (the / operator or the $ operator). The start condition, ^, and <<EOF>> patterns can only occur at the beginning of a pattern, and cannot be surrounded by parentheses. The characters ^ and $ only have their special properties at, respectively, the beginning and end of regular expressions. In all other cases they are treated as a normal characters.
%option debug
%x comment
NAME [[:alpha:]][_[:alnum:]]*
%%
"//".* // ignore
"/*" begin(StartCondition_::comment);
<comment>.|\n // ignore
<comment>"*/" begin(StartCondition_::INITIAL);
^a return 1;
a return 2;
a$ return 3;
{NAME} return 4;
.|\n // ignore
)
Flexc++'s default skeleton files are in /usr/share/flexc++.
By default, flexc++ generates the following files:
flexc++(1), flexc++api(3)