#======================================================================== #
# Text::MetaText
#
# Version 0.22
#
# Copyright (C) 1996-1998 Andy Wardley <abw@kfs.org>. # All Rights Reserved.
#
#------------------------------------------------------------------------ #
# Changes
#
# This file lists the changes to the Text::MetaText module from # version 0.01 onwards. Development work on the Template::Base # module, from which Text::MetaText is derived, is not included. #
# The list includes internal changes as well as user-visible ones and # as such may seem unnecessarily verbose or irrelevant in places. #
#========================================================================

#------------------------------------------------------------------------ # Version 0.01 Date: 1997/04/29 12:09:47 #------------------------------------------------------------------------

• Original code migrated from ancestral Template::Base module.

#------------------------------------------------------------------------ # Version 0.02 Date: 1997/04/29 12:12:07 #------------------------------------------------------------------------

• Cleaned up and optimised much of the existing Template::Base code.
• Added user-definable MAGIC tokens, including different tokens for start/end of MetaText directive.
• Updated filter functions definition
• Added user-definable warning function (ERROR)
• Added user-definable debug function (DEBUG) providing multi-level reporting
• Replaced old-style typeglobs with FileHandles
• Modified splitparams() to recognise single parameters (e.g. "noprint") as flags rather than requiring a balanced "name = value". Flags are added thusly: $directive->{ PARAMS }->{ $flag } = 0;
• Further modified splitparams() to protect multi-word quoted sequences that may be values to pass to a function without being interpreted as many individual flags. i.e. "foo bar baz" should not be interpreted as ("foo", "bar", "baz").
• Added "noprint" flag to BLOCK definition to prevent inclusion of the BLOCK definition in the processed output. This changes in V0.10 where the default behaviour becomes "noprint" and printing is activated with the "print" flag.

#------------------------------------------------------------------------ # Version 0.03 Date: 1997/04/30 08:20:12 #------------------------------------------------------------------------

• Added ROGUE option to indicate how unrecognised tokens should be handled.
• Added primitive LOOP construct.

#------------------------------------------------------------------------ # Version 0.04 Date: 1997/04/30 14:34:03 #------------------------------------------------------------------------

• Fully implemented LOOP construct. LOOP segments are defined between %% LOOP %% and %% ENDLOOP %% and _pre-process()'d into the symbol table as LOOP.name (name defaults to a unique zero-padded 5-digit number). This is a test state. Ideally, looping constructs should be built into INCLUDE directives. Lo and behold! See the next comment...
• Removed LOOP directive (ha!) and built functionality into the standard INCLUDE directive. This now supports the "repeat" flag, with an optional value (i.e. "repeat = 5"). Without the value, the parameters supplied to the INCLUDE are split by a delimiter (comma, ',' by default, but can be set with "delimiter = <string>") and the loop repeats until all the data has been used. In the case that the block loops more times than a given parameter has values (i.e. repeat=5 data="10,15,20") then the last value is repeated. NOTE: LOOP constructs have since been removed from MetaText. They may return one day.
• Formalised case sensitivity. Keywords and special directive variables (e.g. REPEAT, IF, etc) are forced to upper case, parameter names to lower case. Identifiers (which may reference a case-sensitive external file) and parameter values are left as found. (This changes again in version 0.5)
• Modified parsedirective() to handle all special parameters in a generic way. Previously, a special function, extractconditional() was employed to identify "if=<condition>" statements. This is no longer required.

#------------------------------------------------------------------------ # Version 0.05 Date: 1997/05/02 06:48:31 #------------------------------------------------------------------------

• Further enhanced ROGUE error reporting to re-insert cleaned-up version of original tag (default), delete the tag in entirety (delete) and in either case, warn about the bad token (warn).
• Changed case sensitivity handling (again). I decided it was incorrect to force all parameter names and identifiers to lower case. Why not let the users deferentiate between "foo" and "FOO" if they want to? There is now a CASE configuration option. If it evaluates true, MetaText runs in case-sensitive mode, preserving the case of all identifiers and parameters (except for directive control params such as REPEAT, DELIMITER etc. which are always forced to UPPER). When CASE is false, all parameters and identifiers are converted to lower case. This will cause problems if you try to INCLUDE an external file whose name is not fully lower case. i.e. "INCLUDE Foo/Bar" becomes "INCLUDE foo/bar". For this reason, case sensitive is the default mode. This changes once again in version 0.17 where case sensitivity is finally rationalised once and for all (I hope).
• Modified splitparams() to allow single quotes ''' to be used to encompass multi-word values in addition to the usual double quotes '"'. Cleaned up the regex code considerably, pre-defining various elements to make the final expression readable and easily understandable <gasp!>
• Modified splitparams() to allow variables to reference other variables (e.g. alias = $name). Added interpolatetags() to interpolate all these values before passing them to process(), _substitute() and _evaluate. Modified _evaluate() to understand and account for references and cleaned up much of the evaluation code. (This changes in version 0.6)
• Added DELIMITER option, allowing the user to specify the default DELIMITER for splitting strings. Specific delimiters can still be specified within individiual operations.
• Added some extra characters to the characters class that can delimit directories in a DIRECTORY or DATADIR string. Currently [|;,:].

#------------------------------------------------------------------------ # Version 0.06 Date: 1997/05/02 14:43:18 #------------------------------------------------------------------------

• Fixed a minor bug in _tidy() that was stripping quotes from quoted parameter lists.
• Shifted lower case conversion (if $self->{ CASE }) out of splitparams() and put it into parsedirective() to allow slightly more flexibility in parameter parsing.
• Removed variable interpolation from basic block parameters. It was getting very complicated to track what was what. Despite coding a reasonably elegant solution, I decided that it made the code more complex for very little gain. Condition statements (if="<condition>") may include variable references as may INCLUDE identifiers, e.g. %% INCLUDE $name %%

#------------------------------------------------------------------------ # Version 0.07 Date: 1997/05/06 16:04:45 #------------------------------------------------------------------------

• Added generic _interpolate() function for expanding variable values within a string. Block identifiers are now passed through this function to permit directives such as "%% INCLUDE $name %%". Ambiguous names can be resolved by surrounding the variable name in braces, as per the Unix shell syntax. e.g. "foo${bar}baz".
• Modified _evaluate() to use hash array to reference anonymous code subs to do comparisons (==, !=, <=, etc) and boolean operations (and, or, xor, etc).
• Quotes are implicitly removed from around absolute values in conditional equations. e.g. with the following conditional: if="name=\"John Doe\"" the variable 'name' is implictly changed from '"John Doe"' to 'John Doe'. The original escaped quotes ARE required to quote the whitespace within the variable value.
• Added regex matching (=~ and !~) to _evaluate() function for conditionals.
• Relaxed valid/invalid variable checking within _evaluate() to allow the simple expression 'if="$variable"' to simply return 0 if $variable isn't defined rather than bleat about it.

#------------------------------------------------------------------------ # Version 0.08 Date: 1997/05/08 08:34:41 #------------------------------------------------------------------------

• Fixed bug where surrounding quotes weren't being stripped from if="" values in all cases.
• Changed _evalute() to evaluate parameter values (as well as parameter names) case-insensitively unless CASE sensitive flag is set.
• Added "in" comparator in _evaluate() to determine if a value is in a given list. e.g. "$name in \"fred,tom,harry\"". Uses the current DELIMITER value (or the default) to delimit the list.
• Changed name of debug function from _debug() to _DEBUG() to enhance readability.

#------------------------------------------------------------------------ # Version 0.09 Date: 1997/05/15 09:13:00 #------------------------------------------------------------------------

• Added an "UNLESS" option to INCLUDE directives for convenience, effectively giving the opposite to the "IF" directive. An INCLUDE directive may contain both an IF and an UNLESS statement in which case, the IF is evaluated first, and the UNLESS second. If the IF condition fails, the UNLESS will not be evaulated. If the UNLESS evaluates true, the INCLUDE will be skipped, even if the IF condition returned TRUE.
• Added INDEX directive to build a list of INCLUDE directives within a block. This is removed in a later version, not least of all because it proved to be unwieldy and of limited use.
• Optimised and tidied up the main _process() loop.

#------------------------------------------------------------------------ # Version 0.10 Date: 1997/05/19 08:37:05 #------------------------------------------------------------------------

• Made a couple of minor changes to get Perl 5.004 running -w without warnings.
• Enhanced _index() to only index INCLUDE directives whose identifiers match the "element=<regex>,<regex>..." pattern specified in the INDEX directive. Case sensitivity to the pattern match depends on $self->{ CASE }.
• Changed the rather generic "DIRECTORY" configuration option to "ELEMPATH" (element path) which more accurately describes what it actually refers to. Pity I couldn't get it right second time around - I change it to LIB in version 0.12.
• Finally fixed and rationalised the handling of carriage returns at the end of lines. The problem was, particularly with DEFINE directives, where the directive was removed during processing leaving a blank line it it's place. A number of DEFINE directives on subsequent lines (as is quite common) leaves a great big chunk of whitespace. The solution is to ignore any carriage return that occurs immediately after the closing directive magic token. Any trailing characters after the directive, including whitespace, will get a carriage return added to it. The original carriage return gets eaten by the regex, in case you wondered what happened to it.
• Changed the "print" INCLUDE option (in which the print format could be specified) to be "format" which makes more sense, IMHO.
• Removed the print format and filter code out of process() and into it's own function, postprocess(). This has several advantages. Firstly, it means that the code can conveniently be called for other directives apart from INCLUDE (SUBST now passes it's output through the printf filter as well). Secondly, the directive object itself can be passed to postprocess() meaning that the "filter" and "format" options can now be stored as internal directive values rather than entries in the tags table.
• Changed the print format code to attempt to recognise the difference between a printf() format and a time2str() format. A format that matches "%[^s]" is passed through time2str() from Date::Format. Other formats containing no other marker than "%s" are assumed to be printf() formats. A special "%P" marker in the format is ignored, but silently invokes the printf() option regardless. "%s" is handled identically by either printf() or time2str().
• Modified _substitute() slightly to look for certain tag(s) before attempting to call the tag as a function. The tag(s) and their associated return values are:

TIME Returns current system time in seconds sinch the epoch

• Changed AUTOLOAD to return undef if the original (unresolvable) directive should be left as is. The value is passed up through _substitute() to the relevant SUBST block in _process(). The original directive is reconstructed here and the postprocess() step is bypassed.
• Added POD documentation.

#------------------------------------------------------------------------ # Version 0.11 Date: 1997/07/22 10:18:15 #------------------------------------------------------------------------

• Fixed fix in AUTOLOAD that was keeping directives from being deleted when ROGUE=delete was defined.
• Fixed bug in process() which was returning an output list rather than concatenating the contents when __END__ was detected. The usual return point had been changed some time ago.
• Added CHOMP configuration item which allows the user to specify the end-of-line processing behaviour. With CHOMP defined to any true value, a newline immediately following a directive will be chomped off. Modified preprocess() to handle this.
• Corrected, enhanced and added further POD documentation.
• Added 'postproc' debug flag for examining postprocess() activity and data.

#------------------------------------------------------------------------ # Version 0.12 Date: 1997/11/28 07:56:24 #------------------------------------------------------------------------

• Replaced the DEBUGLEVEL constant sub-routines for the new "use constant...";
• Updated the tests (a little).
• Added the EXECUTE configuration option to control how the processor attempts to resolve unknown identifiers. With EXECUTE true, the _substitute() method tried to run it as a package method. With EXECUTE set > 1, it will also try to run it as a function in the main package, assuming that the previous method call failed.
• Added PRINT as a special directive parameter (for BLOCK) rather than have it as just another parameter.
• Re-organised the code a little, listing methods roughly in the order in which they're used.
• Replaced the "ELEMPATH" configuration variable with "LIB".
• Moved the revision history (this list) and the bugs/future enhancements lists into separate files: "Changes" and "Todo", respectively.
• Changes parseparams() to unescape any escaped characters, rather than just quotes.

#------------------------------------------------------------------------ # Version 0.13 Date: 1997/12/05 15:58:02 #------------------------------------------------------------------------

• Modified Makefile.PL to install metapage as well.
• Updated and completed documentation for module.

#------------------------------------------------------------------------ # Version 0.14 Date: 1998/01/30 15:50:37 #------------------------------------------------------------------------

• Fixed some significant bugs in the _evaluate() method and added 'eval' script to test general evaluation conditions.
• Fixed regex comparitor match bug in _evaluate() which was causing the simple evaluation 'if="variable"' to fail.
• Fixed a number of typos and made a few enhancements to the Text::MetaText documentation.
• Added documentation to metapage utility.

#------------------------------------------------------------------------ # Version 0.15 Date: 1998/02/04 08:57:14 #------------------------------------------------------------------------

• Added 'ignore' and 'accept' options to metapage (1.06)
• First public beta release.

#------------------------------------------------------------------------ # Version 0.16 Date: 1998/02/16 11:21:02 #------------------------------------------------------------------------

• Fixed 'FILETIME' bug in metapage which wasn't being set with '-t' option.
• Fixed another _evaluate() bug which caused conditionals of the form if="$undefined == 0" to return a false negative.
• Added above test to t/*/eval
• Separated directive parsing code out to the Text::MetaText::Directive module.

#------------------------------------------------------------------------ # Version 0.17 Date: 1998/02/25 16:24:15 #------------------------------------------------------------------------

• Changed directory search order so that the current directory ('.') is searched after the LIB directories. This can be over-ridden by explicitly stating a file in the current directory as "./file"
• Added symbolname(), symboldefined() and symbolentry() to give consistant access to the symbol table. Operations that may modify the symbol name (such as case folding when working CASE insensitively) are now localised to symbolname().
• Added variablename() and variablevalue() to provide consistent access to variable names and values. Support for CASE (in)sensitivity is implemented here. Modified _interpolate(), integrateparams() and _substitute() to use them.
• Removed AUTOLOAD method and no longer derive Text::MetaText from AutoLoader. Previously, AUTOLOAD had been used to resolved unrecognised SUBST symbols. This is now handled inside _substitute().
• Added public methods process_file() and process_text(). The latter processes the contents of the text string passed. The original process() function remains for backward compatibility but simply calls process_file(). Added the private _process() method which is called by the above methods.
• Added private methods parsefile(), parsetext() and parse() which are called by the respective processXXXX() methods.
• Added getline() and ungetline() to act as lexical tokenisers for feeding to parseXXXX() methods.
• Fixed bug in _parse() which was concatentating consecutive lines without adding spaces in between them.
• Reworked and updated test suite.
• Fixed a copule of tyops in the domucentation.
• Modified metapage to work with the improved CASE sensitivity handling. It seems that CASE sensitive variable handling was largely broken. I never noticed because I never use it case sensitively. Don't know why I ever implemented it really... This is a quick kludge to get it working until I get around to implementing CASEVARS in Version 0.18.
• Updated documentation to include new changes.

#------------------------------------------------------------------------ # Version 0.18 Date: 1998/03/13 15:22:48 #------------------------------------------------------------------------

• Changed ROGUE option to store a hash array of options rather than a list which had to be grep'd every time. Added a t/rogue.t test script and modified test.pl to include a post-processing hook so that the test script can write MetaText warnings/errors to the output file. This can then be compared against the expected output (t/expect/test.n) to see if the warning was expected or not.
• Added "__MTEND__" as an alias for "__END__". Useful when defining a text string in a script and you want to use "__END__" but can't because perl will think it's meant for it.
• Added CASEVARS configuration option which specifies a list of variables whose names shouldn't be folded to lower case when running case insensitively. This is used by metapage to supply "system" variables such as FILENAME, FILETIME, FILEPATH, etc. When defined as CASEVARS, they must be specified in the same (UPPER) case even if CASE sensitivity is OFF. The variables 'filename', 'filetime' and 'filepath' may also be user defined and their definiton and use will not be affected by these additional variables. i.e. CASEVARS don't stomp on normal vars.
• Modified metapage to use CASEVARS option
• Updated documentation and cleaned up few miscellaneous rough edges

#------------------------------------------------------------------------ # Version 0.19 Date: 1998/04/02 09:25:16 #------------------------------------------------------------------------

• Rationalised error handling and added an internal ERROR variable to store error state. "warnings" are reported via _warn() as per usual, and fatal errors are reported via a new method _error() which sets the internal ERROR variable and then calls _warn() to report the error. Any ERROR handling function defined is now stored in the ERRORFN internal rather than the previous ERROR which is now used as described.
• Updated metapage to version 1.11. Changes include:
  • quoting values in config file [define] section is now ok
  • added "no execution" mode (-n)
  • better reporting of MetaText errors
  • improved file statistic and verbose messages
  • several miscellaneous improvements to internal code
  • updated docs to reflect changes
• Second public beta release consisting of:

Text::MetaText v0.19
Text::MetaText::Directive v0.02
metapage v1.11

#------------------------------------------------------------------------ # Version 0.20 Date: 1998/04/07 16:14:51 #------------------------------------------------------------------------

• Fixed bug in postprocess() which was incorrectly treating an input of '0' as an undefined value and converting to ''.
• Added the public declare($input, $name) method which allows text strings to be pre-parsed into the symbol table. The method also allows $input to be a reference to an array which can contain text strings or MetaText Directive references as its elements. This is effectively the 'compiled' or 'parsed' version of an input. There is currently no way to cleanly create Directives for this, so using this form of declare() requires a little black magic until I get around to sorting out the Directive class. The people who want to use it already know how to do it so they should be happy.
• Added test for declare() method.
• Minor documentation updates.

#------------------------------------------------------------------------ # Version 0.21 Date: 1998/04/28 15:31:56 #------------------------------------------------------------------------

• Added a MetaText Factory object (Text::MetaText::Factory) which is responsible for parsing and creating directives. This abstracts some of the code that was previously in Directive into an object which can better manage the process (by trapping errors, for example). It makes the MetaText module more configurable by allowing the Factory and Directive classes to be sub-classed and bound into MetaText at run time.
• Removed parsing code from Directive class (Text::MetaText::Directive). The Directive class is now a simple data container serving as a suitable base class for derivation.
• Added a FACTORY configuration option to _configure() which allows a Factory instance to be bound in at run time.
• Added the _factory() method to return a reference to the internal Factory instance. If a Factory is not bound at run time via the configuration option, FACTORY, then a default Factory object (Text::MetaText::Factory) is created and stored the first time a _factory() method call is made.
• Changed directive KEYWORD member to TYPE to better describe what it actually is. Modified _parse() and _process() accordingly.

#------------------------------------------------------------------------ # Version 0.22 Date: 1998/09/01 11:23:14 #------------------------------------------------------------------------

• Added 4 pre-defined formats: dquoted, squoted, quoted (same as dquoted) and money. These are specified as such: %% whatshesaid format=quoted %%.
• Added parseerror() method which adds file/line reporting to the error message passed.
• Removed all trailing newlines from error messages. One is subsequently printed by the _warn() function but the message is stored without it.
• Added HAS_CONDITION and HAS_POSTPROC parameters to the Directive (added in the Directive constructor) that are used within _process() to optimise the cases where conditionals and/or post-process formatting aren't required.
• Doing the above tickled a well-hidden bug. It seems that CHOMP wasn't doing exactly what it should in removing empty lines after directives. However, the postprocess() method was effectively reversing this process by chomping off the final newline. When postprocess() became optional, the bug showed itself and was duly whacked across the head with a large stick.
• Added TRIM configuration and directive parameters which follow a better heuristic for cleaning up extraneous newlines surrounding a BLOCK definition. The CHOMP and TRIM behaviour really needs to be rationalised further.
• In implementing the above, I changed the way in which "control" parameters are listed internally for various directives. It is now possible to specify control params on a per-directive basis rather than the previous all-or-nothing approach. This should make it easier to extend the parameters for a specific directive without affecting the others.
• Heavily updated the existing test suite and added a few more tests to reflect the changes to CHOMP and TRIM.
• Fixed a bug in _process() which was causing the condition "if=0" to return a false positive.
• Documented TRIM option and pre-defined format types: quoted, dquoted, squoted.
• Documented declare() method and added "VARIABLE INTERPOLATION" section.
• Added the File::Recurse module to Makefile.PL as a pre-requisite for metapage.
• Third public beta release consisting of:

Text::MetaText v0.22
Text::MetaText::Directive v0.04
Text::MetaText::Factory v0.02
metapage v1.11