Issues fixed after release 20211029

Fix tokenization issue c109

Automated random testing produced an error tokenizing the following code fragment:

    s s(..)(.)sss
    ;

This is equivalent to 's/(..)(.)//s' with 's' as the delimiter instead of '/'. It was tokenized correctly except when the final 's' was followed by a newline, as in the example. When the delimiter is a letter rather than a punctuation character, perltidy exercises some seldom-used code which had an off-by-one loop limit. This has been fixed.

12 Nov 2021.

Fix tokenization of $$^, issue c106

Automated random testing produced an error tokenizing the following fragment:

   my$seed=$$^$^T;

The first ^ should have been tokenized as the bitwise xor operator but was not. This is fixed with this update.

8 Nov 2021

Fix coding error, issue c104

Automated random testing produced an error with something like the following input line taken from an obfuscated perl script:

    open(IN, $ 0);

The '0' was missing in the output:

    open( IN, $ );

The tokenization was correct, but a line of code in the formatter which removes the space between the '$' and the '0' should have used a 'defined' when doing a check:

    $token .= $word if ($word);             # OLD: error

This if test fails on '0'. The corrected line is

    $token .= $word if ( defined($word) );  # NEW: fixes c104

This fixes the problem and gives the correct formatted result

    open( IN, $0 );

8 Nov 2021.

Fix undefined variable reference, c102

Random testing produced an undefined variable reference for the following input

    make_sorter ( sort_sha => sub {sha512 ( $_} );
    make_sorter ( sort_ids => sub {/^ID:(\d+)/} );

when formatted with the following input parameters:

    --space-function-paren
    --maximum-line-length=26
    --noadd-newlines

Notice that the input script has a peculiar syntax error - the last two closing tokens of the first line are transposed. (Ironically, this snippet is taken from code which accompanied the book Perl Best Practices). The perltidy tokenizer caught the syntax error, but the formatter touched an undefined variable while attempting to do the formatting. It would be possible to just skip formatting for errors like this, but it can sometimes help finding bugs to see an attempted formatting. So the formatter coding has been corrected to avoid the undefined variable reference.

This fixes issue c102.

5 Nov 2021.

Some blocks with side comments exceed line length

In some rare cases, one-line blocks with side comments were exceeding the line length limit. These usually had a semicolon between the closing block brace and the side comment. For example:

        my $size
            = do { local $^W; -f $local && -s _ }; # no ALLO if sending data from a pipe

This update breaks the one-line block in an attempt to keep the total length below the line length limit. The result on the above is:

        my $size = do {
            local $^W;
            -f $local && -s _;
        };    # no ALLO if sending data from a pipe

Note that this break can be prevented by including the flag --ignore-side-comment-lengths or -iscl.

3 Nov 2021.

Issues fixed after release 20210625

Fix c090, inconsistent warning messages for deprecated syntax

For something like the following snippet, a warning about deprecated syntax was either going into the error file or the log file, depending on formatting. This has been fixed.

   do $roff ( &verify($tpage) );

20 Oct 2021, 72e4bb1.

Fix c091, incorrect closing side comment

An error was discovered and corrected in the behavior of the --closing-side-comment (-csc) flag when only subs were being marked with the setting -cscl='sub'. The problem was that in rare cases a closing paren could be marked with '## end'. The cause of the problem is that the pattern matching regex which was generated for this case happens to match an empty string, and it could happen that certain parens had empty strings as block names. This was fixed in two ways. First, the regex was fixed so that it cannot match an empty string. Second, a test for an empty string was added.

20 Oct 2021, aa1a019.

Issue c089, improve vertical alignment for lists without parens

An update was made to improve vertical alignment in situations where parens are omitted around lists. The goal is to make lists without parens align as they would if they were contained in parens. Some examples:

    # OLD, no parens, no alignment:
    glVertex3d $cx + $s * $xs, $cy, $z;
    glVertex3d $cx, $cy + $s * $ys, $z;
    glVertex3d $cx - $s * $xs, $cy, $z;
    glVertex3d $cx, $cy - $s * $ys, $z;

    # OLD, with parens and aligned:
    glVertex3d( $cx + $s * $xs, $cy,            $z );
    glVertex3d( $cx,            $cy + $s * $ys, $z );
    glVertex3d( $cx - $s * $xs, $cy,            $z );
    glVertex3d( $cx,            $cy - $s * $ys, $z );

    # NEW, no parens but aligned
    glVertex3d $cx + $s * $xs, $cy,            $z;
    glVertex3d $cx,            $cy + $s * $ys, $z;
    glVertex3d $cx - $s * $xs, $cy,            $z;
    glVertex3d $cx,            $cy - $s * $ys, $z;

    # OLD
    mkTextConfig $c, $x, $y, -anchor => 'se', $color;
    mkTextConfig $c, $x + 30, $y, -anchor => 's',  $color;
    mkTextConfig $c, $x + 60, $y, -anchor => 'sw', $color;
    mkTextConfig $c, $x, $y + 30, -anchor => 'e', $color;

    # NEW
    mkTextConfig $c, $x,      $y,      -anchor => 'se', $color;
    mkTextConfig $c, $x + 30, $y,      -anchor => 's',  $color;
    mkTextConfig $c, $x + 60, $y,      -anchor => 'sw', $color;
    mkTextConfig $c, $x,      $y + 30, -anchor => 'e',  $color;

    # OLD
    permute_test [ 'a', 'b', 'c' ],   '/', '/', [ 'a', 'b', 'c' ];
    permute_test [ 'a,', 'b', 'c,' ], '/', '/', [ 'a,', 'b', 'c,' ];
    permute_test [ 'a', ',', '#', 'c' ], '/', '/', [ 'a', ',', '#', 'c' ];
    permute_test [ 'f_oo', 'b_ar' ], '/', '/', [ 'f_oo', 'b_ar' ];

    # NEW
    permute_test [ 'a', 'b', 'c' ],      '/', '/', [ 'a', 'b', 'c' ];
    permute_test [ 'a,', 'b', 'c,' ],    '/', '/', [ 'a,', 'b', 'c,' ];
    permute_test [ 'a', ',', '#', 'c' ], '/', '/', [ 'a', ',', '#', 'c' ];
    permute_test [ 'f_oo', 'b_ar' ],     '/', '/', [ 'f_oo', 'b_ar' ];

    # OLD:
    is $thingy, "fee",           "source filters apply to evalbytten strings";
    is "foo",   $unfiltered_foo, 'filters leak not out of byte evals';
    is $av->[2], "NAME:my_xop",          "OP_NAME returns registered name";
    is $av->[3], "DESC:XOP for testing", "OP_DESC returns registered desc";
    is $av->[4], "CLASS:$OA_UNOP",       "OP_CLASS returns registered class";
    is scalar @$av, 7, "registered peep called";
    is $av->[5], "peep:$unop", "...with correct 'o' param";
    is $av->[6], "oldop:$kid", "...and correct 'oldop' param";

    # NEW
    is $av->[2],    "NAME:my_xop",          "OP_NAME returns registered name";
    is $av->[3],    "DESC:XOP for testing", "OP_DESC returns registered desc";
    is $av->[4],    "CLASS:$OA_UNOP",       "OP_CLASS returns registered class";
    is scalar @$av, 7,                      "registered peep called";
    is $av->[5],    "peep:$unop",           "...with correct 'o' param";
    is $av->[6],    "oldop:$kid",           "...and correct 'oldop' param";

20 Oct 2021, 1dffec5.

Issue c087, breaking after anonymous sub

This update keeps both of these configurations stable for all cases except when the -lp option is used. For the -lp option, both become one-line blocks (the second case) to prevents the -lp indentation style from being lost. This update was made to minimize changes to existing formatting.

    $obj = {
        foo => sub { "bar" }
    };

    $obj = { foo => sub { "bar" } };

17 Oct 2021, f05e6b5.

Improve formatting of some Moose structures

In some structures used in Moose coding, some asymmetrical container breaks were being caused by the braces being tokenized as hash braces rather than block braces. This was also causing some unwanted warning messages.

    # OLD
    ::is(
        ::exception { has '+bar' => ( default => sub { 100 } );
        },
        undef,
        '... extended the attribute successfully'
    );

    # NEW
    ::is(
        ::exception {
            has '+bar' => ( default => sub { 100 } );
        },
        undef,
        '... extended the attribute successfully'
    );

This fixes issue c074.

12 Oct 2021, 7e873fa.

Fix issue c081, -cscw preventing deletion of closing side comments

Random testing revealed a problem in which an old closing side comment was not being deleted when it fell below the interval specified on -csci=n and the -cscw flag was also set.

For example, the following snippet has been formatted with -csc -csci=1. The interval -csci=1 insures that all blocks get side comments:

    if ($x3) {
        $zz;
        if ($x2) {
            if ($x1) {
                ..;
            } ## end if ($x1)
            $a;
            $b;
            $c;
        } ## end if ($x2)
    } ## end if ($x3)

If we then run with -csc -csci=6, the comment ## end if ($x1) will fall below the threshold and be removed (correctly):

    if ($x3) {
        $zz;
        if ($x2) {
            if ($x1) {
                ..;
            }
            $a;
            $b;
            $c;
        } ## end if ($x2)
    } ## end if ($x3)

But if we also add the -cscw flag (warnings) then it was not being removed. This update fixes this problem (issue c081).

2 Oct 2021, 25ef8e8

Partial fix for issue git #74 on -lp at anonymous subs

In the following snippet, the final one-line anonymous sub is not followed by a comma. This caused the -lp mode to revert to standard indentation mode because a forced line break was being made after the closing sub brace:

    # OLD, perltidy -lp
    $got = timethese(
        $iterations,
        {
          Foo => sub { ++$foo },
          Bar => '++$bar',
          Baz => sub { ++$baz }
        }
    );

An update was made to check for and fix this.

    # NEW, perltidy -lp
    $got = timethese(
                      $iterations,
                      {
                         Foo => sub { ++$foo },
                         Bar => '++$bar',
                         Baz => sub { ++$baz }
                      }
    );

But note that this only applies to one-line anonymous subs. If an anonymous sub is broken into several lines due to its length or complexity, then these forced line breaks cause indentation to revert to the standard indentation scheme.

22 Sep 2021, 4fd58f7.

Testing with random parameters produced a formatting instability related to the -vmll flag. The problem was due to a subtle difference in the definition of nesting depth and indentation level. The vertical aligner was using nesting depth instead of indentation level to compute the maximum line length when -vmll is set. In some rare cases there is a difference. The problem was fixed by passing the maximum line length to the vertical aligner so that the calculation is only done in the formatter. This fixes b1209.

20 Sep 2021, acf1d2d.

Fix issue b1208

Testing with random parameters produced a formatting instability which could be triggered when there is a short line length limit and there is a long side comment on the closing brace of a sort/map/grep/eval block. The problem was due to not correctly including the length of the side comment when testing to see if the block could fit on one line. This update fixes issue b1208.

18 Sep 2021, 0af1321.

Fix issue git #73

The -nfpva parameter was not working correctly for functions called with pointers. For example

    # OLD: perltidy -sfp -nfpva
    $self->method                ( 'parameter_0', 'parameter_1' );
    $self->method_with_long_name ( 'parameter_0', 'parameter_1' );

    # FIXED: perltidy -sfp -nfpva
    $self->method ( 'parameter_0', 'parameter_1' );
    $self->method_with_long_name ( 'parameter_0', 'parameter_1' );

The problem had to do with how the pointer tokens '->' are represented internally and has been fixed.

17 Sep 2021,e3b4a6f.

Fix unusual parsing error b1207

Testing with random parameters produced another instability caused by misparsing an 'x' operator after a possible file handle. This is very similar to b1205, but involves a sigil immediately following a times operator.

To illustrate some cases, consider:

    sub x {print "arg is $_[0]\n"}
    my $num = 3;
    my @list=(1,2,3);
    my %hash=(1,2,3,4);
    open (my $fh, ">", "junk.txt");
    print $fh x$num;   # prints a GLOB $num times to STDOUT
    print $fh x9;      # prints 'x9' to file 'junk.txt'
    print $fh x@list;  # prints a GLOB 3 times to STDOUT
    print $fh x%hash;  # prints a GLOB 2 times to STDOUT

Note in particular the different treatment of the first two print statements.

This update fixes case b1207.

15 Sep 2021, 107586f.

Fix formatting instability b1206

Testing with random parameters produced an instability due welding with a very short line length and large value of -ci. This is similar to issues b1197-b1204 and fixed with a similar method.

14 Sep 2021, 9704cd7.

Fix unusual parsing error b1205

Testing with random parameters produced an instability caused by misparsing an 'x' operator after a possible file handle. Testing with Perl showed that an 'x' followed by a '(' in this location is always the 'times' operator and never a call to a function 'x'. If x is immediately followed by a number it is subject to the usual weird parsing rules at such a location.

To illustrate, consider what these statements do:

    open( my $zz, ">", "junk.txt" );
    sub x { return $_[0] }  # never called
    print $zz x(2);    # prints a glob 2 times; not a function call
    print $zz x 2;     # prints a glob 2 times
    print $zz x2;      # syntax error
    print $zz x;       # syntax error
    print $zz z;       # prints 'z' in file 'junk.txt'

This update fixes case b1205.

13 Sep 2021, cfa2515.

Use stress_level to fix cases b1197-b1204

Testing with random input parameters produced a number of cases of unstable formatting. All of these involved some combination of a short maximum line length, a large -ci and -i, and often one or more of -xci -lp and -wn. These parameters can force lines to break at poor locations. All of these cases were fixed by introducing a quantity called the 'stress_level', which is the approximate indentation level at which the line break logic comes under high stress and become unstable. For default parameters the stress level is about 12, but unusual parameter combinations can make it much less, even as low as zero. For code which is at an indentation level greater than this depth, some defensive actions are taken to avoid instability, such as temporarily turning off the -xci flag when the indentation depth exceeds the stress level. Most actual working code will not be influenced by this logic. Actual code which has a very deep indentation level can avoid problems by using a long line length, a short number of indentation spaces, or even the whitespace-cycle parameter.

This update fixes issues b1197 b1198 b1199 b1200 b1201 b1202 b1203 b1204

12 Sep 2021, 0ac771e.

Fix unusual hanging side comment issue, c070

This issues can be illustrated with the following input script:

    {
        if ($xxx) {
          ...
        } ## end if ($xxx ...
        # b <filename>:<line> [<condition>]
    }


    # OLD: perltidy -fpsc=21
    {
        if ($xxx) {
            ...;
        } ## end if ($xxx ...
                        # b <filename>:<line> [<condition>]
    }

The comment '# b ..' moved over to the column 21 to the right as if it were a side comment. The reason is that it accidentally got marked as a hanging side comment. It should not have been because the previous side comment was a closing side comment. This update fixes this:

    # NEW: perltidy -fpsc=21
    {
        if ($xxx) {
            ...;
        } ## end if ($xxx ...

        # b <filename>:<line> [<condition>]
    }

This fixes issue c070.

10 Sep 2021, ec6ccf9.

Fix parsing issue c068

This issue is illustrated with the following line:

   my $aa = $^ ? "defined" : "not defined";

If we tell perltidy to remove the space before the '?', then the output will no longer be a valid script:

   # perltidy -nwls='?':
   my $aa = $^? "defined" : "not defined";

The problem is that Perl considers '$^?' to be a special variable. So Rerunning perltidy on this gives an error, and perl also gives an error. This update fixes the problem by preventing a space after anything like '$^' from being removed a new special variable would be created.

This fixes issue c068.

7 Sep 2021, 9bc23d1.

Fix parsing problem issue c066

This issue is illustrated with the following line (rt80058):

   my $ok=$^Oeq"linux";

Running perltidy generated a warning message which is caused by the lack of space before the 'eq'. This update fixes the problem.

4 Sep 2021, df79a20.

Fix unusual parsing problem issue c065

Testing produced an unusual parsing problem in perltidy which can be illustrated with the following simple script:

    my $str = 'a'x2.2;
    print $str,"\n";

Normally an integer would follow the 'x' operator, but Perl seems to accept any valid number and truncates it to an integer. So in this case the number 2.2 is truncated to 2 and the output is 'aa'.

But perltidy, with default parameters, formats this as

    my $str = 'a' x 2 . 2;
    print $str,"\n";

which prints the result "aa2". The problem is fixed with this update. With the update, the result is

    my $str = 'a' x 2.2;
    print $str, "\n";

which is equivalent to the original script.

This fixes issue c065.

4 Sep 2021, f242f78.

Fix formatting instability issues b1195, b1196

Testing with random parameters produced two similar cases of unstable formatting which are fixed with this update.

28 Aug 2021, ab9ad39.

Fix formatting instability issue b1194

Testing with random parameters produced a case of unstable formatting which is fixed with this update.

This fixes case b1194, and at the same time it simplifies the logic which handles issues b1183 and b1191.

21 Aug 2021, 62e5b01.

Fix some problems involving tabs characters, case c062

This update fixes some problems found in random testing with tab characters. For example, in the following snippet there is a tab character after 'sub'

    do sub      : lvalue {
        return;
      }

Running perltidy on this repeatedly keep increasing the space between 'sub' and ':'

    # OLD: perltidy
    do sub       : lvalue {
        return;
      }

    # OLD: perltidy
    do sub        : lvalue {
        return;
      }

etc.

    # NEW: perltidy
    do sub : lvalue {
        return;
      }

Problems like this can occur if string comparisons use ' ' instead of the regex \s when working on spaces. Several instances of this were located and corrected.

This fixes issue c062.

18 Aug 2021, d86787f.

Correct parsing error, case c061

Testing with random input produced an error condition involving a side comment placed between a sub name and prototype, as in the following snippet:

    sub
    witch   # sc
    ()   # prototype may be on new line ...
    { print "which?\n" }
    witch();

The current version of perltidy makes an error:

    # OLD: perltidy
    sub witch   # sc ()    # prototype may be on new line ...
    { print "which?\n" }
    witch();

This update corrects the problem:

    # NEW: perltidy
    sub witch    # sc
      ()         # prototype may be on new line ...
    { print "which?\n" }
    witch();

This fixes case c061;

18 Aug 2021, 3bb2b2c.

Improved line break, case c060

The default formatting produced an undesirable line break at the last '&&' term in the following:

    my $static = grep {
             $class       =~ /^$_$/
          || $fullname    =~ /^$_$/
          || $method_name =~ /^$_$/
          && ( $class eq 'main' )
    } grep { !m![/\\.]! } $self->dispatch_to;    # filter PATH

This update corrects this to give

    my $static = grep {
             $class       =~ /^$_$/
          || $fullname    =~ /^$_$/
          || $method_name =~ /^$_$/ && ( $class eq 'main' )
    } grep { !m![/\\.]! } $self->dispatch_to;    # filter PATH

15 Aug 2021, 9d1c8a9.

Fix error check caused by -wn -iscl, case c058

Testing with random parameters triggered an an internal error check. This was caused by a recent coding change which allowed a weld across a side comment. The problem was in the development version, not in the latest released version, and is fixed with this update. This closes issue c058.

14 Aug 2021, 5a13886.

Fix formatting instability, b1193

Testing with random parameters produced unstable formatting involving parameters which included -lp -sbvtc=1. This update fixes this problem, case b1193.

13 Aug 2021, d4c3425.

Fix error in tokenizer, issue c055

The ultimate cause of the undefined variable reference in the previous issue was found to be a typo in the tokenizer. This update finishes fixing issue c055.

10 Aug 2021, 2963db3

Fix undefined variable reference in development version

In testing, the following unbalanced snippet caused a reference to an undefined value in the current development version (but not in the current released version).

        if($CPAN::Config->{term_is_latin}){
                $swhat=~s{([\xC0-\xDF])([\x80-\xBF])}{chr(ord($1)<<6&0xC0|ord($2)&0x3F)}eg;}if($self->colorize_output){if($CPAN::DEBUG&&$swhat=~/^Debug\(/){
                        $ornament=$CPAN::Config->{colorize_debug}||"black on_cyan";}

A check has been added to fix this.

10 Aug 2021, a3f9774.

Fix formatting instability, b1192

Testing with random parameters produced unstable formatting with the following snippet when run with some unusual parameters:

          @EXPORT =
              ( @{$EXPORT_TAGS{standard}}, );

It was also be formatted as

          @EXPORT = (
                      @{$EXPORT_TAGS{standard}},
          );

The problem was that a list formatting style was turning on and off due to the the needless terminal comma within the parens. A patch was made to ignore commas like this when deciding if list formatting should be used.

This fixes case b1192.

10 Aug 2021, b949215.

Fix formatting instability, b1191

Random testing produced an instability involving an unusual parameter combination and the following input code:

    $_[0]eq$_[1]
      or($_[1]=~/^([!~])(.)([\x00-\xff]*)/)
      and($1 eq '!')
      ^(eval{($_[2]."::".$_[0])=~/$2$3/;});

This update fixes case b1191.

9 Aug 2021, 16b4575.

Fix error parsing sub attributes without spaces, b1190

Testing with random parameters produced an instability which was caused by incorrect parsing of a sub attribute list without spaces, as in

        sub:lvalue{"a"}

This update fixes case b1190.

9 Aug 2021, 7008bcc.

Fix rare loss of vertical alignment in welded containers, c053

This update corrects a rare loss of vertical alignment in welded containers.

To illustrate the issue, the normal formatting of the following snippet is

    # perltidy -sil=1 
    ( $msg, $defstyle ) = do {
            $i == 1 ? ( "First", "Color" )
          : $i == 2 ? ( "Then", "Rarity" )
          :           ( "Then", "Name" );
    };

If it appears within a welded container, the alignment of the last line was being lost:

    # OLD: perltidy -wn -sil=1
    { {

        ( $msg, $defstyle ) = do {
                $i == 1 ? ( "First", "Color" )
              : $i == 2 ? ( "Then",  "Rarity" )
              : ( "Then", "Name" );
        };
    } }

The corrected result is

    # NEW: perltidy -wn -sil=1
    { {

        ( $msg, $defstyle ) = do {
                $i == 1 ? ( "First", "Color" )
              : $i == 2 ? ( "Then", "Rarity" )
              :           ( "Then", "Name" );
        };
    } }

Several other minor vertical alignment issues are fixed with this updated. The problem was that two slightly different measures of the depth of indentation were being compared in the vertical aligner.

This fixes case c053.

8 Aug 2021, 97f02ee.

Fix edge case of formatting instability, b1189.

Testing with random parameters produced a case of unstable formatting involving welding with parameter -notrim-qw. The problem was that the -notrim-qw flag converts a qw quote into a quote with fixed leading whitespace. The lines of these types of quotes which have no other code are are output early in the formatting process, since they have a fixed format, so welding does not work. In particular, the closing tokens cannot be welded if they are on a separate line. This also holds for all types of non-qw quotes. So welding can only be done if the first and last lines of a non-qw quote contain other code. A check for this was added.

For example, in the following snippet the terminal '}' is alone on a line:

    is eval(q{
        $[ = 3;
        BEGIN { my $x = "foo\x{666}"; $x =~ /foo\p{Alnum}/; }
        $t[3];
    }
    ), "a";

# In the previous version this was half-welded: # OLD: perltidy -wn -sil=0

    is eval( q{
        $[ = 3;
        BEGIN { my $x = "foo\x{666}"; $x =~ /foo\p{Alnum}/; }
        $t[3];
    }
      ),
      "a";

The new check avoids welding in this case, giving

    # NEW: perltidy -wn -sil=0
    is eval(
        q{
        $[ = 3;
        BEGIN { my $x = "foo\x{666}"; $x =~ /foo\p{Alnum}/; }
        $t[3];
    }
      ),
      "a";

Welding can still be done if the opening and closing container tokens have other code. For example, welding can be done for the following snippet:

    is eval(q{
        $[ = 3;
        BEGIN { my $x = "foo\x{666}"; $x =~ /foo\p{Alnum}/; }
        $t[3];
    }), "a";

And welding can still be done on all qw quotes unless the -notrim-qw flag is set.

This fixes case b1189.

7 Aug 2021, e9c25f2.

Fix edge cases of formatting instability, b1187 b1188.

Testing with random parameters produced some cases of instability involving -wn -lp and several other parameters. The mechanism involved an interaction between the formatter and vertical aligner.

This fixes cases b1187 and b1188.

3 Aug 2021, 5be949b.

Fix edge case of formatting instability, b1186.

Testing with random parameters produced a case of instability involving parameter -lp -pvt=n and a short maximum line length.

This fixes case b1186.

2 Aug 2021, f3dbee1.

Fix edge case of formatting instability, b1185.

Testing with random parameters produced a case of welding instability involving parameters -wn, -vt=2, -lp and a short maximum line length.

This fixes case b1185.

1 Aug 2021, d2ab2b7.

Fix edge case of formatting instability, b1183.

Testing with random parameters produced a case of welding instability involving parameters -wn, -vt=n, -lp. This update fixes the problem.

This fixes case b1183.

30 Jul 2021, 055650b.

Fix edge case of formatting instability, b1184.

Testing with random parameters produced a case of welding instability involving a tripple weld with parameters -wn, -vt=n, -lp. This update fixes the problem.

This fixes case b1184.

29 Jul 2021, 6dd53fb.

Fix edge case of formatting instability, b1182.

Testing with random parameters produced a case of welding instability involving unusual parameters and welding long ternary expressions. This update fixes the problem.

This fixes case b1182.

28 Jul 2021, 01d6c40.

Fix edge case of formatting instability.

Random testing with random input parameters produced cases of formatting instability involving welding with unusual parameter settings. This update makes a small tolarance adjustment to fix it.

This fixes cases b1180 b1181.

28 Jul 2021, b38ccfc.

Fix rare problem with formatting nested ternary statements

This update fixes an extremely rare problem in formatting nested ternary statements, illustrated in the following snippet:

  # OLD: There should be a break before the '?' in line 11 here:
  WriteMakefile(
      (
          $PERL_CORE ? ()
          : (
              (
                  eval { ExtUtils::MakeMaker->VERSION(6.48) }
                  ? ( MIN_PERL_VERSION => '5.006' )
                  : ()
              ),
              (
                  eval { ExtUtils::MakeMaker->VERSION(6.46) } ? (
                      META_MERGE => {
                          #
                      }
                    )
                  : ()
              ),
          )
      ),
  );

  # NEW: Line 12 correctly begins with a '?'
  WriteMakefile(
      (
          $PERL_CORE ? ()
          : (
              (
                  eval { ExtUtils::MakeMaker->VERSION(6.48) }
                  ? ( MIN_PERL_VERSION => '5.006' )
                  : ()
              ),
              (
                  eval { ExtUtils::MakeMaker->VERSION(6.46) }
                  ? (
                      META_MERGE => {
                          #
                      }
                    )
                  : ()
              ),
          )
      ),
  );

This fixes issue c050, 63784d8.

22 Jul 2021.

Fix conflict of -bom and -scp parameters

Automated testing with random parameters produced a case of instability caused by a conflict of parameters -bom and -scp. In the following script the -bom command says to keep the tokens ')->' on a new line, whereas the -scp command says to stack the closing paren on the previous line.

        $resource = {
                id => $package->new_from_mana(
                        $result->{data}
                )->id
        };

The parameters are:

    --break-at-old-method-breakpoints
    --indent-columns=8
    --maximum-line-length=60
    --stack-closing-paren

This caused an instability which was fixed by giving priority to the -bom flag. The stable state is then

        $resource = { id => $package->new_from_mana(
                        $result->{data}
        )->id };

This fixes case b1179.

21 Jul 2021, 4ecc078.

Fix problems with -kgb in complex structures

This update fixes two problems involving the -kgb option.

The first problem is that testing with random parameters produced some examples of formatting instabilities involving applying --keyword-group-blanks to complex structures, particularly welded structures. The -kgb parameters work well on simple statements or simple lists, so a check was added to prevent them from working on lists which are more than one level deep. This fixes issues b1177 and b1178 without significantly changing the -kgb functionality.

The second problem is that a terminal blank line could be misplaced if the last statement was a structure. This is illustrated with the following snippet:

    sub new {
      my $class = shift;
      my $number = shift || croak "What?? No number??\n";
      my $classToGenerate = shift || croak "Need a class to generate, man!\n";
      my $hash = shift; #No carp here, some operators do not need specific stuff
      my $self = { _number => $number,
                   _class => $classToGenerate,
                   _hash => $hash };
      bless $self, $class; # And bless it
      return $self;
    }

    # OLD: perltidy -kgb -sil=0 gave
    sub new {

        my $class           = shift;
        my $number          = shift || croak "What?? No number??\n";
        my $classToGenerate = shift || croak "Need a class to generate, man!\n";
        my $hash = shift;   #No carp here, some operators do not need specific stuff
        my $self = {
            _number => $number,

            _class => $classToGenerate,
            _hash  => $hash
        };
        bless $self, $class;    # And bless it
        return $self;
    }

The blank line which has appeared after the line '_number =>' was intended to go after the closing brace but a line count was off. This has been fixed:

    # NEW: perltidy -kgb -sil=0 gives
    sub new {

        my $class           = shift;
        my $number          = shift || croak "What?? No number??\n";
        my $classToGenerate = shift || croak "Need a class to generate, man!\n";
        my $hash = shift;   #No carp here, some operators do not need specific stuff
        my $self = {
            _number => $number,
            _class  => $classToGenerate,
            _hash   => $hash
        };

        bless $self, $class;    # And bless it
        return $self;
    }

This fixes issue c048.

19 Jul 2021, 071a3f6.

Fix to keep from losing blank lines after a code-skipping section

A problem in the current version of perltidy is that a blank line after the closing code-skipping comment can be lost if there was a blank line before the start of the code skipping section. For example, given the following code:

    $x = 1;

    #<<V
    % # = ( foo => 'bar', baz => 'buz' );
    print keys(%#), "\n";
    #>>V

    @# = ( foo, 'bar', baz, 'buz' );
    print @#, "\n";

running perltidy gives:

    $x = 1;

    #<<V
    % # = ( foo => 'bar', baz => 'buz' );
    print keys(%#), "\n";
    #>>V
    @# = ( foo, 'bar', baz, 'buz' );
    print @#, "\n";

Notice that the blank line after the closing comment #>>V is missing. What happened is that the formatter is counting blank lines and did not 'see' the code skipping section. So the blank after the closing comment looked like the second blank in a row, so it got removed since the default is --maximum-consecutive-blank-lines=1.

This update fixes this by resetting the counter. This fixes case c047. A simple workaround until the next release is to include the parameter

--maximum-consecutive-blank-lines=2, or -mbl=2.

It can be removed after the next release.

18 Jul 2021, 9648e16.

Fix possible welding instability in ternary after fat comma

Random testing produced a case of formatting instability involving welding within a ternary expression following a fat comma:

    if ( $op and $op eq 'do_search' ) {
        @{$invoices} =
          GetInvoices(
              shipmentdatefrom =>
              $shipmentdatefrom ? output_pref( {
                             str => $shipmentdatefrom,
                             dateformat => 'iso'
              } )
              : undef,
              publicationyear => $publicationyear,
          );
    }

when the following specific parameters were used

    --extended-continuation-indentation
    --line-up-parentheses
    --maximum-line-length=38
    --variable-maximum-line-length
    --weld-nested-containers

This update fixes this issue, case b1174.

18 Jul 2021, 12ae46b.

Fix mis-tokenization before pointer

Random testing produced the following case in which formatting was unstable because the variable '$t' was being mis-tokenized as a possible indirect object.

    --break-before-all-operators
    --ignore-old-breakpoints
    --maximum-line-length=22
    -sil=0

    my $json_response
      = decode_json $t
      ->tx->res->content
      ->get_body_chunk;

This update fixes cases b1175, b1176.

17 Jul 2021, 4aa1318.

Fix to make -wn and -bbxx=n flags work together

Testing with random parameters produced some cases where the combination of -wn and various -bbxx=n flags were not working together. To illustrate, consider the following script (-sil=1 just means start at 1 indentation level)

    # perltidy -sil=1
    $$d{"day_name"} = [
        [
            "lundi",    "mardi",  "mercredi", "jeudi",
            "vendredi", "samedi", "dimanche"
        ]
    ];

With welding we get:

    # -sil=1 -wn
    $$d{"day_name"} = [ [
        "lundi",    "mardi",  "mercredi", "jeudi",
        "vendredi", "samedi", "dimanche"
    ] ];

With -bbsb=3 (--break-before-square-brackets) we get:

    # -sil=1 -bbsb=3
    $$d{"day_name"} =
      [
        [
            "lundi",    "mardi",  "mercredi", "jeudi",
            "vendredi", "samedi", "dimanche"
        ]
      ];

So far, so good. But for the combination -bbsb=3 -wn we get

    # OLD: ERROR
    # -sil=1 -bbsb=3 -wn
    $$d{"day_name"} = [ [
        "lundi",    "mardi",  "mercredi", "jeudi",
        "vendredi", "samedi", "dimanche"
    ] ];

which is incorrect because it ignors the -bbsb flag. The corrected result is

    # NEW: OK
    # -sil=1 -bbsb=3 -wn
    $$d{"day_name"} =
      [ [
        "lundi",    "mardi",  "mercredi", "jeudi",
        "vendredi", "samedi", "dimanche"
      ] ];

This update fixes case b1173. It works for any number of welded containers, and the -bbxxi=n flags also work correctly.

16 Jul 2021, c71f882.

Fix problem with side comment after pattern

Testing with randomly placed side comments produced an error illustrated with the following snippet:

    testit
    /A (*THEN) X | B (*THEN) C/x#sc#
    ,
    "Simple (*THEN) test"
    ;

If 'testit' is an unknown bareword then perltidy has to guess if the '/' is a division or can start a pattern. In this case the side comment caused a bad guess. This is case c044 and is fixed with this update. There are no other known issues with side comments at the present time but testing continues.

13 Jul 2021, 8b36de8.

Fix problem with side comment after pointer, part 3

This update fixes some incorrect tokenization produced by a side comment placed between a pointer and a bareword as in this snippet:

    sub tzoffset {};

    ...

    my $J
    +=
    (
    $self
    ->#sc
    tzoffset
    / (
    24
    *
    3600
    )
    );

If a sub declaration for the bareword has been seen, the following '/' was being rejected as an operator. This update fixes this case, which is issue c043.

13 Jul 2021, cab7ed3.

Avoid line breaks before a slash in certain cases

This is a modification to the previous update for case c039 which prevents a line break before a '/' character which follows a bareword or possible indirect object. This rule will be only be used to prevent creating new line breaks. Existing line breaks can remain.

11 Jul 2021, 7afee47.

Fix error parsing sub attributes with side comment

Testing with side comments produced an error in the following snippet:

    sub plugh () :#
      Ugly('\(") : Bad;

This is fixed in this update, case c038.

11 Jul 2021, 80f2a3a.

Fix case b1172, a failure to converge

Random testing produced case b1172, a failure to converge with unusual parametrs. This update fixes this case. There are no other known cases of instability at the present time but testing continues.

10 Jul 2021, 47e7f9b.

Avoid line breaks before a slash in certain cases

This update prevents a line break before a '/' character which follows a bareword or possible indirect object. The purpose is reduce the chance of introducing a syntax error in cases where perl is using spaces to distinguish between a division and the start of a pattern.

This fixes case c039.

10 Jul 2021, 461199c.

Removed warning message if ending in code skipping section

In the previous version, a warning was produced if a 'code-skipping' opening comment '#<<V' was not followed by a closing comment '#>>V'. But the related 'format-skipping' commands do not give a warning if a '#<<<' comment is not ended with a '#>>>' closing comment. In order to be able to smoothly change between these options, it seems best to remove the warning about a missing '#>>V'. There is still a message in the log file about this, so if there is any uncertainty about it, a log file can be saved and consulted.

10 Jul 2021, 461199c.

Improve logic for distinguishing a pattern vs a division

Testing with side comments produced the following snippet which caused a error due to the side comment on the '/'

    $bond_str
    =
    VERY_WEAK #sc#
    / #sc#
    1.05
    ;

Several related examples were found in which side comments triggered errors. For example

    ok
    /[^\s]+/#sc#
    ,
    'm/[^\s]/ utf8'
    ;

This update fixes these problems, case c040.

9 Jul 2021, ffe4351.

Fix problem caused by side comment after ++

Testing with side comments produced an incorrect error message for this snippet:

    xxx 
    ++#
    $test, 
    Internals::SvREADONLY( %$copy) , 
    "cloned hash restricted?" ;

The problem was caused by the side comment between the '++' and '$test'. The same problem occurs for '--' instead of '++'. This is fixed with this update, case c042.

8 Jul 2021, 20cc9a0.

Fix problem caused by side comment after pointer, part 2

This is related to the previous issue, c037. The following snippet was misparsed at the old style ' package separater due to the side comment following the pointer.

    @ret
    =
    $o
    ->#
    SUPER'method
    (
    'whatever'
    )
    ;

This is fixed in this update, case c041.

7 Jul 2021, 1806772.

Fix problem caused by side comment after pointer

The following test script

    is(
    $one
    ->#sc#
    package
    ,
    "bar"
    ,
    "Got package"
    )
    ;

Caused the following error message:

  4: package
     ^
  found package where operator expected

The problem was caused by a side comment between the pointer '->' and the word 'package'. This caused package to be misparsed as a keyword, causing the error.

This is fixed in this update, case c037, 96f2ebb.

Fix error parsing '%#' and similar combinations

Perltidy was correctly distinguishing between '$#' and '$ #' but not between '@#' and '@ #' and '%#' and '% #'. The coding for parsing these types of expressions has been corrected. Some simple examples:

    # this is a valid program, '%#' is a punctuation variable
    %# = ( foo => 'bar', baz => 'buz' );
    print keys(%#), "\n";

    # but this is a syntax error (space before # makes a side comment)
    # (perltidy was ignoring the space and forming '%#' here)
    % # = ( foo => 'bar', baz => 'buz' );
    print keys(%#), "\n";

    # this is a valid program, '@#' is a punctuation variable
    @# = ( foo , 'bar', baz , 'buz' );
    print @#, "\n";

    # this is a valid program, the space makes the '#' a side comment
    # perltidy formed %# here, causing an error
    % #
    var = ( foo => 'bar', baz => 'buz' );
    print keys(%var), "\n";

This fixes case c036.

6 Jul 2021, e233d41.

Fix error parsing '&#'

The following test script caused an error when perltidy did not correctly parse the tight side comment after the '&' (it parsed correctly if there was a space before the '#').

    print$my_bag
    &#sc#
    $your_bag
    ,
    "\n"
    ;

This update fixes this issue, c033.

5 Jul 2021, 0d784e0.

Fix error parsing format statement

The following test script caused an error when perltidy took 'format' to start a format statement.

    my$ascii#sc#
    =#sc#
    $formatter#sc#
    ->#sc#
    format#sc#
    (#sc#
    $html#sc#
    )#sc#
    ;#sc#

This was fixed by requiring a format statement to begin where a new statement can occur. This fixes issue c035.

5 Jan 2021, 2ef16fb.

Fix some incorrect error messages due to side comments

Testing with large numbers of side comments caused perltidy to produce some incorrect error messages. Two issues are fixed with this update. First, a side comment between a pointer '->' and the next identifier caused a message. Second, in some cases a comment after an opening paren could cause a message. The following snippet is an example.

    $Msg#sc#
    ->#sc#
    $field#sc#
    (#sc#
    )#sc#
    ;#sc#

This update fixes cases c029 and c030.

4 Jul 2021, caffc2c.

Fix undefined var ref involving --format-skipping

Testing produced a situation in which version 20200625 could cause an undefined variable to be accessed (the variable 'Ktoken_vars') as in this snippet:

    #!/usr/bin/perl
    #<<<
        my $ra= (
            [ 'Shine', 40 ], [ 'Specular', [ 1, 1, 0.3, 0 ] ] );
    #<<<
    ...

The conditions for this to happen are:

  (1) format skipping (#<<<) begins before the first line of code, and
  (2) the format skipping section contains the two successive characters ', ['.

The undefined variable was 'Ktoken_vars'. This problem was introduced by commit 21ef53b, an update which fixed case b1100. This undefined variable access does influence the formatted output.

This update fixes this problem.

4 Jul 2021, 82916fe.

Check for side comment within package statement

Testing with randomly placed side comments caused perltidy to produce an incorrect warning when a side comment is placed before the end of a package statement:

    package double # side comment
    ;

This update fixes this. 3 Jul 2021, c00059a.

Fix problem with -comma-arrow-breakpoint=n flag

Testing revealed a formatting irregularity which was caused when the flag -cab=2 got applied to a container which was not a list. This is fixed with update, which fixes case b939a.

1 Jul 2021, 021b938.

Fix a formatting instability

Testing showed that a previously fixed case of instability, b1144, which was fixed 21 Jun 2021, 1b682fd, was unstable again. This update is a small change which fixes this. There are no other known unstable cases at this time but automated testing runs continue to search for instabilities.

1 Jul 2021, 021b938.

Fixed use of uninitialized value

The previous Tokenizer update caused the use of an unitialized value when run on case b1053:

 Use of uninitialized value $next_nonblank_token in pattern match (m//) at /home/steve/bin/Perl/Tidy/Tokenizer.pm line 7589.
 Use of uninitialized value $nn_nonblank_token in pattern match (m//) at /home/steve/bin/Perl/Tidy/Tokenizer.pm line 3723.
 b1053 converged on iteration 2

This update fixes this.

1 Jul 2021, ea139bd.

Fix token type of colon introducing anonomyous sub attribute list

In the following example

    print "not " unless ref +(
        map {
            sub : lvalue { "a" }
        } 1
    )[0] eq "CODE";

the colon after 'sub' was being marked as part of a label rather than the start of an attribute list. This does not cause an error, but the space before the ':' is lost. This is fixed in this update.

Note that something like 'sub :' can also be a label, so the tokenizer has to look ahead to decide what to do. For example, this 'sub :' is a label:

    my $xx = 0;
    sub : {
        $xx++;
        print "looping with label sub:, a=$xx\n";
        if ( $xx < 10 ) { goto "sub" }
    }

In this case, the goto statement needs quotes around "sub" because it is a keyword.

29 Jun 2021, d5fb3d5.

Minor adjustments to improve formatting stability

Testing with random input parameters produced several new cases of formatting instability involving unusual parameter combinations. This update fixes these cases, b1169 b1170 b1171, and all previously discovered cases remain stable with the update.

28 Jun 2021, e1f22e0.

Remove limit on -ci=n when -xci is set, see rt #136415

This update undoes the update c16c5ee of 20 Feb 2021, in which the value of -ci=n was limited to the value of -i=n when -xci was set. Recent improvements in stability tolerances allow this limit to be removed.

28 Jun 2021, 1b3c5e9.

Minor optimization

Added a quick check to bypass a needless sub call.

26 Jan 2021, e7822df.

Eliminate token variable _LEVEL_TRUE_

It was possible to eliminate this token variable by changing the order of welding operations. This reduces the number of variables per token from 12 to 11.

26 Jun 2021, 1f4f78c.

Eliminate token variable _CONTAINER_ENVIRONMENT_

Testing with NYT_Prof shows that the number of variables per token has a direct effect on efficiency. This update reduces the number of token variables from 13 to 12, and also simplifies the coding. It was possible to compute this variable from the others, so it was redundant.

26 Jun 2021, 300ca1e.

Issues fixed after release 20210402

Release 20210625

24 Jun 2021, a4ff53d.

Adjust tolerances to fix some unstable edge cases

Testing with random input parameters produced a number of edge cases of unstable formatting which were traced to the parameter combinations which included -lp and some other unusual settings.

This fixes cases b1103 b1134 b1135 b1136 b1138 b1140 b1143 b1144 b1145 b1146 b1147 b1148 b1151 b1152 b1153 b1154 b1156 b1157 b1163 b1164 b1165

There are no other known cases of formatting instability at the present time, but testing with random input parameters will continue.

21 Jun 2021, 1b682fd.

Adjust tolerances to fix some unstable edge cases

Testing with random input parameters produced a number of edge cases of unstable formatting which were traced to the parameter combinations which included -bbxi=2 and -cab=2. A small adjustment to length tolerances was made to fix the problem.

This fixes cases b1137 b1149 b1150 b1155 b1158 b1159 b1160 b1161 b1166 b1167 b1168.

19 Jun 2021, 4d4970a.

Added flag -atnl, --add-terminal-newline, see git #58

This flag, which is enabled by default, allows perltidy to terminate the last line of the output stream with a newline character, regardless of whether or not the input stream was terminated with a newline character. If this flag is negated, with -natnl, then perltidy will add a terminal newline to the the output stream only if the input stream is terminated with a newline.

Negating this flag may be useful for manipulating one-line scripts intended for use on a command line.

This update also removes the description of the obsolete --check-syntax flag from the man pages and help text.

18 Jun 2021, 6f83170.

Allow --delete-side-comments to work with -nanl

The -nanl flag (--noadd-newlines) was preventing side comments from being deleted, for example:

    # perltidy -dsc -nanl
    calc()    # side comment

The same issue was happening for --delete-closing-side comments. This has been fixed.

18 Jun 2021, dbfd802.

Update welding rule to avoid unstable states

Testing with random input parameters produced a formatting instability involving an unusual parameter combination:

    --noadd-whitespace
    --break-before-paren=3
    --continuation-indentation=8
    --delete-old-whitespace
    --line-up-parentheses
    --weld-nested-containers

and the following code

        if(defined$hints{family}){
            @infos=({
                     family=>$hints{family},
                     socktype=>$hints{socktype},
                     protocol=>$hints{protocol},
            });
        }

This update fixes the problem, case b1162.

18 Jun 2021, 76873ea.

Convert some weld sub calls to hash lookups

This is a minor optimization. These subs are eliminated: is_welded_right_at_K, is_welded_left_at_K, weld_len_right_at_K.

17 Jun 2021, 1691013.

Update LineSink.pm to allow undefined line endings

This update is necessary to eventually prevent an unwanted terminal newline being added to a file.

17 Jun 2021, 2600533.

Fix incorrect sub call

This fixes an incorrect call which could cause an incorrect weld.

16 Jun 2021, 068a28b.

Add --code-skipping option, see git #65

Added a new option '--code-skipping', requested in git #65, in which code between comment lines '#<<V' and '#>>V' is passed verbatim to the output stream without error checking. It is simmilar to --format skipping but there is no error checking, and is useful for skipping an extended syntax.

16 Jun 2021, 99ec876.

Handle nested print format blocks

Perltidy was producing an error at nested print format blocks, such as

    format NEST =
    @<<<
    {
        my $birds = "birds";
        local *NEST = *BIRDS{FORMAT};
        write NEST;
        format BIRDS =
    @<<<<<
    $birds;
    .
    "nest"
      }
    .

It was ending the first format at the first '.' rather than the second '.' in this example. This update fixes this, issue c019.

13 Jun 2021.

Allow stacked labels without spaces

When labels are stacked in a single line, such as

A:B:C:

the default is to space them:

A: B: C:

This update allows the spaces to be removed if desired:

# perltidy -naws -dws A:B:C:

13 Jun 2021, c2a63b2

Fix edge cases of instability involving -wn -lp

Random testing produced some cases of instability involving -wn -lp and some unusual additional parameters. These were traced to a test for welding, and were fixed by refining a certain tolerance. This fixes cases b1141, b1142.

12 Jun 2021, 125494b.

Remove incorrect warning at repeated function paren call

This update removes an incorrect error messagge at the construct ')('. To illustrate, the following is a valid program:

    my @words = qw(To view this email as a web page go here);
    my @subs;
    push @subs, sub { my $i=shift; $i %= @words; print "$words[$i] "; return $subs[0]};
    $subs[0](0)(1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11);
    print "\n";

However perltidy was giving an error message at the ')(' combination, which is unusual in perl scripts. This update fixes this.

These are function call parens, so logically they should be under control of the -sfp or --space-function-parens parameter. I wrote a patch to do this, but decided not to implement it. The reason is that, as noted in the manual, subtle errors in perl scripts can occur when spaces are placed before parens. So, to avoid possible problems, the -sfp parameter will be restricted to spaces between a bareword [assumed to be a function] and a paren.

This update is in Tokenizer.pm and fixes case c017.

6 Jun 2021, 6551d65.

Add warning when lexical sub names match some builtins

This update adds a warning when lexical subs have names which match some builtin names which will almost certainly cause a parsing error in the current version of perltidy. For example, the following program is valid and will run, but perltidy will produce an error.

    use feature qw(lexical_subs);
    use warnings; no warnings "experimental::lexical_subs";
    {
      my sub y { print "Hello from y: $_[0]\n"; }
      y(1);
    }

6 Jun 2021, 32729fb.

Minor cleanups

This update fixes a case of formatting instability recently found with random testing. It also does some minor coding cleanups.

This fixes case b1139.

5 Jun 2021, b8527ab.

Revised data structures for welding

This update replaces the data structures used for the welding option with simpler but more general structures. This cleans up the code and will simplify future coding. No formatting changes should occur with this update.

4 Jun 2021, 4a886c8.

improved treatment of lexical subs

This update improves the treatment of lexical subs. Previously they were formatted correctly but an error would have been produced if the same name was used for lexical subs defined in different blocks.

For example, running the previous version of perltidy on the following:

    use feature qw(lexical_subs);
    use warnings; no warnings "experimental::lexical_subs";
    {
        my sub hello { print "Hello from hello1\n" }
        {
            my sub hello { print "Hello from hello2\n" }
            hello();
        }
        hello();
    }
    {
        my sub hello { print "Hello from hello3\n" }
        hello();
    }

gave the (incorrect) error message:

    6: already saw definition of 'sub hello' in package 'main' at line 4
    12: already saw definition of 'sub hello' in package 'main' at line 6

This update fixes that.

1 Jun 2021, 85ecb7a.

add v-string underscores; warn of leading commas

This update cleans up a couple of open issues in the tokenizer.

A warning message will be produced for a list which begins with a comma:

            my %string = (
              ,       "saddr",    $stest,  "daddr",
              $dtest, "source",   $sname,  "dest")

This warning had been temporarily deactivated.

Underscores in v-strings without a leading 'v' are now parsed correctly.

Several comments have been updated.

31 May 2021, ef44e70.

Fix parsing error at operator following a comma

The following lines are syntactically correct but some were producing a syntax error

    print "hello1\n", || print "hi1\n";
    print "hello2\n", && print "bye2\n";
    print "hello3\n", or print "bye3\n";
    print "hello4\n", and print "bye4\n";

For example, the first line produced this message

    1: print "hello1\n", || print "hi1\n";
                       - ^
    found || where term expected (previous token underlined)

This has been fixed. This fixes case c015.

27 May 2021, b537a72.

Added optional o in octal number definitions

An optional letter 'o' or 'O' in the octal representation of numbers, which was added in perl version 5.33.5, is now recogized. The leading zero is still required.

For example:

    $a = 0o100;
    $a = 0O100;

26 May 2021, 544df8c.

Fix several problems with -lp formatting

This update fixes several problems with -lp formatting which are all somewhat related.

ISSUE #1 (cases c002 and c004): A problem involving -lp -wn and certain qw lists

The last line of a welded qw list was being outdented even if it contained text as well as the closing container token. This update fixes the problem and simplifies the logic.

A few examples (all use 'perltidy -wn -lp'):

    # OLD and NEW: OK, closing qw paren is on separate line
    $roads->add_capacity_path( qw( CoolCity 10 ChocolateGulch 8
                               PecanPeak 10 BlueberryWoods 6
                               HotCity
    ) );

    # OLD: poor; outdented text not aligned with previous text
    $roads->add_capacity_path( qw( CoolCity 10 ChocolateGulch 8
                               PecanPeak 10 BlueberryWoods 6
    HotCity ) );

    # NEW:
    $roads->add_capacity_path( qw( CoolCity 10 ChocolateGulch 8
                               PecanPeak 10 BlueberryWoods 6
                               HotCity ) );

    # OLD:
    $roads->add_capacity_path( qw( ChocolateGulch 3 StrawberryFields 0
    StrawberryFields ) );

    # NEW:
    $roads->add_capacity_path( qw( ChocolateGulch 3 StrawberryFields 0
                               StrawberryFields ) );

    # OLD:
    my $mon_name = ( qw(January February March April
                     May June July August
    September October November December) )[$mon];

    # NEW
    my $mon_name = ( qw(January February March April
                     May June July August
                     September October November December) )[$mon];

ISSUE #2 (case c007): A rare program error could be triggered with -lp -xci

In some very rare circumstances it was possible to trigger a "Program error" message. The program still ran to completion. The conditions for this to occur were that flags -lp -xci were set, and that there was a container of sort/map/grep blocks, and there was a side comment on the closing paren. For example:

    # OLD: perltidy -lp -xci, gave an error message and extra indentation here
    my %specified_opts = (
          (
                          map { /^no-?(.*)$/i ? ($1 => 0) : ($_ => 1) }
                          map { /^--([\-_\w]+)$/ } @ARGV
          ),    # this comment caused an error with flags -lp -xci
    );

    # NEW: perltidy -lp -xci, no error
    my %specified_opts = (
          (
             map { /^no-?(.*)$/i ? ( $1 => 0 ) : ( $_ => 1 ) }
             map { /^--([\-_\w]+)$/ } @ARGV
          ),    # this comment caused an error with flags -lp -xci
    );

ISSUE #3 (case c008): In some unusual cases the -lp formatting style was not being applied when it should have been. For example (text is shifted right):

    # NEW: perltidy -lp
    $result = runperl(
        switches => [ '-I.', "-w" ],
        stderr   => 1,
        prog     => <<'PROG' );
    SIG
    PROG

    # NEW: perltidy -lp
    $result = runperl(
                       switches => [ '-I.', "-w" ],
                       stderr   => 1,
                       prog     => <<'PROG' );
    SIG
    PROG

25 May 2021, 6947fe9.

Modify welding rules

This is an update to the patch 19 Apr 2021, eeeaf09. It restricts that patch to -lp formatting mode.

This fixes case b1131.

21 May 2021, a4ec4c1.

Fix inconsistency involving counting commas

Random testing produced a formatting instability involving the combination of flags -bbp=2 -xci -vt=2 -bvtc=2. The problem was traced to an error in counting the number of line ending commas in lists.

This fixes case b1130.

15 May 2021, 90cceb1.

Slightly modify line breaks for -lp indentation

Random testing produced an edge case of formatting instability for -lp indentation which was traced to checking for an old line break at a '=>'. This has been fixed. Some existing formatting with deeply nested structures may be slightly changed due to the fix, but most existing formatting will be unchanged.

This fixes b1035.

15 May 2021, dd42648.

Rewrite coding for -bom flag

Random testing produced some examples of formatting instability involving the -bom flag in combination with certain other flags which are fixed with this update. As part of this update, a previous update to fix case b977 (21 Feb 2021, commit 28114e9) was revised to use a better criterion for deciding when not to keep a ')->' break. The previous criterion was that the opening and closing containers should be separated by more than one line. The new criterion is that they should contain a list. This still fixes case b977. Another case, b1120, was fixed by requiring that only parentheses expressions be considered for keeping a line break, not '}->' or ']->'.

Some issues are illustrated in the following examples using '-bom -gnu'. In the first example the leading ')->' was being lost due to the old b977 fix:

    # input:
    $show = $top->Entry( '-width' => 20,
                       )->pack('-side' => 'left');

    # OLD: perltidy -gnu -bom
    $show = $top->Entry('-width' => 20,)->pack('-side' => 'left');

    # NEW: perltidy -gnu -bom
    $show = $top->Entry(
                        '-width' => 20,
                       )->pack('-side' => 'left');

In the following example a leading '->' was being lost. The NEW version keeps the leading '->' but has to give up on the -lp alignment because of complexity:

        # input
        $_make_phase_arg = join(" ",
                           map {CPAN::HandleConfig
                                 ->safe_quote($_)} @{$prefs->{$phase}{args}},
                          );

        # OLD: perltidy -gnu -bom
        $_make_phase_arg = join(" ",
                                map { CPAN::HandleConfig->safe_quote($_) }
                                  @{$prefs->{$phase}{args}},
                               );

        # NEW: perltidy -gnu -bom
        $_make_phase_arg = join(
            " ",
            map {
                CPAN::HandleConfig
                  ->safe_quote($_)
            } @{$prefs->{$phase}{args}},
        );

In the following example, a leading ')->' was being converted to a leading '->' due to the old b977 fix:

    # Starting script
    $lb = $t->Scrolled("Listbox", -scrollbars => "osoe"
                      )->pack(-fill => "both", -expand => 1);

    # OLD: perltidy -bom -gnu
    $lb = $t->Scrolled( "Listbox", -scrollbars => "osoe" )
      ->pack( -fill => "both", -expand => 1 );

    # NEW: perltidy -bom -gnu
    $lb = $t->Scrolled(
                       "Listbox", -scrollbars => "osoe"
                      )->pack(-fill => "both", -expand => 1);

In the following example, a leading ')->' was being lost, again due to the old b977 fix:

    $myDiag->Label(-text => $text,
                                  )->pack(-fill => 'x',
                                                  -padx => 3,
                                                  -pady => 3);

    # OLD: -gnu -bom
    $myDiag->Label(-text => $text,)->pack(
                                          -fill => 'x',
                                          -padx => 3,
                                          -pady => 3
                                         );

    # NEW -gnu -bom
    $myDiag->Label(
                   -text => $text,
      )->pack(
              -fill => 'x',
              -padx => 3,
              -pady => 3
             );

This update fixes case b1120 and revises the fix for b977.

13 May 2021, d0ac5e9.

Adjust tolerances for some line length tests

Random testing produced some edge cases of unstable formatting involving the -lp format. These were fixed by using a slightly larger tolerance in the length test for containers which were broken in the input file.

This fixes cases b1059 b1063 b1117.

13 May 2021, 24a11d3.

Do not apply -lp formatting to containers with here-doc text

If a container contains text of a here-doc then the indentation is fixed by the here-doc text, so applying -lp formatting does not work well. So this update turns off the -lp formatting in this case.

But note that if a container contains a here target but not the here text so it still gets the -lp indentation:

    # perltidy -lp
    &WH::Spell::AddSpell(
                          "Cause Light Wounds", "WFP",
                          "CauseLightWounds",   <<'EOH');
    ...
    EOH

This fixes case b1081.

10 May 2021, 4f7a56b.

Fix some edge welding cases

Some adjustments in welding coding was made to maintain stability for some unusual parameter combinations.

This fixes cases b1111 b1112.

9 May 2021, 68f619a.

Improve tolerance for welding qw quotes

The tolerance for welding qw quotes has been update to be the same as used for welding other tokens. This fixes case b1129.

9 May 2021, d1de85f.

Revise weld tolerances, simplify code, fix welded ci

The welding process uses a tolerance to keep results stable. Basically, a zero tolerance is used if it looks like it is welding an existing weld, while a finite tolerance is used otherwise. The effect is to reject a few marginal welds to gain stability. The coding to do this was simplified and the tolerance was made more precise to fix case b1124.

Another change with this update is that at welded containers, the value of the -ci flag of an outer opening token is transferred to the inner opening token. This may improve some indentation in a few cases if the -lp flag is also used. It has no effect if -lp is not used.

    # OLD: perltidy -wn -gnu
    emit_symbols([qw(
     ctermid
     get_sysinfo
     Perl_OS2_init
     ...
     CroakWinError
    )]);

    # NEW: perltidy -wn -gnu
    emit_symbols([qw(
        ctermid
        get_sysinfo
        Perl_OS2_init
        ...
        CroakWinError
    )]);

9 May 2021, ad8870b.

Correct brace types mismarked by tokenizer, update

This is a generalization of commit 7d3bf4 in which the tokenizer sends a signal to the formatter if the type of a brace following an unknown bareword had to be guessed. The formatter has more information and can fix the problem. This fixes case b1128.

8 May 2021, b3eaf23.

Added warning when -ci has to be reduced; ref rt #136415

In commit c16c5ee an update was made to prevent instability with -xci when the value of -ci exceeds -i (which is not recommended). This update adds a warning message to avoid confusing the user.

7 May 2021, e9e14e4.

Improve indentation of welded multiline qw quotes

Formatting of multiline qw lists with welding works best if the opening and closing qw tokens are on separate lines, like this:

    # perltidy -wn
    my $mon_name = ( qw(
        January February March April
        May June July August
        September October November December
    ) )[$mon];

    # perltidy -wn -lp
    my $mon_name = ( qw(
                     January February March April
                     May June July August
                     September October November December
    ) )[$mon];

Otherwise formatting can be poor, particularly if the last line has text and a closing container.

    # OLD: perltidy -wn
    my $mon_name = ( qw(January February March April
        May June July August
    September October November December) )[$mon];

Note that perltidy does not change the line breaks within multiline quotes, so they must be changed by hand if desired.

This update indents the last line of a multiline quote which contains both text and the closing token, such as:

    # NEW: perltidy -wn
    my $mon_name = ( qw(January February March April
        May June July August
        September October November December) )[$mon];

This update is only when the -lp flag is not used. If -lp is used and the last line contains text, the last line is still outdented:

    $ OLD and NEW: perltidy -wn -lp
    my $mon_name = ( qw(January February March April
                     May June July August
    September October November December) )[$mon];

This is difficult to fix. The best solution is to place the closing qw qw containers on a separate line.

This fix is for case c002.

6 May 2021, 176f8a7.

Test length of closing multiline qw quote before welding

Random testing produced an unstable state which was due to not checking for excessive length of the last line of a multiline qw quote. A check was added, this fixes issue b1039.

5 May 2021, b72ad24.

Update welding rule to avoid blinking states

Random testing with unusual parameter combinations produced some unstable welds. For example case b1106 has these parameters

    --noadd-whitespace
    --continuation-indentation=6
    --delete-old-whitespace
    --line-up-parentheses
    --maximum-line-length=36
    --variable-maximum-line-length
    --weld-nested-containers

and was switching between these two states:

            return map{
                ($_,[$self->$_(@_[1..$#_])])
            }@every;

            return map { (
                $_, [ $self->$_( @_[ 1 .. $#_ ] ) ]
            ) } @every;

An existing rule, WELD RULE 2, was updated to prevent welding to an intact one-line weld, as in the first snippet, if it is on a separate line from the first opening token. With this change, both of these states are stable.

This update fixes cases b1082 b1102 b1106 b1115.

4 May 2021, 07efa9d.

Fix problem of conflict of -otr and -lp

Several random test cases produced an instability involving -otr and -lp. In -lp mode, when an opening paren follows an equals and is far to the right, a break is made at the equals to minimize the indentation of the next lines. The -otr flag is a suggestion that an opening paren should be place on the right. A check has been made to avoid this in -lp mode following an equals, since that defeats the purpose of the original break.

This fixes cases b964 b1040 b1062 b1083 b1089.

4 May 2021, 24a0d32.

Add option -pvtc=3, requested in rt136417

A new integer option, n=3, has been added to the vertical tightness closing flags. For a container with n=3, the closing token will behave as for n=0 if the opening token is preceded by an '=' or '=>', and like n=1 otherwise.

3 May 2021, 93063a7.

Fix vertical alignment issue in rt136416

This update fixes a problem with unwanted vertical alignment rasied in rt#136416. The example is

    use File::Spec::Functions 'catfile', 'catdir';
    use Mojo::Base 'Mojolicious',        '-signatures';

An update was made to reject alignments in use statements with different module names. The test file t/snippets/align35.in has more examples.

3 May 2021, 048126c.

Fix some rare issues with the -lp option

Random testing produced some rare cases of unstable formatting involving the -lp option which are fixed with this update. This is a generalization of commit edc7878 of 23 Jan 2021. This fixes cases b1109 b1110.

2 May 2021, a8d1c8b.

Correct brace types mismarked by tokenizer

This is a generalization of commit 7d23bf4 to fix some additional cases found in random testing in which the type of a curly brace could not be determined in the tokenizer and was not being corrected by the formatter.

This fixes cases b1125 b1126 b1127.

2 May 2021, dac97cb.

Avoid instability of combination -bbx=2 -lp and -xci

Random testing produced several cases in which the flags -bbx=2 -lp and -xci were causing formatting instability. The fix is to locally turn off -xci when -lp and -bbx=2 are in effect. This is an extension of commit 2b05051.

This fixes cases b1090 b1095 b1101 b1116 b1118 b1121 b1122 b1099

1 May 2021, 4cb81ba.

Restrict use of flag -cab=3 to simple structures

Logic was added to turn off -cab=3 in complex structures. Otherwise, instability can occur. When it is overridden the behavior of the closest match, -cab=2, is used instead.

For example, using these parameters for case b1113

    --noadd-whitespace
    --break-before-hash-brace-and-indent=2
    --break-before-hash-brace=1
    --comma-arrow-breakpoints=3
    --continuation-indentation=9
    --maximum-line-length=76
    --variable-maximum-line-length

formatting of the following snippet was unstable:

    $STYLESHEET{'html-light'}={
        'tags'=>{
            'predefined identifier'=>
                     {
                'start'=>'<font color="#2040a0"><strong>',
                'stop'=>'</strong></font>'
                     },
        }
    };

This update fixes cases b1096 b1113.

29 Apr 2021, 32a1830.

Update docs for git #64 regarding -lp and side comments

The wording regarding when -lp reverts to the default indentation scheme has been revised to include side comment as follows:

In situations where perltidy does not have complete freedom to choose line breaks it may temporarily revert to its default indentation method. This can occur for example if there are blank lines, block comments, multi-line quotes, or side comments between the opening and closing parens, braces, or brackets.

The word 'may' is significant for side comments. In a list which is just one level deep side comments will work (perhaps with -iscl if side comments are long). For example this is ok

    # perltidy -lp
    $gif->arc(
               50, 50,     # Center x, y.
               30, 30,     # Width, Height.
               0,  360,    # Start Angle, End Angle.
               $red
    );

But if a list is more than one level deep then the default indentation is used.

28 Apr 2021, 49977b8.

Fix line break rules for uncontained commas + cleanups

This is an adjustment of update 344519e which had to do with breaking lines with commas which were not inside of a container. In a few cases it was producing very long lines when -l=0 was set. The solution was to remove the concatenation operator from the list of operators at which breaks were prevented.

Other updates are: Remove unused indentation table. Correct maximum_line_length table for -vmll when -wc is also set. Also fix whitespace rule for '$ =' within a signature to fix case b1123.

26 Apr 2021, d014c2a.

Fix problem with -wn and -wc=n

Random testing produced some cases in which the -wn flag was unstable when -wc=n was used with very small n. This has been fixed.

This fixes cases: b1098 b1107 25 Apr 2021, 92babdf.

Adjust line break rules for uncontained commas

Random testing produced case c1119 which was unstable due to the formatting rules for breaking lines at commas which occur outside of containers. The rules were modified to fix the problem.

20 Apr 2021, 344519e.

Fix a bad line break choice at a slash

Random testing produced case c001 in which the following snipppet

   ok $mi/(@mtime-1) >= 0.75 && $ai/(@atime-1) >= 0.75 &&
             $ss/(@mtime+@atime) >= 0.2;

when processed with these parameters

    --maximum-line-length=20
    --nowant-right-space=' / '
    --want-break-before='* /'

produced the following result

    ok $mi
      /( @mtime - 1 ) >=
      0.75
      && $ai
      /( @atime - 1 )
      >= 0.75
      && $ss
      /( @mtime +
          @atime ) >=
      0.2;

using 'perl -cw' on this snippet gives a syntax error

    syntax error at /tmp/issues.t line 5, near "/( "
        (Might be a runaway multi-line // string starting on line 2)

The error is due to perl's weird parsing rules near a possible indrect object. This is a situation where perltidy must ignore a user spacing and line break request. This should have been done but in this case a flag to prevent this was not being propagated to later stages of formatting. This has been fixed.

20 Apr 2021, 4fbc69a.

Fix rare problem with -lp -wn

Random testing produced case b1114 which gave unstable formatting with these parameters

    --noadd-whitespace
    --indent-columns=8
    --line-up-parentheses
    --maximum-line-length=25
    --weld-nested-containers

and this snippet

    is(length(pack("j", 0)),
        $Config{ivsize});

Fixed 19 Apr 2021, eeeaf09.

Fix issue git#63

The following lines produced an error message due to the side comment

    my $fragment = $parser->    #parse_html_string
      parse_balanced_chunk($I);

Fixed 18 Apr 2021, c2030cf.

Avoid welding at sort/map/grep paren calls

Random testing produced several cases of unstable welds in which the inner container something like 'sort ('. The problem was that there are special rules which prevent a break following such a paren. The problem was fixed by preventing welds at these locations.

This update fixes cases b1077 b1092 b1093 b1094 b1104 b1105 b1108.

17 Apr 2021, d679b48.

Fix issue git#62

This fixes issue git #62. A similar issue for the % operator was fixed. 17 Apr 2021, f80d677.

Fix problem involving -bbx=2 -xci -osbr and similar -otr flags

Random testing produced case b1100 in which the output style produced by the --opening-token-right flags interfered with counting line-ending commas, and this in turn caused the -bbx flag to turn off the -xci flag. This problem was fixed.

15 Apr 2021, 21ef53b.

Fix rare line break problem

Random testing produced case b1097 with this parameter set

    --brace-vertical-tightness-closing=1
    --continuation-indentation=8
    --indent-columns=10
    --maximum-line-length=36

and either this output

          my (@files) = @{
                    $args{-files} };

or this output

          my (@files) =
                  @{ $args{-files}
                  };

The different results were caused by the unusual combination of parameters. The problem was fixed by not allowing the formatter to consider existing breaks at highly stressed locations such as these.

15 Apr 2021, 9f15b9d.

Fix problem parsing anonymous subs with attribute lists

Random testing produced case b994 with unstable formatting:

    do
    sub :
    lvalue
    {
    return;
    }

when run with parameters:

    --continuation-indentation=0
    --ignore-old-breakpoints
    --maximum-line-length=7
    --opening-anonymous-sub-brace-on-new-line

The line 'sub :' was being correctly parsed but the following opening block brace was not correctly marked as an anonymous sub brace. This fixes cases b994 and b1053.

15 Apr 2021, 84c1123.

Correct brace types mismarked by tokenizer

Testing with random parameters produced a case in which a brace following an unknown bareword was marked by the tokenizer as a code block brace rather than a hash brace. This can cause poor formatting. The problem was solved by having the tokenizer send a signal to the formatter if a block type was guessed. The formatter has more information and can fix the problem. This fixes case b1085.

11 Apr 2021, 7d23bf4.

Unify coding for welded quotes and other welded containers

Random testing produced some cases where welded quotes were not converging. These were found to be due to the same problem previouly encountered and fixed for normal containers. The problem was fixed by moving the corrected coding to a new common sub.

This update fixes cases b1066 b1067 b1071 b1079 b1080.

10 Apr 2021, 5d73dd5.

Slight change in weld length calculation

Random testing produced some cases of instability with some unusual input parameter combinations involving the -wn parameter. This was fixed by revising a line length calculation. This fixes cases b604 and b605.

9 Apr 2021, a25cfaa.

Improve treatment of -vmll with -wn

Random testing showed a weakness in the treatment of the -vmll flag in combination with the -wn flag. This has been fixed.

This fixes cases b866 b1074 b1075 b1084 b1086 b1087 b1088

8 Apr 2021, a6effa3.

Merge weld rule 6 into rule 3

One of the welding rules, RULE 6, has been merged into RULE 3 for generality. This rule restricts welding to an opening container followed by a bare word, which can cause instability in some cases. The updated code is less restrictive and fixes some cases recently found with random testing, b1078 b1091.

8 Apr 2021, f28ab55.

Moved logic of previous update to the FileWriter module

The previous update regarding blank line generation was not sufficiently general to handle all possible parameter combinations. The problem was solved and simplified by moving the logic to a lower level, in the FileWriter module.

6 Apr 2021, 756e930.

Fix problem with excess blank line generation with -blao

Random testing produced some cases where excess blank lines could be generated with the parameter -blank-lines-after-opening-block. Case b1073 has the following script

    sub stop {

        1;
    }

and the following parameters

    --blank-lines-after-opening-block=2
    --maximum-consecutive-blank-lines=10
    --maximum-line-length=15

When run, blank lines keep getting generated until the maximum is reached. This has been fixed.

6 Apr 2021, 9216298.

Fix edge case involving -wn and -lp or -bbao

A change was made to a welding rule involving the -lp option or -wbb='=', and very short maximum line lengths. This correctly fixes case b1041. It replaces a fix for this case reported on 2 Apr 2021.

5 Apr 2021.

Modify a condition for applying -bbx=2

Random testing produced some edge cases in which formatting with the -bbx=2 flags, in combination with certain other parameters, was not converging. An existing criterion for the -bbx=2 flag to apply is that there be a broken sub-list with at least one line-ending comma. This was updated to also require either a fat comma or one additional line-ending comma. This filters out some problem cases without changing much existing formatting.

This update fixes cases b1068 b1069 b1070 b1072 b1073 b1076.

5 Apr 2021, 16c4591.

Improve previous -wn update

The previous update produced some problems in testing which are corrected with this update.

5 Apr 2021, ffef089.

Fix rare convergence problem with -wn

Random testing produced some cases in which unusual parameter combinations caused lack of convergence for the -wn flag. The problem was fixed by adjusting a tolerance in the line length calculation.

This fixes cases b1041 b1055.

2 Apr 2021, a8b6259.

Avoid conflict of -bli and -xci

Random testing produced a case with the combination -bli and -xci which did not converge. This was fixed by turning off -xci for braces under -bli control.

This fixes case b1065.

2 Apr 2021, d20ea80.

Issues fixed after release 20210111

Avoid conflict of -bbp=2 and -xci

Random testing produced a number of cases of unstable formatting when both -xci and -bbp=2 or similar flags were set. The problem was that -xci can cause one-line blocks to break open, causing the -bbp=2 flag to continually switch formatting. The problem is fixed by locally turning off -xci at containers which do not themselves contain broken containers.

This fixes cases b1033 b1036 b1037 b1038 b1042 b1043 b1044 b1045 b1046 b1047 b1051 b1052 b1061.

30 Mar 2021, 2b05051.

Fix rule for welding with barewords

Random testing produced a case which was not converging due to a rule which avoids welding when a bareword follows. The rule was modified to allow an exception for an existing one-line weld. A non-fatal typo was also discovered and fixed.

This fixes cases b1057 b1064.

29 Mar 2021, d677082.

Fix conflict between -wba='||' and -opr

Random testing produced a problem with convergence due to a conflict between two parameters for the following code

    my $lxy =
      ( @$cx - @$cy ) ||
      (
        length ( int ( $cx->[-1] ) ) -
        length ( int ( $cy->[-1] ) ) );

when using these parameters

    --break-after-all-operators
    --maximum-line-length=61
    --opening-paren-right

Both the '||' and the '(' want to be at the end of a line according to the parameters. The problem is resolved by giving priority to the '||'. This fixes case b1060.

29 Mar 2021, 6921a7d.

Follow user requests better to break before operators

Random testing produced some cases in which user requests to break before selected operators were not being followed. For example

    # OLD: perltidy -wbb='.='
    $value .=
      ( grep /\s/, ( $value, $next ) )
      ? " $next"
      : $next;

    # FIXED: perltidy -wbb='.='
    $value
      .= ( grep /\s/, ( $value, $next ) )
      ? " $next"
      : $next;

This fixes case b1054.

28 Mar 2021, 94f0877.

Fix problems with combinations of -iob -lp

This is an correction to the update of 13 Mar 2021, 71adc77. Random testing produced several additional problems with convergence involving the combination -iob -lp. This update fixes the problem by overriding -iob at some breakpoins which are essential to the -lp parameter.

This update fixes these old cases: b1021 b1023

and these new cases: b1034 b1048 b1049 b1050 b1056 b1058

27 Mar 2021, cc94623.

Add flag -lpxl=s to provide control over -lp formatting

The flag -lpxl=s provides control over which containers get -lp formatting. A shortcut flag -lfp is also added for limiting -lp to simple function calls.

Updated 25 Mar 2021, bfc00fp.

Fix error message for multiple conflicting specifications in -wnxl

There was an error in the coding for an error message which checks for conflicting requests in the -wnxl parameter.

Fixed 21 Mar 2021, 2ef97a2.

Fix issue git #57, Warn_count was not initialized

This update fixes issue git #57, in which a warning flag was not getting zeroed on each new call to perltidy.

19 Mar 2021, b6d296a.

Fix rare problem with combination -lp -wn -naws

This update fixes case b1032 by includeing lines starting with 'if', 'or', and || among the stable breakpoints for welding when -lp -naws flags are also set.

This update also modifies update 7a6be43 of 16 Mar 2021 to exclude list items when checking token lengths. This reduces changes to existing formatting while still fixing the problem in case b1031.

18 Mar 2021, 6200034.

Fix definition of list within list for -bbx flags

Testing produced a blinking state involving a -bbx=2 flag with an unusual combination of other parameters. The problem was traced to the definition of a list containing another list being too restrictive. This update fixes case 1024.

17 Mar 2021, 7f5da0a.

Fix problem with -xci and long tokens

Testing produced an unstable situation involving the -xci flag and tokens which exceed the maximum line length. This fix identifies this situation and locally deactivates the -xci flag. This fixes case b1031.

16 Mar 2021, 7a6be43.

Fix error in parsing use statement curly brace

Testing with random parameters produced some cases where the -xci option was not producing stable results when the maximum line length was set to a very small value. The problem was traced to the tokenizer misparsing a hash brace of a use statement as a code block type. This influences the value of continuation indentation within the braces. The problem was fixed.

This fixes cases b1022 b1025 b1026 b1027 b1028 b1029 b1030

16 Mar 2021, 6371be2.

Fix problems with combinations of -iob -lp -wn -dws -naws

Testing with random parameters produced some situation where the parameter -iob interfered with convergence when parameters -lp and/or -wn were also set. The combination -wn -lp -dws -naws also produced some non-converging states in testing. This update fixes these issues.

The following cases are fixed: b1019 b1020 b1021 b1023

13 Mar 2021, 71adc77.

Simplify sub weld_nested_containers

This update consolidates a number of specialized rules for welding into fewer, simpler rules which accomplish the same effect.

These cases are fixed with this update: b186 b520 b872 b937 b996 b997 b1002 b1003 b1004 b1005 b1006 b1013 b1014

There are no current open issues with the weld logic.

10 Mar 2021, cf3ed23.

Adjust line length tolerance for welding

A minor tolerance adjustment was needed to fix some edge welding cases.

This fixes cases b995 b998 b1000 b1001 b1007 b1008 b1009 b1010 b1011 b1012 b1016 b1017 b1018

7 Mar 2021, b9166ca.

Fix problem with -vtc=n and outdented long lines

Random testing produced an issue with -vtc=1 and an outdented long line. The parameters for b999 are

    --maximum-line-length=75
    --paren-vertical-tightness-closing=1

File 'b999.in' state 1 is

                while ( $line =~
    s/^([^\t]*)(\t+)/$1.(" " x ((length($2)<<3)-(length($1)&7)))/e
                  )

and state 2 is

                while ( $line =~
    s/^([^\t]*)(\t+)/$1.(" " x ((length($2)<<3)-(length($1)&7)))/e)

The problem was fixed by turning off caching for outdented long lines. This fixes case b999.

7 Mar 2021, 3da7e41.

Fix problem with combination -lp and -wbb='='

Random testing produced case b932 in which the combination -lp and -wbb='=' was not stable.

File 'b932.par' is:

    --line-up-parentheses
    --maximum-line-length=51
    --want-break-before='='

File 'b932.in' in the desired state is:

    my @parts
      = decompose( '(\s+|/|\!|=)',
                   $line, undef, 1, undef, '["\']' );

The alternate state is

    my @parts = decompose( '(\s+|/|\!|=)',
                     $line, undef, 1, undef, '["\']' );

The problem was that the -lp code which set a line break at the equals did not check the -wba flag setting.

This update fixes case b932.

7 Mar 2021, 63129c3.

Fix edge formatting cases with parameter -bbx=2

Random testing produced some cases where formatting with parameters of the form --break-before-..=2 can lead to unstable final states. The problem lies in the definition of a broken list. The problem is fixed by defining a broken list for this particular flag to be a list with at least one non-terminal line-ending comma. This insures that the list will remain broken on subsequent iterations. This fixes cases b789 and b938.

6 Mar 2021, 360d669.

Add flag -fpva, --function-paren-vertical-alignment

A flag -fpva, --function-paren-vertical-alignment, is added to prevent vertical alignment of function parens when the -sfp flag is used. This is on by default, so that existing formatting remains unchanged unless the user requests that vertical alignment not occur with -nfpva.

5 Mar 2021, 312be4c.

Fix for issue git #53, do not align spaced function parens

Introducing a space before a function call paren had a side effect of allowing the vertical aligner to align the parens, as in the example.

    # OLD and NEW, default without -sfp:
    log_something_with_long_function( 'This is a log message.', 2 );
    Coro::AnyEvent::sleep( 3, 4 );

    # OLD: perltidy -sfp 
    log_something_with_long_function ( 'This is a log message.', 2 );
    Coro::AnyEvent::sleep            ( 3, 4 );

    # NEW: perltidy -sfp 
    log_something_with_long_function ( 'This is a log message.', 2 );
    Coro::AnyEvent::sleep ( 3, 4 );

This update changes the default to not do this vertical alignment. This should have been the default but this side-effect was missed when the -sfp parameter was added. Note that parens following keywords are likewise not vertically aligned.

5 Mar 2021, 00431bf.

Fix issue git#54 involving -bbp=n and -bbpi=n

In this issue, different results were obtained depending upon the existence of a comma in a list. To fix this, the definition of a list was adjusted from requiring one or more commas to requiring either a fat comma or a comma.

At the same time, a known problem involving the combination -lp -bbp=n -bbpi=n was fixed. This fixes cases b826 b909 b989.

4 Mar 2021, 872d4b4.

Fix several minor weld issues

Some edge cases for the welding parameter -wn have been fixed. There are no other currently known weld issues. Some debug code for welding has been left in the code for possible future use.

This fixes cases b109 b110 b520 b756 b901 b937 b965 b982 b988 b991 b992 b993

3 Mar 2021, cfef087.

Update tokenizer recognition of indirect object

This is the parameter file b990.pro: --noadd-whitespace --continuation-indentation=0 --maximum-line-length=7 --space-terminal-semicolon

Applying perltidy -pro=b990.pro to the following snippet gave two states

    # State 1
    print
    H;

    # State 2
    print H
    ;

The tokenizer was alternately parsing 'H' as either possble indirect object, 'Z', or indirect object, 'Y'. Two fixes were tested. The first was to modify the tokenizer to recognize a ';' as well as a space as a direct object terminator. An alternative fix is to not allowing a break before type 'Y' so that the tokenizer kept parsing as type 'Y'. Both fixes work, but the latter fix would change existing formatting by the -extrude option, so the first fix was used. With this fix, the stable state is 'State 1' above.

This update is a generalization of the update "Fixed blinker related to line break at indirect object" of 16 Jan 2021.

This fixes case b990. 1 Mar 2021, 49cd66f.

Do not start a batch with a blank token

Perltidy does final formatting in discrete batches of tokens, where a batch is a continuous section of the total token list. A batch begins a new line and will be broken into one or more lines. If a batch starts with a blank token it will simply be skipped on on output. However, some rare problems have been found in random testing which can occur if a batch starts with a blank. An example is case b984 which has the following parameters:

        # this is file 'b984.pro'
        --block-brace-vertical-tightness=2
        --indent-columns=10
        --maximum-line-length=27
        --outdent-keywords
        --variable-maximum-line-length

          # OLD: perltidy -pro=b984.pro
          unless (
                    exists $self->{
                              'accession_number'} )
          {         return "unknown";
          }

          # NEW: perltidy -pro=b984.pro
          unless (
                    exists $self->{
                              'accession_number'} )
          {       return "unknown";
          }

Both look OK, but the OLD version did not outdent the keyword 'return' as requested with the -okw flag.

This update fixes cases b149 b888 b984 b985 b986 b987.

28 Feb 2021, 8aaf599.

Avoid double spaces in -csc text output

Random testing produced some rare cases where two spaces could occur in -csc text. This happened when there were multiple lines and the formatter placed a leading blank in one of the continuation lines as padding. This has been fixed.

For example

    while (
        <>;
      )
    {
       ...
    } ## end while ( <>; )

Previously, the last line had an extra space after the ';'

    } ## end while ( <>;  )

Another example

    while (
        do {
            { package DB; @a = caller( $i++ ) }
        }
      )
    {
      ...
    } ## end while ( do { { package DB...}})

Previously the last line had an extra space between the opening braces:

    } ## end while ( do {  { package DB...}})

27 Feb 2021, b22e891.

Remove control of debug flag -fll

Random testing produced an unstable state when a debug flag, -nfll, was set. The only time it is appropriate to set this flag is if the -extrude option is set, so a check was added to verify this. This fixes case b935.

27 Feb 2021, 9155b3d.

Restrict previous update to just -vmll

The previous update was found to occasionally needlessly change existing formatting with very long long lines. So it is restricted to just when -vmll is set. For example, it is ok to keep the long quote following the opening paren in the following case.

  # perltidy -gnu
  ok( "got to the end without dying (note without DEBUGGING passing this test means nothing)"
    );

26 Feb 2021, 2b88464.

Add a gap calculation in line length tests with -vmll

This fixes case b965. The -vmll flag can produce gaps in lines which need to be included in weld line length estimates.

26 Feb 2021, a643cf2.

Update rule for spacing paren after constant function

Random testing produced an unstable state for the following snippet (case b934)

        sub pi();
        if (
            $t >
            pi( )
              )

when run with these parameters:

  --continuation-indentation=6
  --maximum-line-length=17
  --paren-vertical-tightness-closing=2

The formatting was stable without the first line, which declares 'pi' to be a constant sub. The problem was fixed by updating a regex to treat the spacing of a paren following a sub the same for the two token types, 'U' or 'C' (constant function).

This fixes case b934, 12bfdfe.

26 Feb 2021.

Improve line length test for the -vtc=2 option

This is a small change to the update of 13 Feb 2021, f79a4f1. Random testing produced additional blinking states caused by the combination of -vtc=2 and -vmll flag, plus several others. The problem was that a line length check in the vertical aligner was being skipped as an optimization if it didn't appear necessary. The unusual properties of the -vmll flag require that the check always be done.

This fixes cases b656 b862 b971 b972.

26 Feb 2021, 80107e0.

Improve one-line block length tests

Some oscillating states produced in random parameter tests were traced to problems with forming one-line blocks. A more precise length test was added to fix this.

This fixes cases b562 b563 b574 b777 b778 b924 b936 b975 b976 b983.

In the process of fixing this issue, a glitch was discovered in the previous coding of the -bl (braces-left) flag that caused somewhat random results for block types sort/map/grep/eval. The problem was a conflict between the logic for forming one-line blocks and the logic for applying the -bl flag. Usually, -bl formatting was not applied to these block types, but occasionally it was. To minimize changes in existing formatting, in the new version the -bl flag is not applied to these block types. A future flag could be added to give user control over which of these block types are under -bl control.

25 Feb 2021, 92bec8d.

Add tolerance to one-line block length tests

Testing with random input parameters produced some cases in which a stable solution could not be found due to attempts to form one-line blocks near the end of a long line. The problem was fixed by adding a small tolerance to the line length test. This does not change existing formatting.

This fixes cases b069 b070 b077 b078.

21 Feb 2021, 0b97b94.

Restrict -bom at cuddled method calls

The -bom flag tries to keep old breakpoints at lines beginning with '->' and also with some lines beginning with ')->'. These latter lines can lead to blinking states in cases where the opening paren is on the previous line. To fix this, a restriction was added that the line difference between the opening and closing parens should be more than 1.

This fixes case b977.

21 Feb 2021, 28114e9.

Add weld rule to avoid conflict between -wn and -bom

Testing with ramdom input parameters produced states which were oscillating because of a conflict between the -wn and -bom parameters. The problem was resolved by giving the -bom parameter priority over -wn.

These cases are fixed with this update: b966 b973

20 Feb 2021.

Limit the value of -ci=n to that of -i=n when -xci is set

Testing with random input parameters produced a number of oscillating states which had both parameter -xci as well as a value of -ci=n which exceeded the value of -i=n. To correct this, perltidy will silently reduce the -ci value to the -i value when -xci is also set. This should not change existing formatting because a value of -ci greater than -i would not normally be used in practice.

These cases are fixed with this update: b707 b770 b912 b920 b930 b933 b939 b940 b941 b942 b978 b974 b979 b980 b981

20 Feb 2021, c16c5ee.

Modify length tolerance for welding to qw lists

Several cases of alternating states were produced in random testing which were caused by line length limits being reached when welding to qw lists. This was fixed by adding a small tolerance to line length tests.

This fixes cases b654 b655 b943 b944 b967 b968 b969 b970.

19 Feb 2021, 0baafc8.

Modify space rule between binary plus or minus and a bareword

The update of 13 Feb 2021, cf414fe, has been modified to be less restrictive. Space between a binary plus and minus and a bareword may now be removed in some cases where no tokenization ambiguity exists. 18 Feb 2021, a8564c8.

Do not apply -xci if it would put tokens beyond the maximum line length

This update fixes cases b899 b935. 17 Feb 2021, b955a7c.

Do not weld to a hash brace

The reason is that it has a very strong bond strength to the next token, so a line break after it may not work. Previously we allowed welding to something like '@{' but even that caused blinking states (cases b751, b779).

This will not change much existing code. This update fixes cases b751 b779.

16 Feb 2021, eb2f4e7.

Avoid line breaks after token type 'G'

Random testing with very short maximum line lengths produced some blinking states which were traced to the tokenizer alternately parsed an unknown bareword as type 'w' or type 'G', depending on whether or not an opening block brace immediately followed on the same line. To fix this, a rule was added which prevents a line break between a type 'G' token and an opening code block brace.

This update fixes these cases: b900 b902 b928 b929

15 Feb 2021, f005a95.

Restrict breaking at old uncontained commas

Random testing with very short maximum line lengths produced some blinking states which were traced to duplicating old comma breakpoints which were not really good breakpoints. A patch was made to be more selective.

These cases are fixed with this update: b610 b757 b931

15 Feb 2021, 98b41a0.

Modify line length test for the -vtc=2 option

The line length test which was added Feb 13 2021 turns out to be more restrictive than necessary. A modification was made to only apply it if a new one-line block would be formed. This prevents it from needlessly changing existing formatting.

The following cases were re-activated after this update: b654 b655 b656 b862

15 Feb 2021, 4673fdd.

Use increased line length tolerance if ci exceeds i

In testing perltidy with random input parameters, some blinking states occurred when the value of -ci was significantly larger than the value of -i. (In actual practice, -ci is not normally set greater than -i). This update adds a tolerance to line length tests which avoids this problem. This fixes the following cases

b775 b776 b826 b908 b910 b911 b923 b925 b926 b927

14 Feb 2021, 8451f2f.

Keep space between binary plus or minus and a bareword

This update makes a space between a binary + or - and a bareword an essential whitespace. Otherwise, they may be converted into unary + or - on the next pass, which can lead to blinking states. Fixes cases b660 b670 b780 b781 b787 b788 b790.

13 Feb 2021, cf414fe.

Prevent breaks after unary plus and minus

Some alternating states were produced when extremely maximum line lengths forced a break after a unary plus or minus. Fixes cases b670 b790.

13 Feb 2021, cf414fe.

Add line length test for the -vtc=2 option

Random testing produced a number of cases of blinking states which were caused when the -vtc=2 flag caused the vertical aligner to combine lines which exceeded the allowable line length. These long lines were then getting reduced in size on every other iteration. A line length test was added in the vertical aligner to prevent this. This fixes these cases:

b654 b655 b656 b657 b761 b762 b763 b764 b765 b766 b767 b768 b769 b862 b904 b905 b906 b907 b913 b914 b915 b916 b917 b918 b919

13 Feb 2021, f79a4f1.

Define left side bond strengths for unary plus and minus

Random testing produced a blinking state which was traced to the unary plus not having a defined strength in the line break algorithm. This was fixed by setting it to be the same as the left strength of a plus. This fixes case b511. 12 Feb 2021, 58a7895.

Fix problem with breaking at an = sign

Random testing produced some blinking cases which were related to detecting an old good breakpoint at an equals. If the user requested that a break be done before an equals, and the input script had a break after an equals, then that break should not have been marked as a good existing break point before a keyword. This update fixes cases b434 b903.

11 Feb 2021, f9a8543.

Fix conflict of -kbl=0 and essential space after =cut

Random testing produced a case where a blank line after an =cut was alternately being deleted and added due to a conflict with the flag setting -keep-old-blank-lines=0. This was resolved by giving prioritiy to the essential blank line after the =cut line.

This fixes case b860. 11 Feb 2021, 8c13609.

Do not break one-line block at here target

A blinking state produced by random testing was traced to a line of coding which which unnecessarily prevented one-line blocks from being formed when a here-target was encountered. This has been fixed.

For example, the code block in the following contains a here target and was being broken into two lines:

    unless ($INC{$file}) {
        die <<"END_DIE" }

These will now be output with the blocks intact, like this

    unless ($INC{$file}) { die <<"END_DIE" }

This fixes case b523. 11 Feb 2021, 6d5bb74.

Skip processing -kgb* flags in lists or if -maximum-consecutive-blank-lines=0

Random testing produced an alternating state which was caused by -kgb flags being active on keywords which were in a list rather than a code block. A check was added to prevent this. Also, the -kgb* flags have no effect if no blank lines can be output, so a check was added for this situation. This fixes case b760.

10 Feb 2021, 177fc3a.

Modify tolerance in testing for welds

Random testing with unusual parameters produced some blinking weld states which were fixed by modifying a tolerance used in a line length test. The following cases were fixed with this update:

b746 b748 b749 b750 b752 b753 b754 b755 b756 b758 b759 b771 b772 b773 b774 b782 b783 b784 b785 b786

9 Feb 2021, a4609ac.

Modified rule for breaking lines at old commas

Random testing produced some blinking cases resulting from the treatment of old line breaks at commas not contained within containers. The following cases were fixed with this update:

b064 b065 b068 b210 b747

This change has no effect on scripts with normal parameter values. 9 Feb 2021, 5c23661.

Restrict references to old line breaks

A number of cases of blinking states were traced to code which biased the breaking of long lines to existing breaks. This was fixed by restricting this coding to just use old comma line break points.

The following cases were fixed with this update:

b193 b194 b195 b197 b198 b199 b216 b217 b218 b219 b220 b221 b244 b245 b246 b247 b249 b251 b252 b253 b254 b256 b257 b258 b259 b260 b261 b262 b263 b264 b265 b266 b268 b269 b270 b271 b272 b274 b275 b278 b280 b281 b283 b285 b288 b291 b295 b296 b297 b299 b302 b304 b305 b307 b310 b311 b312 b313 b314 b315 b316 b317 b318 b319 b320 b321 b322 b323 b324 b325 b326 b327 b329 b330 b331 b332 b333 b334 b335 b336 b337 b338 b339 b340 b341 b342 b343 b344 b345 b346 b347 b348 b349

8 Feb 2021, 66be455.

Fix rare problem involving interaction of -olbn=n and -wn flags

Random testing revealed a rare alternating state which could occur when both flags --one-line-block-nesting=n and --weld-nested-containers are set, and the maximum line length is set very low. The problem was fixed by ignoring the first flag at welded tokens. This should not have any effect on scripts with realistic parameter values.

The following case was fixed with this update: b690.

6 Feb 2021, 3e96930.

add rule to avoid welding at some barewords

A rule was added to prevent certain rare blinking states involving welding. The rule is that if an opening container is immediately followed by a bareword which is unknown, a weld will be avoided.

The following cases were fixed with this update: b611 b626.

6 Feb 2021, 6b1f44a

further simplify -bbxi=n implementation

This update adds a new variable which indicates if a container is permanently broken due to a side comment or blank line. This helps reduce the number of cases where the -bbxi=n flag cannot be applied. Another change was to always apply the -bbx=n flag, even if the -bbxi=n flag cannot be applied. These two flags now operate almost exactly as in previous versions but without the blinking problem. The only difference is that now the -bbxi=n flag with n>0 will revert to n=0 for some short containers which might not be broken open.

reset -bbxi=2 to -bbxi=0 if -lp is set to avoid blinking states

The options of the form bbxi=2, such as break-before-paren-and-indent=2, have been found to cause blinking states if the -lp flag is set. Both of these options are fairly rare. To correct this the -bbxi=2 flag is now reset to -bbxi=0 if the -lp flag is set. Note that -bbxi=2 and -bbxi=0 give the same formatting result with the common setting -ci=4 and -i=4.

The following cases were fixed with this update:

b396 b397 b398 b429 b435 b457 b502 b503 b504 b505 b538 b540 b542 b617 b618 b619 b620 b621

3 Feb 2021, 67ab0ef.

rewrite sub break_before_list_opening_containers

sub break_before_list_opening_containers was rewritten to reduce the chance of producing alternating states.

The following cases were fixed with this update:

b030 b032 b455 b456 b458 b459 b460 b461 b462 b536 b622 b651 b652 b653 b708 b709 b710 b713 b714 b719 b723 b724 b725 b726 b727 b729 b731 b733 b735 b736 b737 b738 b739 b740 b743 b744

3 Feb 2021, 5083ab9.

redefine list to have at least one internal comma

Random testing produced some blinking states which could be fixed by changing the definition of a list, for formatting purposes, to have one or more interior commas rather than simply one or more commas. The difference is that something with a single terminal comma, like '( $x, )', is no longer classified as a list. This makes no difference except when perltidy is stress tested with unusual parameters.

The following cases were fixed with this update:

b116 b119 b122 b174 b179 b187 b361 b369 b370 b372 b376 b427 b428 b448 b449 b450 b451 b452 b453 b469 b473 b474 b475 b476 b477 b479 b480 b481 b482 b497 b552 b553 b554 b558 b559 b634 b637 b642 b644 b645 b647 b650 b661 b662 b663 b664 b666 b677 b685 b688 b698 b699 b700 b702 b703 b704 b711 b712 b715 b716 b717 b718 b721 b730 b734 b741 b742

1 Feb 2021, 35078f7.

rewrite and combine coding for -bbx=n and -bbxi=n

Random testing produced a large number of blinking states involving parameters such as --break-before-parens=n and --break-before-parens-and-indent=n and similar pairs. The problem was traced to the fact that the former parameter was implemented late in the pipeline whereas the latter parameter was implemented early in the pipeline. Normally there was no problem, but in some extreme cases, often involving very short maximum line lengths, this could produce alternating output states. The problem was resolved by combining the implementation of both flags in a single new sub to avoid any inconsistencies. The following cases were fixed with this update:

b018 b066 b071 b079 b090 b105 b146 b149 b158 b160 b161 b163 b164 b166 b167 b169 b170 b171 b172 b178 b185 b190 b192 b203 b206 b222 b223 b224 b237 b359 b362 b377 b379 b381 b382 b389 b395 b407 b408 b409 b410 b411 b412 b414 b417 b418 b419 b421 b433 b438 b443 b444 b478 b483 b484 b486 b490 b492 b493 b494 b496 b506 b507 b517 b521 b522 b524 b525 b535 b537 b539 b541 b543 b546 b550 b551 b555 b564 b566 b567 b569 b570 b572 b573 b575 b576 b577 b578 b579 b580 b582 b586 b587 b588 b589 b590 b591 b592 b593 b603 b607 b609 b613 b615 b616 b623 b624 b630 b635 b636 b638 b639 b640 b641 b643 b646 b648 b649 b658 b659 b665 b667 b668 b669 b671 b672 b673 b674 b675 b676 b678 b679 b680 b681 b682 b683 b684 b686 b687 b689 b691 b692 b693 b694 b695 b696 b697 b701 b705 b706 b720 b722 b728 b732 b745

31 Jan 2021, 10e8bfd.

adjust line length and token count tolerances for -wn

Most remaining edge cases of blinking states involving the -wn parameter have been fixed by adjusting some tolerances in sub weld_nested_containers. The following cases are fixed with this update:

b156 b157 b186 b196 b454 b520 b527 b530 b532 b533 b534 b612 b614 b625 b627

This update has no effect for realistic parameter settings.

30 Jan 2021, d359a60.

fix additional edge blinker cases involving -wn

Some blinking cases produced in random testing were traced to welding in very short lines (length = 20 for example) in which a weld was made to a square bracket containing just a single parameter, so that it had no good internal breaking points. A rule was added to avoid welding to a square bracket not containing any commas. The following cases were fixed with the update:

b002 b003 b005 b006 b007 b009 b010 b014 b015 b017 b020 b111 b112 b113 b124 b126 b128 b151 b153 b439 b606

29 Jan 2021, 33f1f2b.

fix additional edge blinker cases involving -wn

Random testing produced some blinking states which were traced to the precision of a line length test. In sub weld_nested_containers, the test

    $do_not_weld ||= $excess_length_to_K->($Kinner_opening) > 0;

was changed to allow a 1 character margin of error:

    $do_not_weld ||= $excess_length_to_K->($Kinner_opening) >= 0;

The following cases were fixed with this update:

b025 b075 b091 b109 b110 b152 b154 b155 b162 b168 b176 b422 b423 b424 b425 b426 b565

29 Jan 2021, 33f1f2b.

fix some edge blinker cases involving -wn

Random testing produced some blinking states which were eliminated by a simplification of the definition of a one_line_weld in sub weld_nested_containers. The following cases were fixed with this update:

b131 b134 b136 b205 b233 b238 b284 b350 b352 b358 b385 b487 b604 b605

29 Jan 2021, 33f1f2b.

fix some edge blinker cases involving -bbxi=n esp. with -boc

The following cases were fixed with this update:

b041 b182 b184 b366 b367 b368 b370 b371 b382 b420 b432 b438 b464 b466 b467 b468 b500 b501 b508 b509 b510 b512 b513 b514 b515 b516 b526 b528 b529 b531 b544 b545 b547 b548 b549 b556 b557 b568 b571 b581 b583 b584 b585 b594 b595 b596 b597 b598 b599 b600 b601 b602 b608 b041 b182 b184 b355 b356 b366 b367 b368 b371 b420 b432 b464 b465 b466 b467 b468 b500 b501 b508 b509 b510 b512 b513 b514 b515 b516 b526 b528 b529 b531 b544 b545 b547 b548 b549 b556 b557 b568 b571 b581 b583 b584

28 Jan 2021.

fix problem with combination -cab=2 and bbhbi=n

Random testing produced a number of cases in which the combination -cab=2 and bbhbi=n and similar flags were in conflict, causing alternating states. This was fixed by not changing ci for containers which can fit entirely on one line, which is what -cab=2 says to do. The following cases were fixed with this update:

b046 b061 b081 b084 b089 b093 b130 b133 b135 b138 b142 b145 b147 b150 b165 b173 b191 b211 b294 b309 b360 b363 b364 b365 b373 b386 b387 b388 b392 b437 b440 b472 b488 b489

27 Jan 2021, 6d710de.

fix problem with -freeze-whitespace

Random testing produced a case in which the --freeze-whitespace flag (which is mainly useful for testing) could cause a blank space which kept increasing. The problem was caused by the "vertical tightness" mechanism. Turning it off when the -freeze-whitespace-flag is on fixed the problem. The following cases were fixed with this update:

b037 b038 b043 b059 b060 b067 b072 b215 b225 b267 b273 b276 b279 b282 b289 b292 b300 b303 b354 b374 b375 b383 b384 b402 b403 b404 b405 b436 b441 b445 b446 b471 b485 b498 b499

27 Jan 2021, 6d710de.

Avoid blinking states associated with -bbpi and similar flags

Random testing with extreme parameter values revealed blinking states associated with the -bbpi and related flags. The problem was that continuation indentation was sometimes being added according to the flag but the lists were not actually being broken. After this was fixed the following cases ran correctly:

b024 b035 b036 b040 b042 b047 b049 b050 b051 b052 b053 b054 b057 b062 b063 b073 b074 b076 b080 b083 b085 b086 b087 b088 b102 b103 b104 b106 b107 b108 b115 b118 b121 b123 b125 b127 b132 b137 b139 b140 b141 b143 b144 b148 b159 b175 b177 b180 b181 b188 b189 b200 b201 b202 b204 b207 b212 b213 b214 b226 b227 b228 b229 b230 b232 b239 b240 b241 b243 b248 b250 b255 b277 b286 b287 b290 b293 b298 b301 b306 b308 b328 b351 b353 b357 b378 b380 b390 b391 b393 b394 b399 b400 b401 b406 b413 b415 b416 b430 b431 b442 b447 b463 b470 b491 b495

27 Jan 2021, 96144a3.

Revise coding for the --freeze-whitespace option

Random testing produced some blinking states which were traced to an incorrect implementation of the --freeze-whitespace option (which is mainly useful in stress testing perltidy). A related flag, --add-whitespace is involved. This update corrects these problems. Test cases include b057, b183, b242. 24 Jan 2021, 9956a57.

Fix for issue git #51, closing qw paren not outdented when -ndnl is set

The problem is that a bare closing qw paren was not being outdented if the flag '-nodelete-old-newlines is set. For example

    # OLD (OK, outdented): perltidy -ci=4 -xci
    {
        modules => [
            qw(
                JSON
            )
        ],
    }


    # OLD (indented) : perltidy -ndnl -ci=4 -xci
    {
        modules => [
            qw(
                JSON
                )
        ],
    }

    # FIXED: perltidy -ndnl -ci=4 -xci
    {
        modules => [
            qw(
                JSON
            )
        ],
    }

The problem happened because the -ndnl flag forces each line to be written immediately, so the next line (which needs to be checked in this case) was not available when the outdent decision had to be made. A patch to work around this was added 24 Jan 2021, 52996fb.

Some issues with the -lp option

Random testing revealed some problems involving the -lp option which are fixed with this update.

The problem is illustrated with the following snippet

    # perltidy -lp
    Alien::FillOutTemplate(
                "$main::libdir/to-$main::desttype/$main::filetype/spec",
                "$workdir/$fields{NAME}-$fields{VERSION}-$fields{RELEASE}.spec",
                %fields
    );

which alternately formats to this form

    # perltidy -lp
    Alien::FillOutTemplate(
                "$main::libdir/to-$main::desttype/$main::filetype/spec",
                "$workdir/$fields{NAME}-$fields{VERSION}-$fields{RELEASE}.spec",
                %fields );

when formatted with the single parameter -lp. A number of similar examples were found in testing. The problem was traced to the treatment of the space which perltidy tentatively adds wherever there is a newline, just in case the formatted output has different line breaks. The problem was that the indentation level of these spaces was being set as the level of the next token rather than the previous token. Normally the indentation level of a space has no effect, but the -lp option does use it and this caused the problem. This was fixed 23 Jan 2021, edc7878.

added rule for -wn, do not weld to a hash brace

In random testing, the following two alternating states

    # State 1
    {
        if ( defined
        ($symbol_table{$direccion}) )
    }
    
    # State 2
    {
        if (defined (
                $symbol_table{
                    $direccion}
            )
        )
    }

were occurring with the following particular parameter set

    --weld-nested-containers
    --maximum-line-length=40
    --continuation-indentation=7
    --paren-tightness=2
    --extended-continuation-indentation

The problem was traced to welding to the opening hash brace. A rule was added to prevent this, and testing with a large body of code showed that it did not significantly change existing formatting. With this change, the above snippet formats in the stable state

    {
        if (defined(
            $symbol_table{$direccion}
        ))
    }

20 Jan 2021, 4021436.

Do not let -kgb option delete essential blank after =cut

A blinking state was found in random testing for the following snippet

    =head1 TODO
    
    handle UNIMARC encodings
    
    =cut
    
    use strict;
    use warnings;

when run with the following parameters

    --keyword-group-blanks-size='2.8'
    --keyword-group-blanks-before=0

The space after the =cut was alternately being added as an essential blank which is required by pod utilities, and then deleted by these parameters. This was fixed 17 Jan 2021, b9a5f5d.

Turn off -boc flag if -iob is set

In random testing, the cause of a blinker was traced to both flags --ignore-old-breakpoints and --break-at-old-comma-breakpoints being set. There is a warning message but the -boc flag was not actually being turned off. This was fixed 17 Jan 2021, b9a5f5d.

Modified spacing rule for token type Y

A space following a token type 'Y' (filehandle) should not be removed. Otherwise it might be converted into type 'Z' (possible filehandle). If that were to happen, the space could not be added back automatically in later formatting, so the user would have to do it by hand. This fix prevents this from happening. 17 Jan 2021, bef9a83.

In random testing a blinker was reduced to the following snippet

    {
             print FILE
              GD::Barcode
              ->new();
    }

which switched to the following state on each iteration

    {
             print FILE GD::Barcode
              ->new();
    }

with the following parameters

    --maximum-line-length=20
    --indent-columns=9
    --continuation-indentation=1

The problem was that the token 'FILE' was either parsed as type 'Y' or 'Z' depending on the existence of a subsequent space. These have different line break rules, causing a blinker. The problem was fixed by modifying the tokenizer to consider a newline to be a space. Updated 16 Jan 2021, d40cca9.

Turn off -bli if -bar is set

A conflict arises if both -bli and -bar are set. In this case a warning message is given and -bli is turned off. Updated 15 Jan 2021, ef69531.

A blinking state was discovered in testing between the following two states

    my$table=
         [[1,2,3],[2,4,6],[3,6,9],
         ];

    my$table=
        [[1,2,3],[2,4,6],[3,6,9],];

with these parameters

    --continuation-indentation=5
    --maximum-line-length=31
    --break-before-square-bracket-and-indent=2
    --break-before-square-bracket=1
    --noadd-whitespace

The problem was found to be caused by the -bbsb parameters causing the indentation level of the first square bracket to change depending upon whether the term was broken on input or not. Two fixes would correct this. One is to turn off the option if the -ci=n value exceeds the -i=n value. The other is to require a broken container to span at least three lines before turning this option on. The latter option was made to sub 'adjust_container_indentation'. With this change the snippet remains stable at the second state above. Fixed 14 Jan 2021, 5c793a1.

In random testing with convergence a 'blinker' (oscillating states) was found for the following script

    sub _prompt {

          print $_[0];
          return (
                readline
                       (*{$_[1]})!~
                       /^q/i)
                 ; # Return false if user types 'q'

    }

with the following specific parameters:

    --maximum-line-length=32
    --indent-columns=6
    --continuation-indentation=7
    --weld-nested-containers
    --extended-continuation-indentation
    --noadd-whitespace

The other state was

    sub _prompt {

          print $_[0];
          return (
                readline(
                      *{
                            $_
                                   [
                                   1
                                   ]
                      }
                )!~/^q/i
                 )
                 ; # Return false if user types 'q'

    }

All of the listed parameters are required to cause this, but the main cause is the very large continuation indentation and short line length. Welding was being turned on and off in this case. Normally welding is not done if all containers are on a single line, but an exception was made to detect a situation like this and keep the welded string together. Updated 13 Jan 2021, 5c793a1.

Fixed incorrect guess of division vs pattern

A syntax error was produced in random testing when perltidy was fed the following line:

    sub _DR () { pi2 /360 } sub _RD () { 360 /pi2 }

The bareword 'pi2' was not recognized and the text between the two slashes was a taken as a possible pattern argument in a parenless call to pi2. Two fixes were made to fix this. Perltidy looks for 'pi' but not 'pi2', so the first fix was to expand its table to include all variations of 'pi' in Trig.pm. Second, the fact that the first slash was followed by a number should have tipped the guess to favor division, so this was fixed. As it was, a backup spacing rule was used, which favored a pattern.

The formatted result is now

    sub _DR () { pi2 / 360 }
    sub _RD () { 360 / pi2 }

This update was made 13 Jan 2021, a50ecf8.

Correct formula for estimating line length with -wn option

A formula used to estimating maximum line length when the -wn option is set was missing a term for continuation indentation. No actual changes in formatting have been seen. This update made 12 Jan 2021.

The following blinker was found in random testing. The following statement (with @j starting at level 0)

    @j = ( $x, $y, $z );

run with the following parameters

    --indent-columns=5
    --continuation-indentation=7
    --maximum-line-length=20
    --break-before-paren-and-indent=2
    --break-before-paren=2
    --maximum-fields-per-table=4

caused an oscillation between two states. An unusual feature which contributed to the problem is the very large ci value. This is fixed in a patch made 12 Jan 2021, 9a97dba.

Issues fixed after release 20201207

Improve indentation of multiline qw quotes when -xci flag is set

The indentation of multiline qw quotes runs into problems when there is nesting, as in the following.

    # OLD: perltidy -xci -ci=4
    for my $feep (
        qw{
        pwage      pwchange   pwclass    pwcomment
        pwexpire   pwgecos    pwpasswd   pwquota
        }
        )

The problem is that multiline qw quotes do not get the same indentation treatment as lists.

This update fixes this in the following circumstances:

  - the leading qw( and trailing ) are on separate lines
  - the closing token is one of ) } ] >
  - the -xci flag is set

The above example becomes

    # NEW: perltidy -xci -ci=4
    for my $feep (
        qw{
            pwage      pwchange   pwclass    pwcomment
            pwexpire   pwgecos    pwpasswd   pwquota
        }
        )

The reason that the -xci flag is required is to minimize unexpected changes to existing scripts. The extra indentation is removed if the -wn flag is also given, so both old and new versions with -wn give

    # OLD and NEW: perltidy -wn -xci -ci=4
    for my $feep ( qw{
        pwage      pwchange   pwclass    pwcomment
        pwexpire   pwgecos    pwpasswd   pwquota
    } )

This update added 8 Jan 2021, 474cfa8.

Improve alignment of leading equals in rare situation

A rare case in which a vertical alignment opportunity of leading equals was missed has been fixed. This involved lines with additional varying alignment tokens, such as 'unless' and second '=' in lines 1-3 below. In this example lines 4 and 5 were not 'looking' backwards to align their leading equals.

    # OLD:
    $them = 'localhost' unless ( $them = shift );
    $cmd  = '!print'    unless ( $cmd  = shift );
    $port = 2345        unless ( $port = shift );
    $saddr = 'S n a4 x8';
    $SIG{'INT'} = 'dokill';

    # NEW
    $them       = 'localhost' unless ( $them = shift );
    $cmd        = '!print'    unless ( $cmd  = shift );
    $port       = 2345        unless ( $port = shift );
    $saddr      = 'S n a4 x8';
    $SIG{'INT'} = 'dokill';

Fixed 5 Jan 2021, 9244678.

Moved previous patch to a better location

The previous patch was moved to a location where it only applies if there is a side comment on the line with a closing token. This minimizes changes to other side comment locations.

Further improvement in rules for forgetting last side comment location

The code for forgetting the last side comment location was rewritten to improve formatting in some edge cases. The update also fixes a very rare problem discovered during testing and illustrated with the following snippet. The problem occurs for the particular combination of parameters -sct -act=2 and when a closing paren has a side comment:

    OLD: perltidy -sct -act=2
    foreach $line (
        [0, 1, 2], [3, 4, 5], [6, 7, 8],    # rows
        [0, 3, 6], [1, 4, 7], [2, 5, 8],    # columns
        [0, 4, 8], [2, 4, 6])                                     # diagonals

    NEW: perltidy -sct -act=2
    foreach $line (
        [0, 1, 2], [3, 4, 5], [6, 7, 8],    # rows
        [0, 3, 6], [1, 4, 7], [2, 5, 8],    # columns
        [0, 4, 8], [2, 4, 6])    # diagonals

In the old version the last side comment was aligned before the closing paren was attached to the previous line, causing the final side comment to be far to the right. A patch in the new version just places it at the default location. This is the best than can be done for now, but is preferable to the old formatting. 3 Jan 2021, e57d8db.

Improve rule for forgetting last side comment location

The code which aligns side comments remembers the most recent side comment and in some cases tries to start aligning at that column for later side comments. Sometimes the old side comment column was being remembered too long, causing occasional poor formatting and causing a noticeable and unexpected drift of side comment locations to the right. The rule for forgetting the previous side comment column has been modified to reduce this problem. The new rule is essentially to forget the previous side comment location at a new side comment with different indentation level or significant number of lines without side comments (about 12). The previous implementation forgetting changes in indentation level across code blocks only. Below is an example where the old method gets into trouble and the new method is ok:

        # OLD:
        foreach my $r (@$array) {
            $Dat{Data}{ uc $r->[0] } = join( ";", @$r );    # store all info
            my $name = $Dat{GivenName}{ uc $r->[0] } || $r->[1];

            # pass array as ad-hoc string, mark missing values
            $Dat{Data}{ uc $r->[0] } = join(
                ";",
                (
                    uc $r->[0], uc $name,                   # symbol, name
                    $r->[2],    $r->[3], $r->[4],           # price, date, time
                    $r->[5],    $r->[6],                    # change, %change
                    $r->[7],    "-", "-", "-",    # vol, avg vol, bid,ask
                    $r->[8],               $r->[9],     # previous, open
                    "$r->[10] - $r->[11]", $r->[12],    # day range,year range,
                    "-",                   "-", "-", "-", "-"
                )
            );                                          # eps,p/e,div,yld,cap
        }

The second side comment is at a deeper indentation level but was not being forgotten, causing line length limits to interfere with later alignment. The new rule gives a better result:

        # NEW:
        foreach my $r (@$array) {
            $Dat{Data}{ uc $r->[0] } = join( ";", @$r );    # store all info
            my $name = $Dat{GivenName}{ uc $r->[0] } || $r->[1];

            # pass array as ad-hoc string, mark missing values
            $Dat{Data}{ uc $r->[0] } = join(
                ";",
                (
                    uc $r->[0], uc $name,               # symbol, name
                    $r->[2],    $r->[3], $r->[4],       # price, date, time
                    $r->[5],    $r->[6],                # change, %change
                    $r->[7],    "-", "-", "-",          # vol, avg vol, bid,ask
                    $r->[8],               $r->[9],     # previous, open
                    "$r->[10] - $r->[11]", $r->[12],    # day range,year range,
                    "-",                   "-", "-", "-", "-"
                )
            );    # eps,p/e,div,yld,cap
        }

The following exampel shows an unexpected alignment in the cascade of trailing comments which are aligned but slowly separating from their closing containers:

    # OLD:
    {
        $a = [
            Cascade    => $menu_cb,
            -menuitems => [
                [ Checkbutton => 'Oil checked', -variable => \$OIL ],
                [
                    Button   => 'See current values',
                    -command => [
                        \&see_vars, $TOP,

                    ],    # end see_vars
                ],        # end button
            ],            # end checkbutton menuitems
        ];                # end checkbuttons cascade
    }

This was caused by forgetting side comments only across code block changes. The new result is more reasonable:

    # NEW:
    {
        $a = [
            Cascade    => $menu_cb,
            -menuitems => [
                [ Checkbutton => 'Oil checked', -variable => \$OIL ],
                [
                    Button   => 'See current values',
                    -command => [
                        \&see_vars, $TOP,

                    ],    # end see_vars
                ],    # end button
            ],    # end checkbutton menuitems
        ];    # end checkbuttons cascade
    }

This change will cause occasional differences in side comment locations from previous versions but overall it gives fewer unexpected results so it is a worthwhile change. 29-Dec-2020, 76993f4.

Fixed very minor inconsistency in redefining lists after prune step

In rare cases it is necessary to update the type of lists, and this influences vertical alignment. This update fixes a minor inconsistency in doing this. In some rare cases with complex list elements vertical alignment can be improved. 27 Dec, 2020, 751faec.

            # OLD
            return join( '',
                $pre,   '<IMG ',   $iconsizes{$alt} || '',
                $align, 'BORDER=', $nav_border,
                ' ALT="', $alt,        "\"\n",
                ' SRC="', $ICONSERVER, "/$icon",
                '">' );

            # NEW
            return join( '',
                $pre,     '<IMG ',     $iconsizes{$alt} || '',
                $align,   'BORDER=',   $nav_border,
                ' ALT="', $alt,        "\"\n",
                ' SRC="', $ICONSERVER, "/$icon",
                '">' );
Improved vertical alignment of some edge cases

The existing rules for aligning two lines with very different lengths were rejecting some good alignments, such as the first line of numbers in the example below:

    # OLD:
    @gg_3 = (
        [
            0.0, 1.360755E-2, 9.569446E-4, 9.569446E-4,
            1.043498E-3, 1.043498E-3
        ],
        [
            9.569446E-4, 9.569446E-4, 0.0, 7.065964E-5,
            1.422811E-4, 1.422811E-4
        ],
        ...
    );

    # NEW:
    @gg_3 = (
        [
            0.0,         1.360755E-2, 9.569446E-4, 9.569446E-4,
            1.043498E-3, 1.043498E-3
        ],
        [
            9.569446E-4, 9.569446E-4, 0.0, 7.065964E-5,
            1.422811E-4, 1.422811E-4
        ],
        ...
    );

The rule in sub 'two_line_pad' was updated to allow alignment of any lists if the patterns match exactly (all numbers in this case). Updated 27-Dec-2020, 035d2b7.

Avoid -lp style formatting of lists containing multiline qw quotes

The -lp formatting style often does not work well when lists contain multiline qw quotes. This update avoids this problem by not formatting such lists with the -lp style. For example,

    # OLD, perltidy -gnu
    @EXPORT = (
        qw(
          i Re Im rho theta arg
          sqrt log ln
          log10 logn cbrt root
          cplx cplxe
          ),
        @trig,
              );


    # NEW, perltidy -gnu
    @EXPORT = (
        qw(
          i Re Im rho theta arg
          sqrt log ln
          log10 logn cbrt root
          cplx cplxe
        ),
        @trig,
    );

27-Dec-2020, 948c9bd.

improve formatting of multiline qw

This update adds a sequence numbering system for multiline qw quotes. In the perltidy tokenizer normal container pair types, like { }, (), [], are given unique serial numbers which are used as keys to data structures. qw quoted lists do not get serial numbers by the tokenizer, so this update creates a separate serial number scheme for them to correct this problem. One formatting problem that this solves is that of preventing the closing token of a multiline quote from being outdented more than the opening token. This is a general formatting rule which should be followed. Without a sequence number, the closing qw token could not lookup its corresponding opening indentation so it had to resort to a default, breaking the rule, as in the following:

    # OLD, perltidy -wn
    # qw line
    if ( $pos == 0 ) {
        @return = grep( /^$word/,
            sort qw(
              ! a b d h i m o q r u autobundle clean
              make test install force reload look
        ) ); #<-- outdented more than 'sort'
    }

    # Here is the same with a list instead of a qw; note how the
    # closing sort paren does not outdent more than the 'sort' line.
    # This is the desired result for qw.
    # perltidy -wn
    if ( $pos == 0 ) {
        @return = grep( /^$word/,
            sort (

                '!',          'a', 'b', 'd', 'h', 'i', 'm', 'o', 'q', 'r', 'u',
                'autobundle', 'clean',
                'make',       'test', 'install', 'force', 'reload', 'look'
            ) );  #<-- not outdented more than 'sort'
    }

    # NEW (perltidy -wn)
    if ( $pos == 0 ) {
        @return = grep( /^$word/,
            sort qw(
              ! a b d h i m o q r u autobundle clean
              make test install force reload look
            ) ); #<-- not outdented more than sort
    }

Here is another example # OLD: $_->meta->make_immutable( inline_constructor => 0, constructor_name => "_new", inline_accessors => 0, ) for qw( Class::XYZ::Package Class::XYZ::Module Class::XYZ::Class

        Class::XYZ::Overload
    );  #<-- outdented more than the line with 'for qw('

    # NEW:
    $_->meta->make_immutable(
        inline_constructor => 0,
        constructor_name   => "_new",
        inline_accessors   => 0,
      )
      for qw(
      Class::XYZ::Package
      Class::XYZ::Module
      Class::XYZ::Class

      Class::XYZ::Overload
      ); #<-- outdented same as the line with 'for qw('

26 Dec 2020, cdbf0e4.

improve list marking method

In the process of making vertical alignments, lines which are simple lists of items are treated different from other lines. The old method for finding and marking these lines had a few problems which are corrected with this update. The main problem was that the old method ran into trouble when there were side comments. For example, the old method was not marking the following list and as a result the two columns of values were not aligned:

    # OLD
    return (
        $startpos, $ldelpos - $startpos,         # PREFIX
        $ldelpos,  1,                            # OPENING BRACKET
        $ldelpos + 1, $endpos - $ldelpos - 2,    # CONTENTS
        $endpos - 1, 1,                          # CLOSING BRACKET
        $endpos, length($$textref) - $endpos,    # REMAINDER
    );

    # NEW
    return (
        $startpos,    $ldelpos - $startpos,           # PREFIX
        $ldelpos,     1,                              # OPENING BRACKET
        $ldelpos + 1, $endpos - $ldelpos - 2,         # CONTENTS
        $endpos - 1,  1,                              # CLOSING BRACKET
        $endpos,      length($$textref) - $endpos,    # REMAINDER
    );

Another problem was that occasionally unwanted alignments were made between lines which were not really lists because the lines were incorrectly marked. For example (note padding after first comma)

    # OLD: (undesirable alignment)
    my ( $isig2, $chisq ) = ( 1 / ( $sig * $sig ), 0 );
    my ( $ym,    $al, $cov, $bet, $olda, $ochisq, $di, $pivt, $info ) =
      map { null } ( 0 .. 8 );

    # NEW: (no alignment)
    my ( $isig2, $chisq ) = ( 1 / ( $sig * $sig ), 0 );
    my ( $ym, $al, $cov, $bet, $olda, $ochisq, $di, $pivt, $info ) =
      map { null } ( 0 .. 8 );

This update was made 22 Dec 2020, 36d4c35.

Fix git #51, closing quote pattern delimiters not following -cti flag settings

Closing pattern delimiter tokens of qw quotes were not following the -cti flag settings for containers in all cases, as would be expected, in particular when followed by a comma. For example, the closing qw paren below was indented with continuation indentation but would not have that extra indentation if it followed the default -cpi setting for a paren:

    # OLD:
    @EXPORT = (
        qw(
          i Re Im rho theta arg
          sqrt log ln
          log10 logn cbrt root
          cplx cplxe
          ),
        @trig
    );

    # NEW
    @EXPORT = (
        qw(
            i Re Im rho theta arg
            sqrt log ln
            log10 logn cbrt root
            cplx cplxe
        ),
        @trig
    );

This update makes closing qw quote terminators follow the settings for their corresponding container tokens as closely as possible. For a closing '>' the setting for a closing paren will now be followed. Other closing qw terminators will remain indented, to minimize changes to existing formatting. For example ('>' is outdented):

    @EXPORT = (
        qw<
          i Re Im rho theta arg
          sqrt log ln
          log10 logn cbrt root
          cplx cplxe
        >,
        @trig
    );

but (';' remains indented):

    @EXPORT = (
        qw;
          i Re Im rho theta arg
          sqrt log ln
          log10 logn cbrt root
          cplx cplxe
          ;,
        @trig
    );

This update was added 18 Dec 2020 and modified 24 Dec 2020, 538688f.

Update manual pages regarding issue git #50

Additional wording was added to the man pages regarding situations in which perltidy does not change whitespace. This update was added 17 Dec 2020.

Rewrote sub check_match

Moved inner part of sub check_match into sub match_line_pair in order to make info available earlier. This gave some minor alignment improvements. This was done 16 Dec 2020, 7ba4f3b.

    # OLD:
    @tests = (
        @common,     '$_',
        '"\$_"',     '@_',
        '"\@_"',     '??N',
        '"??N"',     chr 256,
        '"\x{100}"', chr 65536,
        '"\x{10000}"', ord 'N' == 78 ? ( chr 11, '"\013"' ) : ()
    );

    # NEW:
    @tests = (
        @common,       '$_',
        '"\$_"',       '@_',
        '"\@_"',       '??N',
        '"??N"',       chr 256,
        '"\x{100}"',   chr 65536,
        '"\x{10000}"', ord 'N' == 78 ? ( chr 11, '"\013"' ) : ()
    );
Improved vertical alignments by avoiding pruning step

There is a step in vertical alignment where the alignments are formed into a tree with different levels, and some deeper levels are pruned to preserve lower level alignments. This usually works well, but some deeper alignments will be lost, which is what was happening in the example below. It turns out that if the tree pruning is skipped when alignment depths increase monotonically across lines, as in the example, then better overall alignment is achieved by the subsequent 'sweep' pass.

    # OLD
    my $cmd = shift @ARGV;
    if    ( $cmd eq "new" )         { $force_new = 1; }
    elsif ( $cmd eq "interactive" ) { $interactive = 1; $batch       = 0; }
    elsif ( $cmd eq "batch" )       { $batch       = 1; $interactive = 0; }
    elsif ( $cmd eq "use_old" )     { $use_old = 1; }
    elsif ( $cmd eq "show" )        { $show    = 1; last; }
    elsif ( $cmd eq "showall" )     { $showall = 1; last; }
    elsif ( $cmd eq "show_all" )    { $showall = 1; last; }
    elsif ( $cmd eq "remove" )      { $remove  = 1; last; }
    elsif ( $cmd eq "help" )        { $help    = 1; last; }

    # NEW
    my $cmd = shift @ARGV;
    if    ( $cmd eq "new" )         { $force_new   = 1; }
    elsif ( $cmd eq "interactive" ) { $interactive = 1; $batch       = 0; }
    elsif ( $cmd eq "batch" )       { $batch       = 1; $interactive = 0; }
    elsif ( $cmd eq "use_old" )     { $use_old     = 1; }
    elsif ( $cmd eq "show" )        { $show        = 1; last; }
    elsif ( $cmd eq "showall" )     { $showall     = 1; last; }
    elsif ( $cmd eq "show_all" )    { $showall     = 1; last; }
    elsif ( $cmd eq "remove" )      { $remove      = 1; last; }
    elsif ( $cmd eq "help" )        { $help        = 1; last; }

This update was made 14 Dec 2020, 44e0afa.

Improved some marginal vertical alignments

This update fixed a rare situation in which some vertical alignment was missed. The problem had to do with two lines being incorrectly marked as a marginal match. A new routine, 'match_line_pairs' was added to set a flag with the information needed to detect and prevent this. This fix was made 13 Dec 2020, 9a8e49b.

    # OLD
    $sec = $sec + ( 60 * $min );
    $graphcpu[$sec] = $line;
    $secmax  = $sec  if ( $sec > $secmax );
    $linemax = $line if ( $line > $linemax );

    # NEW
    $sec            = $sec + ( 60 * $min );
    $graphcpu[$sec] = $line;
    $secmax         = $sec  if ( $sec > $secmax );
    $linemax        = $line if ( $line > $linemax );
Do not align equals across changes in continuation indentation

A rule was added to prevent vertical alignment of lines with leading '=' across a change in continuation indentation. Sometimes aligning across a change in CI can come out okay, but sometimes it can be very poor. For example:

    # BAD:
    $!               = 2, die qq/$0: can't stat -${arg}'s "$file"./
        unless $time = ( stat($file) )[$STAT_MTIME];

    # FIXED:
    $! = 2, die qq/$0: can't stat -${arg}'s "$file"./
      unless $time = ( stat($file) )[$STAT_MTIME];

The second line is a continuation of the first, and this update prevents this alignment. The above 'BAD' formatting was in the previous developmental version of perltidy, not the previous release. This update added 12 Dec 2020, 5b56147.

Improve vertical alignment in some two-line matches

When two lines would be perfectly aligned except for the line length limit, previously they would only be aligned if they had a common leading equals. The update removes this restriction and allows as many alignments to be made as possible. The results are generally improved. This update was made 11 Dec 2020, f3c6cd8. Some examples:

# In this example the side comments were limiting the matches

    # OLD
    shift @data if @data and $data[0] =~ /Contributed\s+Perl/;    # Skip header
    pop @data if @data and $data[-1] =~ /^\w/;    # Skip footer, like

    # NEW
    shift @data if @data and $data[0]  =~ /Contributed\s+Perl/;    # Skip header
    pop @data   if @data and $data[-1] =~ /^\w/;    # Skip footer, like

# The same is true here.

    # OLD
    if ($tvg::o_span) { $tvg::hour_span = $tvg::o_span; }
    if ( $tvg::hour_span % 2 > 0 ) { $tvg::hour_span++; }    # Multiple of 2

    # NEW
    if ($tvg::o_span)              { $tvg::hour_span = $tvg::o_span; }
    if ( $tvg::hour_span % 2 > 0 ) { $tvg::hour_span++; }    # Multiple of 2

In the next example, the first comma is now aligned but not the second, because of the line length limit:

    # OLD
    is( MyClass->meta, $mc, '... these metas are still the same thing' );
    is( MyClass->meta->meta, $mc->meta, '... these meta-metas are the same thing' );

    # NEW
    is( MyClass->meta,       $mc, '... these metas are still the same thing' );
    is( MyClass->meta->meta, $mc->meta, '... these meta-metas are the same thing' );

In this last example, the first comma is not aligned, but alignment resumes after the second comma.

    # OLD
    is( $obj->name, $COMPRESS_FILE, "   Name now set to '$COMPRESS_FILE'" );
    is( $obj->prefix, '', "   Prefix now empty" );

    # NEW
    is( $obj->name, $COMPRESS_FILE, "   Name now set to '$COMPRESS_FILE'" );
    is( $obj->prefix, '',           "   Prefix now empty" );
Improve vertical alignment in some marginal matches

In perltidy a 'marginal match' occurs for example when two lines share some alignment tokens but are somewhat different. When this happens some limits are placed on the size of the padding spaces that can be introduced. In this update the amount of allowed padding is significantly increased for certain 'good' alignment tokens. Results of extensive testing were favorable provided that the change is restricted to alignments of '=', 'if' and 'unless'. Update made 10 Dec 2020, a585f0b.

    # OLD
    my @roles = $self->role_names;
    my $role_names = join "|", @roles;

    # NEW
    my @roles      = $self->role_names;
    my $role_names = join "|", @roles;

    # OLD
    $sysname .= 'del' if $self->label =~ /deletion/;
    $sysname .= 'ins' if $self->label =~ /insertion/;
    $sysname .= uc $self->allele_ori->seq if $self->allele_ori->seq;

    # NEW
    $sysname .= 'del'                     if $self->label =~ /deletion/;
    $sysname .= 'ins'                     if $self->label =~ /insertion/;
    $sysname .= uc $self->allele_ori->seq if $self->allele_ori->seq;
Improve vertical alignment of lines ending in fat comma

A minor adjustment was made to the rule for aligning lines which end in '=>'. When there are just two lines in an alignment group, the alignment is avoided if the first of the two ends in a '=>'. Previously, alignment was avoided if either ended in a '=>'. The old rule was preventing some good alignments in a later stage of the iteration. In the following example, the last two lines are processed separately because they do not match the comma in 'sprintf'. The new rule allows the fat comma alignment to eventually get made later in the iteration. Update made 9 Dec 2020, ca0ddf4.

    # OLD
    $template->param(
        classlist => $classlist,
        ...,
        suggestion => $suggestion,
        totspent   => sprintf( "%.2f", $totspent ),
        totcomtd   => sprintf( "%.2f", $totcomtd ),
        totavail   => sprintf( "%.2f", $totavail ),
        nobudget => $#results == -1 ? 1 : 0,
        intranetcolorstylesheet =>
          C4::Context->preference("intranetcolorstylesheet"),
        ...
    );

    # NEW
    $template->param(
        classlist => $classlist,
        ...,
        suggestion              => $suggestion,
        totspent                => sprintf( "%.2f", $totspent ),
        totcomtd                => sprintf( "%.2f", $totcomtd ),
        totavail                => sprintf( "%.2f", $totavail ),
        nobudget                => $#results == -1 ? 1 : 0,
        intranetcolorstylesheet =>
          C4::Context->preference("intranetcolorstylesheet"),
        ...
    );
Avoid processing a file more than once

In the unlikely event that a user enters a filename more than once on the command line to perltidy, as for 'file1.pl' here

  perltidy file1.pl file1.pl 

then that file will be processed more than once. This looks harmless, but if the user was also using the -b (backup) parameter, then the original backup would be overwritten, which is not good. To avoid this, a filter has been placed on the list of files to remove duplicates. 9 Dec 2020, 646a542.

Fix for issue git #49, exit status not correctly set

The exit status flag was not being set for the -w option if the -se or if the -q flag were set. Issue git #44 was similar but a special case of the problem. The problem was fixed 8 Dec 2020, cb6028f.

Issues fixed after release 20201202

Fix for issue git #47

This issue has to do with the --weld-nested-containers option in the specific case of formatting a function which returns a list of anonymous subs. For example

    $promises[$i]->then(
        sub { $all->resolve(@_); () },
        sub {
            $results->[$i] = [@_];
            $all->reject(@$results) if --$remaining <= 0;
            return ();
        }
    );

A bug introduced in v20201202 caused an incorrect welding to occur when the -wn flag was set

    $promises[$i]->then( sub { $all->resolve(@_); () },
        sub {
        $results->[$i] = [@_];
        $all->reject(@$results) if --$remaining <= 0;
        return ();
        } );

This bug has been fixed, and code which has been incorrectly formatted will be correctly formatted with the next release. The bug was a result of a new coding introduced in v20201202 for fixing some issues with parsing sub signatures. Previously they were sometimes parsed the same as prototypes and sometimes as lists, now they are always parsed as lists. Fixed 6 Dec 2020, 6fd0c4f.

Issues fixed after release 20201001

removed excess spaces in a package declaration

Testing revealed that for a line such as

   package        Bob::Dog;

which has extra spaces or even tabs after the keyword 'package', the extra spaces or tabs were not being removed. This was fixed 28 Nov 2020, 008443d. The line now formats to

    package Bob::Dog;
do not automatically delete closing side comments with --indent-only

For the parameter combination --indent-only and --closing-side-comments, old closing side comments were getting deleted but new closing side comments were not made. A fix was made to prevent this deletion. This fix was made 27 Nov 2020, 957e0ca.

fix to stop at 1 iteration when using --indent-only

Previously, for the combination --indent-only and -conv, two iterations would be done. Only one iteration is necessary in this case. Fix made 23 Nov 2020, ae493d8.

fix for formatting signed numbers with spaces

In developing an improved convergence test, an issue slowing convergence was found related to signed numbers as in the following line,

    @london = (deg2rad(-  0.5), deg2rad(90 - 51.3));

The leading '-' here is separated from the following number '0.5'. This is handled by tokenizing the minus as type 'm' and the number as type 'n'. The whitespace between them is removed in formatting, and so the space is gone in the output. But a little problem is that the default rule for placing spaces within the parens is based on the token count, after the first formatting the result is

    @london = ( deg2rad( -0.5 ), deg2rad( 90 - 51.3 ) );

The next time it is formatted, the '-0.5' counts as one token, resulting in

    @london = ( deg2rad(-0.5), deg2rad( 90 - 51.3 ) );

Notice that the space within the parens around the '-0.5' is gone. An update was made to fix this, so that the final state is reached in one step. This fix was made 23 Nov 2020, f477c8b.

fix to prevent conversion of a block comment to hanging side comment

A rare situation was identified during testing in which a block comment could be converted to be a hanging side comment. For example:

    sub macro_get_names {    #
    #
    # %name = macro_get_names();  (key=macrohandle, value=macroname)
    #
        local (%name) = ();
        ...
    }

For the following specific contitions the block comment in line 2 could be converted into a hanging side comment, which is undesirable:

   1. The line contains nothing except for a '#' with no leading space
   2. It follows a line with side comment
   3. It has indentation level > 0

An update was made to prevent this from happening. There are two cases, depending on the value of --maximum-consecutive-blank-lines, or -mbl. If this value is positive (the default) then a blank line is inserted above the block comment to prevent it from becoming a hanging side comment. If this -mbl is zero, then the comment is converted to be a static block comment which again prevents it from becoming a hanging side comment. This fix was made 23 Nov 2020, 2eb3de1.

improved convergence test

A better test for convergence has been added. When iterations are requested, the new test will stop after the first pass if no changes in line break locations are made. Previously, at least two passes were required to verify convergnece unless the output stream had the same checksum as the input stream. Extensive testing has been made to verify the correctness of the new test. This update was made 23 Nov 2020, 29efb63.

fixed problem with vertical alignments involving 'if' statements

An update was made to break vertical alignment when a new sequence of if-like statements or ternary statements is encountered. This situation was causing a loss of alignment in some cases. For example

  OLD:
    $m1 = 0;
    if ( $value =~ /\bset\b/i )      { $m0 = 1; }
    if ( $value =~ /\barithmetic/i ) { $m1 = 1; }
    if    ( $m0 && !$m1 ) { $CONFIG[1] = 0; }
    elsif ( !$m0 && $m1 ) { $CONFIG[1] = 1; }
    else                  { $ok        = 0; last; }

 NEW:
    $m1 = 0;
    if    ( $value =~ /\bset\b/i )      { $m0        = 1; }
    if    ( $value =~ /\barithmetic/i ) { $m1        = 1; }
    if    ( $m0 && !$m1 )               { $CONFIG[1] = 0; }
    elsif ( !$m0 && $m1 )               { $CONFIG[1] = 1; }
    else                                { $ok        = 0; last; }

This update was made 15 Nov 2020, 2b7784d.

added option -wnxl=s to give control of welding by the -wn parameter

The parameter string can restrict the types of containers which are welded. This was added 11 Nov 2020 in 'added -wnxl=s for control of -wn', 2e642d2.

merged pull request git #46

The man page gave the incorrect string for -fse. This was fixed 11 Nov 2020 in 1f9869e.

recognize overloaded RPerl operators to avoid error messages

RPerl uses some bareword operators which caused error messages. An update was made to avoid this problem in files containing 'use RPerl'. This update was made 6 Nov 2020, f8bd088.

fix issue git #45, -wn and -vtc=n now work together

When -wn was set, the -vtc=n flag was being ignored. This was a simple fix made 5 Nov 2020 in 'fix issue git #45, -wn and -vtc=n now work together', 1fbc381.

implement request RT #133649, added parameters -kbb=s and -kba=s

These parameters request that old breakpoints be kept before or after selected token types. For example, -kbb='=>' means that newlines before fat commas should be kept. This was added 4 Nov 2020.

added parameters -maxue=n and maxle=n

These parameters had tentatively been hardwired in the tokenizer. Now the user can control them or turn the checks off altogether.

Fix problem parsing '$$*'

In random testing, an error was encountered parsing the following line

  $self->{"mod_id"}=($$*1001)%(10**(rand()*6));
                       ---^
  found Number where operator expected (previous token underlined)

The line parsed correctly with a space between the '$$' and the '*'. The problem had to do with an error in some newer code for postfix dereferencing, and this was fixed on 2 Nov 2020, 'fix problem scanning '$$'; revise call to operator_expected', 49d993b.

Update for git #44, fix exit status for assert-tidy/untidy

The exit status was always 0 for --assert-tidy if the user had turned off error messages with -quiet. This was fixed by gluesys/master in 'fix exit status for assert-tidy/untidy options', 625d250.

Fixed problem parsing extruded signature

A parsing error was encountered in a test parsing the following extruded signature:

  sub foo2
  (
  $
  first
  ,
  $
  ,
  $
  third
  )
  {
  return
  "first=$first, third=$third"
  ;
  }

The second '$' combined with the ',' on the next line to form a punctuation variable. This was fixed 20 Oct 2020 in 'fixed problem parsing extruded signature', 9b454f6.

The file parses correctly now, with formatted output

  sub foo2 ( $first, $, $third ) {
      return "first=$first, third=$third";
  }
Fixed several uses of undefined variables found in testing

Several instances of incorrect array indexing were found in testing and fixed. These each involved incorrectly indexing with index -1. They were found by placing undefs at the end of arrays. None of these was causing incorrect formatting. They were fixed 26 Oct 2020 in 'fixed several instances of incorrect array indexing', c60f694.

Prevent syntax error by breaking dashed package names

In stress testing perltidy with the -extrude option, the following test snippet

  use perl6-alpha;

was broken into sepate lines

  use
  perl6
  -
  alpha
  ;

A rule was added to prevent breaking around a dash separating two barewords. Rerunning gives

  use
  perl6-alpha
  ;

This was fixed 26 Oct 2020 in 'prevent breaking package names with trailing dashes', 9234be4.

Prevent syntax error by breaking dashed barewords

In stress testing perltidy with the -extrude option, using the following test snippet

  my %var;
  {
      $var{-y}  = 1;
      $var{-q}  = 1;
      $var{-qq} = 1;
      $var{-m}  = 1;
      $var{y}   = 1;
      $var{q}   = 1;
      $var{qq}  = 1;
      $var{m}   = 1;
  }

a syntax error was created when newlines were placed before or after the dashes. It is necessary to always keep a dash on the same line with its surrounding tokens. A rule was added to do this. The new 'extruded' result for the above snippet is:

  my%var
  ;
  {
  $var{-y}
  =
  1
  ;
  $var{-q}
  =
  1
  ;
  $var{-qq}
  =
  1
  ;
  $var{-m}
  =
  1
  ;
  $var{y}
  =
  1
  ;
  $var{q}
  =
  1
  ;
  $var{qq}
  =
  1
  ;
  $var{m}
  =
  1
  ;
  }

This update was added 26 Oct 2020, 'prevent syntax error by breaking dashed barewords', e121cae.

more types of severe errors will prevent formatting

Files for which 'severe errors' are found have always been output verbatim rather than being formatted. The definition of 'severe error' has been expanded to include a final indentation level error greater than 1, more than 2 brace errors, and more than 3 "unexpected token type" parsing errors. The goal is to avoid formatting a non-perl script or a perl script with severe errors. So for example the following snippet has a level error of 2

  {{{{
  }}

was previously output with default parameters as

  { 
      {
          {
              {}
          }

along with an error message. But now it is just output verbatim as

  {{{{
  }}

along with an error message. This update was added 25 Oct 2020, 'avoid formatting files with more types of severe errors', 2a86f51.

added 'state' as keyword

A statement such as the following was generating an error message at the colon:

   state $a : shared;

The problem was that 'state' was not in the list of keywords. This has been fixed and the line now parses without error. The 'state.t' test file for perl 5.31 now formats without error. This was added 18 Oct 2020 in "add 'state' as keyword", d73e15f.

sub signatures no longer parsed with prototypes

Simple signatures (those without commas) were being parsed with code originally written for prototypes. This prevented them from being formatted with the usual formatting rules. This was changed so that all signatures are now formatted with the normal formatting rules. For example:

 # Old, input and after formatting:
 sub t123 ($list=wantarray) { $list ? "list" : "scalar" }

 # New, after formatting
 sub t123 ( $list = wantarray ) { $list ? "list" : "scalar" }

Notice that some spaces have been introduced within the signature. Previously the contents of the parens not changed unless the parens contained a list.

This change introduced a problem parsing extended syntax within signatures which has been fixed. In the following snippet, the ':' caused a parsing error which was fixed.

  # perltidy -sal='method'
  method foo4 ( $class : $bar, $bubba ) { $class->bar($bar) }

The ':' here is given a type of 'A'. This may be used to change the spacing around it. For example:

  # perltidy -sal='method' -nwls='A'
  method foo4 ( $class: $bar, $bubba ) { $class->bar($bar) }

This update was added 18 Oct 2020, in 'format all signatures separately from prototypes', e6a10f3. The test file 'signatures.t' distributed with perl5.31 formats without error now.

fix parsing problem with $#

A problem with parsing variables of the form $# and $#array was found in testing and fixed. For most variables the leading sigil may be separated from the remaining part of the identifier by whitespace. An exception is for a variable beginning with '$#'. If there is any space between the '$' and '#' then the '#' starts a comment. So the following snippet is has valid syntax and is equivalent to $ans=40;

    my $ #
    #
    ans = 40;

This was being misparsed and was fixed 17 Oct 2020, in 'fixed parsing error with spaces in $#' a079cdb.

fix missing line break for hash of subs with signatures

During testing the following error was found and fixed. Given the following input snippet:

    get(
        on_ready => sub ($worker) {
            $on_ready->end;
            return;
        },
        on_exit => sub ( $worker, $status ) { return; },
    );

The resulting formatting was

    get(
        on_ready => sub ($worker) {
            $on_ready->end;
            return;
        }, on_exit => sub ( $worker, $status ) { return; },
    );

Notice that the break after the comma has been lost. The problem was traced to a short-cut taken by the code looking for one-line blocks. The unique circumstances in which this occurred involved a hash of anonymous subs, one with a signature with multiple parameters and short enough to be a one-line block, as in the last sub definition line. This was fixed 17 Oct 2020 in 'fix missing line break for hash of subs with signatures', 51428db.

fix issues with prototype and signature parsing

Problems with parsing prototypes and signatures were found during testing and fixed 17 Oct 2020 in 'fixed problem parsing multi-line signatures with comments', 017fd07. For example the following snippet was mis-parsed because of the hash mark.

    sub test ( # comment ))) 
        $x, $x) { $x+$y }

Complex signature expressions such as the following are now parsed without error:

    sub t086
        ( #foo)))
        $ #foo)))
        a #foo)))
        ) #foo)))
        { $a.$b }
improve guess for pattern or division

The following line caused a tokenization error in which the two slashes were parsed as a pattern.

   my $masksize = ceil( Opcode::opcodes / 8 );    # /

This problem was discovered in random testing. When a slash follows a bareword whose prototype is not known to perltidy, it has to guess whether the slash starts a pattern or is a division. The guessing logic was rewritten and improved 14 Oct 2020 in 'rewrote logic to guess if divide or pattern', afebe2f.

fix -bos to keep isolated semicolon breaks after block braces

The flag -bos, or --break-at-old-semicolon-breakpoints, keeps breaks at old isolated semicolons. For example

    $z = sqrt($x**2 + $y**2)
      ;

In testing it was found not to be doing this after braces which require semicolons, such as 'do' and anonymous subs. This was fixed 12 Oct 2020 in 'fix -bos to work with semicolons after braces', 03ee7fc. For example

    my $dist = sub {
        $z = sqrt( $x**2 + $y**2 )
          ;
      }
      ;
keep break after 'use overload'

If a line break occurs after use overload then it will now be kept. Previously it was dropped. For example, this would be kept intact:

                use overload
                    '+' => sub {
                        print length $_[2], "\n";
                        my ( $x, $y ) = _order(@_);
                        Number::Roman->new( int $x + $y );
                    },
                    '-' => sub {
                        my ( $x, $y ) = _order(@_);
                        Number::Roman->new( int $x - $y );
                    },
                    ...

This keeps the list from shifting to the right and can avoid problems in formatting the list with certain styles, including with the -xci flag. Fixed 12 Oct 2020 in 'keep break after use overload statement', 8485afa.

added flag -xci to improve formatting when -ci and -i are equal, issue git #28

This flag causes continuation indentation to "extend" deeper into structures. If you use -ci=n and -i=n with the same value of n you will probably want to set this flag. Since this is a fairly new flag, the default is -nxci to avoid disturbing existing formatting.

terminal braces not indenting correctly with -bli formatting, issue git #40

This problem is illustrated with the following snippet when run with -bli -blil='*'

    #-bli -bli list='*'
    try
      {
        die;
      }
    catch
      {
        die;
      };    # <-- this was not indenting

This was due to conflicting rules and was fixed 1 Oct 2020 in commit 'fix issue git #40, incorrect closing brace indentation with -bli', a5aefe9.

At the same time, it was noted that block types sort/map/grep and eval were not following -bli formatting when -blil='*' and this was fixed. For example, with corrected formatting, we would have

  # perltidy -bli -blil='*'
    eval
      {
        my $app = App::perlbrew->new( "install-patchperl", "-q" );
        $app->run();
      }
      or do
      {
        $error          = $@;
        $produced_error = 1;
      };

Issues fixed after release 20200907

This is a detailed log of changes since the release 20200907. All bugs were found with the help of automated random testing.

Keep any space between a bareword and quote

In random testing, the -mangle option introduced a syntax error by deleting the space between barewords and quotes (test file 'MxScreen'), such as:

  oops"Your login, $Bad_Login, is not valid";

Sub 'is_essential_whitespace' was updated to prevent this on 27 Sep 2020, in 'keep any space between a bareword and quote', f32553c.

Fixed some incorrect indentation disagreements reported in LOG file

The .LOG file reports any disagreements between the indentation of the input and output files. This can help locate brace errors. These were incorrect when some of the options were used, including --whitespace-cycle, -bbhb, -nib. This was corrected 24 Sep 2020, 'fixed incorrect log entry for indentation disagreement', 3d40545. At the same time, locations of closing brace indentation disagreements are now tracked and reported in the .ERR file when there is a brace error. This can help localize the error if the file was previously formatted by perltidy.

If an =cut starts a POD section within code, give a warning

Previously only a complaint was given, which went into the log file and was not normally seen. Perl silently accepts this but it can cause significant problems with pod utilities, so a clear warning is better. This situation arose in testing on random files in combination with a -dp flag and it took some time to understand the results because of the lack of a warning.

Switched from using an eval block to the -can() function for sub finish_formatting>

This is not a bug, but is cleaner coding and insures that error messages get reported. This change was made 20 Sep 2020, 'switch from eval { } to ->can('finish_formatting')', 28f2a4f.

fixed uninitialized value reference

The following message was generated during automated testing

 Use of uninitialized value $cti in numeric eq (==) at /home/steve/bin/Perl/Tidy/Formatter.pm line 12079.
 Use of uninitialized value $cti in numeric eq (==) at /home/steve/bin/Perl/Tidy/Formatter.pm line 12089.
 Use of uninitialized value $cti in numeric eq (==) at /home/steve/bin/Perl/Tidy/Formatter.pm line 12097.

The problem could be simplified to running perltidy -wn on this snippet:

     __PACKAGE__->load_components( qw(
>         Core
> 
>     ) );

This was fixed 20 Sep 2020 in 'fixed_uninitialized_value', 8d6c4ed.

fix incorrect parsing of certain deprecated empty here-docs

The following snippet was being incorrectly parsed:

 print <<
 # Hello World 13!
 
   ;
 print "DONE\n";

This is a deprecated here-doc without a specified target but currently still a valid program. It would have been correctly parsed if the semicolon followed the '<<' operator rather than the here-doc.

This was found in random testing and fixed 16 Sep 2020. A warning message about deprecated here-doc targets was added.

make the arrow a vertical alignment token, git #39

The -> can now be vertically aligned if a space is placed before it with -wls='->'. Added 15 Sep 2020 as part of previous item, 9ac6af6.

add flags -bbhb=n, -bbsb=n, =bbp=n, git #38

These flags give control over the opening token of a multiple-line list. They are described in the man pages, perltidy.html. Added 15 Sep 2020 in "added flags -bbhb=n, -bbsb=n, -bbq=n, suggestion git #38". 9ac6af6.

Allow vertical alignment of line-ending fat comma

A change was made to allow a '=>' at the end of a line to align vertically, provided that it aligns with two or more other '=>' tokens. This update was 14 Sep 2020, 'Allow line-ending '=>' to align vertically', ea96739.

fixed uninitialized value reference

The following message was generated when running perltidy on random text:

 Use of uninitialized value $K_semicolon in subtraction (-) at /home/steve/bin/Perl/Tidy/Formatter.pm line 16467.

This was fixed 14 Sep 2020, included in 'Allow line-ending '=>' to align vertically', ea96739.

Do not create a zero size file by deleting semicolons

A rule was added to prevent a file consisting of a single semicolon

 ;

from becoming a zero length file. This could cause problems with other software. Fixed 13 Sep 2020, 'do not create a zero length file by deleting semicolons', b39195e.

fixed uninitialized value reference

The following message was generated when running perltidy on random text:

 Use of uninitialized value $cti in numeric eq (==) at /home/steve/bin/Perl/Tidy/Formatter.pm line 11926.
 Use of uninitialized value $cti in numeric eq (==) at /home/steve/bin/Perl/Tidy/Formatter.pm line 11936.
 Use of uninitialized value $cti in numeric eq (==) at /home/steve/bin/Perl/Tidy/Formatter.pm line 11944.

This was fixed 13 Sep 2020 in 'fixed unitialized variable problem ', adb2096.

fixed uninitialized value reference

The following message was generated when running perltidy on random text:

 substr outside of string at /home/steve/bin/Perl/Tidy/Tokenizer.pm line 7362.
 Use of uninitialized value in concatenation (.) or string at /home/steve/bin/Perl/Tidy/Tokenizer.pm line 7362.

This was fixed 13 Sep 2020 in 'fixed unitialized variable problem', 5bf49a3.

fixed uninitialized value reference

The following message was generated when running perltidy on random text:

 Use of uninitialized value $K_opening in subtraction (-) at /home/steve/bin/Perl/Tidy/Formatter.pm line 16467.

This was fixed 13 Sep 2020 in 'fix undefined variable reference', 1919482.

hashbang warning changed

The following snippet generated a warning that there might be a hash-bang after the start of the script.

 $x = 2;
 #!  sunos does not yet provide a /usr/bin/perl
 $script = "$^X $script";

To prevent this annoyance, the warning is not given unless the first nonblank character after the '#!' is a '/'. Note that this change is just for the warning message. The actual hash bang check does not require the slash.

Fixed 13 Sep 2020, 'prevent unnecessary hash-bang warning message' 4f7733e and 'improved hash-bang warning filter', fa84904.

uninitialized index referenced

An unitialized index was referenced when running on a file of randomly generated text:

  Use of uninitialized value $K_oo in subtraction (-) at /home/steve/bin/Perl/Tidy/Formatter.pm line 7259.

This was fixed 12 Sep 2020 in 'fixed undefined index', 616bb88.

Oops message triggered

The parameter combination -lp -wc triggered an internal bug message from perltidy:

 398: Program bug with -lp.  seqno=77 should be 254 and i=1 should be less than max=-1
 713: The logfile perltidy.LOG may contain useful information
 713: 
 713: Oops, you seem to have encountered a bug in perltidy.  Please check the
 713: BUGS file at http://perltidy.sourceforge.net.  If the problem is not
 713: listed there, please report it so that it can be corrected.  Include the
 ...

The problem is that the parameters --line-up-parentheses and --whitespace-cycle=n are not compatible. The fix is to write a message and turn off the -wc parameter when the both occur. This was fixed 8 Sep 2020 in "do not allow -wc and -lp together, can cause bugs", 7103781.

Internal fault detected by perltidy

This snippet after processing with the indicated parameters triggered a Fault message in store-token-to-go due to discontinuous internal index values :

  perltidy --noadd-newlines --space-terminal-semicolon

  if ( $_ =~ /PENCIL/ ) { $pencil_flag= 1 } ; ;
  $yy=1;

This triggered the message:

 ==============================================================================
 While operating on input stream with name: '<stdin>'
 A fault was detected at line 7472 of sub 'Perl::Tidy::Formatter::store_token_to_go'
 in file '/home/steve/bin/Perl/Tidy/Formatter.pm'
 which was called from line 8298 of sub 'Perl::Tidy::Formatter::process_line_of_CODE'
 Message: 'Unexpected break in K values: 591 != 589+1'
 This is probably an error introduced by a recent programming change. 
 ==============================================================================

The deletion of the extra, spaced, comma had created an extra space in the token array which had not been forseen in the original programming. It was fixed 10 Sep 2020 in "fixed very rare fault found with automated testing", eb1b1d9.

Error parsing deprecated $# variable

This problem can be illustrated with this two-line snippet:

  $#
  eq$,?print"yes\n":print"no\n";

Perltidy joined '$#' and 'eq' to get $#eq, but should have stopped at the line end to get $# followed by keyword 'eq'. (Note that $# is deprecated). This was fixed 11 Sep 2020 in "fixed several fringe parsing bugs found in testing", 85e01b7.

Error message parsing a file with angle brackets and ternaries

This problem can be illustrated with the following test snippet which was not correctly parsed.

 print$$ <300?"$$<300\n":$$<700?"$$<700\n":$$<2_000?"$$<2,000\n":$$<10_000?"$$ <10,000\n":"$$>9,999\n";

The problem is related to the '<' symbol following the '$$' variable, a possible filehandle, and is similar to a previous bug. The problem was corrected 11 Sep 2020 in "fixed several fringe parsing bugs found in testing", 85e01b7. The line now correctly formats to

 print $$ < 300  ? "$$<300\n"
   : $$ < 700    ? "$$<700\n"
   : $$ < 2_000  ? "$$<2,000\n"
   : $$ < 10_000 ? "$$ <10,000\n"
   :               "$$>9,999\n";
code crash with cuddled-else formatting on unbalanced files

A file with incorrect bracing which effectively gave negative indentation caused a crash when a stack was referenced with a negative index. The problem was fixed 8 Sept 2020 in "convert array to hash to avoid trouble with neg levels in bad files", a720e0d.

error message 'Unterminated angle operator?'

This error can be demonstrated with this line.

  print $i <10 ? "yes" : "no";

Perl has some strange parsing rules near a possible filehandle, and they change over time. The '<' here is a less than symbol, but perltidy expected that it might be the start of an angle operator, based on the old rules, and gave a warning. The formatting was still correct, but the warning was confusing. This has been fixed 8 Sep 2020 in 'remove confusing warning message', 0a4d725.

Line broken after here target

This problem is illustrated with the following snippet

  $sth= $dbh->prepare (<<"END_OF_SELECT") or die "Couldn't prepare SQL" ;
      SELECT COUNT(duration),SUM(duration) 
      FROM logins WHERE username='$user'
  END_OF_SELECT

When run with a short line length it got broken after the here target, causing an error. This was due to a recent program change and fixed 7 Sep 2020 in 'fixed bug where long line with here target got broken', 8f7e4cb.

undefined variable named 'test2'

An uninitialized value was being referenced and triggered this message:

 undefined test2, i_opening=5, max=18, caller=Perl::Tidy::Formatter ./perltidy-20200907.pl 13465
 Use of uninitialized value $test2 in numeric eq (==) at ./perltidy-20200907.pl line 19692.

Fixed 8 Sep 2020 in 'fixed rare problem with stored index values for -lp option', 4147c8c.

Line order switched at start of quoted text

This problem arose in several scripts involving the parameter --line-up-parentheses pluse one or more of the vertical tightness flags. It can be illustrated with the following snippet:

    perltidy --line-up-parentheses --paren-vertical-tightness=1

    if (
        ( $name, $chap ) =
        $cur_fname =~ m!^Bible/
          .*?/          # testament
          .*?/          # range of books
          (.*?)/        # book name
          .*?           # optional range of verses
          (\d+)$!x
      )
    {
        $cur_name = "$name $chap";
    }

This gave

    if (( $name, $chap ) =
          .*?/          # testament
        $cur_fname =~ m!^Bible/
          .*?/          # range of books
          (.*?)/        # book name
          .*?           # optional range of verses
          (\d+)$!x
      )
    {
        $cur_name = "$name $chap";
    }

Notice the incorrect line order. The problem was an incorrect order of operations in the vertical aligner flush, leaving a line stranded and coming out in the wrong order. This was fixed 11 Sep 2020.

crash due to bad index named '$j_terminal_match'

This crash was due to an index error which caused a non-existent object to be referenced. The problem is fixed 2020-09-07 in "fix problem of undefined values involving j_terminal_match", c5bfa77. The particular parameters which caused this were:

    --noadd-newlines --nowant-left-space='=' 
an issue with the -x flag

This is not a bug but did take some time to resolve. The problem was reduced to the following script run with the -x flag (--look-for-hash-bang)

 print(SCRIPT$headmaybe . <<EOB . <<'EOF' .$tailmaybe),$!;
 #!$wd/perl
 EOB
 print "\$^X is $^X, \$0 is $0\n";
 EOF

The resulting file had a syntax error (here-doc target EOB changed).

 print(SCRIPT$headmaybe . <<EOB . <<'EOF' .$tailmaybe),$!;
 #!$wd/perl
 EOB print "\$^X is $^X, \$0 is $0\n";
 EOF

The problem is that the -x flag tells perltidy not to start parsing until it sees a line starting with '#!', which happens to be in a here-doc in this case.

A warning was added to the manual 7 Sept 2020 in "add warning about inappropriate -x flag", fe66743.

error parsing sub signature

This problem was reduced to the following snippet:

 substr
 (
  $#
 )

The deprecated variable '$#' was being parsed incorrectly, and this was due to an error in which the word 'substr' followed by a paren was taken as the start of a sub signature. The problem was fixed 8 Sep 2020 in 'fix problem parsing sub prototypes' 569e05f. The code

  $container_type =~ /^sub/;

was corrected to be

  $container_type =~ /^sub\b/;
uninitialized value message, found 7 Sep 2020

Unitialized values were referenced. An index was not being tested. Fixed 8 Sep 2020 in "fix undefined variable", 9729965.

 Use of uninitialized value $Kon in array element at /home/steve/bin/Perl/Tidy/Formatter.pm line 4022.
 Use of uninitialized value $Kon in array element at /home/steve/bin/Perl/Tidy/Formatter.pm line 4023.
 Use of uninitialized value $Ko in subtraction (-) at /home/steve/bin/Perl/Tidy/Formatter.pm line 4023.

Open Issues

These are known issues which have not been fixed.

lexical subs not fully supported

Basic parsing of lexical subs works but some aspects of lexical subs are not yet functional. One of these is that unlike regular subs, lexical subs can override names of builtin functions.

First consider the following snippet

  sub s { 
    my $arg=$_[0];
    print "s called with arg $arg\n";
  }
  s(1);
  s(2);

The 's' in the two last lines is the builtin s function, not the sub. Both perltidy and perl make the same assumption here. This program happens to still run but prints nothing. It will not run if the last semicolon is removed.

Now consider the following snippet in which the sub has a preceding 'my'

  use feature 'lexical_subs', 'signatures';
  my sub s { 
    my $arg=$_[0];
    print "s called with arg $arg\n";
  }
  s(1);
  s(2);

The builtin function 's' is replaced by the sub s here, and the program runs. Perltidy will format this but it is assuming that the s in the two last lines are the builtin s function. If the last semicolon is removed, there will be an formatting error. So perltidy and perl make different assumptions in this case.

Another issue is that perltidy does not yet remember the extent of the scope of a lexical sub.

issues with paren-less calls

Consider the following snippet:

  use Test::More;
  ok open($stdin, "<&", $1), 'open ... "<&", $magical_fileno', ||  _diag $!;

Note the unusual situation of a comma followed by an '||'. Perltidy will format this satisfactorily but it will write an error message. The syntax is correct, however. Perl knows the prototype of the 'ok' function, which is called here without parens, so the last comma marks the last arg and is needed to keep the || from attaching to the last arg.

Full support of peren-less calls will probably never be implemented in perltidy because it would require that it parse all of the modules used to find the prototypes. This would make it impossible to run perltidy on small snippets of code from within an editor.

The problem can be avoid if parens are used:

  ok ( open($stdin, "<&", $1), 'open ... "<&", $magical_fileno') ||  _diag $!;
multiple sub paren calls

Perltidy currently flags as an error a closing paren followed by an opening paren, as in the following

  $subsubs[0]()(0)

This syntax is ok. The example is from test 'current_sub.t' in perl5.31.