Escaped whitespace is treated as as whitespace, rather than literals

[Xophmeister]

The grammar, as of d1a1a3f, swallows any escaped horizontal whitespace, treating them as actual whitespace rather than literals. For example:

# This should output:
# <x>
# < >
# <	>
# <x>
printf "<%s>\n" x \  \	 x

The parse tree for the above looks like this:

program [0, 0] - [1, 0]
  command [0, 0] - [0, 25]
    name: command_name [0, 0] - [0, 6]
      word [0, 0] - [0, 6]
    argument: string [0, 7] - [0, 15]
      string_content [0, 8] - [0, 14]
    argument: word [0, 16] - [0, 17]
    argument: word [0, 24] - [0, 25]

We see only three arguments -- the printf format string and the two xs -- rather than five, including the escaped space and tab between the xs.

Whitespace literals are semantically different from whitespace used in tokenisation. For example:

echo \ # this is not a comment

[Xophmeister]

Note: This only seems to occur when the escaped whitespace literal is at the beginning of a token. This parses correctly, for example:

# This should output:
# <a b>
# <c >
printf "<%s>\n" a\ b c\