GNU bug report logs - #57604
[ef]grep usage -> POSIXLY_CORRECT?

Previous Next

To reply to this bug, email your comments to 57604 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: bug-grep <at> gnu.org
Subject: [ef]grep usage -> POSIXLY_CORRECT?
Date: Mon, 5 Sep 2022 16:07:35 -0600

It is painful to see the unconditional complaint (grep 3.8) about
[ef]grep usage when those programs have been present since the beginning
of Unix. I have hundreds of scripts that use [ef]grep since for many
years they were either the only, or then later the most portable, way to
get the behavior.

So, I suggest giving that warning only if POSIXLY_CORRECT is set.  Seems to
me this is a case for which that is intended. -k

Message #8 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Karl Berry <karl <at> freefriends.org>
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Mon, 5 Sep 2022 20:28:49 -0500

On 9/5/22 17:07, Karl Berry wrote:
> I have hundreds of scripts that use [ef]grep since for many
> years they were either the only, or then later the most portable, way to
> get the behavior.

Actually, egrep and fgrep were not entirely portable even before POSIX 
deprecated them in 2001. For example, in 1998 they were absent from SCO 
Unix unless you installed optional packages; see 
<https://www.scosales.com/ta/kb/105933.html>.

Similar problems are more common today now that POSIX no longer requires 
egrep and fgrep, for example, see OpenWRT bug 17803 
<https://github.com/openwrt/packages/issues/17803>. These days, my 
impression is that it's more portable to use grep -E than to use egrep. 
Of course neither is perfectly portable, but if a script is going to use 
just one or just the other, grep -E is the way to go nowadays.

>   I suggest giving that warning only if POSIXLY_CORRECT is set.
That wouldn't be right, as POSIXLY_CORRECT governs misfeatures required 
by POSIX, which isn't the case here: egrep and fgrep are extensions to 
POSIX.

For the bug report that prompted those warnings, please see:

https://bugs.gnu.org/49996

Message #11 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: eggert <at> cs.ucla.edu
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Tue, 6 Sep 2022 14:33:00 -0600

    Actually, egrep and fgrep were not entirely portable 

They were portable to every system I ever used (admittedly not the
entire world), before POSIX got involved.

    These days, my impression is that it's more portable to use grep -E
    than to use egrep.

I agree. And IMHO that is a bug POSIX gratuitously introduced.

    That wouldn't be right, as POSIXLY_CORRECT governs misfeatures required 
    by POSIX, which isn't the case here: 

IMHO it is the case. The misfeature being fixed is POSIX unblessing
[ef]grep.

But, whatever. Since it bothers you to use POSIXLY_CORRECT, let's invent
some other envvar that turns off the warning, like
"PLEASE_LET_ME_USE_EFGREP_I_DONT_CARE_ABOUT_POSIX", and Arnold and I
will set it and life can go on.

    https://bugs.gnu.org/49996

I'm unconvinced. What Simon's bug report says is "hey, why not deprecate
[ef]grep because it's time". Well, IMHO it's not time, and will never be
time, and "deprecation" merely means "cause trouble for users for no
real reason". My "pet issue" is the exact opposite of Simon's ...

Without some way to turn off those warnings, GNU [ef]grep become
unusable without editing to remove the comment. It is completely
infeasible, not to mention a tremendous waste of time, to edit
everywhere on my systems that use them, after 40 years of historical
usage.

I'm not saying scripts intended to be portable should not be changed (as
we know, they have had to be, because POSIX forced it). But all the
scripts in the world which are *not* needed to be portable don't need to
be changed. They just need to keep working and not be randomly broken by
outside forces.

[ef]grep are fundamental names for the utility. Please reconsider. -k

Message #14 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Karl Berry <karl <at> freefriends.org>
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Tue, 6 Sep 2022 15:52:13 -0500

On 9/6/22 15:33, Karl Berry wrote:
> Since it bothers you to use POSIXLY_CORRECT, let's invent
> some other envvar that turns off the warning, like
> "PLEASE_LET_ME_USE_EFGREP_I_DONT_CARE_ABOUT_POSIX", and Arnold and I
> will set it and life can go on.

Python has PYTHONWARNINGS. I suppose GNU Grep could have a GREP_WARNINGS 
variable that could suppress these diagnostics from egrep and fgrep, 
along with diagnostics like these:

$ grep '\Q' /dev/null
grep: warning: stray \ before Q
$ grep '[:alpha:]' /dev/null
grep: character class syntax is [[:space:]], not [:space:]

We've been moving away from environment variables like GREP_OPTIONS, 
though, for security reasons. Perhaps we should let others comment.

In the meantime you can implement egrep and fgrep the way you like, as 
shell scripts in your private bin directory, as this is something you'll 
need for grep 3.8 anyway. And once you have that simple workaround, 
maybe we won't need the complexity of GREP_WARNINGS.

Message #17 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: arnold <at> skeeve.com
To: karl <at> freefriends.org, eggert <at> cs.ucla.edu
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 07 Sep 2022 01:47:39 -0600

I totally agree here with Karl, obsoleting fgrep and egrep is the
wrong way to go.  It's built into too many scripts world-wide, as
well as burned into the "finger ROMs" of too many users. Not to mention
all the books / training materials that talk about them that people
may have.

Thanks,

Arnold

Karl Berry <karl <at> freefriends.org> wrote:

>     Actually, egrep and fgrep were not entirely portable 
>
> They were portable to every system I ever used (admittedly not the
> entire world), before POSIX got involved.
>
>     These days, my impression is that it's more portable to use grep -E
>     than to use egrep.
>
> I agree. And IMHO that is a bug POSIX gratuitously introduced.
>
>     That wouldn't be right, as POSIXLY_CORRECT governs misfeatures required 
>     by POSIX, which isn't the case here: 
>
> IMHO it is the case. The misfeature being fixed is POSIX unblessing
> [ef]grep.
>
> But, whatever. Since it bothers you to use POSIXLY_CORRECT, let's invent
> some other envvar that turns off the warning, like
> "PLEASE_LET_ME_USE_EFGREP_I_DONT_CARE_ABOUT_POSIX", and Arnold and I
> will set it and life can go on.
>
>     https://bugs.gnu.org/49996
>
> I'm unconvinced. What Simon's bug report says is "hey, why not deprecate
> [ef]grep because it's time". Well, IMHO it's not time, and will never be
> time, and "deprecation" merely means "cause trouble for users for no
> real reason". My "pet issue" is the exact opposite of Simon's ...
>
> Without some way to turn off those warnings, GNU [ef]grep become
> unusable without editing to remove the comment. It is completely
> infeasible, not to mention a tremendous waste of time, to edit
> everywhere on my systems that use them, after 40 years of historical
> usage.
>
> I'm not saying scripts intended to be portable should not be changed (as
> we know, they have had to be, because POSIX forced it). But all the
> scripts in the world which are *not* needed to be portable don't need to
> be changed. They just need to keep working and not be randomly broken by
> outside forces.
>
> [ef]grep are fundamental names for the utility. Please reconsider. -k
>
>
>

Message #20 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Simon Josefsson <simon <at> josefsson.org>
To: Karl Berry <karl <at> freefriends.org>
Cc: eggert <at> cs.ucla.edu, 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 07 Sep 2022 09:55:24 +0200

[Message part 1 (text/plain, inline)]

Karl Berry <karl <at> freefriends.org> writes:

> But, whatever. Since it bothers you to use POSIXLY_CORRECT, let's invent
> some other envvar that turns off the warning, like
> "PLEASE_LET_ME_USE_EFGREP_I_DONT_CARE_ABOUT_POSIX", and Arnold and I
> will set it and life can go on.
>
>     https://bugs.gnu.org/49996
>
> I'm unconvinced. What Simon's bug report says is "hey, why not deprecate
> [ef]grep because it's time". Well, IMHO it's not time, and will never be
> time, and "deprecation" merely means "cause trouble for users for no
> real reason". My "pet issue" is the exact opposite of Simon's ...
>
> Without some way to turn off those warnings, GNU [ef]grep become
> unusable without editing to remove the comment. It is completely
> infeasible, not to mention a tremendous waste of time, to edit
> everywhere on my systems that use them, after 40 years of historical
> usage.
>
> I'm not saying scripts intended to be portable should not be changed (as
> we know, they have had to be, because POSIX forced it). But all the
> scripts in the world which are *not* needed to be portable don't need to
> be changed. They just need to keep working and not be randomly broken by
> outside forces.
>
> [ef]grep are fundamental names for the utility. Please reconsider. -k

I can understand your frustration, and maybe it would have been a
simpler solution to undeprecate the tools in documentation and have them
be official GNU-enhancements as they de-facto has been for the past 20+
years.  My pet issue was the inconsistency between documentation and
reality, and I don't care strongly how to resolve it.

On the other hand, it seems your main problem here is the warning: and I
agree that is annoying and should be fixed.  There doesn't seem to be
any way to disable it now?  It would be nice to fix that.

I think the simplest way to fix that is to drop the tools rather than to
issue warnings: on some current systems (I think it was Alpine), there
is no egrep and fgrep any more, and dealing with non-existing tools
(discover the error, then use egrep -E or add a personal alias for
interactive use) is easier than dealing with annoying warnings that
cannot be disabled.

/Simon

[signature.asc (application/pgp-signature, inline)]

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Simon Josefsson <simon <at> josefsson.org>
To: Paul Eggert via Bug reports for GNU grep <bug-grep <at> gnu.org>
Cc: 57604 <at> debbugs.gnu.org, Karl Berry <karl <at> freefriends.org>
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 07 Sep 2022 10:02:04 +0200

[Message part 1 (text/plain, inline)]

Paul Eggert via Bug reports for GNU grep <bug-grep <at> gnu.org> writes:

> $ grep '\Q' /dev/null
> grep: warning: stray \ before Q
> $ grep '[:alpha:]' /dev/null
> grep: character class syntax is [[:space:]], not [:space:]

Is the use of diagnostic warnings like this supported by POSIX?
Personally, I'd rather have tools exit with an error code on invalid
uses rather than issuing warning messages.

/Simon

[signature.asc (application/pgp-signature, inline)]

Message #29 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gary Johnson <garyjohn <at> spocom.com>
To: bug-grep <at> gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 7 Sep 2022 05:47:53 -0700

On 2022-09-07, Simon Josefsson via Bug reports for GNU grep wrote:

> On the other hand, it seems your main problem here is the warning: and I
> agree that is annoying and should be fixed.  There doesn't seem to be
> any way to disable it now?  It would be nice to fix that.
> 
> I think the simplest way to fix that is to drop the tools rather than to
> issue warnings: on some current systems (I think it was Alpine), there
> is no egrep and fgrep any more, and dealing with non-existing tools
> (discover the error, then use egrep -E or add a personal alias for
> interactive use) is easier than dealing with annoying warnings that
> cannot be disabled.

The warnings I see don't come from anything I've written; they've
been coming from third-party tools.  I'd rather not modify those.

Disabling the warnings is fairly simple.  Do one of the following.

1.  Edit the egrep and fgrep scripts to remove the warnings.
2.  Copy the egrep and fgrep scripts to ~/bin (someplace in PATH
    earlier than where the grep installation put them) and edit
    those copies to remove the warnings.

Regards,
Gary

Message #32 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gary Johnson <garyjohn <at> spocom.com>
To: bug-grep <at> gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 7 Sep 2022 05:36:10 -0700

On 2022-09-07, Simon Josefsson via Bug reports for GNU grep wrote:

> Personally, I'd rather have tools exit with an error code on invalid
> uses rather than issuing warning messages.

The warning messages are merely annoying.  Returning a non-zero exit
status would break scripts, and in many cases, silently.

Regards,
Gary

Message #35 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Simon Josefsson <simon <at> josefsson.org>
Cc: 57604 <at> debbugs.gnu.org, karl <at> freefriends.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 7 Sep 2022 14:26:07 -0500

On 9/7/22 03:02, Simon Josefsson via Bug reports for GNU grep wrote:
>> $ grep '\Q' /dev/null
>> grep: warning: stray \ before Q
>> $ grep '[:alpha:]' /dev/null
>> grep: character class syntax is [[:space:]], not [:space:]
> Is the use of diagnostic warnings like this supported by POSIX?

Yes, POSIX says that \Q produces undefined results, and that [:alpha:] 
can be treated as either [[:alpha:]], [:alph], or an error (the last is 
the GNU behavior).

> I'd rather have tools exit with an error code on invalid
> uses rather than issuing warning messages.

GNU grep does that for [:alpha:]. At some point it should do the same 
for \Q, as \Q's behavior is squirrelly anyway. I thought it might be too 
disruptive to do it for \Q right away so in 3.8 \Q merely gets warned about.

Message #38 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 7 Sep 2022 16:35:14 -0600

    sj> Personally, I'd rather have tools exit with an error code on
    invalid uses rather than issuing warning messages.

For me (and according to historical practice), an unnecessary \ is not
invalid. So, not surprisingly, I, on the other hand, would rather have
grep provide a way to turn off the warning (or go back to not warning by
default, but I guess that argument has been rejected).  --karl

Message #41 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 7 Sep 2022 16:35:17 -0600

    [ef]grep

I guess my basic issue is that I don't understand the benefit of the new
warning.  It causes a lot of trouble.  What is the countervailing
positive benefit?

    $ grep '\Q' /dev/null
    grep: warning: stray \ before Q

It would be nice to be able to turn those off too. (It hit me today.)

Those warnings are even worse than [ef]grep in a way, since they are not
changeable just by editing the scripts, but built-in behavior.

    We've been moving away from environment variables like GREP_OPTIONS, 
    though, for security reasons. 

Personally, I don't see a security issue with the putative GREPWARNINGS?
Unlike GREP_OPTIONS, whose removal is understandable (though it sure was
convenient, alas). --karl

Message #44 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Wed, 7 Sep 2022 16:35:18 -0600

    simpler solution to undeprecate the tools in documentation and have
    them be official GNU-enhancements as they de-facto has been for the
    past 20+ years

+20

    My pet issue was the inconsistency between documentation and reality, 

Agreed.

    I think the simplest way to fix that is to drop the tools

Not that anyone cares, but IMHO the simplest way is to delete the new
warning lines. End of problem.

    issue warnings: on some current systems (I think it was Alpine)
    is no egrep and fgrep any more

To my understanding, GNU should not cater to the lowest common
denominator, or be unfriendly to users. To quote from the coding
standards:

  Standard C and POSIX prohibit many kinds of extensions.  Feel
  free to make the extensions anyway ...
  ...
  But we do not follow either of these specifications [ANSI C and POSIX]
  rigidly, and there are specific points on which we decided not to
  follow them, so as to make the GNU system better for users.
  ...
  In particular, don't reject a new feature, or remove an old one,
                                             ^^^^^^^^^^^^^^^^^^^^
  merely because a standard says it is ``forbidden'' or ``deprecated''.

--karl

Message #47 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Karl Berry <karl <at> freefriends.org>
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Thu, 8 Sep 2022 02:21:42 -0700

Hi Karl,

Sorry to cause you grief, but...

On Wed, Sep 7, 2022 at 7:49 PM Karl Berry <karl <at> freefriends.org> wrote:
>
>     [ef]grep
>
> I guess my basic issue is that I don't understand the benefit of the new
> warning.  It causes a lot of trouble.  What is the countervailing
> positive benefit?

Some must care about portability, and these warnings help them do a better job.
As Gary mentioned above, it's easy to disable them.

>     $ grep '\Q' /dev/null
>     grep: warning: stray \ before Q

> It would be nice to be able to turn those off too. (It hit me today.)

I would argue that it is even more important to retain these
stray-backslash warnings, because they tend to highlight real bugs.
Consider these uses of \d:

  $ echo d | grep-3.7 '\d'
  d
  $ echo d | grep-3.8 '\d'
  grep: warning: stray \ before d

Anyone used to PCRE regexps (who isn't, these days) knows that its
"\d" is intended to match a digit, not the letter "d". With grep-3.7,
you'd get misbehavior and no warning about your error. With grep-3.8,
you'll get the diagnostic and maybe switch to using "grep -P", where
"\d" works as expected -- switching from \d to [0-9] hurts readability
and feels like dumbing-down, especially when there are two or more \d
uses. Using PCRE's \Q...\E groups *without -P* is another issue that
is now diagnosed.

For example, the following upstream projects have misuses of grep that
are exposed by running this:

  git grep 'grep .*\\[dQE]' | grep -ve '-[[:alnum:]]*P'

- linux
scripts/checkpatch.pl:          `grep -Eq
"\\"\\^\Q$vendor\E,\\.\\*\\":" $vp_file`;

- gcc
libgo/go/cmd/go/testdata/script/mod_get_lazy_indirect.txt:grep
'rsc.io/quote v\d+\.\d+\.\d+ // indirect$' go.mod
libgo/go/cmd/go/testdata/script/mod_get_lazy_indirect.txt:! grep
'rsc.io/quote v\d+\.\d+\.\d+$' go.mod
libgo/go/cmd/go/testdata/script/mod_get_lazy_indirect.txt:grep
'rsc.io/quote v\d+\.\d+\.\d+$' go.mod
libgo/go/cmd/go/testdata/script/mod_get_lazy_indirect.txt:! grep
'rsc.io/quote v\d+\.\d+\.\d+ // indirect$' go.mod

Message #50 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: jim <at> meyering.net
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Thu, 8 Sep 2022 17:01:01 -0600

Hi Jim,

    Some must care about portability, 
    
Certainly agreed. Even I do, sometimes :). But that does not mean
everyone needs to, in every situation.  As I said, I fail to understand
the benefit of making the warning unconditional.

So far as I can see, it's also against GNU principles, as I wrote,
though evidently you don't agree.

    and these warnings help them do a better job.

When people want extreme POSIX compliance, they should set
POSIXLY_CORRECT. That's what it's there for, and that's when I think the
warnings should be issued, as I said at the beginning.

But since Paul rejected that, ok, a different variable that lets us turn
them off (GREPWARNINGS=efgrepok or whatever) would at least provide some
palliation. I don't understand why you two are opposed to this simple
remediation.

    As Gary mentioned above, it's easy to disable them.

Obviously it is trivial to edit the scripts or have a different version
in PATH for my own machine(s).  But those are no substitute for having a
supported way to use the distributed [ef]grep without warnings.

    I would argue that it is even more important to retain these
    stray-backslash warnings, because they tend to highlight real bugs.

"tend" being the key word there. But anyway, I see your point, and won't
argue that one further, since the efgrep warnings are what's causing me
the agony. -k

Message #53 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Guillem Jover <guillem <at> debian.org>
To: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 9 Sep 2022 14:16:10 +0200

Hi!

Just to note since grep 3.8 got uploaded into Debian, this has been a
problematic change. There are now packages that fail to work such as
apt-file (https://bugs.debian.org/1019329), and the emission of tons
of warnings all over the place, in many cases hard to know where these
are coming from, if the invoking command is nested deep.

Transitioning away from fgrep/egrep seems like it would be painful as
that involves lots of upstream projects:

  https://codesearch.debian.net/search?q=fgrep&literal=1
  https://codesearch.debian.net/search?q=egrep&literal=1

(And https://codesearch.debian.net/search?q=rgrep&literal=1)

In Debian the grep package has been blocked from migrating from
unstable to testing, and this change might get reverted:

  https://bugs.debian.org/1019335

There's a check by shellcheck, but unfortunately not everyone runs that
over shell code.

(Personally I'm happy to adapt code I maintain for this kind of thing,
but from reading older discussions upstream I get the feeling the
theoretical reach of these tools usage was perhaps not calibrated against
actual usage?)

Thanks,
Guillem

Message #56 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Guillem Jover <guillem <at> debian.org>
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 9 Sep 2022 11:41:49 -0500

On 9/9/22 07:16, Guillem Jover wrote:
> There are now packages that fail to work such as
> apt-file (https://bugs.debian.org/1019329),

From what I can see, that bug report doesn't say that apt-file fails to 
work, only that apt-file issues a warning and then goes on to work.

> Transitioning away from fgrep/egrep seems like it would be painful as
> that involves lots of upstream projects:

I glanced at those, and didn't see any projects that will stop working, 
only projects that will see annoying warnings. Admittedly I didn't look 
at all the examples, but in the first page of 
<https://codesearch.debian.net/search?q=fgrep&literal=1> (your first 
citation) all the code examples should continue to work.

Could you give examples of programs that actually stop working? That 
would help us consider remedies.

Message #59 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Karl Berry <karl <at> freefriends.org>
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Sun, 11 Sep 2022 13:41:22 -0700

On Thu, Sep 8, 2022 at 4:01 PM Karl Berry <karl <at> freefriends.org> wrote:
> Hi Jim,
>
>     Some must care about portability,
>
> Certainly agreed. Even I do, sometimes :). But that does not mean
> everyone needs to, in every situation.  As I said, I fail to understand
> the benefit of making the warning unconditional.
>
> So far as I can see, it's also against GNU principles, as I wrote,
> though evidently you don't agree.
>
>     and these warnings help them do a better job.
>
> When people want extreme POSIX compliance, they should set
> POSIXLY_CORRECT. That's what it's there for, and that's when I think the
> warnings should be issued, as I said at the beginning.
>
> But since Paul rejected that, ok, a different variable that lets us turn
> them off (GREPWARNINGS=efgrepok or whatever) would at least provide some
> palliation. I don't understand why you two are opposed to this simple
> remediation.
>
>     As Gary mentioned above, it's easy to disable them.
>
> Obviously it is trivial to edit the scripts or have a different version
> in PATH for my own machine(s).  But those are no substitute for having a
> supported way to use the distributed [ef]grep without warnings.
>
>     I would argue that it is even more important to retain these
>     stray-backslash warnings, because they tend to highlight real bugs.
>
> "tend" being the key word there. But anyway, I see your point, and won't
> argue that one further, since the efgrep warnings are what's causing me
> the agony. -k

Hi Karl,

It would help if you could point to some malfunction.

Consider the alternative.

Should we release a new version of grep that provides a documented way
(say a configure-time option) to disable a warning about a
long-deprecated feature so you don't have to manually tweak the
four-line fgrep and egrep scripts? AFAIK, these new warnings cause no
malfunction.

Wouldn't it be better to fix the roots of the problem rather than
piling another kludge on top to disable the annoying warnings? Think
about the next steps: when more and more distros cease to distribute
the egrep and fgrep crutches, what will people do? Eventually, we'll
all break the habit, at least in scripts. If you want to use it in
personal scripts or on the command line, create your wrapper script or
alias/function.

Message #62 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Sam <sam <at> cmpct.info>
To: Jim Meyering <jim <at> meyering.net>
Cc: 57604 <at> debbugs.gnu.org, Karl Berry <karl <at> freefriends.org>
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Sun, 11 Sep 2022 21:45:47 +0100

[Message part 1 (text/plain, inline)]

> On 11 Sep 2022, at 21:41, Jim Meyering <jim <at> meyering.net> wrote:
> 
> On Thu, Sep 8, 2022 at 4:01 PM Karl Berry <karl <at> freefriends.org> wrote:
>> Hi Jim,
>> 
>>    Some must care about portability,
>> 
>> Certainly agreed. Even I do, sometimes :). But that does not mean
>> everyone needs to, in every situation.  As I said, I fail to understand
>> the benefit of making the warning unconditional.
>> 
>> So far as I can see, it's also against GNU principles, as I wrote,
>> though evidently you don't agree.
>> 
>>    and these warnings help them do a better job.
>> 
>> When people want extreme POSIX compliance, they should set
>> POSIXLY_CORRECT. That's what it's there for, and that's when I think the
>> warnings should be issued, as I said at the beginning.
>> 
>> But since Paul rejected that, ok, a different variable that lets us turn
>> them off (GREPWARNINGS=efgrepok or whatever) would at least provide some
>> palliation. I don't understand why you two are opposed to this simple
>> remediation.
>> 
>>    As Gary mentioned above, it's easy to disable them.
>> 
>> Obviously it is trivial to edit the scripts or have a different version
>> in PATH for my own machine(s).  But those are no substitute for having a
>> supported way to use the distributed [ef]grep without warnings.
>> 
>>    I would argue that it is even more important to retain these
>>    stray-backslash warnings, because they tend to highlight real bugs.
>> 
>> "tend" being the key word there. But anyway, I see your point, and won't
>> argue that one further, since the efgrep warnings are what's causing me
>> the agony. -k
> 
> Hi Karl,
> 
> It would help if you could point to some malfunction.

We've hit one malfunction in Gentoo: https://bugs.gentoo.org/868384.

A program was using libgcrypt-config via CMake and ended up
failing because of the warnings.

(The program's usage is IMO ill-advised and it should use pkg-config,
but that's beside the point).

> 
> Consider the alternative.
> 
> Should we release a new version of grep that provides a documented way
> (say a configure-time option) to disable a warning about a
> long-deprecated feature so you don't have to manually tweak the
> four-line fgrep and egrep scripts? AFAIK, these new warnings cause no
> malfunction.
> 
> Wouldn't it be better to fix the roots of the problem rather than
> piling another kludge on top to disable the annoying warnings? Think
> about the next steps: when more and more distros cease to distribute
> the egrep and fgrep crutches, what will people do? Eventually, we'll
> all break the habit, at least in scripts. If you want to use it in
> personal scripts or on the command line, create your wrapper script or
> alias/function.
> 

I honestly think at this point, it'd be better to just deem them GNU
extensions.

Best,
sam

[signature.asc (application/pgp-signature, attachment)]

Message #65 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruce Dubbs <bruce.dubbs <at> gmail.com>
To: bug-grep <at> gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Sun, 11 Sep 2022 15:58:33 -0500

On 9/11/22 14:45, Sam wrote:
> 
> 
>> On 11 Sep 2022, at 21:41, Jim Meyering <jim <at> meyering.net> wrote:
>>
>> On Thu, Sep 8, 2022 at 4:01 PM Karl Berry <karl <at> freefriends.org> wrote:
>>> Hi Jim,
>>>
>>>     Some must care about portability,
>>>
>>> Certainly agreed. Even I do, sometimes :). But that does not mean
>>> everyone needs to, in every situation.  As I said, I fail to understand
>>> the benefit of making the warning unconditional.
>>>
>>> So far as I can see, it's also against GNU principles, as I wrote,
>>> though evidently you don't agree.
>>>
>>>     and these warnings help them do a better job.
>>>
>>> When people want extreme POSIX compliance, they should set
>>> POSIXLY_CORRECT. That's what it's there for, and that's when I think the
>>> warnings should be issued, as I said at the beginning.
>>>
>>> But since Paul rejected that, ok, a different variable that lets us turn
>>> them off (GREPWARNINGS=efgrepok or whatever) would at least provide some
>>> palliation. I don't understand why you two are opposed to this simple
>>> remediation.
>>>
>>>     As Gary mentioned above, it's easy to disable them.
>>>
>>> Obviously it is trivial to edit the scripts or have a different version
>>> in PATH for my own machine(s).  But those are no substitute for having a
>>> supported way to use the distributed [ef]grep without warnings.
>>>
>>>     I would argue that it is even more important to retain these
>>>     stray-backslash warnings, because they tend to highlight real bugs.
>>>
>>> "tend" being the key word there. But anyway, I see your point, and won't
>>> argue that one further, since the efgrep warnings are what's causing me
>>> the agony. -k
>>
>> Hi Karl,
>>
>> It would help if you could point to some malfunction.
> 
> We've hit one malfunction in Gentoo: https://bugs.gentoo.org/868384.
> 
> A program was using libgcrypt-config via CMake and ended up
> failing because of the warnings.
> 
> (The program's usage is IMO ill-advised and it should use pkg-config,
> but that's beside the point).
> 
>>
>> Consider the alternative.
>>
>> Should we release a new version of grep that provides a documented way
>> (say a configure-time option) to disable a warning about a
>> long-deprecated feature so you don't have to manually tweak the
>> four-line fgrep and egrep scripts? AFAIK, these new warnings cause no
>> malfunction.
>>
>> Wouldn't it be better to fix the roots of the problem rather than
>> piling another kludge on top to disable the annoying warnings? Think
>> about the next steps: when more and more distros cease to distribute
>> the egrep and fgrep crutches, what will people do? Eventually, we'll
>> all break the habit, at least in scripts. If you want to use it in
>> personal scripts or on the command line, create your wrapper script or
>> alias/function.
>>
> 
> I honestly think at this point, it'd be better to just deem them GNU
> extensions.

After discussing this a bit in the LFS community, we tend to agree.  For us in the 
'from source' community, editing two short scripts is pretty trivial, but it really
shouldn't be needed.

  -- Bruce

Message #68 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: jim <at> meyering.net
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Sun, 11 Sep 2022 16:56:36 -0600

    AFAIK, these new warnings cause no malfunction.

Only if you've already changed your entire world not to use [ef]grep.
Then of course the warnings have no effect :).

Otherwise, they certainly do. I have scripts that run [ef]hundreds of
times (in loops, on lots of files).  grep is fundamental to shell script
programming, after all. Thus unconditional-on-every-invocation the
warning renders the script unusable.

In other cases, outputting to stderr changes the behavior of the script,
as Sam wrote in msg #62.

See also msg #53 in this bug report:
  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=57604#53
for effects far worse than anything on my own little systems.

In general, when a program has behaved in a given way for many years,
and you change the behavior, it just seems fundamental to me to provide
a way for users to get the previous behavior. GNU has been known for
backward compatibility and user-friendliness.

    Wouldn't it be better to fix the roots of the problem 

Well, we differ about "the root of the problem". To me, the original
deprecation was the mistake. I would have argued against it if I'd known
about it, but I missed it. Although I doubt you would have been swayed
by anything I could say, given this thread.

But never mind that, you're hardly likely to change your mind now. Thus
all I'm asking for is a way for [ef]grep to keep working
(forever). There is no harm in the programs. A configure-time option
would be fine.

    personal scripts or on the command line, create your wrapper script
    or alias/function.

As I said before, obviously these workarounds are trivial to do on a
person-by-person basis. Less so for whole systems.

But the general point is, why must you force work on the rest of the
world? In the real world (as opposed to the POSIX_ME_HARDER world that
you apparently want everyone to live in), the warning decreases
portability: scripts that worked before no longer work. There is nothing
forcing you to give that warning unconditionally.  Why is this in the
spirit of GNU? As I wrote from the coding standards ...

If distros choose to stupidly remove [ef]grep, that's their (and their
users') problem. There's nothing that says you need to contribute to it,
and plenty that says you shouldn't. -k

Message #71 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: arnold <at> skeeve.com
To: sam <at> cmpct.info, jim <at> meyering.net
Cc: 57604 <at> debbugs.gnu.org, karl <at> freefriends.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Tue, 13 Sep 2022 12:59:44 -0600

Sam <sam <at> cmpct.info> wrote:

> I honestly think at this point, it'd be better to just deem them GNU
> extensions.

I agree with this, wholeheartedly. FWIW.

Arnold

Message #74 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Santiago Ruano Rincón <santiagorr <at> riseup.net>
To: 57604 <at> debbugs.gnu.org
Cc: sam <at> cmpct.info, arnold <at> skeeve.com, jim <at> meyering.net, karl <at> freefriends.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Thu, 15 Sep 2022 15:57:22 +0200

[Message part 1 (text/plain, inline)]

El 13/09/22 a las 12:59, arnold <at> skeeve.com escribió:
> Sam <sam <at> cmpct.info> wrote:
> 
> > I honestly think at this point, it'd be better to just deem them GNU
> > extensions.
> 
> I agree with this, wholeheartedly. FWIW.
> 
> Arnold

FWIW, {e,f}grep have been reintroduced in Debian with grep 3.8-2.

Now, the stray \ warning is causing trouble. libtool has a problematic
grep \-L:
https://sources.debian.org/src/dpkg/1.21.9/m4/libtool.m4/#L6472
which is embedded in an important number of packages:
https://codesearch.debian.net/search?q=Configured+with%3A%22+%7C+%24GREP+%22+%5C-L&perpkg=1

Jochen Sprickerhof has already reported the problem in libtool,
including a patch:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1019725
https://lists.gnu.org/archive/html/libtool-patches/2022-09/msg00001.html
but it will take a lot of time before all of the concerned packages get
a fixed libtool.

I am willing to temporarily silent those warnings, once the affected
packages have been identified, so we can submit bugs and fix them.
See: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1019724

Cheers,

 -- Santiago

[signature.asc (application/pgp-signature, inline)]

Message #77 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Guillem Jover <guillem <at> debian.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 16 Sep 2022 03:05:05 +0200

On Fri, 2022-09-09 at 11:41:49 -0500, Paul Eggert wrote:
> On 9/9/22 07:16, Guillem Jover wrote:
> > There are now packages that fail to work such as
> > apt-file (https://bugs.debian.org/1019329),
> 
> From what I can see, that bug report doesn't say that apt-file fails to
> work, only that apt-file issues a warning and then goes on to work.

Ah, you are right, that might have coincided with a query I did that
returned nothing then, sorry about that!

> > Transitioning away from fgrep/egrep seems like it would be painful as
> > that involves lots of upstream projects:
> 
> I glanced at those, and didn't see any projects that will stop working, only
> projects that will see annoying warnings. Admittedly I didn't look at all
> the examples, but in the first page of
> <https://codesearch.debian.net/search?q=fgrep&literal=1> (your first
> citation) all the code examples should continue to work.
> 
> Could you give examples of programs that actually stop working? That would
> help us consider remedies.

It's true that most of those instances are probably not going to fail.
But what is definitely being affected are autopkgtests from Debian
packages for example. By default those consider any output to stderr
a signal to mark the test as failing. So the new grep failures are
causing unrelated tests to fail now.

Some are going to be hard to fix locally, or quickly everywhere, for
example the one in libtool, as until it is fixed, relibtoolizing will
have not effect, and afterwards that will get fixed only as long as
the packaging always forcibly relibtoolizes (or autoreconfigures).
<https://bugs.debian.org/1019725>


As I've mentioned earlier, personally, I definitely want to be able to
see those kinds of warnings so that I can fix or change stuff I
maintain, or report bugs with patches. But unfortunately it seems this
is causing enough disruption that all the new warnings might end up
being disabled in Debian. I think it's been discussed earlier that
environment variables are not desired? But I think it would still be nice
to be able to control those warnings globally/externally, so that even
if say a project like Debian ends up disabling them, people can still
enable them to be able to diagnose and track those down.

Thanks,
Guillem

Message #80 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Guillem Jover <guillem <at> debian.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 16 Sep 2022 01:30:16 -0700

On Thu, Sep 15, 2022 at 9:28 PM Guillem Jover <guillem <at> debian.org> wrote:
> On Fri, 2022-09-09 at 11:41:49 -0500, Paul Eggert wrote:
> > On 9/9/22 07:16, Guillem Jover wrote:
> > > There are now packages that fail to work such as
> > > apt-file (https://bugs.debian.org/1019329),
> >
> > From what I can see, that bug report doesn't say that apt-file fails to
> > work, only that apt-file issues a warning and then goes on to work.
>
> Ah, you are right, that might have coincided with a query I did that
> returned nothing then, sorry about that!
>
> > > Transitioning away from fgrep/egrep seems like it would be painful as
> > > that involves lots of upstream projects:
> >
> > I glanced at those, and didn't see any projects that will stop working, only
> > projects that will see annoying warnings. Admittedly I didn't look at all
> > the examples, but in the first page of
> > <https://codesearch.debian.net/search?q=fgrep&literal=1> (your first
> > citation) all the code examples should continue to work.
> >
> > Could you give examples of programs that actually stop working? That would
> > help us consider remedies.
>
> It's true that most of those instances are probably not going to fail.
> But what is definitely being affected are autopkgtests from Debian
> packages for example. By default those consider any output to stderr
> a signal to mark the test as failing. So the new grep failures are
> causing unrelated tests to fail now.
>
> Some are going to be hard to fix locally, or quickly everywhere, for
> example the one in libtool, as until it is fixed, relibtoolizing will
> have not effect, and afterwards that will get fixed only as long as
> the packaging always forcibly relibtoolizes (or autoreconfigures).
> <https://bugs.debian.org/1019725>
>
>
> As I've mentioned earlier, personally, I definitely want to be able to
> see those kinds of warnings so that I can fix or change stuff I
> maintain, or report bugs with patches. But unfortunately it seems this
> is causing enough disruption that all the new warnings might end up
> being disabled in Debian. I think it's been discussed earlier that
> environment variables are not desired? But I think it would still be nice
> to be able to control those warnings globally/externally, so that even
> if say a project like Debian ends up disabling them, people can still
> enable them to be able to diagnose and track those down.

Thanks for the feedback.

We may make a new release with two additions:
- an envvar to control [ef]grep warnings, enabled by default
- a configure-time option to make it disabled by default

This would be an envvar for which we do not commit to any level of
support in future releases.

Message #83 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Guillem Jover <guillem <at> debian.org>
To: Jim Meyering <jim <at> meyering.net>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 16 Sep 2022 14:15:33 +0200

On Fri, 2022-09-16 at 01:30:16 -0700, Jim Meyering wrote:
> On Thu, Sep 15, 2022 at 9:28 PM Guillem Jover <guillem <at> debian.org> wrote:
> > It's true that most of those instances are probably not going to fail.
> > But what is definitely being affected are autopkgtests from Debian
> > packages for example. By default those consider any output to stderr
> > a signal to mark the test as failing. So the new grep failures are
> > causing unrelated tests to fail now.
> >
> > Some are going to be hard to fix locally, or quickly everywhere, for
> > example the one in libtool, as until it is fixed, relibtoolizing will
> > have not effect, and afterwards that will get fixed only as long as
> > the packaging always forcibly relibtoolizes (or autoreconfigures).
> > <https://bugs.debian.org/1019725>
> >
> >
> > As I've mentioned earlier, personally, I definitely want to be able to
> > see those kinds of warnings so that I can fix or change stuff I
> > maintain, or report bugs with patches. But unfortunately it seems this
> > is causing enough disruption that all the new warnings might end up
> > being disabled in Debian. I think it's been discussed earlier that
> > environment variables are not desired? But I think it would still be nice
> > to be able to control those warnings globally/externally, so that even
> > if say a project like Debian ends up disabling them, people can still
> > enable them to be able to diagnose and track those down.
> 
> Thanks for the feedback.
> 
> We may make a new release with two additions:
> - an envvar to control [ef]grep warnings, enabled by default
> - a configure-time option to make it disabled by default

Sorry if I was not more explicit, while the [ef]grep warnings seem to
be causing disruption, the warning affecting libtool is the one about
"stray \ before ". I'm not sure how the other new warnings (AFAIR there
are more?) are affecting things though, but it might be worth handling
all the newly introduced warnings in the same way?

(Debian's grep bug discussing this is <https://bugs.debian.org/1019724>.)

> This would be an envvar for which we do not commit to any level of
> support in future releases.

I think that would be perfectly fine as a transitional measure, yes.
Thanks for considering!

Regards,
Guillem

Message #86 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Simon Josefsson <simon <at> josefsson.org>
To: Jim Meyering <jim <at> meyering.net>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, Guillem Jover <guillem <at> debian.org>,
 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 16 Sep 2022 17:11:35 +0200

[Message part 1 (text/plain, inline)]

Jim Meyering <jim <at> meyering.net> writes:

> This would be an envvar for which we do not commit to any level of
> support in future releases.

Would the envvar be documented?  Would it be a deprecated feature, with
a removal plan?  It seems we traded removing [ef]grep into introducing
new unsupported features which feels a bit unsatisfying...

How about saying that the envvar, together with all remaining traces of
[ef]grep references will be removed in 2025?

The point with the excercise was (at least to me) to remove complexity,
but it seems we will have to wait some more until that can happen.

/Simon

[signature.asc (application/pgp-signature, inline)]

Message #89 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Simon Josefsson <simon <at> josefsson.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, Guillem Jover <guillem <at> debian.org>,
 57604 <at> debbugs.gnu.org
Subject: Re: bug#57604: [ef]grep usage -> POSIXLY_CORRECT?
Date: Fri, 16 Sep 2022 10:12:44 -0700

On Fri, Sep 16, 2022 at 8:12 AM Simon Josefsson <simon <at> josefsson.org> wrote:
> Jim Meyering <jim <at> meyering.net> writes:
>
> > This would be an envvar for which we do not commit to any level of
> > support in future releases.
>
> Would the envvar be documented?  Would it be a deprecated feature, with
> a removal plan?  It seems we traded removing [ef]grep into introducing
> new unsupported features which feels a bit unsatisfying...
>
> How about saying that the envvar, together with all remaining traces of
> [ef]grep references will be removed in 2025?
>
> The point with the excercise was (at least to me) to remove complexity,
> but it seems we will have to wait some more until that can happen.

Hi Simon,
I agree that adding temporary complexity (and then
documenting/announcing it) to aid in the transition feels wrong.
I'm still not sure, but if we were to add such things, they would most
definitely come with a planned removal date, likely before 2025.

Message #92 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Santiago Ruano Rincón <santiagorr <at> riseup.net>
To: 1019724 <at> bugs.debian.org, 57604 <at> debbugs.gnu.org
Cc: Lucas Nussbaum <lucas <at> debian.org>, Jochen Sprickerhof <jspricke <at> debian.org>,
 debian-release <at> lists.debian.org, Paul Gevers <elbrus <at> debian.org>
Subject: Re: Bug#1019724: warning: stray \ before - causes autopkgtest failure
Date: Mon, 19 Sep 2022 14:32:11 +0200

[Message part 1 (text/plain, inline)]

El 18/09/22 a las 07:57, Lucas Nussbaum escribió:
> Hi,
> 
> On 16/09/22 at 21:35 +0200, Paul Gevers wrote:
> > Hi Santiago,
> > 
> > On 15-09-2022 09:26, Paul Gevers wrote:
> > > I am trying to schedule autopkgtests in unstable on amd64 for all source
> > > packages that have one.
> > 
> > All results are now in. Only several test failed due this warning: the known
> > dpkg and cjet and the newly found nagios-plugins-contrib, logcheck and
> > xautolock (see links below).
> > 
> > > Lucas, are you in the position to do an archive rebuild to check for the
> > > grep warnings (see full history at [1]). When you submit bugs, can you
> > > please file them as important, as apparently upstream wants to turn
> > > these warnings into failures in the future, but in Debian Santiago is
> > > going to silence the warning soon, so FTBFS due to this isn't an RC
> > > problem on the short term.
> > > 
> > > Santiago, please keep the grep with warning in unstable until either all
> > > tests are finished or we abort this plan.
> > 
> > To be honest, without bug 1019326 in ucf and bug 1019725 in libtool fixed,
> > I'm not sure how much value an archive rebuild would be at this stage.
> > 
> > So I think it would be good to silence the warning now in Debian. As
> > discussed before, ideally with a way to turn it on (or make it error) such
> > that we can do archive wide QA before the behavior is changed to error out
> > upstream. I can understand it if you want to wait with deploying that
> > optional behavior until upstream has implemented it.

Dear grep developers,

Just to say that I am also in favor to *temporarily* silence the stray-\
warnings. In the Debian context, the most problematic are the failing
autopkgtests, since they could block the transition of a package from
unstable to testing. Even if there are a few packages, as listed above
(including dpkg), that currently have a failing test due to the stray-\
warning, it would be great to avoid new autopkgtests fails due to that.

Also, as you can read below, there are 4235 packages including the
warning in their build logs. Funnily, grep is also in the list :-)

> I did an archive rebuild (not specifically for this). Grepping the logs,
> the following 4235 packages exhibit the warning.
> 
[snip]
> gregwar-captcha grep gretl gri grisbi groff gromacs groonga grub2 grub
[snip]

You can find the warnings in e.g. the last amd64 3.8-2 build:
https://buildd.debian.org/status/fetch.php?pkg=grep&arch=amd64&ver=3.8-2&stamp=1662998047&raw=0

(Thanks Lucas for running the archive rebuild!)

Hopefully, these are "just warnings" for now. I do agree the erroneous
patterns should be fixed, but I would prefer if we could have a smoother
transition before they result in grep errors.
As proposed by Paul Gevers, could you please consider silencing the
stray-\ warning for now, and adding a mechanism (envvar or option, as
you prefer) to enable them?

Cheers,

 -- S

[signature.asc (application/pgp-signature, inline)]

Message #95 received at 57604 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Santiago Ruano Rincón <santiagorr <at> riseup.net>
Cc: Lucas Nussbaum <lucas <at> debian.org>, Jochen Sprickerhof <jspricke <at> debian.org>,
 57604 <at> debbugs.gnu.org, Paul Gevers <elbrus <at> debian.org>,
 1019724 <at> bugs.debian.org, debian-release <at> lists.debian.org
Subject: Re: bug#57604: Bug#1019724: warning: stray \ before - causes
 autopkgtest failure
Date: Mon, 19 Sep 2022 13:43:22 -0700

On 9/19/22 05:32, Santiago Ruano Rincón wrote:

> as you can read below, there are 4235 packages including the
> warning in their build logs. Funnily, grep is also in the list :-)

Grep is on the list because Debian indirectly requires ucf to build 
Grep, and ucf issues the warning about stray \ because ucf mistakenly 
uses a Perlism in a grep regular expression 
<https://bugs.debian.org/1019326>. This particular warning doesn't break 
anything; it merely alerts installers of a screwup that happens to work 
but relies on undefined results.

We're thinking about adding a configure-time option to Grep to disable 
warnings about egrep/fgrep, to address the original Grep bug report 
<https://bugs.gnu.org/57604>. I'm not so sure about disabling warnings 
about bad escapes, as these warnings are so often a win and so rarely a 
loss, as is the case with ucf. Of course there is a tradeoff here 
between (a) having to wade through a bunch of annoying warnings, and (b) 
fixing packages so that they don't rely on undefined results.

Since the main issue here seems to be libtool-related test failures, how 
about patching libtool and letting the affected packages use the patched 
libtool? You can find a patch here:

https://savannah.gnu.org/patch/index.php?10282
https://savannah.gnu.org/patch/download.php?file_id=53720

The libtool test failures are false alarms, so another option would be 
to ignore the failures until libtool gets fixed.


For more on this thorny topic, please see:

https://www.gnu.org/software/grep/manual/html_node/Problematic-Expressions.html

The stray \ issue is the 19th bullet.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.