Strange EOL behavior of mintty with MinGW

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Strange EOL behavior of mintty with MinGW

Albrecht Schlosser-3
Hi,

I'm posting this here since I assume that Andy is reading here
and this seems to be an interaction of mintty with MinGW.

I can reproduce the strange behavior/bug with this simple
command line:

for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done

What I see when using mintty with MinGW is something like
this (elided *correct* lines and shortened):

$ for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done
001 67890123456789012345678901
002 67890123456789012345678901
...
015 67890123456789012345678901
016 67890123456789012345678901

17 67890123456789012345678901
018 67890123456789012345678901
019 67890123456789012345678901
...
047 67890123456789012345678901
048 67890123456789012345678901

49 67890123456789012345678901
050 67890123456789012345678901
051 67890123456789012345678901
...
079 67890123456789012345678901
080 67890123456789012345678901

and so on.

See the additional line break and missing '0' in lines 17 and 49.
It looks as if the first character of a line was stripped/replaced
by a line break (\n).

Note that each line is 31 characters long (32 with \n), and that the
first error happens after 16 lines (total of 512 bytes), but then only
after every 32 lines (1024 bytes).

In contrast, adding or removing one character makes it work okay,
e.g.:

for n in `seq -w 1 115`;do echo "$n 6789012345678901234567890";done

This is all with a (hopefully) current MinGW installation and with
mintty 1.0.1, but I could see it also with mintty 0.9.6 before I
upgraded mintty.

OTOH, this seems to work correct with mintty (0.9.8) on Cygwin.

I also tested it in a standard "DOS" shell and with console2, and
both don't show the odd behavior. If I redirect the output to a
file, all is well (the file is okay), but if I 'cat' the file, I
can see the same problems.

Is this a bug in mintty/MinGW, or do I have a bad MinGW environment?

--
Regards,
Albrecht


------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model
configuration take the hassle out of deploying and managing Subversion and
the tools developers use with it. Learn more about uberSVN and get a free
download at:  http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Strange EOL behavior of mintty with MinGW

Earnie Boyd
Albrecht Schlosser wrote:
>
> Is this a bug in mintty/MinGW, or do I have a bad MinGW environment?
>

Appears to me to be a one off buffering issue either in mintty or the
MSYS DLL.  But that is just a guess; by no means would it be a "bad
MinGW environment" since you get correct results without the tty emulation.

--
Earnie
-- http://www.for-my-kids.com

------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model
configuration take the hassle out of deploying and managing Subversion and
the tools developers use with it. Learn more about uberSVN and get a free
download at:  http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Strange EOL behavior of mintty with MinGW

Andy Koppe
In reply to this post by Albrecht Schlosser-3
On 10 August 2011 12:52, Albrecht Schlosser wrote:

> I can reproduce the strange behavior/bug with this simple
> command line:
>
> for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done
>
> What I see when using mintty with MinGW is something like
> this (elided *correct* lines and shortened):
>
> $ for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done
> 001 67890123456789012345678901
> 002 67890123456789012345678901
> ...
> 015 67890123456789012345678901
> 016 67890123456789012345678901
>
> 17 67890123456789012345678901
> 018 67890123456789012345678901
> 019 67890123456789012345678901
> ...
> 047 67890123456789012345678901
> 048 67890123456789012345678901
>
> 49 67890123456789012345678901
> 050 67890123456789012345678901
> 051 67890123456789012345678901
> ...
> 079 67890123456789012345678901
> 080 67890123456789012345678901
>
> and so on.
>
> See the additional line break and missing '0' in lines 17 and 49.
> It looks as if the first character of a line was stripped/replaced
> by a line break (\n).
>
> Note that each line is 31 characters long (32 with \n), and that the
> first error happens after 16 lines (total of 512 bytes), but then only
> after every 32 lines (1024 bytes).
>
> In contrast, adding or removing one character makes it work okay,
> e.g.:
>
> for n in `seq -w 1 115`;do echo "$n 6789012345678901234567890";done
>
> This is all with a (hopefully) current MinGW installation and with
> mintty 1.0.1, but I could see it also with mintty 0.9.6 before I
> upgraded mintty.
>
> OTOH, this seems to work correct with mintty (0.9.8) on Cygwin.
>
> I also tested it in a standard "DOS" shell and with console2, and
> both don't show the odd behavior. If I redirect the output to a
> file, all is well (the file is okay), but if I 'cat' the file, I
> can see the same problems.
>
> Is this a bug in mintty/MinGW, or do I have a bad MinGW environment?

This happens in rxvt too, except only half as often. I think this is
because mintty passes a 512-byte buffer when reading from the pseudo
terminal device, whereas rxvt passes a 1024 byte buffer. Looking at
--log output from mintty confirms that the missing '0's were replaced
with newlines.

Since both mintty and rxvt are fine in Cygwin, I'm pointing the finger
of suspicion at the MSYS DLL's pseudo terminal device driver, in
particular at its implementation of the ONLCR termios option for
turning NLs into CR/NL pairs.

Andy

------------------------------------------------------------------------------
uberSVN's rich system and user administration capabilities and model
configuration take the hassle out of deploying and managing Subversion and
the tools developers use with it. Learn more about uberSVN and get a free
download at:  http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Strange EOL behavior of mintty with MinGW

Albrecht Schlosser-3
On 10.08.2011 20:34, Andy Koppe wrote:

> On 10 August 2011 12:52, Albrecht Schlosser wrote:
>> I can reproduce the strange behavior/bug with this simple
>> command line:
>>
>> for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done
>>
>> What I see when using mintty with MinGW is something like
>> this (elided *correct* lines and shortened):
>>
>> $ for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done
>> 001 67890123456789012345678901
>> 002 67890123456789012345678901
>> ...
>> 015 67890123456789012345678901
>> 016 67890123456789012345678901
>>
>> 17 67890123456789012345678901
>> 018 67890123456789012345678901
>> 019 67890123456789012345678901
>> ...
>> 047 67890123456789012345678901
>> 048 67890123456789012345678901
>>
>> 49 67890123456789012345678901
>> 050 67890123456789012345678901
>> 051 67890123456789012345678901
>> ...
>> 079 67890123456789012345678901
>> 080 67890123456789012345678901
>>
>> and so on.

...

> This happens in rxvt too, except only half as often. I think this is
> because mintty passes a 512-byte buffer when reading from the pseudo
> terminal device, whereas rxvt passes a 1024 byte buffer. Looking at
> --log output from mintty confirms that the missing '0's were replaced
> with newlines.
>
> Since both mintty and rxvt are fine in Cygwin, I'm pointing the finger
> of suspicion at the MSYS DLL's pseudo terminal device driver, in
> particular at its implementation of the ONLCR termios option for
> turning NLs into CR/NL pairs.

Thanks, Andy, for testing and this interesting information. Indeed,
switching off ONLCR shows no stripped/replaced characters anymore.

$ stty -onlcr
$ for n in `seq -w 1 115`;do echo "$n 67890123456789012345678901";done

shows everything okay, no missing characters, however with a weird
formatting due to missing <cr> characters.

So, here are the next questions:
  - is it time to file a bug report?
  - if yes, where ?

Or does anybody feel inclined to fix this w/o an explicit bug report?

BTW.: Of course this bug does not only happen when writing such fixed
size lines, but this was a simple reproducer. I saw the bug before
from time to time with all sorts of longer output, e.g. directories,
find, diffs, etc..

Thanks for reading so far...

--
Regards,
Albrecht


------------------------------------------------------------------------------
Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
user administration capabilities and model configuration. Take
the hassle out of deploying and managing Subversion and the
tools developers use with it.
http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Strange EOL behavior of mintty with MinGW

Earnie Boyd
Albrecht Schlosser wrote:
> So, here are the next questions:
>   - is it time to file a bug report?

Yes.

>   - if yes, where ?

http://www.mingw.org/reporting_bugs

--
Earnie
-- http://www.for-my-kids.com

------------------------------------------------------------------------------
Get a FREE DOWNLOAD! and learn more about uberSVN rich system,
user administration capabilities and model configuration. Take
the hassle out of deploying and managing Subversion and the
tools developers use with it.
http://p.sf.net/sfu/wandisco-dev2dev
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe
Reply | Threaded
Open this post in threaded view
|

Re: Strange EOL behavior of mintty with MinGW

zennehoy
In reply to this post by Albrecht Schlosser-3
Yes, I realize it's four years later, but as this bug still exists - was a
bug report for this ever filed?



--
View this message in context: http://mingw-users.1079350.n2.nabble.com/Strange-EOL-behavior-of-mintty-with-MinGW-tp6672070p7583670.html
Sent from the MinGW-users mailing list archive at Nabble.com.

------------------------------------------------------------------------------
_______________________________________________
MinGW-users mailing list
[hidden email]

This list observes the Etiquette found at
http://www.mingw.org/Mailing_Lists.
We ask that you be polite and do the same.  Disregard for the list etiquette may cause your account to be moderated.

_______________________________________________
You may change your MinGW Account Options or unsubscribe at:
https://lists.sourceforge.net/lists/listinfo/mingw-users
Also: mailto:[hidden email]?subject=unsubscribe