Quantcast

MSYS "rebase" package

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

MSYS "rebase" package

Keith Marshall-3
A couple of weeks ago, I hinted that I had some questions concerning
this -- more of a request for clarification, really -- but I never
quite got around to posing them.

Chuck, in
http://thread.gmane.org/gmane.comp.gnu.mingw.msys/4772/focus=4728

you said:
> BTW, just as cygwin-rebase can't rebase cygwin1.dll, my msys port of
> rebase can't rebase msys-1.0.dll.

I assume that remains true of the package you eventually released?

> Maybe we need a pure win32 version...

Johannes Schindelin responded to that, offering just such a version,
which he had provided for msys-git.  Given that Chris Sutcliffe's
original problem, (which I myself encountered, identically, when I
upgraded to the current msys-1.0.dll release), entails exactly that
requirement, (that msys-1.0.dll must be rebased), will your rebase
package suffice to address the issue?  (FWIW, msys-1.0.dll is the
ONLY shared object which I have needed to rebase to date).

Since msys-1.0.dll itself seems to be most consistently reported as
problematic, (and if you haven't already done so), could your rebase
package be constructed around Johannes' rebase.exe implementation, so
that it could deliver a solution to this problem?

Further, since in an earlier message in the same thread:
http://thread.gmane.org/gmane.comp.gnu.mingw.msys/4772/focus=4726

you had suggested the tentative solution:

  $ rebase -b 0x30000000 /path/to/msys-1.0.dll

and since PRECISELY that worked for me, (and also for Chris, I
believe), would that particular base address not, perhaps be more
universally acceptable as default, than the current choice?  (I did
note your reasoning for there being no universally CORRECT address).

--
Regards,
Keith.

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Earnie Boyd
Keith Marshall wrote:
>
>    $ rebase -b 0x30000000 /path/to/msys-1.0.dll
>

It might be worth Cesar dropping the base address to the user address
space by default.  I had put it in the system address space without any
issue at the time of the fork but if it is causing issue now, it
wouldn't hurt to move it.  If I remember correctly Cygwin bases the DLL
at 0x60000000 and I used 0x70000000.

Earnie

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
In reply to this post by Keith Marshall-3
On 5/26/2010 8:56 AM, Keith Marshall wrote:\
> you said:
>> BTW, just as cygwin-rebase can't rebase cygwin1.dll, my msys port of
>> rebase can't rebase msys-1.0.dll.
>
> I assume that remains true of the package you eventually released?

Correct. rebase.exe depends on msys.  Mostly, because in order to be
really useful, you need rebaseall, which is a shell script.  So, that
means you can't use rebaseall to rebase the shell interpreter (in this
case, dash) nor the runtime used by dash, regardless of whether
rebase.exe itself relies on that runtime.

>> Maybe we need a pure win32 version...
>
> Johannes Schindelin responded to that, offering just such a version,
> which he had provided for msys-git.

Johannes' version:
version=2.4.2-1
url=http://www.tishler.net/jason/software/rebase
+ a small patch to increase the space allotted for each DLL
That is, this is just an old version of Jason Tishler's original software.

current msys version:
rebase-3.0.1-1
<cygwin mirror>/release/rebase/

(which...just happens to be Jason Tisher's most recent version, anyway).

I guess what you're really asking is, can our rebase.exe (and
peflags.exe) be built as a native win32 app.  I think it can -- but see
below.

> Given that Chris Sutcliffe's
> original problem, (which I myself encountered, identically, when I
> upgraded to the current msys-1.0.dll release), entails exactly that
> requirement, (that msys-1.0.dll must be rebased), will your rebase
> package suffice to address the issue?  (FWIW, msys-1.0.dll is the
> ONLY shared object which I have needed to rebase to date).

Well, for some reason I had always thought that cygwin1.dll itself was
not "rebasable" -- that, even if you used some native tool to change its
ImageBaseAddress, it would die horribly for some esoteric reason
(hardcoded pointer values? dunno...).

So, I've never attempted to rebase either cygwin1.dll or msys-1.0.dll. I
did find one message on the cygwin list from last December, recommending
to use rebase(all) to rebase *a copy* of cygwin1.dll, and then use
native tools to move it into place.  This had something to do with
trying to outsmart a BLODA.

> Since msys-1.0.dll itself seems to be most consistently reported as
> problematic,

Errr...huh?  In my experience, it has *never* been the msys DLL that
couldn't be loaded in the same location in the child as in the parent --
mostly because it is the FIRST dependency to be loaded, so there's
nothing with which it could conflict.  It's more likely to be "Cwd.dll"
or "POSIX.dll" (some perl extension modules loaded almost last), or
sometimes msys-intl-8.dll -- or any of the guile srfi dlls.

> (and if you haven't already done so), could your rebase
> package be constructed around Johannes' rebase.exe implementation, so
> that it could deliver a solution to this problem?

I'm not sure it IS a problem [1] but there's no need to use "Johannes'"
version; I'm already using a more recent version of the same upstream
source.  The real question is, can it be built as a native app, and
SHOULD it be built as a native app?

Downside: building as a native tool means that it wouldn't grok posix
paths -- which is what rebaseall feeds it [2].

[1] As a thought experiment, I wonder if your earlier troubles -- which
you solved by rebasing just msys-1.0.dll -- could ALSO have been solved
by rebasing *everything else* except msys-1.0.dll? [3] Not that such an
approach would be desired, when it's simpler(?) to just rebase one DLL.
(simpler, that is, until you start trying to use the same rebase.exe for
both "fix msys-1.0.dll" and "fix everything else via rebaseall" duties:
see [2]).

[2] rebaseall feeds to rebase a list of DLLs, but not on the command
line.  The list of to-be-rebased DLLs is put in a file, and that file is
passed to rebase via its -T option.  So, since rebaseall is running
under msys-dash, the path to the file-containing-the-list will be
automagically converted to win32, but the *contents* of that file list
will not be.  I think this breaks rebaseall (and peflagsall).

> Further, since in an earlier message in the same thread:
> http://thread.gmane.org/gmane.comp.gnu.mingw.msys/4772/focus=4726
>
> you had suggested the tentative solution:
>
>   $ rebase -b 0x30000000 /path/to/msys-1.0.dll

[3] Ah, well, if you read carefully I was suggesting TWO alternate
approaches:

a) use the (not yet uploaded at that time) msys-rebaseall + msys-dash to
rebase everything (except msys-1.0.dll), or

b) use the cygwin (e.g. non-msys) rebase.exe and rebase JUST
msys-1.0.dll. I didn't really think this would work, and was kinda
surprised that it did.

Since you didn't have access to msys-rebaseall/msys-dash, you did (b)
and it worked. But nobody who was having the problem tested (a).

> and since PRECISELY that worked for me, (and also for Chris, I
> believe), would that particular base address not, perhaps be more
> universally acceptable as default, than the current choice?  (I did
> note your reasoning for there being no universally CORRECT address).

I dunno. Is there some accepted standard (<0x4000000 is .exe, >0x7000000
is system) or something?

Re: the rebase package, I suggest a compromise, based on the following
points:

1) rebaseall and peflagsall both explicitly set PATH=/bin, they will
always use the rebase.exe (peflags.exe) in msys' /bin directory -- even
if a native version exists in /mingw/bin.

2) I believe that neither rebaseall nor peflagsall will work
successfully with a native rebase.exe (peflags.exe).

3) An msys-rebase.exe (msys-peflags.exe) can't rebase (set flags on) the
msys-1.0.dll itself.

So, how about a separate mingw32-rebase package, based on the same
source code, but which provides only the executables and not the
scripts? I'll probably have to update the msys-rebase package's
documentation to mention "the other one", but that's not too hard.

--
Chuck






------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Greg Chicares-2
On 2010-05-26 23:32Z, Charles Wilson wrote:

> On 5/26/2010 8:56 AM, Keith Marshall wrote:\
>> you said:
>>> BTW, just as cygwin-rebase can't rebase cygwin1.dll, my msys port of
>>> rebase can't rebase msys-1.0.dll.
>>
>> I assume that remains true of the package you eventually released?
>
> Correct. rebase.exe depends on msys.  Mostly, because in order to be
> really useful, you need rebaseall, which is a shell script.  So, that
> means you can't use rebaseall to rebase the shell interpreter (in this
> case, dash) nor the runtime used by dash, regardless of whether
> rebase.exe itself relies on that runtime.

Would it help to use a native shell to run that script?
Here's a fellow who has resurrected an old native port of zsh:
  http://zsh-nt.sourceforge.net/

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
On 5/26/2010 8:14 PM, Greg Chicares wrote:
> Would it help to use a native shell to run that script?
> Here's a fellow who has resurrected an old native port of zsh:
>   http://zsh-nt.sourceforge.net/

Errm...doubtful.  The script also uses sed, grep, sort etc.  By the time
you actually invoke rebase, they have all exited, but they need to be
available to the script. I wonder if the stdout/stdin pipes with MSYS
programs, marshalled by 'zsh-nt', would work as desired...

--
Chuck




------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Keith Marshall-3
In reply to this post by Charles Wilson-8
On Thursday 27 May 2010 00:32:53 Charles Wilson wrote:
> I guess what you're really asking is, can our rebase.exe (and
> peflags.exe) be built as a native win32 app.

Yes.

> > Given that Chris Sutcliffe's
> > original problem, (which I myself encountered, identically, when
> > I upgraded to the current msys-1.0.dll release), entails exactly
> > that requirement, (that msys-1.0.dll must be rebased), will your
> > rebase package suffice to address the issue?  (FWIW, msys-1.0.dll
> > is the ONLY shared object which I have needed to rebase to date).
>
> Well, for some reason I had always thought that cygwin1.dll itself
> was not "rebasable" -- that, even if you used some native tool to
> change its ImageBaseAddress, it would die horribly for some
> esoteric reason (hardcoded pointer values? dunno...).

I don't know either; you likely know better than I.

> So, I've never attempted to rebase either cygwin1.dll or
> msys-1.0.dll.

Yet you did recommend doing just that, when Chris originally reported
the problem on the MinGW-MSYS list...

> I did find one message on the cygwin list from last
> December, recommending to use rebase(all) to rebase *a copy* of
> cygwin1.dll, and then use native tools to move it into place.  This
> had something to do with trying to outsmart a BLODA.
>
> > Since msys-1.0.dll itself seems to be most consistently reported
> > as problematic,
>
> Errr...huh?

Well, more accurately, it has been frequently reported, (not
necessarily on our lists), that an MSYS shell cannot be started AT
ALL; right from the outset:

  d:\path\to\msys\bin\sh.exe --login -i

dies with a "cannot reserve space for cygwin's heap" message.

> In my experience, it has *never* been the msys DLL
> that couldn't be loaded in the same location in the child as in the
> parent -- mostly because it is the FIRST dependency to be loaded,
> so there's nothing with which it could conflict.

Well, the message above is distinct from the diagnostic you describe,
as indicative of this failure mode; of course, that in no way implies
that the two do not have a common underlying cause.  However, at the
point where this error occurs, there would appear to be little more
than msys-1.0.dll and bash.exe involved.  (This was before you had
made msys-rebase available; I've no idea if it would have even been
possible to start dash.exe, rather than bash.exe, at this point).

> It's more likely
> to be "Cwd.dll" or "POSIX.dll" (some perl extension modules loaded
> almost last), or sometimes msys-intl-8.dll -- or any of the guile
> srfi dlls.

Hmm.  In my case, it's none of those; at the time when I encountered
this problem I had installed ONLY Console2, msysCORE, msys-coreutils
and msys-bash, so the only likely candidates, besides msys-1.0.dll
itself, would be those DLLs distributed with Console-2.00.144; there
were no other MSYS related DLLs even present, on any file system
attached to the box.  (I didn't try it, at the time, but it may be
instructive to repeat the exercise with Console2 removed from the
equation).

--
Regards,
Keith.

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Earnie Boyd
In reply to this post by Charles Wilson-8
Charles Wilson wrote:

>
>>> Maybe we need a pure win32 version...
>>

>
> I guess what you're really asking is, can our rebase.exe (and
> peflags.exe) be built as a native win32 app.  I think it can -- but see
> below.
>

Yea, I think that is what I've seen before, a rebase that is native MSVCRT.

>
> Errr...huh?  In my experience, it has *never* been the msys DLL that
> couldn't be loaded in the same location in the child as in the parent --
> mostly because it is the FIRST dependency to be loaded, so there's
> nothing with which it could conflict.  It's more likely to be "Cwd.dll"
> or "POSIX.dll" (some perl extension modules loaded almost last), or
> sometimes msys-intl-8.dll -- or any of the guile srfi dlls.
>

I've never had a problem either but their are those that have on the
user list that the rebase of the MSYS dll has helped.  I think it comes
down to which chip set brand is being used.

>> (and if you haven't already done so), could your rebase
>> package be constructed around Johannes' rebase.exe implementation, so
>> that it could deliver a solution to this problem?
>
> I'm not sure it IS a problem [1] but there's no need to use "Johannes'"
> version; I'm already using a more recent version of the same upstream
> source.  The real question is, can it be built as a native app, and
> SHOULD it be built as a native app?
>

And I think it can be a native app without issue.

> Downside: building as a native tool means that it wouldn't grok posix
> paths -- which is what rebaseall feeds it [2].
>

I'm not so sure that this would be an issue.

>
> [2] rebaseall feeds to rebase a list of DLLs, but not on the command
> line.  The list of to-be-rebased DLLs is put in a file, and that file is
> passed to rebase via its -T option.  So, since rebaseall is running
> under msys-dash, the path to the file-containing-the-list will be
> automagically converted to win32, but the *contents* of that file list
> will not be.  I think this breaks rebaseall (and peflagsall).
>

But rebaseall would read the list of dll from the file and spawn the
native rebase so the path would change from POSIX to WIN32 before rebase
sees the path.

>> Further, since in an earlier message in the same thread:
>> http://thread.gmane.org/gmane.comp.gnu.mingw.msys/4772/focus=4726
>>
>> you had suggested the tentative solution:
>>
>>    $ rebase -b 0x30000000 /path/to/msys-1.0.dll
>
> [3] Ah, well, if you read carefully I was suggesting TWO alternate
> approaches:
>
> a) use the (not yet uploaded at that time) msys-rebaseall + msys-dash to
> rebase everything (except msys-1.0.dll), or
>
> b) use the cygwin (e.g. non-msys) rebase.exe and rebase JUST
> msys-1.0.dll. I didn't really think this would work, and was kinda
> surprised that it did.
>
> Since you didn't have access to msys-rebaseall/msys-dash, you did (b)
> and it worked. But nobody who was having the problem tested (a).
>
>> and since PRECISELY that worked for me, (and also for Chris, I
>> believe), would that particular base address not, perhaps be more
>> universally acceptable as default, than the current choice?  (I did
>> note your reasoning for there being no universally CORRECT address).
>
> I dunno. Is there some accepted standard (<0x4000000 is .exe,>0x7000000
> is system) or something?
>

User space < 0x70000000 >= System space

> Re: the rebase package, I suggest a compromise, based on the following
> points:
>
> 1) rebaseall and peflagsall both explicitly set PATH=/bin, they will
> always use the rebase.exe (peflags.exe) in msys' /bin directory -- even
> if a native version exists in /mingw/bin.
>

I consider that a broken feature.  The assumption that the PATH should
be set to /bin is just wrong and should be fixed.

> 2) I believe that neither rebaseall nor peflagsall will work
> successfully with a native rebase.exe (peflags.exe).
>

And I disagree as noted above.

> 3) An msys-rebase.exe (msys-peflags.exe) can't rebase (set flags on) the
> msys-1.0.dll itself.
>
> So, how about a separate mingw32-rebase package, based on the same
> source code, but which provides only the executables and not the
> scripts? I'll probably have to update the msys-rebase package's
> documentation to mention "the other one", but that's not too hard.
>

And fix rebaseall and peflagsall so that it isn't setting the PATH or at
least provide a REBASE_PATH to be prepended to PATH.

Earnie

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
On 5/27/2010 7:49 AM, Earnie wrote:

> Charles Wilson wrote:
>> [2] rebaseall feeds to rebase a list of DLLs, but not on the command
>> line.  The list of to-be-rebased DLLs is put in a file, and that file is
>> passed to rebase via its -T option.  So, since rebaseall is running
>> under msys-dash, the path to the file-containing-the-list will be
>> automagically converted to win32, but the *contents* of that file list
>> will not be.  I think this breaks rebaseall (and peflagsall).
>>
>
> But rebaseall would read the list of dll from the file and spawn the
> native rebase so the path would change from POSIX to WIN32 before rebase
> sees the path.

No, rebase is called only once, and given the path to the file that
contains the list of DLLs.  So, the path to the listfile would get
converted, and then *rebase* reads the contents.  So, MSYS never gets
the chance to convert the paths listed inside the listfile.

The reason for the behavior on the part of rebase (e.g. called once to
act on a list of dll's, instead of being called multiple times to
operate singly on each dll) is so that rebase can do all the
book-keeping.  It determines the memory footprint of each DLL, and
increments (decrements) the base address to use for the *next* DLL by
that amount.  Otherwise, rebase would have to somehow indicate to the
script what the DLL's footprint was, and the script would have to track
that and use different -b 0xNNNNNNNN options on each call.

>> I dunno. Is there some accepted standard (<0x4000000 is .exe,>0x7000000
>> is system) or something?
>>
>
> User space < 0x70000000 >= System space

Thanks. See here: http://www.drdobbs.com/184416272

> Applications (.exe files) start at 0x00400000 for all compilers
> that I tested.
...

> The address range for an application that is not reserved by any
> version of Windows is from 0x00400000 to 0x80000000. The system DLLs
> for Windows are currently based in memory from 0x70000000 to
> 0x78000000 on the Intel processors and from 0x68000000 to 0x78000000
> on the MIPS processors. Other standard DLLs (for OLE support) are
> apparently in the range 0x50000000 to 0x5f000000. When selecting base
> addresses for DLLs, Microsoft suggests that you select them from the
> top of the allowed address range downwards, in order to avoid
> conflicts with memory allocated dynamically by the application (which
> is allocated from the bottom up).
...
> In conclusion, the most suitable address range for DLLs is from
> 0x60000000  through 0x6f000000. Microsoft, seeking portability where
> it cannot be achieved, proposes to reduce the range further to
> 0x60000000  through 0x68000000 in order to accommodate both Intel and
> MIPS processors. (Also note that Microsoft’s upper limit overlaps the
> reserved range of the MIPS processor.) Microsoft’s proposal continues
> with a “first letter” scheme for the selection of the base address,
> which I have summarized in Table 1.

(Unfortunatly, the link to "Table 1" is broken)

>> 1) rebaseall and peflagsall both explicitly set PATH=/bin, they will
>> always use the rebase.exe (peflags.exe) in msys' /bin directory -- even
>> if a native version exists in /mingw/bin.
>>
>
> I consider that a broken feature.  The assumption that the PATH should
> be set to /bin is just wrong and should be fixed.

So, you WANT rebaseall, a particular script provided by a particular
package, which expects specific behavior and options from an executable
that is provided by *that same package*, to use whatever random
executable that happens to have the same name which it might find in the
PATH?  Especially when that PATH is *not* the normal path most MSYS
users are familiar with after they launch a "normal" shell via msys.bat?

I'm sorry, Earnie, but that's...not wise. It's ASKING for trouble.

The rebase.exe provided by the Microsoft SDK doesn't accept the -T
option. rebaseall uses it. That's one reason why PATH is explicitly set
to /bin. Also, remember that the recommended sequence is:

launch a dash shell
  * dash doesn't read the bash startup files, so your PATH is
    probably not what you're used to, in "regular" bash. Plus,
    you haven't run msys.bat, so all the PATH shenanigans it performs
    don't happen either.
  * In my case -- and that of many cygwin users -- I put neither
    the cygwin bin/ nor the MinGW/MSYS bin/ in my overall Windows
    PATH.  I rely on the .startup-files to do that.

run rebaseall
  * which needs to "find" sed, grep, gawk -- and rebase.exe.  Since the
    PATH is in an unusual state, rebaseall explicitly sets it to /bin.
    I don't see much difference between setting PATH=/bin and explicitly
    invoking every tool as /bin/foo.

>> 2) I believe that neither rebaseall nor peflagsall will work
>> successfully with a native rebase.exe (peflags.exe).
>>
>
> And I disagree as noted above.

Because you misunderstand the situation.  rebase is NOT called in a
loop, nor are the DLL paths themselves listed on the command line -- so
MSYS doesn't get to fixup each individual path to the DLLs to be
rebased. The only path that appears in the command to rebase is the path
to the file containing the list -- so that's the only path that gets
fixed up; NOT the paths contained inside that listfile.

Now, both the Microsoft and Tishler versions of rebase CAN be called
with a list of DLLs passed explicitly on the command line.  In this
case, with a native version of rebase invoked within an msys-dash shell,
since each DLL pathname appears on the command line then MSYS would get
a chance to do its thing.

However, rebaseall doesn't do it that way, for two reasons: (1) command
line length limitations.  It's real easy to create a command line that's
too long, when you're listing every .dll in an entire installation. (2)
quoting issues. If the pathnames appeared on the command line, they'd
all need to be escaped/quoted to protect against shell special
characters and word splitting.

>> 3) An msys-rebase.exe (msys-peflags.exe) can't rebase (set flags on) the
>> msys-1.0.dll itself.
>>
>> So, how about a separate mingw32-rebase package, based on the same
>> source code, but which provides only the executables and not the
>> scripts? I'll probably have to update the msys-rebase package's
>> documentation to mention "the other one", but that's not too hard.
>>
>
> And fix rebaseall and peflagsall so that it isn't setting the PATH or at
> least provide a REBASE_PATH to be prepended to PATH.

Look at the code in rebaseall first, and then tell me if you still think
it'd be just great to use a native rebase with rebaseall. If so, then
I'll add REBASE_PATH support.

Oddly, however, nobody in six years on cygwin has ever found it
necessary to use *some other rebase* than the one *shipped with the
rebaseall* script.

--
Chuck


------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Earnie Boyd
Charles Wilson wrote:

> On 5/27/2010 7:49 AM, Earnie wrote:
>> Charles Wilson wrote:
>>> [2] rebaseall feeds to rebase a list of DLLs, but not on the command
>>> line.  The list of to-be-rebased DLLs is put in a file, and that file is
>>> passed to rebase via its -T option.  So, since rebaseall is running
>>> under msys-dash, the path to the file-containing-the-list will be
>>> automagically converted to win32, but the *contents* of that file list
>>> will not be.  I think this breaks rebaseall (and peflagsall).
>>>
>>
>> But rebaseall would read the list of dll from the file and spawn the
>> native rebase so the path would change from POSIX to WIN32 before rebase
>> sees the path.
>
> No, rebase is called only once, and given the path to the file that
> contains the list of DLLs.  So, the path to the listfile would get
> converted, and then *rebase* reads the contents.  So, MSYS never gets
> the chance to convert the paths listed inside the listfile.
>

Ah, yea, well in that scenario you are correct.

> The reason for the behavior on the part of rebase (e.g. called once to
> act on a list of dll's, instead of being called multiple times to
> operate singly on each dll) is so that rebase can do all the
> book-keeping.  It determines the memory footprint of each DLL, and
> increments (decrements) the base address to use for the *next* DLL by
> that amount.  Otherwise, rebase would have to somehow indicate to the
> script what the DLL's footprint was, and the script would have to track
> that and use different -b 0xNNNNNNNN options on each call.
>

Good to know.

>>> I dunno. Is there some accepted standard (<0x4000000 is .exe,>0x7000000
>>> is system) or something?
>>>
>>
>> User space<  0x70000000>= System space
>
> Thanks. See here: http://www.drdobbs.com/184416272
>

Ah, well the MIPS wasn't documented at the time I adjusted the DLL to
0x70000000.  So based on that information the formula is more.

if (CPU_SET == MIPS)
then
   User space < 0x68000000 >= System space
else
   User space < 0x70000000 >= System space
endif

>> In conclusion, the most suitable address range for DLLs is from
>> 0x60000000  through 0x6f000000. Microsoft, seeking portability where
>> it cannot be achieved, proposes to reduce the range further to
>> 0x60000000  through 0x68000000 in order to accommodate both Intel and
>> MIPS processors. (Also note that Microsoft’s upper limit overlaps the
>> reserved range of the MIPS processor.) Microsoft’s proposal continues
>> with a “first letter” scheme for the selection of the base address,
>> which I have summarized in Table 1.

Which is probably why Cygwin used 0x60000000.

>>> 1) rebaseall and peflagsall both explicitly set PATH=/bin, they will
>>> always use the rebase.exe (peflags.exe) in msys' /bin directory -- even
>>> if a native version exists in /mingw/bin.
>>>
>>
>> I consider that a broken feature.  The assumption that the PATH should
>> be set to /bin is just wrong and should be fixed.
>
> So, you WANT rebaseall, a particular script provided by a particular
> package, which expects specific behavior and options from an executable
> that is provided by *that same package*, to use whatever random
> executable that happens to have the same name which it might find in the
> PATH?  Especially when that PATH is *not* the normal path most MSYS
> users are familiar with after they launch a "normal" shell via msys.bat?
>

But if I didn't install to / the rebase executable wouldn't be found in
/bin anyway.

> I'm sorry, Earnie, but that's...not wise. It's ASKING for trouble.

In either scenario there are issues and assuming /bin isn't correct
either.  Perhaps a better scenario would be to set PATH to the directory
name of argv[0].

>
> The rebase.exe provided by the Microsoft SDK doesn't accept the -T
> option. rebaseall uses it. That's one reason why PATH is explicitly set
> to /bin. Also, remember that the recommended sequence is:
>

Ok, an understandable reason but I still think there are better ways to
accomplish the goal.

> launch a dash shell
>    * dash doesn't read the bash startup files, so your PATH is
>      probably not what you're used to, in "regular" bash. Plus,
>      you haven't run msys.bat, so all the PATH shenanigans it performs
>      don't happen either.

So it doesn't read /etc/profile or ~/.profile?  I knew there was some
reason I used bash instead of the default cygwin shell.

>    * In my case -- and that of many cygwin users -- I put neither
>      the cygwin bin/ nor the MinGW/MSYS bin/ in my overall Windows
>      PATH.  I rely on the .startup-files to do that.
>

Yea, that isn't recommended by me either.

> run rebaseall
>    * which needs to "find" sed, grep, gawk -- and rebase.exe.  Since the
>      PATH is in an unusual state, rebaseall explicitly sets it to /bin.
>      I don't see much difference between setting PATH=/bin and explicitly
>      invoking every tool as /bin/foo.

If the tool is installed at / yes, but if the tool is installed in
/usr/local it will be an issue.

>
>>> 2) I believe that neither rebaseall nor peflagsall will work
>>> successfully with a native rebase.exe (peflags.exe).
>>>
>>
>> And I disagree as noted above.
>
> Because you misunderstand the situation.  rebase is NOT called in a
> loop, nor are the DLL paths themselves listed on the command line -- so
> MSYS doesn't get to fixup each individual path to the DLLs to be
> rebased. The only path that appears in the command to rebase is the path
> to the file containing the list -- so that's the only path that gets
> fixed up; NOT the paths contained inside that listfile.
>

Yes, true, I misunderstood.  I thought you meant that rebaseall was
reading the file but it is passing the file to rebase.

> Now, both the Microsoft and Tishler versions of rebase CAN be called
> with a list of DLLs passed explicitly on the command line.  In this
> case, with a native version of rebase invoked within an msys-dash shell,
> since each DLL pathname appears on the command line then MSYS would get
> a chance to do its thing.
>

Yes.

> However, rebaseall doesn't do it that way, for two reasons: (1) command
> line length limitations.  It's real easy to create a command line that's
> too long, when you're listing every .dll in an entire installation. (2)
> quoting issues. If the pathnames appeared on the command line, they'd
> all need to be escaped/quoted to protect against shell special
> characters and word splitting.
>

Yea, that command line limit would be bad.

>>> 3) An msys-rebase.exe (msys-peflags.exe) can't rebase (set flags on) the
>>> msys-1.0.dll itself.
>>>
>>> So, how about a separate mingw32-rebase package, based on the same
>>> source code, but which provides only the executables and not the
>>> scripts? I'll probably have to update the msys-rebase package's
>>> documentation to mention "the other one", but that's not too hard.
>>>
>>
>> And fix rebaseall and peflagsall so that it isn't setting the PATH or at
>> least provide a REBASE_PATH to be prepended to PATH.
>
> Look at the code in rebaseall first, and then tell me if you still think
> it'd be just great to use a native rebase with rebaseall. If so, then
> I'll add REBASE_PATH support.
>
> Oddly, however, nobody in six years on cygwin has ever found it
> necessary to use *some other rebase* than the one *shipped with the
> rebaseall* script.
>

Ok, I'll take a look.  Maybe because most users would use the setup.exe
program to install it and it would end up in the /bin directory is the
reason no one has complained.

Earnie

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Greg Chicares-2
In reply to this post by Charles Wilson-8
On 2010-05-27 15:03Z, Charles Wilson wrote:
> On 5/27/2010 7:49 AM, Earnie wrote:
>> Charles Wilson wrote:
>
>>> I dunno. Is there some accepted standard (<0x4000000 is .exe,>0x7000000
>>> is system) or something?
>>
>> User space < 0x70000000 >= System space
>
> Thanks. See here: http://www.drdobbs.com/184416272
[...]
>> Microsoft’s proposal continues
>> with a “first letter” scheme for the selection of the base address,
>> which I have summarized in Table 1.
>
> (Unfortunatly, the link to "Table 1" is broken)

Here's Table 1:
  http://i.cmpnet.com/ddj/windevnet/images/wdj0012b/0012bt1.gif
[I clicked "Print" on the menu right above the article's name in
the URL you gave--usually that helps with bad links.] The table
here seems to be the same thing:

http://www.gidforums.com/t-994.html

One possible scheme is to choose a base address based on the first
letter of the DLL name.
First letter Base address
A - C 0x60000000
D - F 0x61000000
G - I 0x62000000
J - L 0x63000000
M - O 0x64000000
P - R 0x65000000
S - U 0x66000000
V - X 0x67000000
Y - Z 0x68000000


------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
In reply to this post by Earnie Boyd
On 5/27/2010 9:49 PM, Earnie wrote:

> Charles Wilson wrote:
>> So, you WANT rebaseall, a particular script provided by a particular
>> package, which expects specific behavior and options from an executable
>> that is provided by *that same package*, to use whatever random
>> executable that happens to have the same name which it might find in the
>> PATH?  Especially when that PATH is *not* the normal path most MSYS
>> users are familiar with after they launch a "normal" shell via msys.bat?
>>
>
> But if I didn't install to / the rebase executable wouldn't be found in
> /bin anyway.

Wait.  (a) Why wouldn't you, and (b) are the binary *msys* packages
uploaded to sourceware expected to work properly if they are NOT
installed in the expected directory?

I know that the msys-autotool packages will have a very hard time if
they are not installed into MSYS "/" -- since they will look explicitly
for their share/autoconf/*, share/automakeN.NN/*, and share/aclocal/
files under / == /usr.

Even the mingw-autotool packages will have trouble if they are installed
somewhere other than /mingw ('course, it doesn't matter where that
/mingw is physically located: C:\MinGW or D:\my-backup-mingw\)

I know that msys executables are no longer *required* to be installed
into the /bin where msys-1.0.dll lives, but...even on linux, if you
download an RPM that is supposed to be installed in /usr, and somehow
manage to install it in /usr/local -- you'll be very lucky if it works
at all.  If you want to install an RPM somewhere else, you're expected
to download the SRPM and rebuild.

>> I'm sorry, Earnie, but that's...not wise. It's ASKING for trouble.
>
> In either scenario there are issues and assuming /bin isn't correct
> either.  Perhaps a better scenario would be to set PATH to the directory
> name of argv[0].

Suppose instead of ACTUALLY hardcoding PATH=/bin, the rebaseall script
were, in the source package, named rebaseall.in, and contained
"PATH=@bindir@".  That what, when rebuilt with a different --prefix, it
would work.

Your idea of "use the directory of $0" would be ok, too.

But...you're arguing about the way an upstream application is coded --
it's not as if I wrote rebaseall, or was proposing a patch for
msys-1.0.dll itself.

While we can certainly patch the rebase package for MSYS's purposes --
and I *have*, simply because I wanted to use "dash" instead of "ash",
since the former is more actively maintained upstream -- this seems to
be a departure from the way most MSYS packages are treated. I've just
been describing how the pre-existing, upstream package operates, and why
it does so...not how my spiffy brand-new written-just-for-msys tool is
coded.

>> The rebase.exe provided by the Microsoft SDK doesn't accept the -T
>> option. rebaseall uses it. That's one reason why PATH is explicitly set
>> to /bin. Also, remember that the recommended sequence is:
>
> Ok, an understandable reason but I still think there are better ways to
> accomplish the goal.

Again, this is an upstream behavior. Usually we don't *require* rewrites
of upstream code, unless explicitly *buggy* on msys. That isn't the case
here: if I installed rebase on cygwin into /usr/local, it'd not work
there either. (SO, unless there is a *requirement* that rebaseall work
if installed somewhere other than / == /usr, your suggestions are "gee
it'd be nice if", not "this is broken because".)

>> launch a dash shell
>>    * dash doesn't read the bash startup files, so your PATH is
>>      probably not what you're used to, in "regular" bash. Plus,
>>      you haven't run msys.bat, so all the PATH shenanigans it performs
>>      don't happen either.
>
> So it doesn't read /etc/profile or ~/.profile?  I knew there was some
> reason I used bash instead of the default cygwin shell.

dash reads both /etc/profile and ~/.profile for *login* shells. It
always reads the file whose path is specified in the environment
variable ENV.  (So, if no ENV is set and dash is NOT started as a login
shell, then *no* startup files are read at all).  However, so if
rebase's README said 'start dash as a login shell: dash -l' then that'd
be one thing, but it doesn't. It says:

    1. shutdown all MSYS processes
    2. start dash (do not use bash or rxvt)
    3. execute /bin/rebaseall (in the dash window)

However, I'm not actually sure that's in error.  Many times, people's
login scripts start resident programs; mine launches ssh-agent -- which
will break rebaseall because it expects, and verifies, that NO msys
processes are running except for dash itself.

> If the tool is installed at / yes, but if the tool is installed in
> /usr/local it will be an issue.

So, two questions: (1) is it *required* that pre-compiled MSYS packages
must operate correctly if installed somewhere other than their
compiled-in --prefix? (2) is it *required* that packages which we
provide pre-compiled for MSYS, when RE-compiled from source and
installed using a different --prefix must operate correctly?

Right now, rebase violates both. Personally, I think #1 is a fool's
game.  Now, #2...is another question. Most well-behavior packages obey
it, but the upstream rebase package does not.  It can be patched to do
so, of course -- and THAT patch might actually be accepted upstream
(most of the other MSYS mods -- e.g. explicitly using dash rather than
ash -- wouldn't).

As far as a proposed patch, I could see either

PATH=${REBASE_PATH:-@bindir@}

and creating the real rebaseall script from rebaseall.in at configure, or

tp1=${0%/*}
tp2=${tp1:-.}
PATH=$(cd $tp2 && pwd)

to always use rebase.exe from the same directory in which rebaseall
itself is found. This would facilitate testing an uninstalled rebaseall
(*) -- except that it would not be able to "find" sed, grep, sort..., so
we'd need

PATH=$(cd $tp2 && pwd):@bindir@

anyway. (If somebody didn't install *our* sed, grep, and sort -bin
packages into /, the God help 'em).

(*) You still have to shutdown all other MSYS applications, exit your
normal shell, launch a new dash shell, then explicitly invoke
/path/to/my/builddir/rebaseall...so this isn't really all THAT helpful.

>> Oddly, however, nobody in six years on cygwin has ever found it
>> necessary to use *some other rebase* than the one *shipped with the
>> rebaseall* script.
>
> Ok, I'll take a look.  Maybe because most users would use the setup.exe
> program to install it and it would end up in the /bin directory is the
> reason no one has complained.

Sure. Cygwin distributes precompiled binaries in a "install as directed,
or don't blame us, WJM" fashion.  Traditionally, *msys* packages are
provided the same way; in the days of msys-dtk-1.0.1 + updates, nobody
would've expected the coreutils or perl update "overlay" to work
correctly if it were unpacked in /usr/local/ rather than /.  It MIGHT
have worked -- but if it didn't the answer would have been "Don't Do That."

The core MinGW packages like gcc, binutils, etc, of course, are
different (I exclude the mingw autotool packages, for reasons that now
resemble a dead horse).

--
Chuck

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

NightStrike
In reply to this post by Charles Wilson-8
On Wed, May 26, 2010 at 7:32 PM, Charles Wilson
<[hidden email]> wrote:
> On 5/26/2010 8:56 AM, Keith Marshall wrote:\
>> you said:
>>> BTW, just as cygwin-rebase can't rebase cygwin1.dll, my msys port of
>>> rebase can't rebase msys-1.0.dll.
>>
>> I assume that remains true of the package you eventually released?
>
> Correct. rebase.exe depends on msys.  Mostly, because in order to be

Forgive my ignorance, but why can't you just compile rebase
statically?  Then it won't depend on the msys dll.

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
On 5/28/2010 11:35 AM, NightStrike wrote:
>> Correct. rebase.exe depends on msys.  Mostly, because in order to be
>
> Forgive my ignorance, but why can't you just compile rebase
> statically?  Then it won't depend on the msys dll.

There is no way to link any executable statically to the msys (or
cygwin) runtime.  That is, if you need posix support, then you're gonna
use msys-1.0.dll or cygwin1.dll.  So, there really isn't a beast called
"a static msys executable".  You either have posix support and a DLL
dependency, or you compile as a native application.  Usually, if
somebody says "compile (msys/cygwin) foo.exe statically" they mean "link
statically to all msys/cygwin libraries EXCEPT msys-1.0.dll/cygwin1.dll
and any necessary windows system DLLs".

However, even for "native" apps you'd still end up linking against some
win32 system DLLs.  There really are very few programs on windows that
can be considered truly "statically linked" -- and I don't know exactly
what they'd be able to DO, without access to windows services via those
system DLLs.

--
Chuck

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
In reply to this post by Keith Marshall-3
On 5/27/2010 6:31 AM, Keith Marshall wrote:
> On Thursday 27 May 2010 00:32:53 Charles Wilson wrote:
>> I guess what you're really asking is, can our rebase.exe (and
>> peflags.exe) be built as a native win32 app.
>
> Yes.

It can. I've created

rebase-3.0.1_1-1-mingw32-bin.tar.lzma
rebase-3.0.1_1-1-mingw32-doc.tar.lzma
rebase-3.0.1_1-1-mingw32-lic.tar.lzma
rebase-3.0.1_1-1-mingw32-src.tar.lzma

where the -bin package contains only rebase.exe and peflags.exe, built
as native apps.  Since rebase.exe includes C++ code, I linked it using
-static-libgcc so that there would be no deps on any DLLs (other than
w32 system ones).

>> Well, for some reason I had always thought that cygwin1.dll itself
>> was not "rebasable" -- that, even if you used some native tool to
>> change its ImageBaseAddress, it would die horribly for some
>> esoteric reason (hardcoded pointer values? dunno...).
>
> I don't know either; you likely know better than I.

Well, obviously I was wrong, tho -- since it actually worked (for you,
on the msys "version" of cygwin; and for the correspondent who
originally suggested, over on the cygwin lists, trying it on "real" cygwin.

>> So, I've never attempted to rebase either cygwin1.dll or
>> msys-1.0.dll.
>
> Yet you did recommend doing just that, when Chris originally reported
> the problem on the MinGW-MSYS list...

Right, because I saw that recommendation on the cygwin lists.  It never
would have occured to *me*, independently of that recommendation, to do
so -- since I had this mistaken notion about cygwin being non-rebasable.

>> Errr...huh?
>
> Well, more accurately, it has been frequently reported, (not
> necessarily on our lists), that an MSYS shell cannot be started AT
> ALL; right from the outset:
>
>   d:\path\to\msys\bin\sh.exe --login -i
>
> dies with a "cannot reserve space for cygwin's heap" message.
...
> Well, the message above is distinct from the diagnostic you describe,
> as indicative of this failure mode; of course, that in no way implies
> that the two do not have a common underlying cause.

Hmm. yeah, that actually does sound more like a problem with either (a)
msys itself being loaded in "the wrong place", or (b) some other (win32
system?) dll being loaded "too close" to msys dll, since msys expects
its heap start address to be "just above" the dll itself. And, of
course, in the same location in both parent and child.  IIRC.

>> It's more likely
>> to be "Cwd.dll" or "POSIX.dll" (some perl extension modules loaded
>> almost last), or sometimes msys-intl-8.dll -- or any of the guile
>> srfi dlls.
>
> Hmm.  In my case, it's none of those; at the time when I encountered
> this problem I had installed ONLY Console2, msysCORE, msys-coreutils
> and msys-bash, so the only likely candidates, besides msys-1.0.dll
> itself, would be those DLLs distributed with Console-2.00.144; there
> were no other MSYS related DLLs even present, on any file system
> attached to the box.  (I didn't try it, at the time, but it may be
> instructive to repeat the exercise with Console2 removed from the
> equation).

No, I think you're right. There doesn't seem to be much else loaded in
memory at that stage to cause the problem.

--
Chuck

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Danny Smith
In reply to this post by Greg Chicares-2

One possible scheme is to choose a base address based on the first
letter of the DLL name.
First letter Base address
A - C 0x60000000
D - F 0x61000000
G - I 0x62000000
J - L 0x63000000
M - O 0x64000000
P - R 0x65000000
S - U 0x66000000
V - X 0x67000000
Y - Z 0x68000000


No need to reinvent the wheel here.  The --enable-auto-image-base that Mumut
Khan contributed to binutils does it even better:

/* Use the output file to create a image base for relocatable DLLs.  */

static unsigned long
compute_dll_image_base (const char *ofile)
{
  unsigned long hash = strhash (ofile);
  return 0x61300000 + ((hash << 16) & 0x0FFC0000);
}
 where strhash is a function that hashes the dll filename .

Now the number is 0x61300000  rather than 0x60000000, because  0x61000000 is
cygwin1-dll base address.

For reference, search  the cygwin-apps archives around June/July 2005 for a
discussion or rebase  problems.

Danny


------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Earnie Boyd
In reply to this post by Charles Wilson-8
Charles Wilson wrote:

> On 5/27/2010 9:49 PM, Earnie wrote:
>> Charles Wilson wrote:
>>> So, you WANT rebaseall, a particular script provided by a particular
>>> package, which expects specific behavior and options from an executable
>>> that is provided by *that same package*, to use whatever random
>>> executable that happens to have the same name which it might find in the
>>> PATH?  Especially when that PATH is *not* the normal path most MSYS
>>> users are familiar with after they launch a "normal" shell via msys.bat?
>>>
>>
>> But if I didn't install to / the rebase executable wouldn't be found in
>> /bin anyway.
>
> Wait.  (a) Why wouldn't you, and (b) are the binary *msys* packages
> uploaded to sourceware expected to work properly if they are NOT
> installed in the expected directory?
>

Yes, they should work properly if not installed in the expected
directory.  I know there are a few that expect to find their files by
default in /usr/share but those are exceptions easily worked around.

> I know that the msys-autotool packages will have a very hard time if
> they are not installed into MSYS "/" -- since they will look explicitly
> for their share/autoconf/*, share/automakeN.NN/*, and share/aclocal/
> files under / == /usr.
>

IIRC there is an environment variable that can be set to control where
these are located.

> Even the mingw-autotool packages will have trouble if they are installed
> somewhere other than /mingw ('course, it doesn't matter where that
> /mingw is physically located: C:\MinGW or D:\my-backup-mingw\)
>

Since autotool is mostly scripts I would not have thought to have a
autotool-mingw package installed.  But again IIRC there is an
environment variable to control where to look.

> I know that msys executables are no longer *required* to be installed
> into the /bin where msys-1.0.dll lives, but...even on linux, if you
> download an RPM that is supposed to be installed in /usr, and somehow
> manage to install it in /usr/local -- you'll be very lucky if it works
> at all.  If you want to install an RPM somewhere else, you're expected
> to download the SRPM and rebuild.
>

Ack, auto-packaging.  But I am very much a believer in build it your
self which tends to overcome the obstacles of prepackaged binary
releases.  I have though come to trust apt-get enough to use it on my
Ubuntu installations.  I never liked RPM syntax enough to learn it.

>>> I'm sorry, Earnie, but that's...not wise. It's ASKING for trouble.
>>
>> In either scenario there are issues and assuming /bin isn't correct
>> either.  Perhaps a better scenario would be to set PATH to the directory
>> name of argv[0].
>
> Suppose instead of ACTUALLY hardcoding PATH=/bin, the rebaseall script
> were, in the source package, named rebaseall.in, and contained
> "PATH=@bindir@".  That what, when rebuilt with a different --prefix, it
> would work.
>

Yes, this would work up to a point.  That point being one of ``make
install prefix=/my/depot/release/directory''.  I loathe DESTDIR and
changing prefix at install time is not supposed to disturb the build
process.

> Your idea of "use the directory of $0" would be ok, too.
>

Isn't this more like what other distributed package scripts do?

> But...you're arguing about the way an upstream application is coded --
> it's not as if I wrote rebaseall, or was proposing a patch for
> msys-1.0.dll itself.
>

Ack.

> While we can certainly patch the rebase package for MSYS's purposes --
> and I *have*, simply because I wanted to use "dash" instead of "ash",
> since the former is more actively maintained upstream -- this seems to
> be a departure from the way most MSYS packages are treated. I've just
> been describing how the pre-existing, upstream package operates, and why
> it does so...not how my spiffy brand-new written-just-for-msys tool is
> coded.
>

Ack.  I don't expect you to change it unless you had written it. :)

>>> The rebase.exe provided by the Microsoft SDK doesn't accept the -T
>>> option. rebaseall uses it. That's one reason why PATH is explicitly set
>>> to /bin. Also, remember that the recommended sequence is:
>>
>> Ok, an understandable reason but I still think there are better ways to
>> accomplish the goal.
>
> Again, this is an upstream behavior. Usually we don't *require* rewrites
> of upstream code, unless explicitly *buggy* on msys. That isn't the case
> here: if I installed rebase on cygwin into /usr/local, it'd not work
> there either. (SO, unless there is a *requirement* that rebaseall work
> if installed somewhere other than / == /usr, your suggestions are "gee
> it'd be nice if", not "this is broken because".)
>

Ack.

>>> launch a dash shell
>>>     * dash doesn't read the bash startup files, so your PATH is
>>>       probably not what you're used to, in "regular" bash. Plus,
>>>       you haven't run msys.bat, so all the PATH shenanigans it performs
>>>       don't happen either.
>>
>> So it doesn't read /etc/profile or ~/.profile?  I knew there was some
>> reason I used bash instead of the default cygwin shell.
>
> dash reads both /etc/profile and ~/.profile for *login* shells. It
> always reads the file whose path is specified in the environment
> variable ENV.  (So, if no ENV is set and dash is NOT started as a login
> shell, then *no* startup files are read at all).  However, so if
> rebase's README said 'start dash as a login shell: dash -l' then that'd
> be one thing, but it doesn't. It says:
>
>      1. shutdown all MSYS processes
>      2. start dash (do not use bash or rxvt)
>      3. execute /bin/rebaseall (in the dash window)
>
> However, I'm not actually sure that's in error.  Many times, people's
> login scripts start resident programs; mine launches ssh-agent -- which
> will break rebaseall because it expects, and verifies, that NO msys
> processes are running except for dash itself.
>

Ack.

>> If the tool is installed at / yes, but if the tool is installed in
>> /usr/local it will be an issue.
>
> So, two questions: (1) is it *required* that pre-compiled MSYS packages
> must operate correctly if installed somewhere other than their
> compiled-in --prefix? (2) is it *required* that packages which we
> provide pre-compiled for MSYS, when RE-compiled from source and
> installed using a different --prefix must operate correctly?

1) *required*, no.
2) *required*, yes.  This should be true regardless if it is for MSYS or
some other system especially if it is GPL since the GNU Coding Standards
should have been followed.

>
> Right now, rebase violates both. Personally, I think #1 is a fool's
> game.  Now, #2...is another question. Most well-behavior packages obey
> it, but the upstream rebase package does not.  It can be patched to do
> so, of course -- and THAT patch might actually be accepted upstream
> (most of the other MSYS mods -- e.g. explicitly using dash rather than
> ash -- wouldn't).
>

So, the upstream package needs a fix.

> As far as a proposed patch, I could see either
>
> PATH=${REBASE_PATH:-@bindir@}
>
> and creating the real rebaseall script from rebaseall.in at configure, or
>
> tp1=${0%/*}
> tp2=${tp1:-.}
> PATH=$(cd $tp2&&  pwd)
>
> to always use rebase.exe from the same directory in which rebaseall
> itself is found. This would facilitate testing an uninstalled rebaseall
> (*) -- except that it would not be able to "find" sed, grep, sort..., so
> we'd need
>
> PATH=$(cd $tp2&&  pwd):@bindir@
>

PATH=$(cd $tp2&&  pwd):@bindir@:/bin

> anyway. (If somebody didn't install *our* sed, grep, and sort -bin
> packages into /, the God help 'em).
>

Ack.  That goes with what is stated in the README.

> (*) You still have to shutdown all other MSYS applications, exit your
> normal shell, launch a new dash shell, then explicitly invoke
> /path/to/my/builddir/rebaseall...so this isn't really all THAT helpful.
>
>>> Oddly, however, nobody in six years on cygwin has ever found it
>>> necessary to use *some other rebase* than the one *shipped with the
>>> rebaseall* script.
>>
>> Ok, I'll take a look.  Maybe because most users would use the setup.exe
>> program to install it and it would end up in the /bin directory is the
>> reason no one has complained.
>
> Sure. Cygwin distributes precompiled binaries in a "install as directed,
> or don't blame us, WJM" fashion.  Traditionally, *msys* packages are
> provided the same way; in the days of msys-dtk-1.0.1 + updates, nobody
> would've expected the coreutils or perl update "overlay" to work
> correctly if it were unpacked in /usr/local/ rather than /.  It MIGHT
> have worked -- but if it didn't the answer would have been "Don't Do That."
>

Ack.

> The core MinGW packages like gcc, binutils, etc, of course, are
> different (I exclude the mingw autotool packages, for reasons that now
> resemble a dead horse).
>

Probably smell like one too. ;p

--
Earnie

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Earnie Boyd
Earnie wrote:

>>
>> Wait.  (a) Why wouldn't you, and (b) are the binary *msys* packages
>> uploaded to sourceware expected to work properly if they are NOT
>> installed in the expected directory?
>>
>
> Yes, they should work properly if not installed in the expected
> directory.  I know there are a few that expect to find their files by
> default in /usr/share but those are exceptions easily worked around.
>

I should have instead said:

An expectation of the user is that it works properly.  This is Windows
and the idiots abound.  I, the user, should be able to install anywhere
I please, even though I've been warned not to.

Earnie

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
In reply to this post by Earnie Boyd
On 6/1/2010 8:48 AM, Earnie wrote:
> Charles Wilson wrote
>> I know that the msys-autotool packages will have a very hard time if
>> they are not installed into MSYS "/" -- since they will look explicitly
>> for their share/autoconf/*, share/automakeN.NN/*, and share/aclocal/
>> files under / == /usr.
>
> IIRC there is an environment variable that can be set to control where
> these are located.

And if somebody is knowledgeable enough to know about that, they can
certainly figure out how to install the msys-autotool packages somewhere
other than /.  But I don't think this list should be responsible for
supporting them, OR for making undue accomodations for that.  Frankly, I
don't have the time.

Use them as provided, installed as directed, or don't come crying here.

>> I know that msys executables are no longer *required* to be installed
>> into the /bin where msys-1.0.dll lives, but...even on linux, if you
>> download an RPM that is supposed to be installed in /usr, and somehow
>> manage to install it in /usr/local -- you'll be very lucky if it works
>> at all.  If you want to install an RPM somewhere else, you're expected
>> to download the SRPM and rebuild.
>>
>
> Ack, auto-packaging.  But I am very much a believer in build it your
> self which tends to overcome the obstacles of prepackaged binary
> releases.  I have though come to trust apt-get enough to use it on my
> Ubuntu installations.  I never liked RPM syntax enough to learn it.

Yes, your admiration for the gentoo model was obvious from the day
mingwPORT was devised. I hate it; I want to use my computer to get work
done -- not to fuss around incessantly redoing the same crap somebody
else should have already done.

If I want to install MSYS onto three computers, I don't want to waste
time recompiling it all three times.  Nor do I want to recompile it all
even once, and then do all the work the distributor SHOULD have done,
and create my own installation image for the other two computers.

Or, worse, expect MSYS novices on my development team to do any of the
above.

> Yes, this would work up to a point.  That point being one of ``make
> install prefix=/my/depot/release/directory''.

No, actually the GCS explicitly requires that if you change the "prefix"
at 'make install' time, that the images installed into this new prefix
are NOT modified from whatever was established by the original prefix at
configure time.

So (assuming rebase actually HAD a configure stage; it doesn't currently
but the above proposal assumes that one would be added), 'make install
prefix=/tmp/for/packaging' would still work.  Assuming the end user
actually unpacked the tarball thus created into the original configured
prefix.

> I loathe DESTDIR and
> changing prefix at install time is not supposed to disturb the build
> process.

Err...yes...that statement contradicts what you said above.

> Isn't this more like what other distributed package scripts do?

Sometimes.  .pc files have prefix=@prefix@; they don't do fancy
where-am-i path shenanigans.  .la files have explicitly absolute paths
based on @prefix@ too.  Some /bin/scripts do it one way, some another.

>> Right now, rebase violates both. Personally, I think #1 is a fool's
>> game.  Now, #2...is another question. Most well-behavior packages obey
>> it, but the upstream rebase package does not.  It can be patched to do
>> so, of course -- and THAT patch might actually be accepted upstream
>> (most of the other MSYS mods -- e.g. explicitly using dash rather than
>> ash -- wouldn't).
>>
>
> So, the upstream package needs a fix.

OK.

> PATH=$(cd $tp2&&  pwd):@bindir@:/bin

Well, all of this pre-supposes that a configure script is added.  So
it's a bigger upstream change than you might think.

I'll look into it.

--
Chuck

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Charles Wilson-8
In reply to this post by Earnie Boyd
On 6/1/2010 8:58 AM, Earnie wrote:

> Earnie wrote:
>>>
>>> Wait.  (a) Why wouldn't you, and (b) are the binary *msys* packages
>>> uploaded to sourceware expected to work properly if they are NOT
>>> installed in the expected directory?
>>>
>>
>> Yes, they should work properly if not installed in the expected
>> directory.  I know there are a few that expect to find their files by
>> default in /usr/share but those are exceptions easily worked around.
>>
>
> I should have instead said:
>
> An expectation of the user is that it works properly.  This is Windows
> and the idiots abound.  I, the user, should be able to install anywhere
> I please, even though I've been warned not to.

In this case, I adopt the cygwin motto: We're Just Mean.  Don't do that;
if you do, understand the consequences: you get to keep all the pieces
if it breaks.

--
Chuck

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: MSYS "rebase" package

Earnie Boyd
Charles Wilson wrote:
> On 6/1/2010 8:58 AM, Earnie wrote:

>> An expectation of the user is that it works properly.  This is Windows
>> and the idiots abound.  I, the user, should be able to install anywhere
>> I please, even though I've been warned not to.
>
> In this case, I adopt the cygwin motto: We're Just Mean.  Don't do that;
> if you do, understand the consequences: you get to keep all the pieces
> if it breaks.
>

Agreed.

Earnie

------------------------------------------------------------------------------

_______________________________________________
MinGW-dvlpr mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/mingw-dvlpr
12
Loading...