preliminary patch for better POSIX installs

[pabs]

Hi all,

nsis doesn't currently install like a normal Unix program - instead it dumps all its stuff in one directory with a few subdirectories. At the moment for the debian package I hack around this by moving files around after the install.

The attached patch against CVS gets about 50% of the way to a better install structure. One problem is that only the subs and makensis get installed when I do scons install. It contains proper DESTDIR (useful for packages), PREFIX and so on semantics, and a portability switch for Windows, where the plugins/etc are stored relative to the exe.

So, I would like to know if this is the kind of thing that should be in the official nsis package. Also, could people review the patch to make sure I'm using scons right, and that I have not done anything stupid.

[kichik]

To have `scons install` install all the files, you have to alias `install` to all of the directories. In the original code, `install` is aliased to $PREFIX and so everything that needs to go into $PREFIX is built and installed when `scons install` is called. I haven't read all of the changes, but it seems you need to add:

code:
defenv.Alias('install', '$NSIS_DATA_DIR')

I'd love to have scons install to the proper locations on POSIX, however it must be backward compatible. Forcing users to replace ${NSISDIR} everywhere is not an option.

I hope I'll get a chance to take a deeper look soon as this is definitely something that belongs in the official package.

[pabs]

About NSISDIR, I guess that for backward compatability sake, it could just use NSIS_DATA_DIR.

Thanks for the info about the aliasing. Another thing - I noticed that scons install-plugins and similar do nothing. First they barf because the directory doesn't exist, and after you create it, they do nothing.

Thanks for your interest, I'll keep plugging away at it.

[pabs]

Added the aliases, but after creating all the directories by hand (not sure how to make scons create the dir structure), I get this:

scons: *** Two environments with different actions were specified for the same target: AdvSplash.dll
File "SConstruct", line 177, in Distribute

[kichik]

Originally posted by pabs
Another thing - I noticed that scons install-plugins and similar do nothing. First they barf because the directory doesn't exist, and after you create it, they do nothing.

Have you specified a PREFIX? Deleting the entire plugins directory or just its content and then using `install-plugins` works fine for me. It creates the directory, installs all of the plugins, and never complains about a missing directory.

The AdvSplash.dll error might be caused by installing the DLL into the same place it's built or by installing it to the same place two times. Try running scons with --debug=dtree. It might give you some vital information.

[pabs]

I've no idea whats going on. With the attached patch, I get the following:

rm -rf build/ .scon* debian/ ; scons -c --debug=dtree
scons: Reading SConscript files ...
Delete("nsis-28-Oct-2005.cvs")
Delete(".instdist")
Delete(".test")
Using GNU tools configuration
Checking for memcpy requirement... yes
Checking for memset requirement... yes
Checking for main() in C library gdi32... no
Checking for main() in C library user32... no
Checking for main() in C library version... no
Checking for main() in C library pthread... yes
Checking for main() in C library stdc++... yes
Checking for main() in C library iconv... no

scons: *** Two environments with different actions were specified for the same target: AdvSplash.dll
File "SConstruct", line 177, in Distribute

[kichik]

You both install and build AdvSplash.dll into the same directory. dest_dir is defined to '.' by default, even on win32. It happens on win32 as well because the `is` operator checks for object identity, not equality.

The operators is and is not test for object identity: x is y is true if and only if x and y are the same object. x is not y yields the inverse truth value.

[pabs]

Fixed that. Still don't understand why scons install doesn't mind creating the dirs for makensis, the stubs, the licence, the config and the todo.txt, but won't do it for anything else, and when I do the mkdir manually, "scons install" installs nothing into Contrib/etc. I seem to remember having a similar problem before attempting this proper POSIX install thing.

[pabs]

Aha, found it:

http://forums.winamp.com/showthread....24#post1704324

If I set the full path for DESTDIR, then I still get the same problem :/

[kichik]

Looks like you're missing a slash:

code:
+defenv.Alias('install', '$DESTDIR/$NSIS_PLUGINS_DIR')

But the others worked, so I'm not sure. Try --debug=dtree, it should explain it all. There's even a GUI for that in the SCons Wiki.

[pabs]

Woooot, not only did I get --debug=dtree working, but I got the whole thing to install properly! I think the main thing was extra an $DESTDIR here and there plus lack of / characters.

Patch, full build log and list of installed files attached.

Build log was got like so (how I'll be doing it in the debian package):

scons NSIS_CONF_DIR=/etc PREFIX=/usr SKIPPLUGINS=System DESTDIR=`pwd`/debian/nsis/ install &> build.txt

Only enhancements needed are to perhaps not install .lib/.def files for the plugins and add a switch to not install licence.txt (debian packages have a standard copyright file). These can wait though, and I can submit a patch later on. Be great if you could test that it all works fine on Windows too, and generally review the patch more deeply.

No luck with System.c btw. If the assembly was split out to separate files and was pure assembly, I could compile it with NASM. Don't know if that'd be an acceptable solution.

[onad]

Thank you so much!

FYI: I'm still busy in between other tasks e.g. NSIS via MSI >:-) CustomAction dll.

I will however make an NSIS Win32 build of installer workable on FreeBSD, even if it takes me a year. Then, to celebrate open a wikipage to celebrate which explains all.

[pabs]

Just curious, but what are the sticking points for nsis working on FreeBSD?

[onad]

Well, I started of just after the switch to scons builds... and before that I had problems of make on a sparc processor under solaris, gave that up. Tried for fun my OSX, build worked ok via "make" only makensis just did not work because of endian isses.

Then patched the makensis code to be OSX compliant, halfway realizing the enormous effort. It was all new to me you see.

Fine, then my mistake move files via Fat32 to FreeBSD, dumb me, Filename capitalisation of whatever.h changed. OK found out. could compile but the other errors.

Then found out that I needed to update scons to the bleeding ege version .. sigh.. then to make it easier needed to install KDE on FreeBSD, fine I'm all set to go. Now if only I had a spare day to toy around a little more of NSIS Build on FreeBSD

But I WILL PREVAIL, I'm persistent

Anyhow, Pabs, thanks for your all sorts of great help.

[onad]

Well compiling unde GCC v2.95.3 will just be probbably impossibly, to many strange defects in the code. Upgrading the FreeBSD 4.11 with GCC 3.3.5 a extreme challinging issue... so re-format one partition and on to FreeBSD v6.0 it is! sigh... At least I will have GCC 3.4.4, an on for a new try.

[kichik]

I'm sorry, I haven't had the time to go over all of it, again. But I do have a couple of notes from another brief look.

It doesn't start building on Windows because NSIS_CONF_DIR is empty on Windows. You should probably only append those symbols when not on Windows. After that it builds, but I haven't tested installation yet.

As for the naming of the variables in the SConscutrct, I think it'd be nicer if they were all named $PREIFX_SOMETHING. It's also a bit of a standard as I see it. Most configure scripts accept --prefix-something in the command line, not --destdir or --something_something_dir.

I think the define names should be a bit more special. Someone might already have BINDIR defined. NSISBINDIR would be better, in my opinion.

As for the System plug-in, if it works, it should be ok. But can nasm really assemble inlined assembly?

[pabs]

I've addressed your concerns about the $PREFIX_* and NSIS*DIR in the attached patch. I also attached a file listing all the occurrences of NSISDIR that I'm not sure what to do with.

Can you show me what you mean about Windows and NSIS_CONF_DIR (now PREFIX_CONF)? I'm not sure I understand the problem.

AFAICT, nasm doesn't do inline assembly - I suggested to split out the assembly code into separate .asm files.

[pabs]

Don't apply this yet, its broken in some situations. I'm also reworking it a bit so it is more consistent.

[kichik]

This code:

code:
for define in ('PREFIX_CONF','PREFIX_BIN','PREFIX_DATA' ...):

    defenv.Append(NSIS_CPPDEFINES = [(define, '"'+defenv[define]+'"')])

Will give an error on Windows because PREFIX_CONF is not part of the environment. It's not part of the environment because its default option value on Windows is None. Same goes to all the others. Those defines shouldn't even be defined on Windows as they're not used.

As for nasm, wouldn't a separate file allow you to define just functions? The current problematic functions have both C code and assembly in them.

[pabs]

Attached a new version of the patch. This one is tested a lot more (including the test target - need to test some more, inc the installer targets). The main highlight of this one is that I've changed makensis so that it checks some environment variables to change the PREFIX_* build-time variables at runtime. This is useful for running tests at compile time (before nsis is installed) and testing an exiting makensis against changed includes and stuff.

[pabs]

New version. Fixed the dist* targets and made only the plugin .dlls install (not the .def and .lib files). Pending some testing on Windows and a review, I think this is ready to go into CVS (sorry I took so long).

One other issue I can think of that would be useful in some parts of this is the lack of an ${EXEEXT} variable in the nsis script language.

[pabs]

New version. Updated for recent cvs changes and fixed a bug with scons -c.

[pabs]

A while ago you wrote about the framework you use for automated daily builds of NSIS. I think it involved wine instead of Windows. I was wondering if there is any info about setting up such an environment on the wiki? This is so that I can test the patch properly.

[kichik]

There's nothing special about it. It's just scons running under wine. All I did was create a cron job that calls scons with wine and uploads the result to SourceForge.

[pabs]

What about the compiler and python? Does it just use mingw32 from the linux install?

[kichik]

All Windows applications. I've copied over the free edition of VC, the Platform SDK and a Python installation. There's no special voodoo.

[pabs]

Couldn't get VCToolkitSetup.exe installed, it failed to install the .NET framework.

[kichik]

I had to make some changes to make it compile. A revised patch against the latest CVS version is attached. The tests failed because makensis.nsi couldn't find ..\makensisw.exe. I haven't had the chance to look into that yet.

Some other comments:

• Even without specifying the install target, all of the files are installed. They are installed to the build directory, but they are still installed. The check for PREFIX in SConstruct should be brought back.
• I think NSIS*DIR should match the real directory name to avoid confusion. For example, NSISPLUGINDIR should be NSISPLUGINSDIR to match Plugins and $PLUGINSDIR.
• Empty defines drive CONST_STR crazy. Lots of errors arise when NSISBINDIR and friends are empty on a Windows build.
• NSIS_CONFIG_RELOCATABLE_PATHS should be defined for the script in the CEXEBuild constructor along with all of the other defines.
• Maybe a more fit define name can be found than NSIS_CONFIG_RELOCATABLE_PATHS. How about NSIS_CONFIG_POSIX_PATHS, NSIS_CONFIG_WINDOWS_PATHS, NSIS_CONFIG_ONE_DIR or something similar?

[pabs]

Fixed NSISPLUGINDIR and added the define in the CEXEBuild constructor. Updated/merged patch attached.

About ..\makensisw.exe, shouldn't it get that from build/release/Makensisw/?

Not sure how to fix the other things. I'll investigate further.

[kichik]

makensis.nsi is supposed to be compiled in an installed copy of NSIS, so it shouldn't be taken from the build directory.

[kichik]

How about installing everything as it's installed on Windows and then creating symlinks in the proper places? Other way around should work as well... This way, the only required changes would be fixing NSISDIR detection and updating the build system to create symlinks.

[pabs]

Hmm, I don't think that is a good idea.

What about a simplification of the current patch where all the subdirectories of PREFIX_DATA cannot be customised. This cuts out all those environment variables and stuff.

[kichik]

Why don't you think it'll be a good solution?

[pabs]

The main reason is that I created this patch was so that I wouldn't have to do such hacks in the debian package.

[kichik]

I can see why you'd consider this a hack, but think of the upsides. Scripts will work out of the box with no need for new defines and the patch will be much simpler. Are there any downsides other than the general feeling of "this is just stitching some patches so it'll connect properly instead of actually porting it"?

[pabs]

Not really.

How about I do the equivalent of the current patch without all the extra defines? The only changes to the current situation would be:

• ${NSISDIR} would point at /usr/local/share/nsis (or /usr/share/nsis on Linux, and contain Contrib, Include, Plugins, Stubs, Bin (win32 only binaries) and Menu (not sure if this should be installed or not).
• The native binaries (makensis/genpat), config file, docs and examples will be located outside of ${NSISDIR}, so you won't be able to load/read/touch them from .nsi scripts (unless I add keep some of the ${NSIS*DIR} variables).
• You will be able to redefine where makensis looks for its data, and config file/etc by using environment variables. This is useful for running scons test without having installed the software
• Since relocatability is fairly foreign to linux/bsd/unix (although it is possible on linux and macos at least), it will be possible to use absolute locations for stuff

Btw, shouldn't RegTool.bin move to Contrib?

[pabs]

Two problems I saw when reverting most of the patch:

• tutorial.but contains this line. That won't be possible any more, and isn't portable anyway (executables don't have the .exe extension on linux/etc). I've just made this NSISDIR for now.

  \c nsExec::ExecToLog '"${NSISBINDIR}\makensis.exe" /CMDHELP'

  Same problem with nsExec's test.nsi and other files.
• compiler.but references licence.txt, which may or may not be installed (debian has other files for copyright info). Reverted to NSISDIR.

[kichik]

Sounds good to me, though, as you've already noticed, there'll be problems with documentation links. Those can be resolved with a symlink located at the documentation location, or halibut's xhtml-rlink-prefix and xhtml-rlink-suffix \cfg options. Those were added for the CHM, but can be used here to add a path prefix to the links in this case as well.

[pabs]

About the documentation links, I think the best option is to post-process the html files with a python function, I'm not sure how to do that though. Writing the function would be easy, just use some regular expressions. Hooking it up to scons is the hard part (at least for me). Any ideas? This could also be used to process the ready-made html files - the System Readme.html and so on.

[pabs]

Attached the cut-down version. Still need to sort out the documentation issue.