Well, Stefan would know better, but as I understand it, the
AC_COIN_HAS_PACKAGE command actually first creates *_LIB and *_CFLAGS
variables for the package named in the first argument and then also
adds those to the *_LIB and *_CFLAGS variables of the library/binary
that depends on the package being defined. So the COINDEPEND_CFLAGS
are just the combination of the flags of the individual packages that
make up CoinDepend. These then get added to XXXLIB_CFLAGS along with
any other dependent project identified in later AC_COIN_HAS_PACKAGE to
form the full set of flags for XXXLIB. Admittedly, this is a bit
confusing and I'm not sure I got it entirely right. Stefan?



This has nothing to do with how you code as a developer. We are
talking about users, not you, when we talk about what should be in the
Makefile for the examples. To a user, all they have to know is that
they want to link to OS. A user should not have to know that, for
instance, you built without support for Bonmin and Couenne this time
because they aren't available and so they don't have to put those on
the link line. How would they know that? The .pc file automatically
contains whatever dependencies were present at the time of the
building of the library, no matter what those were. As far as
development goes, nothing changes in the above.

> 2.  OS depends, for example, on Cgl. But OS also uses SYMPHONY. In SYMPHONY
> you have
>
> Requires: cgl osi-vol osi-dylp osi-clp coinutils = trunk osi
>
> so should I skip Cgl?

If it is possible for OS to use Cgl *directly*, not *indirectly*
because some other project linked to by OS needs it, then it should be
listed. But I don't really see how you could use Cgl directly without
having some project like Cbc, SYMPHONY, Bonmin, Couenne, etc. present,
in which case Cgl would be among the dependencies of *that* project.
Is there a use case where you would want to link to Cgl without one of
the others present?

In any case, this is not something you have to decide since these
dependencies are generated automatically based on what projects you
check for in configure.ac. The checking there works the same way---you
only check for the things that you directly depend on, not the
secondary dependencies. the m4 macros use pkg-config in the same way
as I've been describing to generate the secondary dependencies. So
just don't worry about what goes in the .pc files under
dependencies---it's checks in configure.ac that you have to worry
about. Just eliminate any that you don't need (such as for Cgl?) and
they will not appear in the .pc file either.

> Maybe not if the user has not installed SYMPHONY.
>  Also, I see you have osi-clp, but NOT clp. I guess based on what you said
> about the recursive nature of things this is not necessary since osi-clp has
>
> Requires: clp osi

Yes, SYMPHONY can't use Clp except through Osi, so there is no reason
to check for Clp directly. If we had a native interface to Clp that
could be used if Osi was missing, then Clp could be checked for as a
dependency. 


From Lou:

> What happened to $(COINUTILSSRCDIR)????

	Answer on two levels. First, for the file you were looking at
(CoinUtils/test/Makefile.am), it makes sense to give a relative location
for the headers. If you're building in the test directory, the headers
are in ../src. If you're doing a VPATH build, you need to know where the
source directory is, then a relative path, hence $(srcdir)/../src. The
rule to remember is that srcdir is the source for the directory you're
building. When you're building the test directory, srcdir resolves to
the source for test.

	More generally, what happened to COINUTILSSRCDIR? A short but somewhat
misleading answer would be `It's been replaced by COINUTILS_CFLAGS'. But
a longer answer may give you a better perspective.

	It's been replaced by pkg-config, and a whole bunch of buildtools
macros that query pkg-config and deal with the situation when pkg-config
isn't available (I'll let Stefan speak to that). If you want to know the
header directories for CoinUtils, you ask

  pkg-config --cflags coinutils

and pkg-config consults coinutils.pc to give the answer (correct for my
particular configuration)

-I/devel/Coin/Split-FedGCC64/include/coin
-I/devel/Coin/Split-FedGCC64/include/coin/ThirdParty

These are the places to look for the installed headers for CoinUtils and
Glpk in my local build structure. Notice that pkg-config is helping me
here, keeping track of the fact that my CoinUtils build brings in glpk.
To get the libs needed to link against libCoinUtils,

  pkg-config --libs coinutils

gives

 -L/devel/Coin/Split-FedGCC64/lib/coin  
 -L/devel/Coin/Split-FedGCC64/lib/coin/ThirdParty -lCoinUtils -lbz2 
 -llapack -lblas -lm -lcoinglpk -ldl -lgmp -lz

So you can see why this simplifies things in the makefiles. When you're
building against uninstalled packages pkg-config needs to find
coinutils-uninstalled.pc. Again, the buildtools macros take care of
this, manipulating PKG_CONFIG_PATH so that pkg-config can find the
right .pc file.

You can ask for information about more than one package, e.g.,

  pkg-config --cflags coinutils osi cgl

and you'll get the combined information for all, duplicates removed:

  -I/devel/Coin/Split-FedGCC64/include/coin 
  -I/devel/Coin/Split-FedGCC64/include/coin/ThirdParty
  
If you're saying `but that's exactly the same as the previous answer',
you're exactly right. All the installed header files are in one place.
You don't get the same information repeated three times (for CoinUtils,
Osi, and Cgl). So when configure.ac uses

AC_COIN_HAS_PACKAGE(CoinDepend, [cgl osi coinutils = trunk], [CbcLib
CbcGeneric])

what will happen is that buildtools creates a variable,
COINDEPEND_CFLAGS, and this is set to the output of

  pkg-config --cflags cgl osi coinutils

(with some fussing to make sure it's the trunk of coinutils).
COINDEPEND_CFLAGS will be added to command lines when building CbcLib
and CbcGeneric.

	Hope this helps. There's nothing quite so frustrating as finding
yourself afloat in an unfamiliar sea.


%%%%%%%%%%%

export  PKG_CONFIG_PATH=/Users/kmartin/coin/os-tmp/vpath-debug/lib/pkgconfig 

%%%%%%

Here is something from Ted explaining how the Build System determines if something is a dependency


Well, here's the answer. The check in configure.ac looks like this:

AC_COIN_HAS_PACKAGE(ASL,   [coinasl])   <====== missing the third argument
AC_COIN_HAS_PACKAGE(HSL,   [coinhsl],   [IpoptLib])
AC_COIN_HAS_PACKAGE(Mumps, [coinmumps], [IpoptLib])

So ASL is not going to be listed as a dependency for the Ipopt
*library* and I guess that's because it isn't a dependency of the
*library*, just of the *binary*. Apparently, the AMPL interpreter is
not part of the library (unless this is a mistake). So you can link to
the Ipopt library without liniking to ASL. However, in the case of
ASL, that really is a direct dependency of OS, right? You can directly
read AMPL files, independent of any solver being able to solve
whatever you read in, right?


%%%%%%%%

Another hint from Lou:

	As Stefan says, += doesn't work in all shells. I'm probably telling you
something you know, but the standard idiom looks like this:

OSLIB_LIBS="$LIBS $OSLIB_LIBS"

to prepend. Reverse the order to append. I'm a serial offender on this;
I regularly forget that I need to add to a variable without trashing the
existing value.

						Lou
%%%%%%%%%%%%%%%%%

INFO on Externals

        You'll want to migrate towards Dependencies and the use of the
set_externals script to set svn:externals.  Documentation has definitely fallen
behind.  I guess what you're seeing here is bits of the new order --- the
independent (split) configuration stuff that Stefan's been implementing ---
starting to leak out.  I used the revised scripts described below to generate
the releases you were questioning.

        You can certainly keep on using the Externals file and
BuildTools/stable/0.6 for a while yet.  BuildTools/trunk should work on the
classic configuration with externals, so you should be able to use it on OS in
the classic packaging.  BuildTools/trunk is essential if you're going to
experiment with the split configuration (i.e., each project checked out
independently, without externals).

        There are major changes in the [prepare,commit]_new_[stable,release]
scripts in BuildTools/trunk.  (In fact, *_new_stable do not exist anywhere
except BuildTools/trunk.)  Also for run_autotools.  They are much more flexible
in terms of where they live and how they are run.  In the split configuration,
there's no guarantee that BuildTools will be present in the classic external
position.  The revised *_new_* scripts behave like normal command scripts.  You
can, for instance, copy them into some directory that's already in your command
search path and execute them from there.  Run_autotools is a bit more limited
due to its need for a number of autotools scripts that we distribute in
BuildTools. The revised scripts have long usage messages; they will help you.

        Prepare_new_stable will look to see whether you've supplied Externals or
Dependencies in your trunk.  If you've created Dependencies, it will use it.  If
all you have is Externals, it will generate Dependencies from Externals and add
it to the stable candidate.

        I'm not sure how much will be useful to you, but I've attached some
scripts below that I use to maintain a split configuration (trunks).  Create a
directory (e.g., Split) where you want the checkout to live.  Start with
getSplit, which will do checkout, update, or status.  When it finishes, you'll
have separate subdirectories for BuildTools, CoinUtils, Osi, etc.  If you want
to run the autotools, just type BuildTools/run_autotools in the Split directory.

        ConfigSplit will run configure, make, and make install (install is a
necessary step in a split configuration).  The first 2/3 of the script are of
marginal utility unless you adopt my stylised naming convention for build
directories. But the final third shows how to run make and make install.

	(Stylised naming convention: Suppose you checked out the split 
configuration in <path>/Split. The script expects the build directory to be
	named <path>/Split-<OS><compiler><bits><opt>
For example, Split-FedGCC64Opt is Fedora, GCC, 64-bit, optimised. Check the 
script to see what it recognises and adapt accordingly.)

	getThirdParty will haul down third-party software into the appropriate
ThirdParty subdirectories.

	testSplit will run `make test'
	
This is probably way more than you wanted to know on Saturday morning   :-)   But 
I'm on a flex week, today's a normal workday for me.
	
							Lou
%%% 


Stefan on LIBS

XXXLIB_LIBS is what you should use in your Makefile.am, XXXLIB_PCLIBS is
what you should use in the .pc.in file to setup the Libs entry.
The thing is that XXXLIB_LIBS hold all flags needed to link against
XXXLIB, but for the .pc file you want to distinguish between
dependencies that have a .pc file and those where you just know the
linking flags. So these two informations are stored in XXXLIB_PCREQUIRED
and XXXLIB_PCLIBS.


%%%%%%%%%%%%

Ralphs on naming convention

First of all, if it's working, then the names are correct  :) . Don't
overthink it  :) . The names are whatever *you* choose them to be for
your individual project. Whatever name you give to the corresponding
package in *your* configure.ac is how those variables get named. Since
you have the line

AC_COIN_CHECK_PACKAGE(Clp, [osi-clp]. [OSLib])

in your configure.ac, the variable for Clp libraries will be CLP_LIBS.
If you change the first argument of AC_COIN_CHECK_PACKAGE to "ClpLib,"
then you would use CLPLIB_LIBS. It doesn't matter what any other
project does. You choose your names within your project, though it
would probably be nice if we did it all similarly to avoid too much
confusion. At the moments, I believe we are mostly similar, but the
reason project add "LIB" onto the names internally is to distinguish
between dependencies for the library versus the executable. Since you
don't have to make that distinction (you are only using the library),
you don't need the redundant "LIB" added to the name.

Cheers,

From Stefan, December 5, 2010

The macro figures out all the compiler and linker flags necessary to
link to OsiClp and puts these into the variables CLP_CFLAGS and
CLP_LIBS. Further, it also adds them to OSLIB_CFLAGS and OSLIB_LIBS.
So the series of AC_COIN_CHECK_PACKAGE's in your configure.ac should
accumulate the compiler and linker flags for all dependencies of OS.

Form the X_LIBS variable, the macro creates then a X_DEPENDENCIES
variable, which is the same as X_LIBS, except that everything starting
with a '-' is removed. So i you had X_LIBS="libCoinUtils.la -lm", then
X_DEPENDENCIES will be just "libCoinUtils.la".

In the old system, this happened automatically. When automake saw
"libCoinUtils.la -lm" in an LDADD variable, then it created a
DEPENDENCIES variable which had only "libCoinUtils.la". However, in the
new system, we mostly list variables in LDADD (e.g., $(OSLIB_LIBS)), and
automake cannot convert this into a DEPENDENCIES variable (since at the
time automake is run, it does not know what $(OSLIB_LIBS) will later
be). So one has to set DEPENDENCIES by hand now.

To get the DEPENDENCIES working, a change in the xxx-uninstalled.pc file
is necessary. It needs to list the .la file with full path, not using
the -L/-l syntax . This change is only in trunk yet, so if you use
stable's in your externals, then, e.g., a rebuild of libClp does not
trigger automatically a rebuild of OS, yet.

> > It now seems that make 

%%%%%%%%%%%%%%%%
More on Externals -- I think I understand -- from around December 6 and 7 2010

Priority is what you see if you do
  svn propget svn:externals .

I think the new idea is that one don't have an Externals file anymore,
but only a Dependencies file. Usually, one lists all stable externals in
the Dependencies file and uses BuildTools/set_externals to set the
svn:externals property to the latest releases.
However, if one specifies trunk's in Dependencies, then set_externals
will also keep them as trunk's.

The svn:externals properties are part of the repository, so they are set
on the server. When svn does a checkout, it looks for the value of
svn:externals, and the checks out everything in there too.
The Externals or Dependencies file is just something COIN-OR specific. I
think the use of the Externals file is to make it easier to edit the
svn:externals property. I.e., you edit the file and then set the svn
property to the content of this file (svn propset svn:external . -F
Externals)

Here is the thing to do:


Okay, this really helps. So when things settle down, then I should:

1) trash the OS Externals file


2) set the Dependicies file to point to BuildTools stable.


3) run BuildTools/set_externals

4) do an svn commit

and that will then set the svn:externals properties on the server/repository. 

%%%%%%%%%%%
Stefan on examples

The only way that the examples work under cygwin/cl is if you disable
pkg-config. Examples with cygwin/cl/pkg-config does not work for any
project.
I also added documentation about the examples Makefile's at
https://projects.coin-or.org/BuildTools/wiki/pm-switch#RootXxxexamplesMakefile.in

Andreas noted to me today that the "ifeq" and "+=" that are used in the
examples makefiles now work only for GNU make  :-( . So for Ipopt we will
do something different,
https://projects.coin-or.org/Ipopt/browser/trunk/Ipopt/examples/Cpp_example/Makefile.in?rev=1875

Stefan

%%%%%%%%%%%%%%%%

Setting Externals the old way

Hmm, I haven't tried what happens if set_externals does not find a
release, maybe you just try  :-) .
Otherwise, I think just doing
  svn propset svn:externals -F Dependencies .
would be enough.



%%%%%%%%%%%%%%%%%%%%%%

March 18 Information from Lou

    Looks like Ted, Stefan, and I haven't been good enough at communicating how we're handling externals these days. See
https://projects.coin-or.org/BuildTools/wiki/pm-svn-releases
Here's a summary.

    First, you have the svn:externals property. This is what svn uses when checking out externals. So the question of interest is `How to set svn:externals?'

    Start with Dependencies. In trunk, it can be anything you like. When you run create_new_stable, a fresh copy of Dependencies is created and committed with the new stable. References to trunk externals are replaced with references to stable externals. References to release externals are unchanged. To set svn:externals on the new stable, the set_externals script is used; references to stable externals are converted to releases.

    When you execute prepare_new_release on a stable branch, Dependencies is processed to find the most recent releases for the stable externals, the result is used to set svn:externals, and the release is created. Dependencies still specifies stable branches, svn:externals specifies releases.

    The prepare_new_stable and prepare_new_release scripts have lots of options to affect these conversions. See the help message for the scripts. You can always use svn propset if the scripts are not doing what you want.

    The intended result is that in a stable or release, the Dependencies file should specify stable externals. Specify a particular release only if you absolutely must. The set_externals script processes Dependencies, finds the most recent release for each stable external, and uses that to set svn:externals. The important point is that Dependencies will typically specify other stables, but svn:externals will specify releases. This is a departure from the past, where the convention was that what you saw in Dependencies (and before that, Externals) was the same as svn:externals.

    Clear as mud?

                    Lou 


    When someone checks out OS/stable/2.3, they will get the set of externals that was frozen in svn:externals the last time that someone ran BuildTools/set_externals on the Dependencies file and committed the change to svn:externals. Dependencies doesn't change, only svn:externals, so it's kind of subtle. Once svn:externals is changed (locally or in the repository) the next svn update command will switch to the new externals.

    It's the responsibility of the project developers to run set_externals every so often to keep a stable up-to-date with respect to new releases. Generally this isn't a big issue.

    The problem with Osi (and any project that has an OsiXXX layer) is that there was a major rearrangement of the src directories between 0.103 and 0.104 (or thereabouts). All the OsiXXX code was moved out to project XXX, and the Osi src directory was reorganised. Then we got into a rush to push out a new CoinAll before INFORMS Austin. The net result is that the VS solution files in O.104.1 are wrong. They were not updated to reflect all the movement of source code. The VS solution files in 0.104.2 are correct. Similarly for Cbc (OsiCbc), Clp (OsiClp), DyLP (OsiDylp), Vol (OsiVol), and SYMPHONY (OsiSym). The OsiXXX source for thirdparty and commercial solvers remains in the Osi project.

    This was a clear screw-up on the part of Ted, Stefan, and I, and I can only plead time pressure. Ted and Stefan don't work with VS, and I work in trunk (unless someone complains and forces me to go look at a stable :-) . By the time this problem occurred to me, it was too late to do anything except resolve to fix it in the next release. We knew it was wrong as it was going out the door.

    For what it's worth, I don't agree with reducing stable externals to releases for a stable, but I seem to be in the minority. I use trunk externals in trunk and would prefer using stable externals in stable, to avoid just this sort of problem. I figure anyone who uses a stable can probably cope with the stream of changes coming over from trunk. But I've never been successful in arguing this when the topic comes up in the TLC.

                        Lou


