Re: [Lumiera-work] INFRASTRUCTURE build and test system on the server

Michael [Plouj] Ploujnikov wrote:
> Christian and I already started playing around with the builddrone
> implementations
> (http://git.lumiera.org/gitweb?p=builddrone/ct;a=summary and
> http://git.lumiera.org/gitweb?p=builddrone/plouj;a=summary).
> 
> I was trying to think how to implement a usable script for this
> system, but I ran into a few questions that I can't answer myself:
> 1. How do we organize or track relationships between functions in the
> soup directories which refer one to another. For example, both GitPull
> and Doxygen might use the AddToReport function. Should the AddToReport
> definition be duplicated in the files for both of the other files? How
> will Bash handle sourcing the same function twice? What happens if the
> two versions of AddToReport start to differ? Should we put AddToReport
> in a separate file which is sourced by the files where GitPull and
> Doxygen functions are defined?

Generally this soup shall only define functions and rather not execute 
anything else (it would be possible but can't rely on other functions 
then). Further, this all is very sketchy and has some rough edges which 
need to thought about. Thats more a design and definition issue.

For example: each scriptlet shall log its output in its own log 
(,git.log, ,doxygen.log, ...) actually we might even define that all 
this logs have some well defined format. My suggestion would be asciidoc 
which in the simplest case just places all in a listing block:
echo -e ".Git Pull Log\n-----" >,git.log
echo -e ".Git Pull Errors\n-----" >,git.err
git pull 1>>,git.log 2>>,git.err
echo "-----" >,git.log
echo "-----" >,git.err

then this sciptlet announces the logs it just created
BUILDDRONE_CURRENT_LOG=,git.log
BUILDDRONE_CURRENT_ERR=,git.err

and a AddToReport picks them up, Note that this vars have to be defined 
in the outer environment, while the actual command might be run in a 
subshell or even in a batch job, if necessary this commands should 
export its environment into a file too where it is picked up by the 
master process.

(of course in real we want some instance identifier in the filename 
,git${BUIDDRONE_INSTANCE}.log and you have to store the exitcode 
somewhere too)

> 
> 2. Is it a good idea to have a limit of one function definition per file?
One topic, often this topics are just one function, doxygen for example. 
But for example its feasible to place all git actions into one file. Or 
all scons or automake related jobs can be defined in a scons or 
autotools file.

> 
> 3. How do we pass various functions to the BatchJob function (as
> written on the wiki)? I am certain that we will want to do stuff like:
> 
> BatchJob Make
> BatchJob Doxygen
> BatchJob MakeCheck
> 
> This would require some sort of an ability to pass functions. Too bad
> we can't do echo the function code like: "echo $MakeCheck | BatchJob"!
> It would work since BatchJob currently receives the commands to run on
> its stdin.

i'd opt for:

BatchJob <<EOF
   Make
EOF

but we can support both
if [[ $# -gt 1 ]]; then
	/use args/
else
  	/use stdin/
fi

'batch' itself reads from stdin, so you have to construct a script 
there. I already sketched that in the wiki, first you have to setup the 
functions (source the builddrone script, this needs to be changed that 
it doesnt produces usage when no arguments are given) and environment:

batch <<EOF
source builddrone
BUILDDRONE_INSTANCE=$BUIDDRONE_INSTANCE

/here the real command which shall be executed/

/finally some cleanup and notify the caller that we are finished/
EOF

> 4. How do we or the user of builddrone define what directory a
> specific project script is run? How do we make sure that all functions
> are actually run in that directory? Do we set a custom env. variable,
> which all the functions are expected to obey? Do we just CD into the
> desired directory prior to executing (sourcing) the functions in a
> given project script?
It's up to us to define the rule, i'd keep it open as you seen in my 
latest commits, the directory gets recorded in a variable and we cd into 
it before we start work. I's encourage for actions to use subshells not 
to alter the gobal environment (except for what they must alter)

	Christian