Re: GSoC ShlibMemLoad

> > Some leading questions ...
> >
> > You are using 'printf' in rtld itself to print log/debug messages, and
> > it works there. What is the address of this 'printf' function ?
> >
> > Is the address of the 'printf' function seen by the loaded
> > test-library the same ? (**)
> >
> > If not, why?

> These addresses differ. I am not sure why, but I think that printf in
> log/debug messages uses printf from libc.so which was loaded at the
> beginning of rtld execution by the regular dynamic loader and printf
> in libprintf.so uses printf from libc.so loaded by
> load_needed_objects.

Yes, they differed for me as well, and I am drawing the same conclusion.

> > Do you remember my comments about missing symbols ?
> >
> >        I.e.: The list of libraries we have in our rtld is the list of
> >        memory-loaded libraries (ML/lib) and not of all libraries
> >        (ALL/lib).
> >
> >        And a symbol not found in ML/lib has to be looked for in
> >        ALL/lib too ?
> >
> > What does that same distinction mean for 'load_needed_objects' ?

> I think that load_needed_objects does not know anything about ALL/lib

Yes, I agree here as well. However, I further also believe that
load_needed_objects() will have to be changed to be made aware of
ALL/lib in some kind to prevent this duplicate loading.

I.e. let it first search for the library by itself (*), and when that
fails let it fall back to dlopen() to load the needed object from the
outside. dlopen() will then consult ALL/lib. The main point here is
that the first search should fail to locate libc.so (*), and then
gotten through dlopen(), thus ALL/lib. That way we should be able to
avoid loading it a second time.

And the changes made to find_symdef() or other (the fallback to
dlsym()) should then take care of handling the resolution of symbols
in libc.so for us.

(*) This implies changes to find_library(). In our use case, loading
    from memory, we wish to search virtual filesystems. So we need an
    API for Tcl to define search paths, which will be inside of the
    starkit or starpack, and all the existing code using the LD_*
    and other environment variables we should be able to discard.

> and if our library needs another one it is loaded for the second time.
> Anyway, all libraries I have tested needed just libc.so.

True. And most C packages for Tcl will be similar. Even so, there will
be packages which depend on more shared libraries (Example: TclDOM has
a C-based variant depending on libxml2.so).

> > And (especially) a library like 'libc.so' which is loaded by the OS
> > and thus in ALL/lib?
> >
> >
> > (**) In (one of) the other test libraries you are using a for-loop and
> >     write to print the pid. If you encapsulate that into a function
> >     FOO to generally print unsigned numbers and then put FOO into the
> >     libprintf code you can print any number you need, like function
> >     addresses.  Inside of rtld itself it is of course possible to use
> >     the working printf to print its address.

> Because I have already studied this whole code for quite a long time
> and have not came up with any solution I am thinking of writing an
> email to John Polstra who is an author of the rtld for
> FreeBSD. Maybe he will be able to help out. Do you think it is a
> good idea?

Yes. Getting the help of (one of) the original authors of some code is
(in my opinion) a good idea. He should know the intricacies of the
code.

> Today I also wrote a simple program which generates defs.h file.

That note I did not understand. Can you explain a bit more?

--

-- 
So long,
	Andreas Kupries <akupries@...>
			<http://www.purl.org/NET/akupries/>
	Developer @	<http://www.activestate.com/>
-------------------------------------------------------------------------------

-------------------------------------------------------------------------
Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW!
Studies have shown that voting for your favorite open source project,
along with a healthy diet, reduces your potential for chronic lameness
and boredom. Vote Now at http://www.sourceforge.net/community/cca08