I sympathize with the problem. In addition, although I am not a lawyer, it
is my understanding that Docker’s license is incompatible with Slurm’s
GPL, and thus you cannot distribute such an integration.
FWIW, I’m just starting on my 2nd “pre-retirement” project (PMIx
being the first and ongoing one) to build an open source HPC container
(under the 3-clause BSD license) that will run at user level, provide
bare-metal (QoS managed) access to the OS-bypass fabric, provide direct
injection of user applications, and function ship access to the file
system. I expect to setup a public Github for it in the next week or so,
and hopefully have at least a start in time for SC15.
Anyone interested can drop me a line off-list (rhc at open-mpi.org <http://open-mpi.org/>) and I’ll notify
you when I get things setup. I’m more than happy to have other interested
parties collaborate on it!
> On May 31, 2015, at 5:27 PM, Christopher Samuel
> On 21/05/15 00:38, Michael Jennings wrote:
>> At the risk of further putting words in Chris' mouth (which I risk
>> doing only because I know he'll forgive me if I get it wrong, and it
>> will help him out if I get it right), I'll say what the two of us are
>> asking for is if anyone has a working implementation of running jobs
>> under SLURM which execute inside a Docker container (or similar
>> container technology), and if so, how you wound up choosing to do it!
> Sorry for being absent for a while after starting this thread, pressures
> of work.
> Michael hit the nail on the head for me there.
> The security side of things is an issue, though I'm not sure how much
> the fact that the program is running in a separate UID namespace helps,
> presumably if you've got to give it HPC filesystem access then the
> answer is probably "not at all".
> One of my concerns has always been that as these images age without
> updates then their exposure to known security bugs increases.
> That seems to be born out by this recent survey:
> # Over 30% of Official Images in Docker Hub Contain High Priority
> # Security Vulnerabilities
> # [...] Surprisingly, we found that more than 30% of images in
> # official repositories are highly susceptible to a variety of
> # security attacks (e.g., Shellshock, Heartbleed, Poodle, etc.).
> # For general images – images pushed by docker users, but not
> # explicitly verified by any authority – this number jumps up
> # to ~40% with a sampling error bound of 3%. [...]
> If anything that puts me off liking them even more. :-(
> All the best,
> Christopher Samuel Senior Systems Administrator
> VLSCI - Victorian Life Sciences Computation Initiative
> Email: [email protected] Phone: +61 (0)3 903
> http://www.vlsci.org.au/ http://twitter.com/vlsci