Home
Reading
Searching
Subscribe
Sponsors
Statistics
Posting
Contact
Spam
Lists
Links
About
Hosting
Filtering
Features Download
Marketing
Archives
FAQ
Blog
 
Gmane
From: Fields, Christopher J <cjfields <at> illinois.edu>
Subject: Re: question temp files in blast
Newsgroups: gmane.comp.lang.perl.bio.general
Date: Wednesday 11th December 2013 14:00:09 UTC (over 2 years ago)
I think File::Temp generates the random file string based on the time stamp
(common practice in UNIX), which rounds to the second.  Might be wrong, but
that could be causing the problem, as files could be created at the same
time in threads/forks. See this link, which also discusses solutions:

https://metacpan.org/pod/File::Temp#Forking
 

chris

On Dec 11, 2013, at 6:57 AM, Francisco J. Ossandón
 wrote:

> Hello Dimitar, 
> You expect to have several instances of the script running at the same
> time??
> 
> If there is only 1 instance for the script, it could be easier to assign
an
> increasing counter for the smaller fastas (seq1.fa, seq2.fa... seqX.fa),
and
> then use the fasta filename as base for the blast output filename
> (seq1.blastout.txt, seq2.blastout.txt... seqX.blastout.txt).
> 
> If there are multiple instances, you could add to the filename the
original
> fasta name and the 'time' function return value (I think it would be
> unlikely to process 2 files with the same name and starting at the same
> time). Something like:
> 
> my $in_file = 'original.fa';
> my $time = time;
> my $counter = 0;
> foreach my $fasta_piece (@fasta_pieces) {
> 	$counter++;
> 	my ($file_out) = ($file_in =~ m/^(.+)\.fa$/i);
> 	$file_out = ".$time.seq$counter.fa"; # Resulting in 'original.
> 1386766006.seq1.fa'
> 
> 	my ($blast_result) = ($file_out =~ m/^(.+)\.fa$/i);
> 	$blast_result .= '.blast_out.txt'; # Resulting in 'original.
> 1386766006.seq1.blast_out.txt'
> }
> 
> That would add some specificity (temporal files with same base name) and
> some randomness (counter and execution time). The filenames can be a
little
> long but I like it because all files are grouped by their base name, so I
> can list/copy/move/delete them together.
> 
> Or maybe that's not enough for you needs??
> 
> Cheers,
> 
> Francisco J. Ossandon
> 
> -----Mensaje original-----
> De: [email protected]
> [mailto:[email protected]] En nombre de
> [email protected]
> Enviado el: martes, 10 de diciembre de 2013 22:54
> Para: [email protected]
> Asunto: [Bioperl-l] question temp files in blast
> 
> Hi guys,
> i have a question about StandAloneBlastPlus and File::Temp.
> 
> I encountered a problem which arises from File::Temp in my particular
> script. In previous email i said i forced StandAloneBLastPLus to accept a
> TEMP_DIR which i give through modifying BlastMethods.pm and
> StandAloneBlastPlus.pm. This works but not always and that is because
> File::Temp is using the built in perl function rand() which uses srand().
> 
> Now in brief: my script is splitting a large FASTA into smaller ones and
for
> each of the smaller ones is starting a new thread of BLAST with as many
> threads as desired. Also is creating a special TEMP_DIR for each thread
in
> which the temp blast files are stored: file.fas and the blast_result.
> However because of the rand() some clashing of file names occurs because
> there is not enough randomness and some of my threads die, not always but
> very often.
> 
> So my question is the following. Should i try to modify BlastMethods.pm
and
> StandAloneBlastPlus.pm further so that i can manually specify the file
names
> of the temp files or to use another module like  Math::Random::Secure in
> order to produce a really random number which i can then pass to srand()
> after i create my threads so that there is no temp file names clashing?
> 
> The easiest is to just use additional module but then more dependencies
just
> for one random number. On the other hand if i modify the current modules
i
> will be sure that there wont be a chance to have temp file name clashing
at
> all and no further dependencies.
> 
> I am sorry if my email seems too messy but i tried to put it really
brief.
> 
> Any advice is welcomed!
> 
> Thank you for your time
> 
> Cheers
> Dimitar
> 
> 
> _______________________________________________
> Bioperl-l mailing list
> [email protected]
> http://lists.open-bio.org/mailman/listinfo/bioperl-l
> 
> _______________________________________________
> Bioperl-l mailing list
> [email protected]
> http://lists.open-bio.org/mailman/listinfo/bioperl-l
 
CD: 3ms