In this example, we will use the resource object to keep track of file locations¶
from pprint import pprint
import streamm import os
local = streamm.Resource('local')
local.dir['templates'] = os.path.join(local.dir['home'],'..','templates','')
This reads in the current directory as the root directory for a project
These directories can be created using the make_dir() function
Calculation object can use the location of these
directories and files within them to copy files to the correct locations
local_json = local.export_json()
We can also setup a resource we can access using ssh calls
remote = streamm.Resource('remote')
Set the type to
ssh, this will trigger some if statements in the
calculation object to scp calculation files to the external resource.
remote.meta['type'] = "ssh"
Enter your username and the address of the resource
remote.ssh['username'] = 'my_username' remote.ssh['address'] = 'system_address'
Then add the direcotry structure to the
dir dictionary of the
remote.dir['storage'] = '/storage/%s'%(remote.ssh['username']) remote.dir['scratch'] = '/scratch/%s'%(remote.ssh['username']) remote.dir['home'] = local.dir['home'] remote.dir['launch'] = local.dir['launch'] remote.dir['templates'] = local.dir['templates']
If you are running on a remote resource you can decide whether to set
the properties[‘exe_command’] to
qsub to submit the calculation to a
queuing system or
./ to run the calculation on the same node the
script is running on.
remote.properties['exe_command'] = 'qsub '
ssh_json = remote.export_json()
If you are running a streamm script on a remote resource you will want
to set the type to
remote.meta['type'] = "local"
Also, you will want to set the launch directory to the scratch location
remote.dir['launch'] = remote.dir['scratch']
This for the other examples like
P3HT_ET were input files are copied
launch directory rather than the directly to the scratch
directory in case the example is accessing a remote resource via