BOINCatTACC | TACC project : provides a conduit | GPU library

Install Docker and Docker-compose in their most recent versions. Follow the installation instructions provided by boinc-server-docker. Wait a few seconds until the make_project container sets up the BOINC server and its mysql database This process is complete when a webpage is accessible on SERVER_IP/boincserver. Setup Reef in an external container.
Install Docker and Docker-compose in their most recent versions Note: Both docker and docker-compose require sudo access, to avoid this problem:
Follow the installation instructions provided by boinc-server-docker Most likely, this server will be installed on a cloud server. If that is the case, do not compose up directly. It is necessary to specify the IP of the system using the syntax below. If not using root, it is possible that an error appears about Docker not being able to connect to the daemon. If this is the case, use sudo to obtain root access, and use sudo for all docker and/or docker-compose commands.
Wait a few seconds until the make_project container sets up the BOINC server and its mysql database This process is complete when a webpage is accessible on SERVER_IP/boincserver
Set-up basic administrative credentials Execute docker ps do obtain all the data about the BOINC containers. If the setup is done correctly, there is an apache container. Log into the apache container by using docker exec -it {apache container ID} bash Follow the guide in BOINC HTMLOps to set -up an administrative account with .htpasswd .htpasswd is located in /home/boincadm/project/html/ops/ Login into the administrative BOINC page at SERVER_IP/boincserver_ops with the username and password provided above
Clone this repository If git is not installed, do it by executing: apt-get update && apt-get install git-core -y If not already there, cd /home/boincadm/project Clone via: git clone https://github.com/ritua2/BOINCatTACC cd BOINCatTACC
Establish user email credentials Run the appropriate bash script to set up the email and password The basic set-up only allows for gmail, for all others, modify both ./api/preprocessing.py and ./email_assimilator.py Be sure to have input the correct name and password, to change them again, change /root/.bashrc The same applies to the Docker, MIDAS credentials - MIDAS is a software component for creating docker images from source-code Run the external Reef set-up to connect to an external container. The process is entirely automated and designed so that user files are stored in a different system. Do:
Setup Reef in an external container The following instructions are for normal Reef setup. If there is a need for scalable storage please refer to the steps listed under Distributed Reef. Reef is the storage container where all user files (results or not) are stored Switch to the pocket-reef directory inside the BOINCatTACC directory after exiting the Docker container, and follow some of these instructions on a system with the above IP and password provided above Note: Reef can be setup on any server as long as its IP is accessible from the main server
Distributed Reef is scalable version of Reef and is setup in two stages where the first stage is to setup reef manager node on the boinc VM itself.
The second stage is to setup the storage node(s) on remote VM(s)
Note: Distributed Reef can be setup on any server as long as its IP is accessible from the main server
Run the setup file from the Apache container It will install all the necessary packages, python libraries, set-up the internal Redis database, properly locate the files, set-up the APIs, Reef cloud storage, and automatic job processing The set-up file will also automatically prompt to enter the credentials for the email. Use caution, since an error would require to manually fix the /root/.bashrc file
Create organization accounts A new token must be assigned to each new user (researcher who wants to submit BOINC jobs) There are 2 options on assigning tokens: Create one automatically through create_token.py (deprecated) Create organization accounts Organization accounts are preferred since they allow users to sign up in an easier way Furthermore, users signing through this way will not have to provide further email authorization To create an organization, following the instructions below and enter inputs accordingly An automatic Reef account will be created for each user signing up through this method Organizations themselves are required to submit jobs using the Stampede2 backend Only one organization is needed. However, more can be created in order to separate and differentiate users For more instructions on supplying tokens through organizations, check the API documentation Always use organization name 'TACC' in order to avoid issues with the front-end To do this:
Assign VolCon credentials VolCon is a distributed way of running jobs on the cloud based on docker, without requiring the BOINC client or VirtualBox Although it can run on any system with an unique IP address (such as internal networks accessible from the main BOINC server), it is primarily designed for cloud servers By default, the main BOINC server does not provide VolCon and so any jobs submitted to it will never be run To create a VolCon network, run the following steps (bash commands below): Assign VolCon credentials: provide VolCon password Start VolCon network: In order for VolCon to be more effective, a large set of working nodes is required. Here will show how to manually run a simple mirror container. Assign Runner credentials: Multiple types of VolCon runners can be used at the same type (GPUs or not, different providers, etc). All runners within the same provided organization type are assumed to be similar and so they can be requested in such a way if the user wishes to. Implement VolCon runners by organization: The example here must be run within a VM, can be run on multiple machines without any changes. Note: VolCon is an update over the previous ADTD-P system and, although both share many features, VolCon is not backwards compatible
Disconnecting the APIs, including Reef - NOTE FROM RITU - THESE COMMANDS ARE NOT CURRENTLY WORKING AS EXPECTED: Disconnecting the APIs will not delete any files currently saved in Reef
Pulling the APIs up again:
I just submitted a job and it does not appear in the results ops page boinc-docker-server needs to download the images before any further processing is done. For large images (> 1 Gb) this process can take time if the server has a low amount of RAM.
I received a job with an empty compressed results file Try to obtain your results in a regular file, since those are the files that our file retrieval tool supports. In general, use the general outputs or texts files. Avoid images and binary files for results. Using plotting inside a BOINC job may cause problems, since most graphic libraries assume a screen.