Setting up the Hayabusa 2 Staging Server
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


=====================================
Staging Server Access
=====================================

Machine: hyb2sbmt.jhuapl.edu

Admin User: apluser  (has sudo access)
Password: hyb24ryugu

Content User: sbmt (can modify web content)
Password: hyabusa2!

Logging in from inside the firewall:

$ ssh apluser@hyb2sbmt.jhuapl.edu

or 

$ ssh sbmt@hyb2sbmt.jhuapl.edu


———————————————————————————---------
Create the cloud-based server
———————————————————————————---------

- From inside the firewall, in a web browser, type:

    myaplcloud/

- Click “Access My Cloud” button

- Enter “JHUAPL\<5-2-1>” for User Name and your APLNIS password

- Select “Catalog” tab

- Select “1-Intranet” sub tab

- Click on the “1I - Ubuntu Linux 16.04” button

- Fill out the machine name and any other parameters you’d like to modify of the virtual machine to be created

- Click the “Submit” button

- In a few minutes you will get an email with instructions on how to log on to your newly created VM

    Note that you will need to log on as “apluser” with the default password given, after which you should change it.
    The “apluser” does not have root access, but it does have sudo access, so you can do any admin tasks you need with it.

- Create a “sbmt” user account, using standard Ubuntu commands for creating user accounts

    (see https://www.digitalocean.com/community/tutorials/how-to-add-and-delete-users-on-an-ubuntu-14-04-vps)

    $ sudo adduser sbmt

———————————————————————————---------
Install 3rd-party software on server
———————————————————————————---------

- Install and set up the apache2 web server using standard Ubuntu commands

    (see https://www.digitalocean.com/community/tutorials/how-to-configure-the-apache-web-server-on-an-ubuntu-or-debian-vps)

    $ sudo apt-get update
    $ sudo apt-get install apache2

- From the apluser account, create a Virtual Host module for the sbmt web site

    $ cd /etc/apache2/sites-available/
    $ sudo cp 000-default.conf 001-sbmt.conf

- Edit the new virtual host module (001-sbmt-conf) to point to a new web server subdirectory:

    DocumentRoot /var/www/sbmt

- Replace the default module with the new one

    $ sudo a2ensite /etc/apache2/sites-available/001-sbmt.conf
    $ sudo a2dissite /etc/apache2/sites-available/000-default.conf
    $ sudo service apache2 reload

- Copy the default web server page to the new sbmt server directory and give sbmt access to it

    $ cd /var/www
    $ sudo cp -r html sbmt
    $ sudo chgrp sbmt sbmt
    $ sudo chmod g+w sbmt 
    $ sudo chown sbmt index.html 
    $ sudo chgrp sbmt index.html 
    $ sudo chmod g+w sbmt/index.html

- Modify the content of the sbmt server index.html page to show that it is the SBMT server page

    This can be done from the sbmt user account using any ascii text editor

- Bring up the web server in a browser to test it

    http://hyb2sbmt.jhuapl.edu/

- Enable htaccess control in the web server

    $ cd /etc/apache2
    $ sudo vi apache2.conf

- Make sure the .htaccess file name is set:

    AccessFileName .htaccess

- Set the sbmt directory to allow override:

    <Directory /var/www/>
       Options Indexes FollowSymLinks
       AllowOverride All
       Require all granted
    </Directory>

- Restart the web server

    sudo service apache2 restart

- From the sbmt user account, modify the content of the sbmt server index.html page to show that it is the SBMT deployment server page

    $ exit
    $ ssh sbmt@hyb2sbmt.u-aizu.ac.jp
    <enter sbmt user password>
    $ cd /var/www/sbmt
    $ vi index.html

    modify the index.html file HTML text to say “Small Body Mapping Tool - Hayabusa 2 Deployment Server”

- Bring up the web server http://hyb2sbmt.jhuapl.edu/ in a browser to verify it

———————————————————————————---------
Set up access control
———————————————————————————---------

- Create a new .htaccess file to control access

    $ cd /var/www/sbmt
    $ vi .htaccess

- Add the following contents to the .htaccess file (taken from the APL SBMT web server):

AuthUserFile /var/www/sbmt/.htpasswd-sbmt
AuthType Basic
AuthName "Please Enter SBMT Password"
Require valid-user
Order Allow,Deny
require user ernstcm1
require user peachjm1
require user zimmemi1
require user turnerj1
require user nguyel1 
require user steelrj1
require user hayabusa2
require user naru.hirata
Satisfy any

- Create a new .htpasswd-sbmt file

    $ vi .htpasswd-sbmt

- Add the following content to the .htpasswd-sbmt file (taken from the APL SBMT web server):

ernstcm1:VZvknzxEu.XGo
barnoos1:/IDi916LPUFsQ
naru.hirata:$apr1$f.LcEpkY$BxqH9EL4gsnYj5Fs3C.nj.
turnerj1:cb6KKiFtTB8E.
nguyel1:0sFjqKJ3n2a26
zimmemi1:$apr1$.nmOwGMk$hD54U3o.m8KbGCcG7gnPK.
steelrj1:$apr1$HdEL6Ma2$9QRMZ1t0n8Z9.Zpez5yIV1
hayabusa2:$apr1$4kuBOJt6$rSXX8t8d2mTxZJwDFNFJh/
peachjm1:$apr1$vYb4rTIN$AMOvdiTm9.avxR./Uxenj0

- In a web browser, try to access the http://hyb2sbmt.jhuapl.edu/ and verify that it prompts you for a password

- Enter your SBMT password and verify that you can see the server web page

- Create the sbmt data subdirectories

    $ mkdir sbmt
    $ cd sbmt
    $ mkdir data
    $ cd data

———————————————————————————---------
Upload simulated data to server for testing
———————————————————————————---------

- From the APL DMZ machine, zip up and scp the Earth and Ryugu data and copy to the Hayabusa2 staging server

    $ cd /project/nearsdc/data
    $ zip -r earth earth/hayabusa2
    $ zip -r ryugu.zip ryugu
    $ scp earth.zip sbmt@hyb2sbmt.jhuapl.edu:/var/www/sbmt/data (and enter the password)
    $ scp ryg.zip sbmt@hyb2sbmt.jhuapl.edu:/var/www/sbmt/data (and enter the password)

- From the Hayabusa2 staging server, as sbmt user, unzip the data files:

    $ cd /var/www/sbmt/data
    $ unzip earth.zip
    $ unzip ryugu.zip

- Test the Hayabusa2 Stage release of SBMT and verify that you can access the Earth and Ryugu Bodies

———————————————————————————---------
Install Image Database
———————————————————————————---------

- Set up the image database and confirm it is running

    $ sudo apt install mysql-server

    Specify a MySQL password. I am using the same password as the "apluser" password on the staging server.

    $ sudo netstat -tap | grep mysql

- You should see the following response:

    tcp        0      0 localhost:mysql         *:*                     LISTEN      22355/mysqld

- Install php 

    $ sudo apt install php
    $ sudo apt-get install libapache2-mod-php
    $ sudo systemctl restart mysql.service

- Install phpmyadmin

    $ sudo apt-get update
    $ sudo apt-get install phpmyadmin php-mbstring php-gettext

    In the installation screen that appears, you MUST select "apache2" for server by hitting the space bar
    (it is NOT selected by default) and select <Yes> to configure the database with dbconfig-common

    Specify the login password. I am using the same password as the "apluser" password on the staging server.

    $ sudo phpenmod mcrypt
    $ sudo phpenmod mbstring
    $ sudo systemctl restart apache2

    You can verify that phpMyAdmin is running by browsing to the phpMyAdmin web page:
    
      http://hyb2sbmt.jhuapl.edu/phpmyadmin

    and logging in as "root", using the same password as the "apluser" password (on the staging server).

- Create the database “sbmt” using phmMyAdmin

    Logged in to phpMyAdmin as root, select the “Databases” tab

    Enter “sbmt” for the database name and press the “Create” button

- Create additional users and give them full database privileges so they can populate the database

    Logged in to phpMyAdmin as root, select the “User Accounts” tab

    Click the “Add user account” link

    Enter the user name, password and re-type password. The password will be used by users to log
    in to phpMyAdmin as well as to populate the database with the GenerateDatabaseSql utility.

    Leave the Host name field as “Any host” (%)

    For the Global privileges field, check the “Check all” checkbox

    Press the “Go” button

- Create an “smbt” database user for search only purposes. This will be used by the PHP search script.
  When an image search is executed by the SBMT client.

    Logged in to phpMyAdmin as root, select the “User Accounts” tab

    Click the “Add user account” link

    Enter “sbmt” for the user name, give it the sbmt password and re-type password. The password will
    be used by the PHP script to connect to the database to do queries.

    Leave the Host name field as “Any host” (%)

    For the privileges section, only check the Data “select” and “file” checkboxes.

    Press the “Go” button


- Modify Apache web server to allow PHP access for queries

    $ sudo vi /etc/apache2/mods-enabled/dir.conf

    In dir.conf, add “index.php” to the DirectoryIndex list: 

      DirectoryIndex index.php index.html index.cgi index.pl index.xhtml index.htm

    $ sudo systemctl restart apache2

- Edit the MySQL configuration file to allow remote access for DatabaseGeneratorSql tool

    $ sudo mv /etc/mysql/my.cnf /etc/mysql/my-orig.cnf
    $ sudo vi /etc/mysql/my.cnf

    Add the following text to the my.cnf file:

    [mysql]
    user            = mysql
    socket          = /var/run/mysqld/mysqld.sock
    port            = 3306
    bind-address    = 10.109.29.242

    …where the bind-address value is the IP address of the server machine.

    Restart MySQL:

    $ sudo /etc/init.d/mysql restart

- Open up port on firewall if it is blocking access

    $ /sbin/iptables -A INPUT -i eth0 -p tcp --destination-port 3306 -j ACCEPT

- Modify the searchimages.php script to access the Hayabusa2 server database and upload to server

    This script has to be tailored for each SBMT server with the database name and credentials.
    Also, the internal version of the script uses an out-of-date PHP mysql access library, so newer
    SBMT servers need to use the new version. I have created subfolders in the SBMT repository
    under the the “misc/server/query” folder for each mission to store the PHP script for its
    server. Use the script in the “hayabusa2” subfolder for the Hayabusa2 servers.

    Copy the Hayabusa2 “searchimages.php” file into the /var/www/sbmt/sbmt/query directory on the
    Hayabusa2 server.

    For other servers than Hayabusa2, modify the $host and $password string variabls as appropriate.

———————————————————————————---------
Populate the database with image data
———————————————————————————---------

- Populate the database
    
    1) In Eclipse, switch to a current branch of sbmt and saavtk which has a DatabaseGeneratorSql
       that contains definitions for the test models

    2) Using your Hayabusa2-Stage server database credential (which I sent out earlier), log on to the
       Hayabusa2-Stage phpMyAdmin web page at:

       http://hyb2sbmt.jhuapl.edu/phpmyadmin

    3) Navigate to the "sbmt" database and verify that there are two tables in it:

       ryugucubes_gaskell
       ryuguimages_gaskell

    4) Populate the database

       - If you don't have a .sbmthyb2-stage directory, just run the Hayabusa2-Stage version of the
         SBMT client and quit it)
       - On your local computer, add a text file called "mysql-login.txt" to your home .sbmthyb2-stage
         directory containing the following two lines:

          your-database-user-id
          your-database-password

       - From Eclipse, run the "DatabaseGeneratorSQL - Hayabusa2-Stage" runconfig to create and populate
         the database tables

       - After the generator has finished running, go back to the phpMyAdmin site and verify that there
         are now two additional tables in the sbmt database with "_beta" suffixes

    5) Test the Hayabusa2-Stage client in beta mode

       - in Eclipse, edit the "SmallBodyMappingToolAPL - Hayabusa2-Stage" run configuration to add a
         "--beta" tag to the program arguments

       - Run the "SmallBodyMappingToolAPL - Hayabusa2-Stage" runconfig

       - Go to the Ryugu->H2 Simulated SPC body

       - Select the ONC image tab and do some searches for images

       - Verify that images are returned by the image search and that they load correctly