.. _aws:
===========================================
Amazon Web Services EC2 AMI [2014 version]
===========================================
We are using a wide variety of software in this class, much of which is
probably not found on your computer. It is all open source software (see
:ref:`licences`) and links/instructions
can be found in the section :ref:`software_installation`.
You can also use the :ref:`vm`.
Another alternative is to write and run your programs "in the cloud"
using Amazon Web Services (AWS) Elastic Cloud Computing (EC2).
You can start up an "instance" (your own private computer, or so it appears)
that is configured using an Amazon Machine Image (AMI) that has been
configured with the Linux operating system and containing
all the software needed for this class.
You must first sign up for an account on the `AWS main page
`_. For this you will need a credit
card, but note that with an account you can get 750 hours per month of
free "micro instance" usage in the
`free usage tier `_.
A micro instance is a single processor (that you will probably be sharing
with others) so it's not suitable for trying out parallel computing, but
should be just fine for much of the programming work in this class.
You can start up more powerful instances with 2 or more processors for a cost
starting at less than 3 cents per hour (the m3.large on-demand
instance). See the `pricing guide `_.
For general information and guides to getting started:
* `Getting started with EC2 `_,
with tutorial to lead you through an example.
* `EC2 FAQ `_.
* `Pricing `_. Note: you are charged
per hour for hours (or fraction thereof) that your instance is in
`running` mode, regardless of whether the CPU is being used.
* `High Performance Computing on AWS `_
with instructions on starting a cluster instance.
* `UW eScience information on AWS `_.
Launching an instance with the *uwhpsc* AMI
-------------------------------------------
Quick way
^^^^^^^^^
Navigate your browser to
``_
You should then be on a page where you see you are on Step 2 of 7 at the top
of the page, "Choose instance type".
Then you can skip the next section and proceed to :ref:`aws_instance_type`.
Search for AMI
^^^^^^^^^^^^^^
**Skip this section** if you followed the "quick way" instructions above.
Going through this part may be useful if you want to see how to search for
other AMI's in the future.
Once you have an AWS account, sign in to the
`management console `_
and click on the
EC2 tab, and then select Region US West (Oregon) from the menu
at the top right of the page, next to your user name.
You should now be on the page
``_.
Click on the big "Launch Instance" button.
On the next page, you will see a list of "Quick start"
Amazon Machine Images (AMIs) that
you can select from if you want to start with a fresh VM. For this class
you don't want any of these. Instead click on the "Community AMIs" tab and
then type `uwhpsc2014` in the search bar. Select this image.
You will then be taken to Step 2, "Choose instance type".
.. _aws_instance_type:
Choose instance type
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
On the next page you can select what sort of instance you wish to start (larger
instances cost more per hour). t1-micro is the the size you can run free (as
long as you only have one running).
Click `Continue` on the next few screens through the "instance details"
and eventually you get to one that
asks for a key pair. If you don't already have one, create a new one and
select it here.
You can now skip over steps 3-6 and jump directly to Step 7, "Review and
Launch".
Launch instance / create key pair
-----------------------------------
When you click on "Launch", you will get a page that asks you to "Select and
existing key pair or create a new one". If you don't already have a key
pair, select "Create a new pair" from the menu and follow instructions. If
you give the name `mykey`, for example, then this will download a file
`mykey.pem`. Store this file in a directory where you can find it again,
since you will need this key in order to log in to your instance once it is
running.
You also need to change the permissions on this file so that is readable only
by the account user. In the directory where this file lives, give the
command::
chmod 400 mykey.pem
If the file is more widely readable then you will not be able to use this
key to log into your instance.
Logging on to your instance
---------------------------
Click `View Instances` on the page that appears to
go back to the Management Console. Click on `Instances` on the left menu
and you should see a list of instance you
have created, in your case only one. If the status is not yet `running`
then wait until it is (click on the `Refresh` button if necessary).
*Click on the instance* and information about it should appear at the bottom
of the screen. Scroll down until you find the `Public DNS` information
Go into the directory where your key pair is stored, in a file with a name
like `mykey.pem` and you should be able to `ssh` into your instance using
the name of the public DNS, with format like::
$ ssh -Y -i KEYPAIR-FILE ubuntu@DNS
where KEYPAIR-FILE and DNS must be replaced by the appropriate
things, e.g. something like this::
$ ssh -Y -i mykey.pem ubuntu@ec2-50-19-75-229.compute-1.amazonaws.com
Note:
* You must include `-i keypair-file`
* You must log in as user ubuntu.
* Including -Y in the ssh command allows X window forwarding, so that if you
give a command that opens a new window (e.g. plotting in Python) it will
appear on your local screen.
* See the section :ref:`ssh` for tips if you are using a Mac or Windows
machine.
If you use Windows, see also the Amazon notes on using *putty* found at
``_.
Once you have logged into your instance, you are on Ubuntu Linux that has
software needed for this class pre-installed. See the file `install.sh` in
the running instance to see the commands that were used to install software.
Other software is easily installed using `apt-get install`, as described
in :ref:`software_installation`.
Transferring files to/from your instance
----------------------------------------
You can use `scp` to transfer files between a running instance and
the computer on which the ssh key is stored.
From your computer (not from the instance)::
$ scp -i KEYPAIR-FILE FILE-TO-SEND ubuntu@DNS:REMOTE-DIRECTORY
where DNS is the public DNS of the instance and `REMOTE-DIRECTORY` is
the path (relative to home directory)
where you want the file to end up. You can leave off
`:REMOTE-DIRECTORY` if you want it to end up in your home directory.
Going the other way, you can download a file from your instance to
your own computer via::
$ scp -i KEYPAIR-FILE ubuntu@DNS:FILE-TO-GET .
to retrieve the file named `FILE-TO-GET` (which might include a path
relative to the home directory) into the current directory.
Stopping your instance
----------------------
Once you are done computing for the day, you will probably want to stop your
instance so you won't be charged while it's sitting idle. You can do this
by selecting the instance from the Management Console / Instances, and then
select `Stop` from the `Instance Actions` menu.
You can restart it later and it will be in the same state you left it in.
But note that it will probably have a new Public DNS!
Creating your own AMI
---------------------
If you add additional software and want to save a disk image of your
improved virtual machine (e.g. in order to launch additional images in the
future to run multiple jobs at once), simply click on `Create Image (EBS
AMI)` from the `Instance Actions` menu.
Viewing webpages directly from your instance
--------------------------------------------
An apache webserver should already be running in your instance,
but to allow people (including yourself) to view
webpages you will need to adjust the security settings. Go back to the
Management Console and click on `Security Groups` on the left menu. Select
`launch-wizard-1` and then click on `Inbound`. Click on `+Add rule`.
You should see a list of ports
that only lists 22 (SSH). You want to add port 80 (HTTP). Select HTTP from
the drop-down menu that says `Custom TCP Rule` and then click on `+Add rule`
and `Apply Rule Change`.
Now you should be able to point your browser to `http://DNS` where `DNS` is
replaced by the Public DNS name of your instance, the same as used for the
`ssh` command. So for the example above, this would be ::
http://ec2-50-19-75-229.compute-1.amazonaws.com
The page being displayed can be found in `/var/www/index.html` on your
instance. Any files you want to be visible on the web should be in
`/var/www`, or it is sufficient to have a link from this directory to where
they are located (created with the `ln -s` command in linux).
So, for example, you could do the following (this has already been done if
you start with the uwhpsc2104 AMI)::
$ cd $HOME
$ mkdir public # create a directory for posting things
$ chmod 755 public # make it readable by others
$ sudo ln -s $HOME/public /var/www/public
then you can see the contents of your $HOME/public directory at::
http://ec2-50-19-75-229.compute-1.amazonaws.com/public
Remember to change the DNS above to the right thing for your own instance!