You can set up a Pipeline server on Amazon EC2 within minutes for free. You can sign up Amazon AWS free usage tier and follow the video and instructions below. For captions/subtitles, click top right corner to “CC is on“.
Please note, the Pipeline Server AMI does not include all the tools in its server library. You can install additional tools using the DPS installer, or by yourself (install tools on server, create workflows, copy workflows to the server library location /usr/pipeline/serverLibrary/).
Step 1 – Create an EC2 Instance.
Go to aws.amazon.com, log in to your account, go to management console, select EC2, and click Launch Instance.
Use the Classic Wizard, click on the Community AMIs tab. We already set up a Pipeline server and made a snapshot of Amazon Machine Image. All you have to do is to create a new instance based on that AMI. It takes a minute or so to load this page.
After the page is loaded, type keyword ‘loni pipeline’, and select LONI Pipeline Server (v5.7) (last updated May 17, 2013). Leave everything by default in the following pages.
If it’s the first time you launch an instance, you need to create a new key pair for SSH authentication. Give a name and click Create and Download your key pair. Suppose you downloaded it as
~/mykey.pem (Please make sure to back up this file, as you cannot download it again from Amazon.)
Next we need to create a new security group. Security group allows you to open specific ports for connections.
We give a group name and description. For the Pipeline server, you need SSH (TCP port 22), as well as TCP port 8001. The following screen shows you the summary of your instance, when you click Launch, an instance will be started.
Go to your Instances page to verify the instance. You can also get the hostname of that instance. Suppose it’s
Step 2 – Configure Server
Open terminal, make sure the certificate you downloaded has the proper permissions:
$ chmod 600 ~/mykey.pem
Then SSH to your EC2 instance using your certificate:
$ ssh -i ~/mykey.pem email@example.com
Check the README.txt which contains steps to start your Pipeline server. You need first to update the server address (you only need to do it once every time the server’s hostname is changed):
$ ./updateServer.sh ec2-184-73-1-1.compute-1.amazonaws.com
And then start the server:
If you’d like to change some server preferences, or add some files to the server library, please note the Pipeline server files are located in /usr/pipeline/ and is owned by user ‘pipeline’. The user ‘ec2-user’ has sudo privileges. To run a command as pipeline user, run
sudo sudo -u pipeline command
Step 3 – Testing
Now you have the server running, you can verify by making a connection.
Open your client and create a new connection, put the server hostname ‘
ec2-184-73-1-1.compute-1.amazonaws.com‘ on the Server Address, and put any username. This testing server allows any username with any password to login. After connecting to the server, you can open a workflow under Server Library and run.
To further configure the Pipeline server, refer to Server Guide – Configuration section.
You can check Distributed Pipeline Server installer to update and install additional tools/packages.
You can also create your own image snapshot on Amazon and share it with others. Please check Amazon’s documentation on Creating Your Own AMIs for more information.