Host Your Own Cloud Using These 8 simple steps

This ultimate purpose of this tutorial is to explain how to host your own cloud server at home. This allows you to access your documents, pictures, videos, music, and other media from anywhere as long as you have an internet connection. This guide also serves as an update to my previous guide, which can be found here. My old guide was written almost two years ago and was in dire need of updating. As I have learned more about Linux, I have discovered new tricks and more efficient methods of hosting a cloud. The following list describes the major updates in this guide:

• Software: Ubuntu Server 14.04 64-bit
• Easier file transferring between server and client
• Fixed many permission issues
• Fixed some potential security issues
• Developed automated scripts to backup data to the server from the client
• Updated configuration steps for Apache2
• For those who read my previous cloud guide, I included a software called Ampache, which allowed music to be streamed from the cloud. I will not be including this software configuration initially in this guide. Ampache is currently undergoing major changes and I do not want to write the configuration before the changes are complete.
• Other minor fixes

Although there are many updates in this guide, the overall concept remains the same. By the end of this guide, you should be able to host a cloud from your home using free software and cheap hardware. Finally, I will attempt to explain everything with as much detail as possible.

Let’s get started with prerequisites.

1. Media you wish to upload to the cloud. The device the media is currently stored on will be called the client from now on.

2. A server. The server does not need to be particularly powerful unless you intend to have a huge amount of traffic accessing it. I find that old Dell computers work extremely well as cloud servers. The main parts in the server you need to worry about are the hard drive and memory. The hard drive will store the data on the cloud. The size of the hard drive limits how much data can be stored on your cloud. A large amount of memory in your server is not essential, but can help when transferring large files. I would recommend at least 1gb of memory in a cloud server.

3. Ubuntu Server 14.04 LTS. This is the software that will be used to run the cloud. This software is command line based, meaning there is no GUI. If that worries you, don’t panic! Learning to use the command line is exciting. Assuming you follow the instructions in this guide, you should have no problems. I will not be explaining how to download and burn Ubuntu to disc in this guide, but if you need help, check out Step 2 of my previous guide.

4. A monitor. This is needed during the initial configuration of the server. Once the configuration is complete, you can take the monitor, keyboard, and mouse away.

Also, be sure to place your server next somewhere near an Ethernet connection to the router. It is important to have an Ethernet connection as opposed to Wi-Fi because it is much faster and more reliable, which is helpful if you are streaming media such as movies.

Next, we will install Ubuntu Server 14.04 LTS to the server.

1. Begin by inserting the CD/USB with Ubuntu Server into the server. Be sure the server is set to boot from USB. If you do not know how to boot from CD/USB, please see the guide here. Ubuntu Server will load and the installation process will begin.

2. First, you will be presented with the language screen. Highlight your language and press Enter to continue.

3. The Ubuntu Server boot screen will appear next. This screen has serveral options for installing and configuring Ubuntu Server. Highlight the Install Ubuntu Server option and press Enter to continue.

4. Next, highlight the appropriate language in the list and press Enter to continue.

5. On the next screen, highlight your location and press Enter to continue.

6. The next screen will ask you whether or not you would like Ubuntu to detect your keyboard layout. You are more than welcome to do this, but I usually choose not to. For those using a standard American keyboard, you should not need to use this tool. Press Enter to continue.

7. The next screen is a keyboard configuration prompt. Highlight the type of keyboard you are using and press Enter to continue.

8. Once the keyboard layout is selected, Ubuntu may ask you for a more specific keyboard layout based on the previous selection. In my case, Ubuntu wanted to know just which form of English (US) keyboard I was using. Highlight the appropriate option and press Enter the continue.

9. Next, Ubuntu will ask you to give your server a host-name. The host name refers to how the name the server will be known by on the network. In my cast, I gave my server the host-name “ubuntu” for simplicity’s sake. You can give your server whatever host-name you like, however, keep in mind all commands used in this guide will be based on the host-name “ubuntu”. Press Tab and then Enter to continue.

10. The server will then ask you to enter your name in the field. You can put whatever name you would like in this field as it doesn’t really have any significance whatsoever. I chose to enter “server” in this field. Press Tab and then Enter to continue.

11. Next, the installer will prompt you to enter a username for the server. The username is the log in credential used to access the server. In my example, I chose the username “server”. You can give your server whatever username you would like, however, keep in mind all commands used in this guide will be based on the username “server”. Press Tab and then Enter to continue.

12. The password prompt will appear next. Enter a password for the user you just created in the previous step. Press Tab and Enter to continue. The next prompt will ask you to re-enter your password. Once finished, press Tab and Enter to continue.

13. The next prompt will ask you if you would like to encrypt your home directory. If your computer is stolen, the thief will not be able to access your files. Encryption adds an extra layer of security to your server, but can also slow down performance. I chose not to encrypt my home directory in this example. Press Enter to continue.

14. Next comes the disk partitioning tool. The options you are presented with on your installation may be a bit different than mine since I am using a virtual machine in the example. Assuming your hard drive contains no other operation systems or partitions, you should be good to highlight “Guided – use entire disk and set up LVM” or something similar to that. Press Enter to continue.

15. The next prompt will ask you to select this hard drive you wish to partition. If you only have one hard drive installed in the server, it will be the only one displayed. If you have multiple hard drives installed in the server, be sure to choose the correct disk. Highlight the appropriate option and press Enter to continue.

16. The next screen will ask to confirm your partitioning selections. Review the selections you made and Press Tab and then Enter to continue.

17. (Optional) If you are using a virtual machine to install Ubuntu Server like I am in this example. The next prompt will ask prompt you to enter the percentage of the volume you selected to use for Ubuntu. Enter the appropriate amount and press Tab and Enter to continue.

18. (Optional) If you are using a virtual machine, the next prompt will ask you to confirm your choices before formatting the drive. Review your selections and press Tab and Enter to continue.

19. If you plan to use a proxy with your Ubuntu Server, enter it on the following prompt and press Tab and Enter to continue. If not, just press Enter to continue.

20. Ubuntu will then prompt you to choose how you would like automatic updates to be installed. Choose whichever option you would like. I would strongly recommend you choose to install security updates automatically. Highlight to appropriate option and press Enter to continue.

21. Next, a software selection prompt will appear. This prompt lists many software packages commonly used in conjunction with Ubuntu Server. In our case, we need to install OpenSSH and LAMP Server. Press space on each of those two packages, then press Tab and Enter to continue.

22. Ubuntu will then install all the package you chose along with other necessary packages. During this process, the installer will prompt you to enter a password for MySQL. Enter a password and press Tab and Enter to continue. Another prompt will ask you to re-enter your password. Once finished, press Tab and Enter to continue. Please note, you do not have to enter a password for MySQL, however, I strongly recommend you chose to use a password.

23. Next, Ubuntu will ask you if it can install GRUB to the master boot record. GRUB is the tool Ubuntu uses to load its operating system. It can also be used as a recovery tool. If you have no other operating systems loaded on the server, highlight Yes and press Enter to continue. If there are other OS’es on the server, decide which bootloader you would like to used and make the appropriate selection on this screen.

24. Finally, Ubuntu will finish the installation process and prompt you to reboot the server. Press Enter to continue.

At this point Ubuntu Server 14.04 LTS should be successfully installed. Next, additional packages will be installed to add extra functionality to the server.

Now, Apache2 needs to be configured. This configuration allows the automated scripts to run without running into permissions issues. To begin, turn on your Ubuntu server. Once the server is booted, the terminal will prompt you for a username. Enter the username you created earlier. In my example, I would enter server. Next, enter the password you created for the user. Please note, nothing shows up on the screen when a password is entered. Once successfully logged in, type:

sudo apt-get install apache2-utils

This package will install various utilities available for use with Apache 2. Once that is complete, the default ports apache uses for the web-server need to be changed. By default, Apache 2 listens for web traffic on port 80. Because many ISP’s block incoming traffic on port 80, the default port needs to be changed. If the port is not changed from port 80, the web-server will not be accessible from an outside network, which defeats the purpose of a cloud. To change the default port number type:
cd /etc/apache2

then:

sudo nano ports.conf

This command will open the file named ports.conf. The line that says Listen 80 needs to be changed to Listen 8080. This changes the port apache2 listens on from port 80 to 8080. Most ISP’s do not block incoming traffic on this port. Once finished, press Ctrl+O and then Crtl+X to save the file and exit nano.

There is another file that needs to be edited to confirm the change from port 80 to 8080. Type:

cd sites-enabled

then

sudo nano 000-default.conf

Change the line that says VirtualHost 80 to VirtualHost 8080. Once finished, press Ctrl+O to save and then Ctrl+X to exit.

Next, to ensure there are no permissions issue when automatically copying files from a client to the server, the default directory Apache2 operates on needs to be changed. To change the default directory, the userdir module needs to be installed and activated. To do this, type:

sudo a2enmod userdir

Once the installation finishes, type:

sudo service apache2 restart

This will restart the apache2 service and update the default path apache2 uses.

Next, type cd to change the directory to the home directory. Then type:

mkdir public_html

This will create the folder apache2 uses. Next, restart apache2 one more time to ensure the changes are made.

Type:

sudo service apache2 restart

Finally, create the folders to store the media that will eventually be placed on the cloud. To do this, type:

mkdir public_html/documents

mkdir public_html/pictures

mkdir public_html/movies

mkdir public_html/music

This will create directories for documents, pictures, movies, and music.

Apache2 is now configured, but before the web-server can be tested, the router needs to be configured for port forwarding.

In order to access your cloud from an outside network, port 8080 must be forwarded to the IP address of your server. If port forwarding is skipped, the router will continue to block incoming traffic on port 8080.

To port-forward, the following information is needed:
The router’s default gateway
The server’s local ip address.

To find the default gateway of the network, type the command route. The default gateway of the router will be displayed in the Gateway field. The gateway will be used to access the router configuration utility.

To find the local ip address of the server, type the command ifconfig. Assuming your server is connected the router via a wired connection, the ip address of the server will be listed under the eth0 connection. Look for the field that says inet addr. The number that follows that address is local ip address of the server. It should look something like 192.168.xxx.xxx, where the x’s can be any number between 0 and 255. Write this number down.

Open a web browser and type the default gateway address in the address bar. The router configuration utility of your specific brand of router should open. Most likely, the router will ask for a username and password before any changes can be made to the configuration. A quick Google search can usually yield the default username and password of your router.

Once logged in, look for a port forwarding tab on the webpage. My router’s port forwarding page was listed under Applications & Gaming and then Single Port Forwarding.

In my example, the router requires me to enter the external and internal port number, and the IP address to forward traffic from that port to. In the external and internal port number columns I entered 8080. In the To IP address column I entered 192.168.2.186, which was the local IP of my server. Your configuration should be extremely similar to this.

Once finished, click save to save the settings. This should be the only thing that needs to be configured on most routers. If you have a external firewall connected to your network, you may need to allow traffic through on port 8080.

Now that the router is port forwarding traffic to the server, it’s time for a test. To access your server from an outside network, I recommend using your phone’s 4G service. To test the server, one last ip address is necessary. On a client connected to the local network, type “what is my ip address” into Google. Google will then display your public ip address. On your 4G connected phone, type the following into the address bar of a web browser:

PublicIPAddres:8080

Where the PublicIPAddress is your public ip address. The 8080 part tells the web browser to access the webpage on port 8080 instead of the default port 80.

If apache2 is configured correctly an apache2 welcome page should be displayed. If this is the case, you are ready to move on to the next step. If you do not see an apache2 page, be sure you are using the correct IP address and have followed the steps above correctly. If you are still having issues, please leave a comment below.

To access the documents directory created earlier, you can type:

PublicIPAddress:8080/~Username/documents

Replace Username with the username of your server. In my example, I would enter server. The remaining part of the address is the folder you want to access. For example, if I wanted to access a folder who’s directory was ~/public_html/movies/UP!/, I would type:

PublicIPAddress:8080/~server/movies/UP!/

Although apache2 is configured, it is not very secure. Currently, anyone could access all the files on the webserver. To prevent this from happening, a username and password authentication system will be implemented in the next step.

To prevent the entire world from accessing the media on your cloud, password authentication needs to be implemented. To begin, change directories to public_html by typing:

cd ~/public_html

Then

sudo nano .htaccess

Do not forget the ‘.’ before the htaccess. The dot hides the file from users. This file will tell the webserver that a username and password are required to access the cloud. Nano will open a blank file called .htaccess when the above command is entered. In the window type:

AuthType Basic
AuthName “Password Required”
AuthUserFile /home/Username/.htpasswd
require valid-user

Be sure to replace the username with username of your server. In my example, the username is server.

Once finished, press Ctrl+O to save and Ctrl+X to exit. The next file that needs to be created is .htpasswd. This file will contain a list of the approved log in credentials for the cloud.

First, type cd to change the directory to the home directory.

Next, type:

htpasswd -c /home/Username/.htpasswd User

Be sure to replace the username with the username of your server, just like above. Also replace the word User with any name you would like. The named specified in the User field is used as the username for the cloud. In my example, I use the name admin. Once entered, the server will create the .htpasswd file, and prompt you for a password for the specified user. Enter and confirm your password to continue.

All folders inside the public_html directory are now secured by password authentication.

To add another user to the credential list, DO NOT enter the same command as above, instead type:

htpasswd /home/Username/.htpasswd User

If you type the same command as above, the .htpasswd file you create initially will be overwritten.

To test the password authentication on the web-server. Open a web browser and enter:

PublicIPAddress:8080/~Username/

A box should appear prompting you to enter a username and password in order to access the content on the page. Enter the username and password created earlier, and the content on the page should become available. Please note, all directories inside of the public_html folder do not need a .htaccess file. Because the parent directory has the .htaccess file, all directories beneath it will require authentication for access.

Now that the cloud is secured, you can now safely start adding media to the server.

This step is entirely optional. The purpose of this step is to create a free domain name which maps your public IP address to a domain name. The reason behind this is simple. IP Addresses are difficult to remember, and occasionally, an ISP will change your public IP address. This step will make your public IP Address static, and map it to a domain name. Like I said before, this step is completely optional.

To begin creating a domain name, open this website, and create an account. The account is needed to create a domain name. Once the account is created, log into the website and click the button that says Subdomains on the left side of the page. Next, click the [Add] button. The page that opens is where the domain name will be created.

Leave the Type of Subdomain as A.
In the Subdomain box, enter the name you want to call your website. For example, I will call my website apples.
In the Domain box, choose the domain you want your website to reside under. The option you choose here does not really matter as long as you choose a public domain. In my example, I chose mooo.com
In the Destiantion box, enter your public IP address. To find this address, type “what is my ip address” into Google.

Finally, click Save. The domain should now be mapped to your public ip address. Whenever you wish to access your website, you can now access it using the subdomain and domain names you selected. For example, to access your movies you would type:

www.SubDomainName.Domain.com:8080/~Username/movies/

Where the subdomainname, domain are the names you chose above. If I chose apples as my subdomain name, and mooo.com as my domain, I would enter those names in their respective areas.

Once this step is complete, it is time to transfer media from your client to the server.

In this step, the media residing on the client machine will be transferred to the newly created cloud. They are many different ways to transfer media to the cloud: copy and paste, drag n’ drop, or FTP. While each of the options are great and work well, they aren’t automated. Each time any media is updated on the client machine, it must be manually transferred to the server. To solve this issue, an automated script is needed. I have attached the script I use to this step. My client machine runs Linux, so this is a Linux script. If you are running Windows are your client machine, you can use a software such as FileZilla to manually transfer your files to the server.

To begin, download the script and place it in the /home folder on the client machine. Open the script and edit it to meet the needs of your client machine. In the script there are four rsync commands. Each one corresponds to a media folder: one for documents, photos, media, and music. Below is a description of each element in the script:

rsync -u -r -v -e ssh –progress –delete –chmod=D775 /path/to/your/media/documents/* YOUR_SERVER_NAME@YOUR_DOMAIN NAME:~/public_html/documents/ –exclude=.htaccess –exclude=.htaccess~

rsync – rsync is the service used to transfer the media to the server.

-u – This option skips copying files if they are newer on the server. If for some reason there was a newer file on the server than on the client machine, it would not be overwritten on the server.

-r – This option copies all files recursively.

-v – This option basically tells rsync to let you know what it is doing as opposed to outputting no data to the terminal. Verbosity can be increased by adding additional dashes in front of the v.

-e – This command followed by ssh tells rsync to copy the files over SSH. This option ensures security if files are being copied over the internet. In order to use this option successfully, SSH must be configured on the server. You must also be able to SSH to the server without using a password. A quick Google search should lead you to a tutorial on how to accomplish both those tasks. If you are only copying files over a local network, this command really is not necessary.

–progress – This option shows the progress rsync is making during the copying process.

–delete – This option will delete files on the server if they are no longer on the client machine. This means that if you accidentally delete a file on the client machine, run the script, and want to retrieve that file off the server because you didn’t mean to delete it in the first place, it won’t be there.

–chmod=D775 – This option gives each file the proper permissions for cloud hosting. If this option is taken out, folders within the cloud directories will not show up on the website.

/path/to/your/media/documents/* – This is the path to your documents folder. The /* at the end of the command ensures all files within the documents folder are transferred. Change the script to match the path to each of your media directories.

YOUR_SERVER_NAME@YOUR_DOMAIN NAME:~/public_html/documents/ – This command is the path to the server directory the files will be copied to. Replace YOUR_SERVER_NAME with the name of your server. In the case of this guide, it would be server. Replace YOUR_DOMAIN_NAME with the domain name you created in the previous step. Finally, replace /documents/ with the appropriate media folder you are copying to.

–exclude=.htaccess –exclude=.htaccess~ – Because the –delete option is in place, these files could potentially be deleted off the server because they are not on the client machine. However, since these files protect the server, they should not be deleted. This command keeps them from being deleted.

To run the script, type:
./rsync_backup

If there you get a permission denied error type:
chmod 755 rsync_backup

At this point, the script should run and all media will be transferred to the server. To automate the script, try creating a cron job. Another quick Google search should yield a simple tutorial on how to accomplish that.

The script can be downloaded from my github page located here.

Now that your cloud server is up and running, there are many different projects, software, and scripts to implement on your server. You can try creating a VPN server with my guide here. You can also set up streaming from your server to any machine using XBMC. That procedure is descried in previous cloud guide here. Also, you can try your hand at web developing, and create your own website. Should you wish to stream your music from the cloud, there is an excellent opensource music streaming software called Ampache.

The possibilities are incredibly numerous. If you have an idea of something you wish to put on your server, chances are, there is a tutorial for it somewhere on the Internet, and if there isn’t, try venturing into uncharted territory. Please feel free to leave any questions or comments below as I try to answer all of them as best I can.

Author: Jay  Redge

Via: Instructables