Tuesday, February 10, 2015

Setting up file- and mediaserver with Debian GNU/Linux 7.8 and 6TB of RAID-5 space

This post a continuing saga about my home-server started in this post now more than 6 years ago.

Times have moved along and more and more I have realized that not only is file redundancy important, but having half-a-thousand CD's and DVD's lying around is not the nicest of one's room decoration.
Due to the need to clear one wall where currently most of the mentioned CD's and DVD's are located, I decided to move the more important content into my library server and eliminate the physical media. This, however, needs a lot of storage space. My Library server previously had three 1.0TB Western Digital Green hard drives in RAID-5, which amounted to 2.0TB of space. I decided to triple that to 6.0TB with three 3.0TB Western Digital Green drives. One funny note to make is that the new 3.0TB Green drive also weighs subjectively about three times as much as the previous 1.0TB Green drive.

In the end, the power consumption of the whole system went up as expected, but not that much - from about 58W average to 65W average at idle.

Along with the change of hard drives I also switched the processor and thus also the motherboard and memory. This decision was primarily caused by the inability of the old Athlon 64 3700+ to fully facilitate the Plex media server, especially the transcoding was unacceptably slow, but also the Plex client interface itself was lagging badly both in browser and in my Samsung TV. The one single Mebibyte of memory in the old server might've also played a part on the bad performance.

So the specs of the new server are as follows:
  • AMD A8-6500T APU (4 cores, 45W, 2.1GHz (3.1GHz Turbo), 1.4GHz idle)
  • ASUS A88XM-E
  • 4.0GiB Crucial Ballistix Sport
  • Tagan 420W i-Xeye TCF PSU
  • Antec Three-Hundred case

Server installation


OS choice
The operating system was again a tried and true Debian GNU/Linux, this time around the version 7.8, "Wheezy".

Partitioning
As before, I decided to split the space up to three parts. An adequately sized (30GiB) root partition at the beginning, a 4GiB swap partition at the end and everything between them as the Data partition.

The first time I did the installation I ended up with an error at the point in the installer, where the grub was to be installed. A bit of digging in the net revealed that with 3.0TB drives, GPT must be used and for a boot drive, there has to be a small reserved space for the boot code left on the drive, with the "bios_grub" flag set

Since this kind of partitioning cannot be done from within the Debian installer, use an Ubuntu live CD, open up three terminals in order to be sure to do the exact same steps on all three disks and use the following steps in each:
sudo parted /dev/sda         # use sdb and sdc in the other windows.
mklabel gpt                  # create a new blank GUID partition table.
unit MiB                     # change the units to Mebibytes (notice the 'i' the middle).
print                        # prints the disk information, including the maximum MiB position.
mkpart GRUB 1 2              # create 1 MiB partition between 1 MiB and 2 MiB positions for BIOS boot code and set
set 1 bios_grub on           #   the "bios_grub" flag for that partition.
mkpart ROOT 2 15362          # create 15360 MiB (15 GiB) partition between 2 MiB and 15362 MiB positions for root.
mkpart HOME 15362 2859536    # create 2.7 TiB partition for Data; as the second position, use the max position you got
                             #   above, but subtract 2048 MiB for the swap partition and an additional 2 MiB for some
                             #   spare space at the end.
mkpart SWAP 2859536 2861584  # finally, create a 2 GiB partition for the swap at the end.
quit                         # exit from parted.
Notice that these partitions are used in RAID-5, therefore doubling their sizes, so root will be 30 GiB, swap will be 4 GiB and the Data partition will be 5.4 TiB.

Now, start the Debian installer and in the partitioning step...
  • be sure to leave the first 1.0 MiB partition as it is, do not change it!
  • mark the latter three partitions on each disk as volumes for RAID,
  • combine the corresponding partitions on all three disks into RAID-5 arrays.
  • use ext4 on the 30 GiB partition and use it as the / (root) partition with the noatime flag.
  • also use ext4 on the ~5.4 TiB partition, but set the mount point to be /Data, also use the noatime flag.
  • set the last 4 GiB partition as the swap partition.

After partitioning, at software selection step, I included the web server, file server and ssh server.

At the grub setup step, the Debian installer only installs grub into the first disk, therefore it is important to install grub manually into the other disks first! Switch to one of the other consoles, eg. with ALT+F2, and enter the following commands:
chroot /target /bin/bash
grub-install /dev/sdb
grub-install /dev/sdc
The installer will finish soon thereafter.

Server setup


Enable sudo for your user:
adduser username sudo
Note that this takes effect only after logging out and back in.

Reduce the boot manager delay:
sudo mcedit /etc/default/grub
Find and change the GRUB_TIMEOUT value from 5 to 1 and execute:
sudo update-grub

Change Gnome Classic to be the default graphical shell:
sudo update-alternatives --config x-session-manager
And select the fallback line, usually number 2.

Change the dynamic IP to be static:
In Gnome Classic, in the right at the upper bar, right-click the network icon and choose Edit to configure the network interface. In the window that opens, select your network interface and click Edit in the right. In the following window, at IPv4 settings tab, switch the method from DHCP into Manual, click Add, enter your static IP (eg. 192.168.10.2), netmask (255.255.255.0) and gateway (192.168.10.1), also enter the DNS below and click Save.

Make Debian boot into console mode:
sudo update-rc.d gdm3 disable

Setting up the environment:
mcedit ~/.bashrc
Add the following lines:
stty -ixon  # enables forward bash command history search CTRL-S
export LC_COLLATE="en_US.UTF-8"  # fixes the broken Estonian character map which breaks regexp [a-z]

File server setup


Setting up the Data directory:
sudo addgroup share
sudo adduser username share
cd /
chown nobody:share /Data
chmod 777 /Data
cd /Data
mkdir .transcodetemp  # this directory is for Plex Media Server
chmod 777 .transcodetemp

Setting up anonymous read-only Samba account:
sudo mcedit /etc/samba/smb.conf
Find the section "Share Definitions" and add the following right beneath it:
[library]
   comment = Library
   path = /Data
   browseable = yes
   guest ok = yes
   read only = yes

Setting up NFS share:
sudo mcedit /etc/exports
Add the following line:
/Data 192.168.10.0/24(rw,root_squash,async,no_subtree_check)

Making sure the file owner is displayed correctly over NFS:
sudo mcedit /etc/idmapd.conf
Uncomment the "Domain = xxxx" line and enter your domain in place of the xxxx.
Make sure to use the same domain in your other servers that connect here over NFS!

Reboot the server to make the changes take effect.

Media server setup


Setting up Plex Media Server:
echo "deb http://shell.ninthgate.se/packages/debian wheezy main" | sudo tee /etc/apt/sources.list.d/plexmediaserver.list
wget http://shell.ninthgate.se/packages/shell-ninthgate-se-keyring.key
sudo apt-key add shell-ninthgate-se-keyring.key
sudo apt-get update
sudo apt-get install plexmediaserver
This will also set the Plex Media Server to be automatically started when the server starts.

Other optional steps


Repairing the locales:
When you select a different language at the start of the Debian installer, it configures only the locale for this language. However, for many reasons the English locale should also exist, therefore:
sudo dpkg-reconfigure locales
And check also the en_US.UTF-8 locale in addition to your language locale.