Categoryengineering

Installing Seafile with Docker and Apache 2

Seafile is an open source file sharing software that allows its users to setup their own cloud storage at home. Think Dropbox, but self-hosted.

This post is about deploying an instance of Seafile using Docker on a non-standard port (i.e. not 80 and 443). The reason to deploy the app on a non-standard port is usually because there is already another webserver running on that port (in my case Apache 2). In such scenarios, it is necessary to set up a reverse proxy so the Seafile port can be exposed by Apache on ports 80/443. Under typical circumstances, deploying Seafile on a standard port would mean that the app would host itself without the need for Apache 2 at all. This post also assumes Ubuntu 16.04 as the operating system, but in theory should apply to other Linux OS’s.

The process can be split into multiple parts:

  1. Setup Docker
  2. Deploy Seafile
  3. Configure Apache 2
  4. Setup systemd
  5. Setup e-mail (optional)
  6. Backing up the image
  7. Troubleshooting

Setup Docker

This guide won’t go into depth on how to setup Docker, but rather the setup guide and more information about what Docker is can be found here: https://docs.docker.com/get-started/

Deploy Seafile

The key page in the Seafile documentation for deploying Seafile within Docker can be found here: https://manual.seafile.com/deploy/deploy_with_docker.html. The command I used is as follows:

docker run -d --name seafile \
  -e SEAFILE_SERVER_HOSTNAME=seafile.example.com \
  -e SEAFILE_ADMIN_EMAIL=me@example.com \
  -e SEAFILE_ADMIN_PASSWORD=a_very_secret_password \
  -v /opt/seafile-data:/shared \
  -p 8000:80 \
  seafileltd/seafile:latest

I recommend setting up Seafile with an actual secret password because it’s not easy to change once it’s set up.

Note that the port argument shows 8000:80. It means that Docker will use the host port 8000 and the container will use port 80 – meaning that using Docker will interpret requests from my host OS on port 8000 as requests on port 80 in the container. 8000 was used on my setup because port 80 was already used.

Once the command is executed, the necessary files will be downloaded and the container will start and will be accessible on port 8000.

There is also an option to setup Seafile with an SSL cert from Let’s Encrypt, but I didn’t do this because I already had a cert that I wanted to reuse.

Once the server is up, navigate to https://seafile.example.com/sys/settings/ and change SERVICE_URL and FILE_SERVER_URL to match your domain name.

Configure Apache 2

Adding a reverse proxy on Apache 2 will allow Apache 2 to route requests to port 8000, which will go to port 80 in the container. The main reference for this section is this: https://manual.seafile.com/deploy/deploy_seahub_at_non-root_domain.html. This is done by adding a new site to Apache 2. The following configuration assumes that a LetsEncrypt SSL certificate is used.

cd /etc/apache2/sites-available
sudo nano seafile.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>
        ServerName seafile.example.com
        ServerAdmin me@example.com

        RewriteEngine On

        # ModSecurity does not process requests.  There is a hard limit of 1 GB
        # with ModSecurity.
        SecRuleEngine Off

        # Whitelist internal IPs for mod_evasive
        DOSWhitelist 192.168.1.*

        <Location /media>
                Require all granted
        </Location>

        # Stuff for seafile server
        ProxyPass /seafhttp http://127.0.0.1:8082
        ProxyPassReverse /seafhttp http://127.0.0.1:8082
        RewriteRule ^/seafhttp - [QSA,L]

        # Stuff for seahub
        SetEnvIf Authorization "(.*)" HTTP_AUTHORIZATION=$1
        ProxyPreserveHost On
        ProxyPass / http://127.0.0.1:8000/
        ProxyPassReverse / http://127.0.0.1:8000/

        ErrorLog ${APACHE_LOG_DIR}/seafile_error.log
        CustomLog ${APACHE_LOG_DIR}/seafile_access.log combined

        Include /etc/letsencrypt/options-ssl-apache.conf

        SSLCertificateFile /path/to/cert/file.pem
        SSLCertificateKeyFile /path/to/cert/key/file.pem
</VirtualHost>
</IfModule>

I’ve added my local domain to the whitelist so that mod_evasive doesn’t think my local host is a threat.

There are two sets of ProxyPass and ProxyPassReverse lines. The reason for this is that Seafile is split up into Seahub and Seafile, where one uses port 8082 (by default), while the other uses 8000. These two lines are essentially what allows Apache to read requests from port 443 and pass them on to the container.

Once the file is saved, execute the following

sudo a2ensite seafile.conf
sudo service apache2 reload

Navigating to the site at the Apache 2 defined port should now load Seafile without having to go to port 8000.

Setup systemd

systemd allows the Docker container to be started up on boot, and be controlled like this:

sudo service seafile start/stop/restart

The configuration is as follows:

sudo nano /etc/systemd/system/seafile.service
[Unit]
Description=Seafile Server
After=network.target mysql.service

[Service]
Type=oneshot
ExecStart=/usr/bin/docker container start seafile
ExecStop=/usr/bin/docker container stop seafile
ExecReload=/usr/bin/docker container restart seafile
RemainAfterExit=yes
User=root
Group=root

[Install]
WantedBy=multi-user.target

Once the configuration is saved, shut off the existing running instance of Seafile and then let systemd start it up for you.

sudo systemctl daemon-reload
sudo service seafile start

At this point, Seafile should start automatically on boot, with Apache 2 doing a reverse proxy to the true Seafile port.

Setup e-mail (optional)

Optionally, Seafile can be configured to send e-mails. The reference is here: https://manual.seafile.com/config/sending_email.html. The config file should be under:

/opt/seafile-data/seafile/conf/seahub_settings.py

I happen to be using Gmail SMTP, so at the end of the file I’ve added

EMAIL_USE_TLS = True
EMAIL_HOST = 'smtp.gmail.com'
EMAIL_HOST_USER = 'me@example.com'
EMAIL_HOST_PASSWORD = 'password'
EMAIL_PORT = 587
DEFAULT_FROM_EMAIL = EMAIL_HOST_USER
SERVER_EMAIL = EMAIL_HOST_USER

Backing up the image

After setting everything up, you’ll probably want to backup the Docker image. First, print out the list of running containers

sudo docker ps

Then take a snapshot of the current running state:

docker commit -p  

Running this command lists all the images and should show this newly saved image:

sudo docker images

I don’t have a private docker repository, so I saved the image to the disk and backed up the file by doing the below. Just make sure to actually move the image to an actual backup location.

sudo docker save -o /.tar 

The image should be backed up now. To restore the image, it’s a matter of running this:

sudo docker load -i 

Listing the images as above should show that the image was successfully loaded. Running this starts it back up again.

sudo docker run 

Troubleshooting

Seafile logs can be helpful if it doesn’t start up correctly. These can be found at:

/opt/seafile-data/logs/seafile

There may be a need to ssh into the Docker container. The command for that is:

sudo docker exec -it seafile /bin/bash

Running Selenium Webdriver on Bash for Windows

Bash for Windows has been working pretty great for me until I needed to run Selenium Webdriver on it. I quickly learned that it wouldn’t work just work right out of the box, and the set up for it is quite convoluted.

You will need

Bash setup

Install Firefox:

sudo apt-get install firefox

Export DISPLAY variable in ~/.bashrc. Just add this to the ~/.bashrc:

export DISPLAY=:0

Xming setup

Just download it from the link above and run it.

Download geckodriver

Download geckodriver from the link above and put in the /usr/bin/ folder in Bash for Windows.

Running Selenium

Here is a piece of sample Python code I used to setup the web browser. It will setup the browser, open Google, sit there for 10 seconds and then quit.  Make sure to have Xming running otherwise the browser isn’t going to start.

import time

from selenium import webdriver
from selenium.webdriver import DesiredCapabilities

def execute_with_retry(method, max_attempts):
    e = None
    for i in range(0, max_attempts):
        try:
            return method()
        except Exception as e:
            print(e)
            time.sleep(1)
    if e is not None:
        raise e

capabilities = DesiredCapabilities.FIREFOX
capabilities["marionette"] = True
firefox_bin = "/usr/bin/firefox"
browser = execute_with_retry(lambda: webdriver.Firefox(
    firefox_binary=firefox_bin, capabilities=capabilities), 10)

browser.get("https://www.google.com")

time.sleep(10)

browser.close()

Should you be an engineer?

It’s not easy to decide what career path to take.  Especially for high school students, who don’t yet know what they want to do.  During my time in university, I spent a bit of time in engineering outreach to speak to high school students and first-year university students about studying engineering.

A few days ago, I had the opportunity to go back to my old high school and present to students about studying engineering (link to presentation below), and to help guide them with their career choices.  The idea was to communicate:

  • What engineering is as a profession
  • What engineering is about
  • That they should consider it as a career

At the same time, it wasn’t meant to “persuade” them to study engineering, but more like show them what it’s about so that they can make a more informed decision when they choose their university majors.  I also wanted to tell them this as general advice:

  • Do what you’re interested in
  • Nothing worth doing is ever easy
  • Don’t do it just for the money

Overall, the students seemed to be receptive to what I had to say, and had good questions to ask me about my experiences.  Hopefully, I helped guide them to make a career choice for themselves!

Link to the presentation: here

Valuable Resource for Learning UnrealScript Game Programming

About three years ago, I dabbled briefly in programming for the Unreal Engine (AKA the Unreal Development Kit).  As a student at the time, it was my way to get more experience in the coding scene.  The mistake I made at the time was trying to build a project that was way too advanced for my skill level.   I should have learned the basics.

Looking back, I should have used a book like the “UnrealScript Game Programming Cookbook” by Dave Voyles.  This book is directed towards audiences who have never touched Unreal programming before.  However, it does require readers to have some familiarity with general object oriented programming knowledge.

This book in particular follows a recipe-based approach for teaching the basics of Unreal programming and covers the core topics in programming for UDK.  The recipes are designed to be basic in order to give learners good exposure without too much of a knowledge dump.  Some may argue that having a recipe-based approach doesn’t teach people how to synthesize solutions for themselves.  I see recipes as a good stepping stone.

This book is not meant to teach you everything there is to know about UDK, but enough for people to get started with a good foundation to learn off of.  From the book, I would have liked to see more details about the nuances of the UnrealScript language, as well as a more big picture view of how all the different topics illustrated in the book fit together – perhaps a chapter dedicated to that.  The way it is organized right now is that the big picture is explained, but the explanations are scattered throughout the book.  I am someone who much prefers a more top-down approach on topics like this.

In any case, this book is still a great resource for learning Unreal programming and I would recommend it to people who would like to start their learning adventure in UDK.  Here is a sample chapter of the book for those who are curious in seeing for themselves what this book has to offer (here).  For more information about the book, click here.

RoboCup 2013 in Eindhoven

After a year of hard work redesigning our soccer robots, it was time to take them to the real competition.  Last year, the team went to Mexico City, but this year, we went to Eindhoven, Netherlands to compete. The event was so well known, that even the queen of the Netherlands paid a visit to the competition!

We had spent a whole year upgrading everything.  I designed a new ball kicking system that was more durable, more maintenance free, and more powerful than the year before.  The other mechanical systems were more robust as well.  The electrical system also had a huge upgrade.  Burning and smoking electrical boards were a thing of the past!  We have never had a design this robust – ever!

What is RoboCup?

For those who don’t know what RoboCup is, RoboCup is an international robotics competition aimed to further robotics research.  Their long-term goal is to have robots beat the World Cup Soccer team winners by 2050. There are various leagues within RoboCup, such as in robotic soccer (where we participate), home care, rescue, simulation, etc.  Our team, the UBC Thunderbots, compete in the small-size soccer league, which is probably has the fastest game play of all the different leagues.  The robots are generally cylinders with a diameter of about 18 cm and a height of 15 cm.  They can move at around 4 m/s and kick a golf ball at 8 m/s.

One of the other leagues, the Standard Platform League competes with standardized robots for all teams and the key difference between each is the software that is written for the robots.

Another league, the mid-size league in RoboCup Soccer competes with robots that are about a 1/3 of the size of a person. The game isn’t as quick as the small-size league game and they play with a real soccer ball instead of a golf ball. During the competition, it gathered quite the crowd too.

Preparing for the Games

Most teams arrived at the venue when it opened to get to work right away. Everybody wanted their robots in tip-top shape. Our team did the same. Even though our robots were ready mechanically and electrically, the key game changer was seeing which team had better artificial intelligence for their robots. Whenever got time to test on the field, we took advantage of it to make sure our AI was working well too.

Round Robin

In the round robin stage, each team gets four games against other teams in their bracket. Each bracket is made up of five teams and the skill level is spread out based on the rankings of the previous year.

We lost our first two games against a team called KIKS from the Toyota National College of Technology in Japan, and a team called STOx’s from the Universidad Santo Tomás in Colombia. In both of those games, we were simply outplayed because the other team had better software than we did. At the end of the game against the Japanese team, I spoke a bit of my horrible Japanese to them and they were super surprised and impressed that I knew a bit of Japanese!

Because our robots were finished not long before the competition began, the software team did not get a chance to test the code that was written, and so we found out the problems during the competition. Our robots were never able to drive up to a ball and take possession of it. The robots somehow always missed the ball. When the robot had the ball, it didn’t know what to do with it. It sat there holding onto the ball doing nothing. This was so frustrating for our team and even more so for the software team who wished they had more time to test.

We won our next two games against two lower tier teams. One is called RoboFEI, from the Centro Universitário da FEI in Brazil, and a team called RFC Cambridge from Harvard and MIT. It’s nice to be able to say that we beat a team from these two ivy league schools. Looks good when we look for sponsors too!

Lucky Loser Round

Because we had a 2-2 record during the group stage, we played a fifth game, the so-called Lucky Loser Round, to see if we could qualify for the quarter finals. Our match up was against a team called ODENS from the Osaka Electro-Communication University in Japan.  The game was so close that there was a dispute about the rules and whether what our team did was legitimate. Because our robots were not able to move up to a ball and shoot it at the net during the shootout, the referee introduced a workaround that the other team disputed.

Our referee’s native tongue was Portuguese, we spoke English, and the other team spoke Japanese. I didn’t know enough Japanese to translate. Eventually, members from other teams like the Thai and Chinese team came to help with the dispute. During the discussion that took about an hour, I heard people speak English, Japanese, Portuguese, Thai, Japanese, and Mandarin all at the same time. It seemed like nobody had the full picture of what was happening and how to resolve it.

The dispute was resolved when both teams agreed to a specific workaround that made it possible for our robots to do shootouts.  The game ended with a shootout and we lost on the very last shot of the shootout unfortunately. We almost made it to 8th place.  Still, achieving 9th place out of 22 teams was the best that the team has ever done before.

The Competition Ends

Skuba, from Kasetsart University in Thailand, had won the competition for four years in a row since 2009, but this year they had a surprising finish that was not even the top 4.  The second place team, ZJUNlict from Zhejiang University in China had beat them early in the quarterfinals.  ZJUNlict’s strongest opponent this year was CMDragons, from Carnegie Mellon University in USA.  In the final round, they were tied for the entire game and the game went to shootouts where CMDragons lost by one goal.  ZJUNlict had made first place in the RoboCup 2013 in the Small Size League.

As the competition came to a close, we said goodbye to the people on other teams that we met and as a parting gift from the German team ER-Force, they gave us a German flag with the names of all their team members signed onto it, with the message, “for our amazing friends, the Thunderbots, from team ER-Force”.

I realized now that the competition is over, I probably wouldn’t be going to competitions in the future because I’ve graduated from university. It made me feel a little bit sad that I couldn’t compete anymore, but at least I can still help out the team wherever they need it.

Link to photo album here

© 2019 Henry Poon's Blog

Theme by Anders NorénUp ↑