Sunday, August 6, 2017

AWS EC2: Setup Python 3 + Flask + Gunicorn + Supervisor + Nginx on an Amazon Linux AMI instance

Connect to your Linux instance

AWS user guide

Install Python 3.x

First thing first, update yum
sudo yum update

You can check which Python 3.x package is available:
sudo yum list | grep python3

Install Python 3.5 and pip
sudo yum install python35
sudo yum install python35-pip

Once installed, you can access it via python3 or python35, e.g.
python3 --version

(Optional) create a symbolic link to pip-3.5 as pip3
sudo ln -s /usr/bin/pip-3.5 /usr/bin/pip3

Setup virtual environment

Create a directory for our project
mkdir my_project
cd my_project

Try to create a virtual environment named env, you would get the following error:
pyvenv3 env
Error: Command '['/home/ec2-user/my_project/env/bin/python3.5', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1

There's some issues with ensurepip. So we are going to setup the environment without pip, and then install it manually
pyvenv3 --without-pip env
source env/bin/activate
curl | python
source env/bin/activate

Setup Flask and Gunicorn

Install flask and gunicorn
pip3 install flask gunicorn

Let's create a simple flask app. Create a file named with the following content.
from flask import Flask

app = Flask(__name__)

def hello():
    return "Hello World!"

if __name__ == "__main__":"")

Start the flask server

Open another ssh terminal and ping the server. It should return Hello World!

If you want to check this via your browser, you would need to update the security group through AWS Console and open port 5000 to your IP.

Now close the flask server and launch it through gunicorn instead.
gunicorn --bind hello:app

In the command above, hello is name of the script, while app is the name of the flask app within

Do another sanity check (notice the port has changed to 8000)

Setup Supervisor

Install Supervisor
sudo easy_install supervisor

As a sanity check, print a sample Supervisor configuration

Save the sample to /etc/
sudo echo_supervisord_conf > /etc/supervisord.conf

We need to make 2 changes to the config
1. Make /tmp/supervisor.sock writable for all users by updating chmod under unix_http_server to avoid this error

2. Add the following block for our flask app
command = /home/ec2-user/my_project/env/bin/gunicorn hello:app
directory = /home/ec2-user/my_project/
user = ec2-user

Refresh configuration
supervisorctl reread
supervisorctl update
supervisorctl start hello

You can now use supervisor to control gunicorn
supervisorctl start hello
supervisorctl status hello
supervisorctl stop hello

Do another sanity check

Now make supervisor to run on startup automatically

Create a a new config file
sudo nano /etc/init.d/supervisord

Copy-n-paste the content from here

We need to make 2 changes to the config
1. Update prefix path at line 27 so it can locate supervisor properly

2. Update line 41 from -n to -S (to set a soft limit) in order to avoid this error
ulimit -S 96000

Make it runnable
sudo chmod +x /etc/init.d/supervisord

Make supervisor starts automatically
chkconfig --add supervisord
chkconfig supervisord on
chkconfig --list supervisord

Now if you reboot the instance, supervisord should start automatically. You can check its status.
service supervisord status

Setup nginx

Install nginx
sudo yum install nginx

Create a server config
sudo nano /etc/nginx/conf.d/hello.conf

Copy-n-paste the following content
server_names_hash_bucket_size   128;

server {
    listen 80 default_server;
    #server_name _;
    access_log  /var/log/nginx/hello.log;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

Edit the main conf
sudo nano /etc/nginx/nginx.conf

(Optional) comment out the first 2 lines of the server block to route all traffic to our hello.conf. Otherwise, you would need to set the proper server_name in hello.conf
server {
#listen 80 default_server;
#listen [::]:80 default_server;

Check for errors
sudo nginx -t

Make nginx to run on startup automatically
chkconfig nginx on
chkconfig --list nginx

Check status of nginx
service nginx status

Start nginx
sudo service nginx start
sudo service nginx reload

Once you update your security group setting on AWS Console to open up port 80, you should be able to see the Hello World! message when you browse to your EC2 instance public domain name.

For debugging
sudo tail -n 7 /var/log/messages
sudo tail -n 7 /var/log/nginx/access.log
sudo tail -n 7 /var/log/nginx/error.log
sudo tail -n 7 /var/log/nginx/hello.log
ps -u ec2-user

  1. python 3.x - How do I install Python3 on an AWS EC2 instance? - Stack Overflow
  2. 7 Linux chkconfig Command Examples – Add, Remove, View, Change Services 
  3. 14.04 - pyvenv-3.4 error: returned non-zero exit status 1 - Ask Ubuntu
  4. imperialwicket - AWS: Install Nginx and PHP-FPM on Amazon Linux 
  5. Deploying Gunicorn — Gunicorn 19.7.1 documentation  
  6. 10.04 - Upstart script doesn't start - Ask Ubuntu
  7. Deploying Python flask application using nginx/gunicorn on Amazon Linux EC2 instance - Stack Overflow
  8. How To Serve Flask Applications with Gunicorn and Nginx on Ubuntu 14.04 | DigitalOcean 
  9. Running Supervisor — Supervisor 3.3.3 documentation
  10. Managing Gunicorn Processes With Supervisor – Onur Güzel 
  11. How to Run Flask Applications with Nginx Using Gunicorn – Onur Güzel
  12. ulimit - How do I increase the open files limit for a non-root user? - Ask Ubuntu  
  13. python - How to automatically start supervisord on Linux (Ubuntu) - Server Fault
  14. unix:///var/run/supervisor.sock no such file · Issue #480 · Supervisor/supervisor  
  15. nginx - uwsgi upstart on amazon linux - Stack Overflow
  16. How to see process created by specific user in Unix/linux - Unix & Linux Stack Exchange  
  17. Server names 
  18. How nginx processes a request
  19. Serving Static Content | NGINX
  20. HTTPS + NGINX with self signed SSL certificate / Articles / 

No comments: