Shiny on AWS - Deploy the app on your server

This article belongs to the series How to deploy a Shiny app on AWS, divided into 7 parts. To access the other articles, use the following table of contents:

Let’s recap.

  • We built a Shiny app and sent it on Github.
  • We created an AWS account to rent a server.
  • We prepared the server to welcome our app.

At the end of the previous article, I said you already had a Shiny app running on your server!

Indeed, it appeared by installing Shiny Server.

1. How to access the default Shiny app?

To access it, two steps:

  1. Find the IP address on your server.
  2. Open the port 3838 on the firewall.

Find the IP address on your server

When you start a new AWS server, an IP address is automatically affected to it.

Warning: The IP address changes each time you reboot the server. If you want a fixed IP address, you’ll need to visit the Elastic IP section and allocate an address to your server. We’ll discuss that in the next article!

In your EC2 dashboard, there is a column IPv4 Public IP.

Find it and note the IP address.

IP address on AWS

For me, it’s 3.121.42.9 (the link will not work, I shutdown the server after publication of the article).

And to access the Shiny app, you just need to type this address in your browser AND specify the port number: 3838.

3.121.42.9:3838

Except… it doesn’t work.

Firewall AWS

Open the port 3838 on the firewall

The reason is that for security reasons, AWS blocks all the ports except 22 by default. Port 22 is the one we used to access the server via SSH in the previous article.

To change that, from your EC2 dashboard:

  • Select your instance.
  • In the bottom half of your screen, find the line Security groups and click on the link.
  • In the new window, again the bottom half of the screen, click on Inbound, the 2nd tab. Notice that only the port 22 is open.
  • Click on Edit, and the popup that opens, enter the following settings:
Type Protocol Port Range Source Description
SSH TCP 22 Custom: 0.0.0.0/0  
Custom TCP Rule TCP 3838 Custom: 0.0.0.0/0  

Here are a couple of screenshots in case you’re lost:

Open ports on AWS

.

Open ports on AWS

Once the port is open, the app will load!

Let’s try again:

3.121.42.9:3838

This time…

Default app

Boom!

It works!

Note: The address is a bit ugly right now. You must type an IP address followed by a port number, erk. In the next article, we’ll discuss how to have a readable address and get rid of the port number.

We obtain an HTML page with a couple of Shiny widgets in the right sidebar.

You may get an error message in one of the widgets: “An error has occurred”. This is totally normal, it comes from a missing package.

Okay, cool! But.. how to do put our app now?

2. How to deploy YOUR app on the server?

To do so, we need to get back to the console of the server.

If you forgot, get back to the previous article: Install R and R Shiny

First, let’s explore what we have.

Explore your server

We just noticed that when you first install Shiny Server, you get a default app.

This app must be stored on the server, right?

So where is it?

Simply, it’s in /srv/shiny-server. That’s where we will store all our apps.

Type the following in the console:

$ cd /srv/shiny-server/
$ ls
index.html  sample-apps

Notice there is a directory, sample-apps, and a file, index.html.

The file index.html is what you saw when you accessed the app. And it calls the app that is in sample-apps. If you dig deeper:

$ cd sample-apps/hello
$ ls
server.R  ui.R

You find the usual server.R and ui.R, the foundations of any Shiny app!

By default, Shiny Server uses this directory to display apps. You can change it, but I don’t recommend it.

What we’ll do instead is:

  1. Download our app in a working directory.
  2. Create a shortcut in /srv/shiny-server/.

Download the app on the server from Github

First, let’s get back to the home folder:

$ cd

And let’s clone the app from Github:

$ git clone https://github.com/Huitziii/movie-explorer.git
Cloning into 'movie-explorer'...
remote: Enumerating objects: 9, done.
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9
Unpacking objects: 100% (9/9), done.
Checking connectivity... done.

Good!

Github is really handy when it comes to sharing your app everywhere.

Except that, as we said before, Shiny Server expects the app to be in /srv/shiny-server/.

Create a shortcut in /srv/shiny-server/

The following commands will create the shortcut:

$ cd /srv/shiny-server
$ sudo ln -s ~/movie-explorer .

Now, if you display the list of files, you’ll see the shortcut along the other files:

$ ls
index.html  movie-explorer  sample-apps

And we can remove the other files, by the way:

$ sudo rm index.html
$ sudo rm -R sample-apps

Alright!

We added our app in the right folder so… it should be accessible now!

Let’s test it.

If I type 3.121.42.9:3838 in my browser, I get…

Root folder

Oops.

Not really welcoming. We’ll change that later.

Click on the link, and…

ERROR: An error has occurred. Check your logs or contact the app author for clarification.

Erk :(

Hm.

It seems the server has indeed noticed the changes, but it doesn’t work.

Why?

3. How to configure Shiny Server?

We’ll have to change the config a little bit.

The config file is in /etc/shiny-server/shiny-server.conf.

We’ll open it and try to understand what’s going on:

sudo nano /etc/shiny-server/shiny-server.conf

The file should contain the following:

# Instruct Shiny Server to run applications as the user "shiny"
run_as shiny;

# Define a server that listens on port 3838
server {
  listen 3838;

  # Define a location at the base URL
  location / {

    # Host the directory of Shiny Apps stored in this directory
    site_dir /srv/shiny-server;

    # Log all Shiny output to files in this directory
    log_dir /var/log/shiny-server;

    # When a user visits the base URL rather than a particular application,
    # an index of the applications available in this directory will be shown.
    directory_index on;
  }
}

Let’s tackle it line by line:

  • First line run_as shiny indicates the user behind the Shiny Server. When you logged in through SSH, you acted as user ubuntu, the default one. But when we installed Shiny Server, we created a new user called shiny, and that’s the one running the Shiny Server.
  • Then, we have listen 3838. That’s the port. That’s why we added :3838 at the end of the URL.
  • The line site_dir /srv/shiny-server tells you where you need to put the files of the app. That’s indeed where we created the shortcut.
  • Finally, log_dir /var/log/shiny-server shows you where the logs are stored. That’s super useful in case something’s buggy. We’ll look into it very soon.
  • Oh and, the last line directory_index on, as the comment states, enables the server to show the directories when there is no index.html. That’s what we saw when we went directly on 3.121.42.9:3838. I don’t like it much, so I prefer to deactivate it: directory_index off. Plus, I don’t want everyone to know all the apps I have.

We need to make one important addition to this file. Right at the top, add:

preserve_logs true;

This will enable you to keep the logs under any circumstances.

I don’t know why, but Shiny has the bad habit of deleting log files when it considers nothing important happens. But sometimes it does it even when the app crashes.

To take into account the changes, we need to reload the server:

$ sudo systemctl reload shiny-server

Speaking of logs.. let’s see why our app crashes!

4. How to debug a Shiny app with the logs

To display them:

$ cd /var/log/shiny-server
$ ls
movie-explorer-shiny-20181210-080534-46879.log rmd-shiny-20181023-144836-40079.log

I see two files.

The first one, named movie-explorer, is our app. The second one is from the default app.

The numbers show you the date. Notice the first file was created on 2018-12-10 at 08:05:34. When the log files start to accumulate, it’s very handy to know when it was created.

If you don’t see any file, try to relaunch the app in your browser. That’s the issue of the disappearing log files I was mentioning before. It should be solved with the new config.

If I display the last lines of the log file, I see:

$ sudo tail movie-explorer-shiny-20181210-080534-46879.log
Warning message:
replacing previous import by ‘Rcpp::evalCpp’ when loading ‘later’ 

Listening on http://127.0.0.1:44533
Warning: Error in library: there is no package called ‘ggvis’
  48: stop
  47: library


Execution halted

Ooooh, right!

We haven’t installed the libraries!

Obviously, it doesn’t work!

Plus, we need to install them for the user shiny, as it’s the one running the Shiny Server.

Let’s do it:

$ sudo su - shiny
$ R
> install.packages("ggvis")

You’ll get asked if you want to use a local library, say yes.

Once the package installed, try again launching the app.

This time, you’ll see some progress:

App Shiny crash

Something showed up!

..but then it crashed.

If you go back to the log files, you’ll notice a new file was created, and it says:

$ sudo tail movie-explorer-shiny-20181211-083747-41773.log
The following objects are masked from ‘package:base’:

    intersect, setdiff, setequal, union

Warning: Error in : The dbplyr package is required to communicate with database backends.
  56: <Anonymous>
Error : The dbplyr package is required to communicate with database backends.


Execution halted

Another package is missing! dbplyr.

Note: After installing the package for the user shiny, you can go back to the main user ubuntu by typing exit in the console.

Now you know how to install the package.

Oh, and, spoiler alert, you’ll also need to install RSQLite!

And finally…

Shiny AWS Movie Explorer Final

Yay o/

After:

  • Creating our server on AWS
  • Installing R and R Shiny
  • Sending our app on the server
  • Debugging the app from the logs
  • Configuring the server

We FINALLY can access our app and share it with everyone in the world!

Good job ;)

Now, you probably have some questions left, like…

  • How to get rid of this ugly IP address?
  • How to set up authentification to restrict the access?
  • How to secure the app in HTTPS?
  • How to optimize the app for many concurrent users?

Start with Creating a nice domaine name.

We’ll tackle these in future articles, so stay tuned!

You like my articles?

To get updated when I publish new articles, subscribe here:

Updated:

Comments

Greg

Thank you very much for this series. I currently have issues installing dplyr because of lacking memory during compilation. The installation hangs at “compile mutate.cpp” just like in this post. Have you experienced that as well?

Charles

Hey Greg!

I’ve had this issue before, and the only way I solved it was by increasing the memory of the machine I was working on.

If you’re using an AWS server, you can increase its capacity temporarily to install the packages, and once they’re installed, you can decrease the memory again. You need the extra memory only for installation.

Greg

Hey Charles!

I eventually solved it by creating 1GB of swap space with sudo fallocate. Worked like a charm and didn’t require any changes via the AWS GUI.

Francisco Goitia

Hi. Thank you for the series, it has been very useful. Are you going to tackle how to optimize the app for multiple concurrent users ?

Charles

Hi Francisco! I have a couple articles to write before that, but yes eventually I want to write about optimizing a Shiny app for multiple concurrent users. That’s a common issue and there are very interesting tools that help a lot when you know how to use them.

Leave a Comment

Required fields are marked *

Loading...

Comments are validated manually. The page will refresh after validation.