Sentry: running self-hosted errors tracking system on an AWS EC2

Previously we used cloud-based Sentry version but then reached emails limit and our backend-team left without those notifications which…

Previously we used cloud-based Sentry version but then reached emails limit and our backend-team left without those notifications which are critical for their work.

A self-hosted version was planned a long time ago so now we have a chance to spin it up.

The post below describes how to start self-hosted Sentry on an AWS EC2 instance, configure email and test if from a Python.

Will use Sentry on-premise repository here.

Run AWS EC2, configure SSL from Let’s Encrypt, install NGINX, Docker and Docker Compose.

Those steps are described in details in the Bitwarden: an organization’s password manager self-hosted version installation on an AWS EC2 post, so I’ll skip them here.

Use EC2 type t3.medium as Sentry requires 3 GiB memory as minimal.

Running Sentry

Clone repository:

root@bttrm-sentry:/home/admin# mkdir /opt/sentry
root@bttrm-sentry:/home/admin# cd /opt/sentry/
root@bttrm-sentry:/opt/sentry# git clone https://github.com/getsentry/onpremise.git
root@bttrm-sentry:/opt/sentry# cd onpremise/

Create Docker Volumes for data and PostgreSQL database:

root@bttrm-sentry:/opt/sentry/onpremise# docker volume create — name=sentry-data && docker volume create — name=sentry-postgres
sentry-data
sentry-postgres

Create config-file:

root@bttrm-sentry:/opt/sentry/onpremise# cp -n .env.example .env

Build images:

root@bttrm-sentry:/opt/sentry/onpremise# docker-compose build
smtp uses an image, skipping
memcached uses an image, skipping
redis uses an image, skipping
postgres uses an image, skipping
Building web
Step 1/1 : FROM sentry:9.1-onbuild
9.1-onbuild: Pulling from library/sentry
…
Successfully built 4840fec904c8
Successfully tagged onpremise_worker:latest

Generate SECRET_KEY:

root@bttrm-sentry:/opt/sentry/onpremise# docker-compose run — rm web config generate-secret-key
Creating network “onpremise_default” with the default driver
Pulling smtp (tianon/exim4:)…
latest: Pulling from tianon/exim4
…
Status: Downloaded newer image for tianon/exim4:latest
Pulling memcached (memcached:1.5-alpine)…
1.5-alpine: Pulling from library/memcached
…
Status: Downloaded newer image for memcached:1.5-alpine
Pulling redis (redis:3.2-alpine)…
3.2-alpine: Pulling from library/redis
…
Status: Downloaded newer image for redis:3.2-alpine
Pulling postgres (postgres:9.5)…
9.5: Pulling from library/postgres
…
Creating onpremise_smtp_1 … done
Creating onpremise_postgres_1 … done
Creating onpremise_memcached_1 … done
Creating onpremise_redis_1 … done
y0%***1yz

The last string in the output — с y0%***1yz — is our key.

Edit the .env file and set this SECRET_KEY.

Populate database and create an admin user:

root@bttrm-sentry:/opt/sentry/onpremise# docker-compose run — rm web upgrade
Starting onpremise_redis_1 … done
Starting onpremise_postgres_1 … done
Starting onpremise_memcached_1 … done
Starting onpremise_smtp_1 … done
12:06:24 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
12:06:27 [INFO] sentry.plugins.github: apps-not-configured
Syncing…
Creating tables …
Creating table django_admin_log
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table django_content_type
Creating table django_session
Creating table django_site
Creating table south_migrationhistory
Installing custom SQL …
Installing indexes …
Installed 0 object(s) from 0 fixture(s)
Migrating…
Running migrations for sentry:
- Migrating forwards to 0472_auto__add_field_sentryapp_author.
…
Created internal Sentry project (slug=internal, id=1)
Would you like to create a user account now? [Y/n]: y
Email: admin@example.com
Password:
Repeat for confirmation:
Should this user be a superuser? [y/N]: Y
User created: admin@example.com
Added to organization: sentry
…

Run the stack:

root@bttrm-sentry:/opt/sentry/onpremise# docker-compose up
onpremise_redis_1 is up-to-date
onpremise_memcached_1 is up-to-date
onpremise_postgres_1 is up-to-date
onpremise_smtp_1 is up-to-date
Creating onpremise_worker_1 … done
Creating onpremise_web_1 … done
Creating onpremise_cron_1 … done
…

Check containers:

root@bttrm-sentry:/opt/sentry/onpremise# docker ps

Open a browser and check if Sentry is working:

Log in, here you can configure email but I did it later using AWS SES:

Press Next, check it’s working:

Not sure what’s the worker message here, but it disappeared later.

Now, stop containers and create systemd unit-file - /etc/systemd/system/sentry.service:

root@bttrm-sentry:/opt/sentry/onpremise# systemctl edit — force sentry

Set here:

[Unit]
Description=Sentry service
Requires=docker.service
After=docker.service

[Service]
Restart=always
WorkingDirectory=/opt/sentry/onpremise

# Compose up
ExecStart=/usr/local/bin/docker-compose -f docker-compose.yml up

# Compose down, remove containers and volumes
ExecStop=/usr/local/bin/docker-compose -f docker-compose.yml down -v

[Install]
WantedBy=multi-user.target

In case of errors like:

May 18 12:38:33 bttrm-sentry systemd[1]: sentry.service: Service has more than one ExecStart= setting, which is only allowed for Type=oneshot services. Refusing.

Add one more ExecStart, just empty:

[Unit]
Description=Sentry service
Requires=docker.service
After=docker.service

[Service]
Restart=always
WorkingDirectory=/opt/sentry/onpremise

# Compose up
ExecStart=
ExecStart=/usr/local/bin/docker-compose -f docker-compose.yml up

# Compose down, remove containers and volumes
ExecStop=/usr/local/bin/docker-compose -f docker-compose.yml down -v

[Install]
WantedBy=multi-user.target

Didn’t saw such the error earlier, just googled this solution here>>>.

Run the service:

root@bttrm-sentry:/opt/sentry/onpremise# systemctl start sentry

Check its status:

root@bttrm-sentry:/opt/sentry/onpremise# systemctl status sentry
● sentry.service — Sentry service
Loaded: loaded (/etc/systemd/system/sentry.service; disabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/sentry.service.d
└─override.conf
Active: active (running) since Sat 2019–05–18 12:39:21 UTC; 12min ago
Main PID: 9209 (docker-compose)

Okay, all good.

Add to autostart:

root@bttrm-sentry:/opt/sentry/onpremise# systemctl enable sentry

Email

Email configuration documantation is available her>>>.

Edit docker-compose.yml, set variables, here is AWS SES example:

...
    SENTRY_EMAIL_HOST: email-smtp.us-east-1.amazonaws.com
    SENTRY_EMAIL_PORT: 587
    SENTRY_EMAIL_PASSWORD: BH3***gpM
    SENTRY_EMAIL_USER: AKI***OAQ
    SENTRY_EMAIL_USE_TLS: "true"
    SENTRY_SERVER_EMAIL: no-reply@example.com
...

Pay attention to quotes around the “true”:

SENTRY_EMAIL_USE_TLS: “true”

Save and restart Sentry service:

root@bttrm-sentry:/opt/sentry/onpremise# systemctl restart sentry

Python Sentry

Create a new project:

Get token:

Check from a workstation — install sentry-sdk:

[setevoy@setevoy-arch-work ~ ] $ sudo pip install sentry-sdk
[setevoy@setevoy-arch-work ~ ] $ python
…
>>> import sentry_sdk
>>> sentry_sdk.init(“https://96e***c44@sentry.example.com/2")
<sentry_sdk.hub._InitGuard object at 0x7efd4c487128>
>>> sentry_sdk.capture_message(“Hello World”)
‘736347c534cb4e3eb27fb3e3c0641439’

Check events in Sentry

And email:

Done.

Originally published at RTFM: Linux, DevOps and system administration.