Compare commits

..

8 Commits

Author SHA1 Message Date
38607ec437 Add margin notes 2021-04-09 05:52:30 +00:00
8fb1b29aef Add My Backup Strategy article 2021-04-09 05:18:18 +00:00
21c25413fc Support setting an article as "wide" 2021-04-09 05:17:51 +00:00
3e0e0beb0f Add styling for <code> tags 2021-04-09 05:01:11 +00:00
fac9de10aa Add copyright 2021-04-09 01:11:55 +00:00
e9c31546ab Reduce text contrast
eee -> ddd
fff -> eee
2021-04-09 00:44:39 +00:00
be19efb887 Make the article pages minimal 2021-04-09 00:36:39 +00:00
43b28bc982 Update email 2021-04-08 23:50:08 +00:00
8 changed files with 520 additions and 110 deletions

341
content/backup-strategy.md Normal file
View File

@ -0,0 +1,341 @@
Title: My Backup Strategy
Date: 2021-04-08
Category: Writing
Summary: Details about the backup system for my data.
Wide: true
[TOC]
Regularly backing up all the data I care about is very important to me. This
article outlines my strategy to make sure I never lose essential data.
## Motivation
Backups should be as automatic as possible. This ensures laziness and
forgetfulness won't interfere with the regularity.
All software used to create and store the backups should be free and open source
so I'm not depending on the survival of a company.
Backups need to be tested to ensure they are correct and happening regularly.
Multiple copies of the backups should exist, including at least one offsite to
protect against my building burning down.
Backups should also be incremental when possible (rather than mirror copies) so
an accidental deletion isn't propagated into the backups, making the file
irrecoverable.
## Strategy
I have one backup folder `/mnt/backup` on my media server at home that serves as
the destination for all my backup sources. All scheduled automatic backups write
to their own subfolder inside of it.
This backup folder is then synced to encrypted 2.5" 1 TB hard drives which I
rotate between my bag, offsite, and my parent's house.
## Backup Sources
I use the tool `rdiff-backup` extensively because it allows me to take
incremental backups locally or over SSH. It acts very similar to `rsync` and has
no configuration.
### Email
I have every email since 2010 backed up continuously in case my email provider
disappears.
I use `offlineimap` to sync my mail to the directory `~/email` on my media
server as a Maildir. Since offlineimap is only a syncing tool, the emails need
to be copied elsewhere to be backed up. I run `rdiff-backup` from a weekly cron
job:
<span class="aside">I'll explain what backup_check.txt does below</span>
```
*/15 * * * * offlineimap > /var/log/offlineimap.log 2>&1
00 12 * * 1 date -Iseconds > /home/email/email/backup_check.txt
20 12 * * 1 rdiff-backup /home/email/email /mnt/backup/local/email/
40 12 * * 1 rdiff-backup --remove-older-than 12B --force /mnt/backup/local/email/
```
Here's my `.offlineimaprc` for reference:
```
[general]
accounts = main
[Account main]
localrepository = Local
remoterepository = Remote
[Repository Local]
type = Maildir
localfolders = ~/email
[Repository Remote]
type = IMAP
readonly = True
folderfilter = lambda foldername: foldername not in ['Trash', 'Spam', 'Drafts']
remotehost = example.com
remoteuser = mail@example.com
remotepass = supersecret
sslcacertfile = /etc/ssl/certs/ca-certificates.crt
```
### Notes
I use Standard Notes to take notes and wrote the tool
[standardnotes-fs](https://github.com/tannercollin/standardnotes-fs) to mount my
notes as a file system to view and edit them as plain text files.
I take weekly backups of the mounted file system on my media server with cron:
```
00 12 * * 1 date -Iseconds > /home/notes/notes/backup_check.txt
15 12 * * 1 rdiff-backup /home/notes/notes /mnt/backup/local/notes/
```
### Nextcloud
I self-host a Nextcloud instance to store all my personal documents (non-code
projects, tax forms, spreadsheets, etc.). Since it's only a syncing software,
the files need to be copied elsewhere to be backed up.
I take weekly backups of the Nextcloud data folder with cron:
```
00 12 * * 1 rdiff-backup /var/www/nextcloud/data/tanner/files /mnt/backup/local/nextcloud/
30 12 * * 1 rdiff-backup --remove-older-than 12B --force /mnt/backup/local/nextcloud/
```
### Gitea
I self-host a Gitea instance to store all my git repositories for code-based
projects. My home folder is also a git repo so I can easily sync my config files
and password database between servers and machines.
I take weekly backups of the Gitea data folder with cron:
```
00 12 * * 1 date -Iseconds > /home/gitea/gitea/data/backup_check.txt
10 12 * * 1 rdiff-backup --exclude **data/indexers --exclude **data/sessions /home/gitea/gitea/data /mnt/backup/local/gitea/
35 12 * * 1 rdiff-backup --remove-older-than 12B --force /mnt/backup/local/gitea/
```
### Telegram
Telegram Messenger is my main app for communication. My parents, most of my
friends, and friend groups are on there so I don't want to lose those messages
in case Telegram disappears or my account gets banned.
Telegram includes a data export feature, but it can't be automated. Instead I
run the deprecated software
[telegram-export](https://github.com/expectocode/telegram-export) hourly with
cron:
```
0 * * * * bash -c 'timeout 50m /home/tanner/opt/telegram-export/env/bin/python -m telegram_export' > /var/log/telegramexport.log 2>&1
```
It likes to hang, so `timeout` kills it if it's still running after 50 minutes.
Hasn't corrupted the database yet.
### Phone
[Signal
Messenger](https://play.google.com/store/apps/details?id=org.thoughtcrime.securesms&hl=en_CA&gl=US)
automatically exports a copy of my text messages database, and
[Aegis](https://play.google.com/store/apps/details?id=com.beemdevelopment.aegis&hl=en_CA&gl=US)
allows me to export an encrypted JSON file of my two-factor authentication
codes.
I mount my phone's internal storage as a file system on my desktop using
[adbfs-rootless](https://github.com/spion/adbfs-rootless). I then rsync the
files over to my media server:
```
$ ./adbfs ~/mntphone
$ time rsync -Wav \
--exclude '*cache' --exclude nobackup \
--exclude '*thumb*' --exclude 'Telegram *' \
--exclude 'collection.media' \
--exclude 'org.thunderdog.challegram' \
--exclude '.trashed-*' --exclude '.pending-*' \
~/mntphone/storage/emulated/0/ \
localmediaserver:/mnt/backup/files/phone/
```
Unfortunately this is a manual process because I need to plug my phone in each
time. Ideally it would happen automatically while I'm asleep and the phone is
charging.
### Miscellaneous Files
The directory `/backup/files` is a repository for any kind of files I want to
keep forever. My phone data, old archives, computer files, Minecraft worlds,
files from previous jobs, and so on.
All the files will be included in the 1 TB hard drive backup rotations.
### Web Services
Web services that I run like [txt.t0.vc](https://txt.t0.vc) and
[QotNews](https://news.t0.vc) are backed up daily, weekly, and monthly depending
on how frequently the data changes.
I run `rdiff-backup` on the remote server with cron:
```
00 14 * * * date -Iseconds > /home/tanner/tbot/t0txt/data/backup_check.txt
04 14 * * * rdiff-backup /home/tanner/tbot/t0txt/data tbotbak@remotebackup::/mnt/backup/remote/tbotbak/daily/t0txt/
14 14 * * * rdiff-backup --remove-older-than 12B --force tbotbak@remotebackup::/mnt/backup/remote/tbotbak/daily/t0txt/
24 14 * * 1 rdiff-backup /home/tanner/tbot/t0txt/data tbotbak@remotebackup::/mnt/backup/remote/tbotbak/weekly/t0txt/
34 14 * * 1 rdiff-backup --remove-older-than 12B --force tbotbak@remotebackup::/mnt/backup/remote/tbotbak/weekly/t0txt/
44 14 1 * * rdiff-backup /home/tanner/tbot/t0txt/data tbotbak@remotebackup::/mnt/backup/remote/tbotbak/monthly/t0txt/
55 14 1 * * rdiff-backup --remove-older-than 12B --force tbotbak@remotebackup::/mnt/backup/remote/tbotbak/monthly/t0txt/
```
The user `tbotbak` has write access only to the `/mnt/backup/remote/tbotbak`
directory. It has its own passwordless SSH key that's only permitted to run the
`rdiff-backup --server` command for security.
### Protospace
I run a lot of services for [Protospace](https://protospace.ca/), my city's
makerspace.
The member portal I wrote called [Spaceport](https://my.protospace.ca/) creates
an archive I download daily:
```
40 10 * * * wget --content-disposition \
--header="Authorization: secretkeygoeshere" \
--directory-prefix /mnt/backup/remote/portalbak/ \
--no-verbose --append-output=/var/log/portalbackup.log \
https://api.my.protospace.ca/backup/
```
The main website and [wiki](https://wiki.protospace.ca) that I sysadmin gets
backed up weekly:
```
0 12 * * 1 mysqldump --all-databases > /var/www/dump.sql
15 12 * * 1 date -Iseconds > /var/www/backup_check.txt
20 12 * * 1 rdiff-backup /var/www pshostbak@remotebackup::/mnt/backup/remote/pshostbak/weekly/www/
```
The Protospace [Minecraft
server](http://games.protospace.ca:8123/?worldname=world&mapname=flat&zoom=3&x=74&y=64&z=354)
I run gets backed up daily:
```
00 15 * * * date -Iseconds > /home/tanner/minecraft/backup_check.txt
00 15 * * * rdiff-backup --exclude **CoreProtect --exclude **dynmap /home/tanner/minecraft psminebak@remotebackup::/mnt/backup/remote/psminebak/
30 15 * * * rdiff-backup --remove-older-than 12B --force psminebak@remotebackup::/mnt/backup/remote/psminebak/
```
I also back up our Google Drive with rclone:
```
45 12 * * 1 rclone copy -v protospace: /mnt/backup/files/protospace/google-drive/
```
## Backup Copies
My backup folder `/mnt/backup` now looks like this:
```
/mnt/backup/
├── files
│   ├── docs
│   ├── phone
│   ├── protospace
│   ├── telegram
│   ├── usbsticks
│   └── ... and so on
├── local
│   ├── email
│   ├── gitea
│   ├── nextcloud
│   └── notes
└── remote
├── portalbak
├── pshostbak
├── psminebak
├── tbotbak
└── telebak
```
This directory tree is the master backup and I make a copy of the entire tree
every Saturday to a hard drive.
The directory is copied over with the following script:
```text
#!/bin/bash
cryptsetup luksOpen /dev/sdf external
mount /dev/mapper/external /mnt/external
time rsync -av --delete /mnt/backup/local/ /mnt/external/backup/local/
time rsync -av --delete /mnt/backup/remote/ /mnt/external/backup/remote/
time rdiff-backup --force -v5 /mnt/backup/files/ /mnt/external/backup/files/
python3 /home/tanner/scripts/checkbackup.py
umount /mnt/external
cryptsetup luksClose external
```
I wrote a Python script `checkbackup.py` that goes through each backup and
compares the timestamp in `backup_check.txt` files to the current time. This
makes sure that the cron ran, backups were taken, and transferred over
correctly.
## Rotating Hard Drives
I rotate through 2.5" 1 TB hard drives each Saturday when I do a backup. They
are quite cheap at [$65 CAD](https://www.memoryexpress.com/Products/MX65194)
each so I can have a bunch floating around.
I keep one connected to the server, one in my bag, one offsite, one at my
mother's house, and one at my dad's house. Every Saturday I run the script above
to take a copy and then swap the drive with the one in my bag. It then gets
<span class="aside">I go back home about twice per year</span>
swapped when I visit my offsite location. Same for when I visit my parents. This
means that all hard drives eventually get rotated through with new data and
don't sit too long unpowered.
The drives are all encrypted with full-disk LUKS encryption using a password I'm
unlikely to forget.
I run the check-summing `btrfs` file system on them in RAID-1 to protect against
bitrot. This means I can only use 0.5 TB of storage for my backups, but the data
is stored redundantly.
Here's how I set up new hard drives to do this:
```
$ sudo cryptsetup luksOpen /dev/sdf external
$ sudo mkfs.btrfs -f -m dup -d dup /dev/mapper/external
$ sudo mount /dev/mapper/external /mnt/external/
$ sudo mkdir /mnt/external/backup
$ sudo chown -R tanner:tanner /mnt/external/backup
$ sudo umount /mnt/external
$ sudo cryptsetup luksClose external
```
## Future Improvements
I'm working on a system to automatically back up all my home directories to my
media server. I need this to grab Bash histories and code that's
work-in-progress. I've been burned by not having this once when a server died.
I'd like to automate backing up my phone by connecting it to a Raspberry Pi when
I go to sleep.
I need to get better at fully testing my backups by restoring them on a blank
machine.

View File

@ -27,4 +27,7 @@ by an AC-DC converter.
This entire process happens quicker than half a second, so it feels instant.
![a rusted metalic hand]({static}/images/light-switch/light2.jpg)
<span class="aside">Black stuff's liquid electrical tape</span>
![a rusted metalic hand]({static}/images/light-switch/light3.jpg)

View File

@ -2,6 +2,7 @@ Title: Choosing a Linux Flavour
Date: 2020-10-31
Category: Writing
Summary: A recommendation on which flavour of Linux to run.
Wide: true
[TOC]
@ -14,6 +15,7 @@ I run Debian on my computers and servers.
## Linux Distributions
When people refer to the "flavour of Linux" they are talking about a Linux
<span class="aside">Interjection: it's technically called GNU/Linux</span>
distribution (distro). It mostly describes what software is distributed in its
software repository.

View File

@ -8,7 +8,7 @@ It's also great for when I'm on vacation. The plant is a year old now and
doesn't look as good as it used to (kinda like you). So this machine is like its
life support.
Update: this plant died long ago.
<span class="aside">Update: this plant died long ago</span>
![the device and pump on a 2L pop bottle with a tube running to a flowerpot]({static}/images/plant-waterer/waterer1.jpg)
@ -36,4 +36,6 @@ Another feature was the ability to run the pump backwards. This completely
eliminated the siphoning problem from before. After pumping for a set duration,
it would run backwards until the tube was cleared of water.
<span class="aside">Also dead :(</span>
![the new version beside a big Min aralia plant]({static}/images/plant-waterer/waterer3.jpg)

View File

@ -24,22 +24,41 @@
{% endblock %}
{% block content %}
<p><a href="/">← Return to Home</a></p>
<header>
<h1>{{ article.title }}</h1>
<div class="summary">
{{ article.summary }}
</div>
<p class="metadata">
{{ article.locale_date }}
{% if article.modified %}
— updated {{ article.locale_modified }}
{% endif %}
{% block info %}
Tanner Collin
<p class='theme-select'>
<a onClick="setTheme('light')">Light</a> / <a onClick="setTheme('dark')">Dark</a>
</p>
</header>
<hr />
<article>
{{ article.content }}
</article>
{% endblock %}
{% block content %}
{% if article.wide %}
<div class="content content-wide">
{% else %}
<div class="content">
{% endif %}
<div class="topbar">
{{ info() }}
</div>
<p><a href="/">← Return to Home</a></p>
<header>
<h1>{{ article.title }}</h1>
<div class="summary">
{{ article.summary }}
</div>
<p class="metadata">
{{ article.locale_date }}
{% if article.modified %}
— updated {{ article.locale_modified }}
{% endif %}
</p>
</header>
<hr />
<article>
{{ article.content }}
</article>
</div>
{% endblock %}

View File

@ -44,48 +44,30 @@
</noscript>
</head>
{% if PROD %}
<body class="<?php echo $themeClass; ?>">
{% else %}
<body>
{% endif %}
<div class="container">
<div class="sidebar">
<img src="/theme/me.jpg" class="me" alt="A picture of me smiling" />
<div class="info">
<p>
Tanner Collin
</p>
<p class="contact-icons">
<a href="mailto:site@tannercollin.com" rel="noreferrer noopener"><img alt="email icon" src="/theme/mail.svg" /></a>
<a href="https://t.me/tannercollin" target="_blank" rel="noreferrer noopener"><img alt="telegram logo" src="/theme/telegram.svg" /></a>
<a href="https://github.com/tannercollin" target="_blank" rel="noreferrer noopener"><img alt="github logo" src="/theme/github.svg" /></a>
</p>
<p class='theme-select'>
<a onClick="setTheme('light')">Light</a> / <a onClick="setTheme('dark')">Dark</a>
</p>
</div>
</div>
<div class="content">
<div class="topbar">
<img src="/theme/me.jpg" class="me" alt="A picture of me smiling" />
<div class="info">
Tanner Collin
<p class="contact-icons">
<a href="mailto:site@tannercollin.com" rel="noreferrer noopener"><img alt="email icon" src="/theme/mail.svg" /></a>
<a href="https://t.me/tannercollin" target="_blank" rel="noreferrer noopener"><img alt="telegram logo" src="/theme/telegram.svg" /></a>
<a href="https://github.com/tannercollin" target="_blank" rel="noreferrer noopener"><img alt="github logo" src="/theme/github.svg" /></a>
</p>
<p class='theme-select'>
<a onClick="setTheme('light')">Light</a> / <a onClick="setTheme('dark')">Dark</a>
</p>
</div>
</div>
{% block content %}
{% endblock %}
</div>
{% if PROD %}
<body class="<?php echo $themeClass; ?>">
{% else %}
<body>
{% endif %}
{%- macro info() -%}
{% block info %}{% endblock %}
{%- endmacro -%}
<div class="container">
<div class="sidebar">
{{ info() }}
</div>
{% block content %}
{% endblock %}
</div>
<p class="copyright">
© 20122021 Tanner Collin
</p>
<script>
function setTheme(theme) {
console.log('Setting theme to', theme);

View File

@ -7,43 +7,64 @@
<meta name="summary" content="The personal website of Tanner Collin." />
{% endblock %}
{% block content %}
<p>
Hi, I'm Tanner! I do firmware and web development in Calgary.
</p>
<h2>Contact Info</h2>
<p>
Email: <a href="mailto:site@tannercollin.com">site@tannercollin.com</a> <br />
Telegram: <a href="https://t.me/tannercollin" target="_blank" rel="noreferrer noopener">@tannercollin</a>
</p>
<h2>Resume</h2>
<ul>
<li>Firmware Engineer at <a href="https://cabanablockchain.com" target="_blank" rel="noreferrer noopener">Cabana Blockchain</a>, 2018</li>
<li>Lead Hardware Engineer at <a href="https://criticalcontrol.com/" target="_blank" rel="noreferrer noopener">Critical Control</a>, 20162018</li>
<li>Electrical Engineer at <a href="https://www.opener.aero/" target="_blank" rel="noreferrer noopener">Opener Aero</a>, 20162016</li>
<li>Electrical Engineer Intern at <a href="https://www.pason.com/" target="_blank" rel="noreferrer noopener">Pason Systems</a>, 20142015</li>
<li>BSc. Electrical Engineering from University of Calgary</li>
</ul>
<h2>Projects</h2>
<p>
My main hobby is working on technical projects. I typically design websites or
build tools that make my life easier. Sometimes art.
</p>
<p>
You can find my code on <a href="https://github.com/tannercollin" target="_blank" rel="noreferrer noopener">GitHub</a>.
</p>
{% for article in articles_page.object_list if article.category.name == 'Projects' %}
<h3><a href="{{ article.url }}">{{ article.title }}</a></h3>
<div class="summary">
{{ article.summary }}
</div>
{% endfor %}
{% block info %}
<img src="/theme/me.jpg" class="me" alt="A picture of me smiling" />
<div class="info">
Tanner Collin
<p class="contact-icons">
<a href="mailto:site2@tannercollin.com" rel="noreferrer noopener"><img alt="email icon" src="/theme/mail.svg" /></a>
<a href="https://t.me/tannercollin" target="_blank" rel="noreferrer noopener"><img alt="telegram logo" src="/theme/telegram.svg" /></a>
<a href="https://github.com/tannercollin" target="_blank" rel="noreferrer noopener"><img alt="github logo" src="/theme/github.svg" /></a>
</p>
<p class='theme-select'>
<a onClick="setTheme('light')">Light</a> / <a onClick="setTheme('dark')">Dark</a>
</p>
</div>
{% endblock %}
{% block content %}
<div class="content">
<div class="topbar">
{{ info() }}
</div>
<p>
Hi, I'm Tanner! I do firmware and web development in Calgary.
</p>
<h2>Contact Info</h2>
<p>
Email: <a href="mailto:site2@tannercollin.com">site2@tannercollin.com</a> <br />
Telegram: <a href="https://t.me/tannercollin" target="_blank" rel="noreferrer noopener">@tannercollin</a>
</p>
<h2>Resume</h2>
<ul>
<li>Firmware Engineer at <a href="https://cabanablockchain.com" target="_blank" rel="noreferrer noopener">Cabana Blockchain</a>, 2018</li>
<li>Lead Hardware Engineer at <a href="https://criticalcontrol.com/" target="_blank" rel="noreferrer noopener">Critical Control</a>, 20162018</li>
<li>Electrical Engineer at <a href="https://www.opener.aero/" target="_blank" rel="noreferrer noopener">Opener Aero</a>, 20162016</li>
<li>Electrical Engineer Intern at <a href="https://www.pason.com/" target="_blank" rel="noreferrer noopener">Pason Systems</a>, 20142015</li>
<li>BSc. Electrical Engineering from University of Calgary</li>
</ul>
<h2>Projects</h2>
<p>
My main hobby is working on technical projects. I typically design websites or
build tools that make my life easier. Sometimes art.
</p>
<p>
You can find my code on <a href="https://github.com/tannercollin" target="_blank" rel="noreferrer noopener">GitHub</a>.
</p>
{% for article in articles_page.object_list if article.category.name == 'Projects' %}
<h3><a href="{{ article.url }}">{{ article.title }}</a></h3>
<div class="summary">
{{ article.summary }}
</div>
{% endfor %}
{% endblock %}

View File

@ -15,11 +15,22 @@ a {
pre {
font-size: 1rem;
padding: 1rem;
overflow-x: auto;
}
:not(pre)>code {
padding: 0 2px;
font-size: 0.9rem;
}
.container {
max-width: 56rem;
margin: 2rem auto 0 auto;
margin: 2rem auto 12rem auto;
}
.copyright {
font: 1rem/1.5 Apparatus SIL,serif;
text-align: center;
}
.theme-select {
@ -124,6 +135,10 @@ pre {
max-width: 36rem;
}
.content-wide {
max-width: none;
}
.content p {
font: 1.2rem/1.5 Apparatus SIL,serif;
}
@ -141,6 +156,15 @@ pre {
height: auto;
}
.content .aside {
display: inline;
float: left;
position: relative;
width: 8rem;
margin-left: -9rem;
font-size: 1rem;
}
.toclink:not(:hover)::after {
visibility: hidden;
}
@ -169,7 +193,7 @@ pre {
body {
background-color: #fff;
background-color: #eee;
color: #000;
}
@ -179,11 +203,15 @@ a {
}
pre {
background-color: #eee;
background-color: #ddd;
}
:not(pre)>code {
background-color: #ddd;
}
.toc {
background-color: #eee;
background-color: #ddd;
}
.content p.metadata {
@ -192,18 +220,22 @@ pre {
body.dark {
background-color: #000;
color: #fff;
color: #eee;
}
body.dark a {
color: #fff;
border-bottom: 1px solid #fff;
color: #eee;
border-bottom: 1px solid #eee;
}
body.dark pre {
background-color: #222;
}
body.dark :not(pre)>code {
background-color: #222;
}
body.dark .toc {
background-color: #222;
}
@ -222,7 +254,7 @@ body.dark .contact-icons img {
@media (prefers-color-scheme: dark) {
body.light {
background-color: #fff;
background-color: #eee;
color: #000;
}
@ -232,11 +264,15 @@ body.dark .contact-icons img {
}
body.light pre {
background-color: #eee;
background-color: #ddd;
}
body.light :not(pre)>code {
background-color: #ddd;
}
body.light .toc {
background-color: #eee;
background-color: #ddd;
}
body.light .content p.metadata {
@ -253,18 +289,22 @@ body.dark .contact-icons img {
body {
background-color: #000;
color: #fff;
color: #eee;
}
a {
color: #fff;
border-bottom: 1px solid #fff;
color: #eee;
border-bottom: 1px solid #eee;
}
pre {
background-color: #222;
}
:not(pre)>code {
background-color: #222;
}
.toc {
background-color: #222;
}