24 September 2023

Local Nextcloud with podman

by rusty-snake

Architecture
Caddy (Host)
Add new user for Nextcloud
- tmux
Installation
- Volumes
- Images
- Networks
- Containers
- caddy
- Nextcloud
  - Installation
Backup
What’s next?

Architecture

The overall architecture of this first concept contists of

A caddy webserver on the host started by a hardened systemd service. The rule of this caddy instance is to proper terminate the https connection and forward everything to the Nextcloud pod. This is a grown construct to workaround some problems and might get/should be replaced with something better in the future.
A caddy webserver does static file servering and acts as the reverse proxy to php-fpm.
A Nextcloud php-fpm container.
A Nextcloud container for cron.

flowchart LR
    caddy_host["Caddy (Host)"]---caddy
    subgraph Podman
        caddy("Caddy")-- nextcloud-fastcgi.network ---nextcloud
        nextcloud("Nextcloud")
        nextcloud_cron("Nextcloud (cron)")
    end

Caddy (Host)

Install caddy and harden caddy.service.

Configure caddy

`/etc/caddy/Caddyfile`

# The Caddyfile is an easy way to configure your Caddy web server.
#
# https://caddyserver.com/docs/caddyfile

# Global Options
# https://caddyserver.com/docs/caddyfile/options
{
	# Enable debug mode
	#debug

	# Disable the admin endpoint
	admin off

	# Disable persisted config. Admin API is disabled anyway.
	persist_config off

	log {
		format filter {
			wrap console
			fields {
				request>headers>Requesttoken replace "*** REDACTED ***"
			}
		}
	}

	local_certs
	skip_install_trust
	ocsp_stapling off
}

# As an alternative to editing the above site block, you can add your own site
# block files in the Caddyfile.d directory, and they will be included as long
# as they use the .caddyfile extension.
import Caddyfile.d/*.caddyfile

`/etc/caddy/Caddyfile.d/nextcloud.caddyfile`

# vim: set ft=caddyfile:

192.168.0.3:7777 {
	root * /tmp

	# TLS Options
	tls {
		# Increase minimal TLS version to 1.3.
		protocols tls1.3

		issuer internal {
			lifetime 365d
			sign_with_root
		}
	}

	# Enable HSTS with a duration of 1 year.
	header Strict-Transport-Security "max-age=31536000; includeSubDomains"

	# Restrict allowed ip ranges.
	@disallowed-ips not remote_ip 127.0.0.1 192.168.0.0/24
	abort @disallowed-ips

	@curl header User-Agent curl*
	respond @curl 403 {
		close
	}

	# Add a error page for errors generated by Caddy (e.g. 502 Bad Gateway).
	handle_errors {
		header Content-Type "text/html; charset=utf-8"
		respond "<h1>{err.status_code} {err.status_text}</h1>"
	}

	encode gzip
	reverse_proxy 127.0.0.1:7867
}

SELinux

Because you use SELinux, we have to adjust the policy of caddy (httpd_t) for our usecase.

Allow caddy (`httpd_t`) to bind to port 7777

semanage port -a -t http_port_t -p tcp 7777
semanage port -a -t http_port_t -p udp 7777

Allow caddy (`httpd_t`) to connect to port 7867

setsebool -P httpd_can_network_connect 1

TODO: Replace this boolean with an module allowing something like reverse_proxy_upstream_port_t and define port 7867 to be of this type.

Start caddy

systemctl enable --now caddy.service

Add caddys root certificate to the system trust store¹

cp /var/lib/caddy/.local/share/caddy/pki/authorities/local/root.crt /etc/pki/ca-trust/source/anchors/caddy-root.pem
chmod 644 /etc/pki/ca-trust/source/anchors/caddy-root.pem
update-ca-trust extract

View certificate fingerprint

openssl x509 -noout -fingerprint -sha256 -inform pem -in /var/lib/caddy/.local/share/caddy/certificates/local/192.168.0.100/192.168.0.3.crt

Open port 7777 in your firewall

firewall-cmd --permanent --add-port=7777/tcp

Add new user for Nextcloud

useradd --comment Nextcloud --no-create-home --home-dir /var/lib/nextcloud --shell /sbin/nologin --system --add-subids-for-system --user-group nextcloud
mkdir /var/lib/nextcloud
cp /etc/skel/.bash* /var/lib/nextcloud
chown -R nextcloud:nextcloud /var/lib/nextcloud
chmod 0700 /var/lib/nextcloud
loginctl enable-linger nextcloud

tmux

To administrate the Nextcloud installation, we will run a tmux session in the background that you can attach to.

sudo -u nextcloud tmux attach-session -t nc-admin0

tmux.service

[Unit]
Description=tmux terminal multiplexer
Documentation=man:tmux(1)

[Service]
Type=forking
ExecStart=/usr/bin/tmux start-server; new-session -s nc-admin0 -d
ExecStop=/usr/bin/tmux kill-server
# If you enable tmux.service rather than tmux.socket, add the following.
#Restart=always

[Install]
WantedBy=default.target

tmux.socket

[Unit]
Description=tmux terminal multiplexer socket
Documentation=man:tmux(1)

[Socket]
ListenStream=/tmp/tmux-%U/default
SocketMode=0660
DirectoryMode=0700

[Install]
WantedBy=sockets.target

mkdir -p /var/lib/nextcloud/.config/systemd/user/sockets.target.wants
ln -s /var/lib/nextcloud/.config/systemd/user/tmux.socket /var/lib/nextcloud/.config/systemd/user/sockets.target.wants/tmux.socket

/var/lib/nextcloud/.tmux.conf:

# Required because we configured /sbin/nologin as login shell.
set-option -g default-shell "/bin/bash"

#
## Optional, customizable to meet your own needs
#

bind-key c new-window -c "#{pane_current_path}"
bind-key % split-window -h -c "#{pane_current_path}"
bind-key '"' split-window -v -c "#{pane_current_path}"

set-option -g escape-time 0

bind-key -n M-w select-pane -U
bind-key -n M-a select-pane -L
bind-key -n M-s select-pane -D
bind-key -n M-d select-pane -R

/var/lib/nextcloud/.bashrc additions:

alias systemctl="systemctl --user"

Installation

Volumes

mkdir -p /var/lib/nextcloud/{app,data,caddy-data}

Because the space on my /var was a bit low I moved the data directory to /home.

mkdir /home/.nextcloud-data
chown nextcloud:nextcloud /home/.nextcloud-data
chmod 0700 /home/.nextcloud-data
ln -s /home/.nextcloud-data/data data

Images

caddy-rootless.containerfile:

FROM caddy:2

RUN apk --no-cache upgrade \
	&& adduser --home /var/lib/caddy --gecos "Caddy web server" --shell /sbin/nologin --ingroup www-data --system --uid 443 caddy \
	&& chown -R caddy:www-data /config /data

USER caddy

podman pull docker.io/library/caddy:2 docker.io/library/nextcloud:28-fpm-alpine
buildah bud -f caddy-rootless.containerfile -t caddy-rootless:2

Networks

~/.config/containers/systemd/nextcloud-fastcgi.network:

[Network]
NetworkName=nextcloud-fastcgi
# TODO
#Internal=true
#Options=isolate

Containers

~/.config/containers/systemd/nextcloud-caddy.container

[Unit]
Description=Caddy web server for Nextcloud
Documentation=man:podman-systemd.unit(5) man:caddy(8)
Wants=network-online.target
After=network-online.target

[Container]
Image=localhost/caddy-rootless:2
Pull=never
# TODO
#AutoUpdate=local
ContainerName=nextcloud-caddy

Volume=/var/lib/nextcloud/Caddyfile:/etc/caddy/Caddyfile:Z,ro
Volume=/var/lib/nextcloud/caddy-data:/data:Z,U,noexec
Volume=/var/lib/nextcloud/app:/var/www/html:z,ro,noexec
Network=nextcloud-fastcgi.network
Network=slirp4netns:port_handler=slirp4netns
PublishPort=127.0.0.1:7867:80

DropCapability=ALL
AddCapability=CAP_NET_BIND_SERVICE
NoNewPrivileges=true
#PidsLimit=
ReadOnly=true
PodmanArgs=--read-only-tmpfs=false
#ReadOnlyTmpfs=false
#SeccompProfile=
# hidepid
PodmanArgs=--security-opt=proc-opts=subset=pid

LogDriver=k8s-file
PodmanArgs=--log-opt=path=/var/lib/nextcloud/logs/caddy.log

# TODO
#HealthCmd=
#HealthOnFailure=stop
#HealthStartPeriod=1m
#HealthStartupCmd=
#HealthStartupTimeout=
#HealthTimeout=

[Service]
Restart=always

# TODO:
#ExecStop=
#ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%N.cid
#ExecStopPost=
#ExecStopPost=/usr/bin/podman rm -f -t 10 --cidfile=%t/%N.cid

[Install]
WantedBy=default.target

~/.config/containers/systemd/nextcloud-app.container

[Unit]
Description=Nextcloud
Documentation=man:podman-systemd.unit(5)
Wants=network-online.target
After=network-online.target

[Container]
Image=docker.io/library/nextcloud:28-fpm-alpine
Pull=never
# TODO
#AutoUpdate=local
ContainerName=nextcloud-app

Volume=/var/lib/nextcloud/app:/var/www/html:z,noexec
Volume=/var/lib/nextcloud/data:/srv/nextcloud/data:z,noexec
Network=nextcloud-fastcgi.network

DropCapability=ALL
AddCapability=CAP_SETUID CAP_SETGID CAP_CHOWN CAP_DAC_OVERRIDE CAP_FOWNER
NoNewPrivileges=true
ReadOnly=true
# Temporary to fix inconsistencies
PodmanArgs=--read-only-tmpfs=true
#SeccompProfile=
PodmanArgs=--security-opt=proc-opts=subset=pid

LogDriver=k8s-file
PodmanArgs=--log-opt=path=/var/lib/nextcloud/logs/nextcloud.log

# TODO
#HealthCmd=
#HealthOnFailure=stop
#HealthStartPeriod=1m
#HealthStartupCmd=
#HealthStartupTimeout=
#HealthTimeout=

[Service]
Restart=always

# TODO:
#ExecStop=
#ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%N.cid
#ExecStopPost=
#ExecStopPost=/usr/bin/podman rm -f -t 10 --cidfile=%t/%N.cid

[Install]
WantedBy=default.target

~/.config/containers/systemd/nextcloud-cron.container

[Unit]
Description=Nextcloud (cron)
Documentation=man:podman-systemd.unit(5)
Wants=network-online.target
After=network-online.target

[Container]
Image=docker.io/library/nextcloud:28-fpm-alpine
Pull=never
# TODO
#AutoUpdate=local
ContainerName=nextcloud-cron
PodmanArgs=--entrypoint=/cron.sh

Volume=/var/lib/nextcloud/app:/var/www/html:z,noexec
Volume=/var/lib/nextcloud/data:/srv/nextcloud/data:z,noexec
Network=nextcloud-fastcgi.network

DropCapability=ALL
AddCapability=CAP_SETUID CAP_SETGID
NoNewPrivileges=true
ReadOnly=true
# Temporary to fix inconsistencies
PodmanArgs=--read-only-tmpfs=true
#SeccompProfile=
PodmanArgs=--security-opt=proc-opts=subset=pid

LogDriver=k8s-file
PodmanArgs=--log-opt=path=/var/lib/nextcloud/logs/nextcloud-cron.log

# TODO
#HealthCmd=
#HealthOnFailure=stop
#HealthStartPeriod=1m
#HealthStartupCmd=
#HealthStartupTimeout=
#HealthTimeout=

[Service]
Restart=always

# TODO:
#ExecStop=
#ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%N.cid
#ExecStopPost=
#ExecStopPost=/usr/bin/podman rm -f -t 10 --cidfile=%t/%N.cid

[Install]
WantedBy=default.target

systemctl daemon-reload

caddy

Caddyfile (click to expand)

# Global Options
{
	# Uncomment the next line and restart/reload caddy to enable debug logging.
	#debug

	# Disable the admin endpoint.
	admin off

	# Disable persisted config. Filesystem is readonly and admin API is disabled anyway.
	persist_config off

	log {
		format filter {
			wrap console
			fields {
				request>headers>Requesttoken replace "*** REDACTED ***"
			}
		}
	}

	auto_https off
	local_certs
	skip_install_trust
	ocsp_stapling off

	servers {
		trusted_proxies static 10.0.2.2
	}
}

:80 {
	root * /var/www/html

	# Ensure security and privacy related headers are always set.
	# This is based on Nextcloud's .htaccess and OWASP recommendations.
	header {
		?Referrer-Policy no-referrer
		?X-Content-Type-Options nosniff
		?X-Frame-Options sameorigin
		?X-Permitted-Cross-Domain-Policies none
		?X-Robots-Tag "noindex, nofollow"
		# TODO: Better understand COxP stuff and Sec-Fetch-*.
		# https://github.com/nextcloud/server/issues/37154
		-X-XSS-Protection
	}

	# Cache-Control
	@static-resources path *.css *.js *.mjs *.svg *.gif *.png *.jpg *.ico *.wasm *.tflite *.map *.ogg *.flac
	handle @static-resources {
		map {query.v} {immutable} {
			"" ""
			default "immutable"
		}
		header Cache-Control "max-age=15778463, {immutable}"
	}
	@woff path *.woff *.woff2
	header @woff Cache-Control "max-age=604800"

	# /.well-known/ redirects
	redir /.well-known/carddav /remote.php/dav 301
	redir /.well-known/caldav /remote.php/dav 301
	rewrite /.well-known/acme-challenge /index.php?{query}
	rewrite /.well-known/pki-validation /index.php?{query}
	@well-known {
		path /.well-known/*
		not path /.well-known/carddav
		not path /.well-known/caldav
		not path /.well-known/acme-challenge
		not path /.well-known/pki-validation
	}
	redir @well-known /index.php{path}?{query} 301

	# Special handling for Microsoft DAV clients
	@ms-dav {
		header User-Agent DavClnt*
		path /
	}
	redir @ms-dav /remote.php/webdav 302

	# Add a error page for errors generated by Caddy (e.g. 502 Bad Gateway).
	handle_errors {
		header Content-Type "text/html; charset=utf-8"
		respond "<h1>{err.status_code} {err.status_text}</h1>"
	}

	php_fastcgi nextcloud-app:9000 {
		# Pretty URLs
		env front_controller_active true
		# Leaks php version
		header_down -X-Powered-By
	}
	@static-files path /core/* /apps/* /custom_apps/* /resources/* /themes/* /dist/* /index.html /robots.txt
	file_server @static-files
}

Nextcloud

Installation

podman exec nextcloud-app chown www-data:www-data /srv/nextcloud/data

Open https://192.168.0.3:7777/ in a browser and install Nextcloud. Use /srv/nextcloud/data as your data directory. Afterwards you must/can futher via occ.

podman exec -i -u www-data nextcloud-app sh <<EOF
php ./occ config:system:set trusted_proxies 0 --value=127.0.0.1

php ./occ config:system:set overwritehost --value=192.168.0.3:7777
php ./occ config:system:set overwriteprotocol --value=https
#php ./occ config:system:set overwritewebroot --value=/
php ./occ config:system:set overwrite.cli.url --value=https://192.168.0.3:7777/

php ./occ config:system:set trusted_domains 0 --value=127.0.0.1:7867
php ./occ config:system:set trusted_domains 1 --value=127.0.0.1:7777
php ./occ config:system:set trusted_domains 2 --value=192.168.0.3:7777

# # # # #

php ./occ config:system:set default_phone_region --value=DE

php ./occ config:system:set default_language --value=de_DE
php ./occ config:system:set default_locale --value=de_DE

php ./occ config:app:set richdocuments disable_certificate_verification --value=yes
php ./occ config:app:set password_policy enforceHaveIBeenPwned --value=0
php ./occ config:app:set privacy readableLocation --value=de
php ./occ config:app:set privacy fullDiskEncryptionEnabled --value=1
EOF

More options can be found at https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/config_sample_php_parameters.html.

The reverse proxy config can also be set from the environment:

TRUSTED_PROXIES=127.0.0.1

OVERWRITEHOST=192.168.0.3:7777

OVERWRITEPROTOCOL=https

OVERWRITEWEBROOT=/

OVERWRITECONDADDR=???

OVERWRITECLIURL=https://192.168.0.3:7777/

Backup

TODO: This is a grown mess and should be replaced with something better.

~/backup.sh:

#!/bin/bash
set -e

cd "${HOME:=$(getent passwd "$(id -u)" | cut -d: -f6)}"

NOW="$(date --utc --iso-8601=seconds)"


BACKUP_DIR="$HOME/backups"
BACKUP_FILE="$BACKUP_DIR/nextcloud-snapshot-$NOW.tar.zstd"
BACKUP_SOURCES=(./app/ ./Caddyfile ./caddy-data/)

mkdir -p "$BACKUP_DIR"
# TODO: --zstd vs. -I"zstd -T0" vs. -Izstdmt vs. -Ipzstd
podman unshare \
	tar -C"$HOME" --create --zstd --file "$BACKUP_FILE" "${BACKUP_SOURCES[@]}"


DATA_DIR="/home/.nextcloud-data/data"
SNAPSHOT_DIR="/home/.nextcloud-data/snapshots/$NOW"

mkdir -p "$SNAPSHOT_DIR"
podman unshare \
	cp --reflink=always -a "$DATA_DIR" "$SNAPSHOT_DIR"


ln -sTf "$(basename "$BACKUP_FILE")" "$BACKUP_DIR/latest"
ln -sTf "$(basename "$SNAPSHOT_DIR")" "$(dirname "$SNAPSHOT_DIR")/latest"

TODO: Are permissions, SELinux labels, timestamps, … backed up correctly?

nextcloud-backup.service:

[Unit]
Description=Backup NextCloud
Before=nextcloud-caddy.service nextcloud-app.service nextcloud-cron.service
Conflicts=nextcloud-caddy.service nextcloud-app.service nextcloud-cron.service
RefuseManualStart=true

[Service]
Type=oneshot
ExecStart=/var/lib/nextcloud/backup.sh

[Install]
RequiredBy=nextcloud-caddy.service nextcloud-app.service nextcloud-cron.service

What’s next?

Maintainance

Keep installation up-to-date by pulling the newest images / rebuilding them.
Update Nextcloud image to newer major release before the old one becomes EOL.
Backups
Regularly check the admin page for new warnings like missing indicies which can be added using podman exec -i -u www-data nextcloud-app php ./occ db:add-missing-indices.
Automatic updates (except major updates)

Performance

MariaDB
APCu / Redis / Memcached
Imaginary
Multiple nextcloud containers (in one pod?) and caddy load balancing

Hardening

Disable preview image generation
Ensure that your Nextcloud instance is installed in a DMZ
Configure access logging
Customized seccomp profiles for caddy and nextcloud --security-opt=seccomp=<profile>.json
Limit resource usage by containers: --cpu-*, --device-*, --memory/--memory-reservation/--memory-swap, --pids-limit, experts:--cgroup-conf= and --ulimit
Minimal Images (distroless)
mask paths inside /sys

Features

Configure an Email server
Install more Apps
Setup Nextcloud Office with CODE or ONLYOFFICE
Look at Nextcloud All-in-One and the stuff it provides like fulltextsearch

https://docs.fedoraproject.org/en-US/quick-docs/using-shared-system-certificates/#proc_adding-new-certificates ↩

tags:

rusty-snake's Tips & Tricks

Collection of useful commands and configs.

Local Nextcloud with podman

Architecture

Caddy (Host)

Configure caddy

`/etc/caddy/Caddyfile`

`/etc/caddy/Caddyfile.d/nextcloud.caddyfile`

SELinux

Allow caddy (`httpd_t`) to bind to port 7777

Allow caddy (`httpd_t`) to connect to port 7867

Start caddy

Add caddys root certificate to the system trust store¹

View certificate fingerprint

Open port 7777 in your firewall

Add new user for Nextcloud

tmux

Installation

Volumes

Images

Networks

Containers

caddy

Nextcloud

Installation

Backup

What’s next?

Maintainance

Performance

Hardening

Features

Local Nextcloud with podman

Architecture

Caddy (Host)

Configure caddy

/etc/caddy/Caddyfile

/etc/caddy/Caddyfile.d/nextcloud.caddyfile

SELinux

Allow caddy (httpd_t) to bind to port 7777

Allow caddy (httpd_t) to connect to port 7867

Start caddy

Add caddys root certificate to the system trust store1

View certificate fingerprint

Open port 7777 in your firewall

Add new user for Nextcloud

tmux

Installation

Volumes

Images

Networks

Containers

caddy

Nextcloud

Installation

Backup

What’s next?

Maintainance

Performance

Hardening

Features

`/etc/caddy/Caddyfile`

`/etc/caddy/Caddyfile.d/nextcloud.caddyfile`

Allow caddy (`httpd_t`) to bind to port 7777

Allow caddy (`httpd_t`) to connect to port 7867

Add caddys root certificate to the system trust store¹