Category Archives: FreeBSD

expand AWS EC2 FreeBSD ZFS disk

For testing purpose, I setup a Freebsd instance on AWS, which is using zfs on root.
And then I add 10G disk space to the root volume. Even though I enabled auto-expand for zroot, it seems the 10G space is not added to system. Here are the steps to expand the disk for zroot:

1. Reboot the server.  Even though it’s said that reboot is not necessary, I suggest to reboot the server to make sure server can recognize the new size.

2. As the disk size change, we need to fix the GPT partition table first.

3. Some documents said can use “zpool online -e” to expand the disk. As shown in above picture, the command can’t auto update GPT and assign the space to zfs partition.

4. We need to use gpart to update GPT first, then expand the zfs partition.

Increasing disk/zpool size of FreeBSDZFS disks in Linode

I’m running a FreeBSD instance in Linode. And FreeBSD is not the official supported OS in Linode.The disk type is set to raw to install FreeBSD.

And recently Linode upgraded my instance and the disk size is increased from 40G to 80G.

But when I login to the system, I found that my zpool is still 40G. But the disk is shown as 80G.

I tried to enable zfs autoexpand on my zpool, the same. Then how to increase the disk/zpool size?

First, re-write disk metadata

After this, gpart can show the real disk space info

Then, expand zfs partition

Then, expand the zpool


Don’t forget to write zfs info to disk

FreeBSD: Error adding new user – pw: user ‘mysql’ disappeared during update

Today when I installed mysql5.7 on my new freebsd server, I got below problem:

This issue is not common. And it’s caused by atabases are out of sync.
To fix this issue, just Run

FreeBSD: use mrsas driver to replace mfi driver

My server got a Dell PERC H330 raid card, and I made it’s working in HBA mode to make sure that it can get best performance under FreeBSD with zfs.
But every time when I boot the server, I’ll get below error message and it take a long time to pass the disk check stage.


After research, it seems this timeout error was caused by the old mfi driver.
And LSI has released a new mrsas driver for FreeBSD. So It’s better to switch the driver from mfi to mrsas.

First, add below line into /boot/loader.conf

And then add below device hint into /boot/device.hints . This line is very important. Without this device hint, FreeBSD will use the old mfi driver for raid card even though you enabled maras drvier.

And then, add below line into /boot/loader.conf to disable disk id identity.

Without above two line, after you switch from mfi to mrsas, all the disks will be shown as diskid-*****************。

And don’t forget to update /etc/fstab to change the swap partition from mfi*p* to da*p*. Otherwise you’ll lose your swap partition.

Then, reboot your server, and enjoy.

FreeBSD + Nginx : Enable HTTP/2 and ALPN

For now more and more servers are starting using HTTP/2 which is faster and more secure.
This post is about how to enable HTTP/2 on FreeBSD servers.

Nginx Stable 1.8.* doesn’t support HTTP/2. So we need to install nginx-devel (version 1.9.*) first. If you have already installed Nginx stable, you need to uninstall it first.

And before you install nginx-devel, you need to install openssl from port first. Otherwise nginx will use system based openssl library, and you can’t enable ALPN for http/2. That’s because ALPN requires openssl 1.0.2*, and the system based openssl is version 0.98

So the first step is:

And then add below line into your make.conf to make sure that you’ll use the latest openssl library to build nginx.

Then you can install nginx-devel

Make sure that you select HTTP_SSL and HTTPV2. Please be aware that SPDY is no longer supported by nginx 1.9.*.
Then install it.

Go back to your nginx.conf, and modify it as following:

I’d like to explain a little bit for this configuration.

This is about to enable HTTP/2 and SSL.
You may notice that here I’m using accept_filter=dataready instead of accept_filter=httpready. There are currently two filters in FreeBSD: “dataready” and “httpready” which need to started at boot by adding accf_data_load=”YES” and accf_http_load=”YES” to /boot/loader.conf. dataready waits for the first properly formed packet to arrive from the client before passing the request to nginx. httpready waits not only for the packets, but also for the end of the HTTP header before passing the request onto nginx. Keep in mind “httpready” filter breaks support for ancient HTTP/0.9 because v0.9 does not have any headers. HTTP/0.9 is so old we are not going to worry about support it and since a HTTP/0.9 would not have the newer SSL ciphers anyways.

To configure nginx to use the accept filters in FreeBSD we need to add the arguments to the listen directive. Since http (port 80) is unencrypted we can use the “accept_filter=httpready” accept filter. This is because FreeBSD will need to look at the packet and parse the complete http header. SSL (https port 443) is encrypted so FreeBSD can not parse the packets so we need to use the “accept_filter=dataready” accept filter. Both accept filter examples can be found in the configuration below. To use FreeBSD accept filters you must enable them in /boot/loader.conf to load on boot.

And someone may use the nginx 1.9.* new feature reuseport. I have to say, unfortunately, this new feature doesn’t support FreeBSD. I made several test on my server, it seems if you enable “reuseport” on your server, then all your traffic will be handled by your first worker! The OS can’t balance the workers’ workload. It means if your server is very busy, with reuseport on FreeBSD will significantly slow down your server. You’ll find that your first worker is taking up 100% CPU and the rest are idle! So, at this moment, do not enable “reuseport” on FreeBSD Nginx.

This part is just enable SSL. It’s pretty easy. There is only one thing you need to take care, that is for yourcert.crt, there is no need to put Root certificate into it. Just put your server certificate and intermediate certificate into it to reduce the size of your certificate to reduce the connection time.

These SSL ciphers are recommended by cloudflare. It can support most browsers. But IE6 is not supported.

NGINX caches the session parameters used to create the SSL/TLS connection. This cache, shared among all workers when you include the shared parameter, drastically improves response time for subsequent requests because the connection setup information is already known. Assign a name to the cache and set its size (a 1-MB shared cache accommodates approximately 4,000 sessions).The ssl_session_timeout directive controls how long the session information remains in the cache. The default value is 5 minutes; increasing it to several hours (as in the following example) improves performance but requires a larger cache. Session tickets store information about specific SSL/TLS sessions. When a client resumes interaction with an application, the session ticket is used to resume the session without renegotiation. Session IDs are an alternative; an MD5 hash is used to map to a specific session stored in the cache created by the ssl_session_cache directive.

OCSP stapling, can decreases the time of the SSL/TLS handshake. Traditionally, when a user connects to your application or website via HTTPS, his or her browser validates the SSL certificate against a certificate revocation list (CRL) or uses an Online Certificate Status Protocol (OCSP) record from a certificate authority (CA). These requests add latency and the CAs can be unreliable. With NGINX you can cache the OCSP response to your server and eliminate costly overhead.

At this step you need to create your ssl_trusted_certificate. Please note that for OCSP certificate (ssl_trusted_certificate /usr/local/etc/nginx/ca-certs.crt), you need to put your root certificate and intermediate certificate into it, and no server certificate. And the right order is root certificate first, then intermediate certificate, then second intermediate certificate.

Enable HSTS for your server. Please be aware that if you added this into your configure, then you can’t remove it and go back to http again. Otherwise end users can’t access your website because they will always redirect to https site.

Now, you can go to to test your website. You should be able to get score A+