Val Packett's Blog2024-03-03T06:45:27Zhttps://val.packett.cool/https://val.packett.cool/x/val.1.pngVal Packetthttps://val.packett.cool/path.join Considered Harmful, or openat() All The Things2024-01-08T00:00:00Zhttps://val.packett.cool/blog/use-openat/Say goodbye to path traversal attacks by using modern kernel facilities and get ready for the capabilities-secure future at the same time!<p>Soooo… isn’t it absurd that we have this hierarchical and dynamic structure called the file system, and the way we use it is by pretty much always traversing it from the root, by path, constructing paths using <a href="https://security.stackexchange.com/q/123720">string manipulation tricks</a> to hopefully try to be safe against both <a href="https://owasp.org/www-community/attacks/Path_Traversal">attacks</a> and <a href="https://www.theregister.com/2015/01/17/scary_code_of_the_week_steam_cleans_linux_pcs/">accidents</a>?</p>
<p>Yes. Yes, it is. Very much absurd and ridiculous. Just think about it. And despair. Or actually, don’t despair; instead, I want YOU to start caring about this and to join the fight for changing it! As we’ll see, the system interfaces have actually been evolving to make this situation better.</p>
<p>In this article we’ll learn the history of how Unix-like systems got a way to make stable references to filesystem subtrees and perform operations relative to those, explore the current state of filesystem APIs in various programming languages, and—hopefully—get motivated to improve that.</p>
<h2 id="from-race-conditions" tabindex="-1"><a class="header-anchor" href="#from-race-conditions"><span>From race conditions…</span></a></h2>
<p>As it is with many good things in this life, the <code>*at</code> family of system calls was invented at Sun Microsystems.
Specifically back in the early 00s, when Solaris 10 has introduced the <code>openat</code>/<code>fchownat</code>/<code>fstatat</code>/<code>futimesat</code>/<code>renameat</code>/<code>unlinkat</code>/<code>utimensat</code> calls as a method to avoid race conditions / time of check vs time of use issues.
Namely, this kind:</p>
<pre><code class="language-c">#define STATE_DIR "/var/db/thing/"
if (stat(STATE_DIR "ok", &sb) != 0) return false;
// in between these calls some other process screws with what /var/db/thing means
// e.g. overwriting what a symlink points to
int db_fd = open(STATE_DIR "data", O_RDWR);
// we ended up referring to one thing at time of check and another at time of use
</code></pre>
<p>Before, it was only possible to avoid these by changing the current working directory and using relative paths from there (!), which was quite an ugly hack, especially bad because the working directory is <em>per-process</em> state, so it wasn’t even thread-safe. That’s actually kinda really funny, isn’t it? “You can have ONE stable reference to a directory, as a treat,” said Unix.</p>
<p>The solution they came up with was using directory file descriptors and adding operations <em>relative to them</em>, hence <code>*at</code>:</p>
<pre><code class="language-c">int state_dir = open("/var/db/thing", O_DIRECTORY /* checks that it is a dir */);
if (fstatat(state_dir, "ok", &sb, 0) != 0) return false;
int db_fd = openat(state_dir, "data", O_RDWR);
</code></pre>
<p>From there, it has found its way into Linux, initially as userspace-only <a href="https://lists.gnu.org/archive/html/bug-gnulib/2005-11/msg00043.html">procfs trickery</a> but finally available as a set of real system calls in version <a href="https://kernelnewbies.org/Linux_2_6_16">2.6.16</a> from 2006.
Then the POSIX.1-2008 standard has included these <code>*at</code> calls and they were picked up by all the BSDs: first DragonFly and FreeBSD around 2008-09, OpenBSD a bit later in 2011, and the holdouts were NetBSD until late 2014 and macOS until late 2015.</p>
<h2 id="to-sandboxing" tabindex="-1"><a class="header-anchor" href="#to-sandboxing"><span>…to sandboxing…</span></a></h2>
<p>Suspiciously soon after these calls were added to FreeBSD, the <a href="https://www.cl.cam.ac.uk/research/security/capsicum/">Capsicum project</a> at Cambridge came up with a way to build capability-based security for Unix.
FreeBSD was the main target for prototyping, and the work got upstreamed for 9.0 and enabled by default in 10.0.
There also was a Linux port sponsored by Google, with an implementation of the consumer side in Chromium, but all that sadly didn’t go anywhere.</p>
<p>So, what is the idea there, anyway?</p>
<p>Turns out, exactly these <code>*at</code> calls are at the core of it.
You see, while the original use case for kernel-level handles to directories was race condition avoidance, the researchers realized the great sandboxing potential of them.
The <a href="https://en.wikipedia.org/wiki/Capability-based_security">capability model</a> has turned out to map very well onto the POSIX API: file descriptors are already unforgeable handles that indicate access to a resource.
They can be passed between processes, inherited by child processes… and <code>openat</code> is exactly how you go from a more-privileged filesystem capability (e.g. a descriptor to <code>/etc</code> signifying access to the whole subtree under <code>/etc</code>) to a less-privileged one (a descriptor to <code>/etc/passwd</code> signifying access to only that file).</p>
<p>Well, provided you restrict it to actually only do the lookup beneath the directory and never escape it.
<em>Capability mode</em> does exactly that, along with disabling all access to <em>global namespaces</em>, i.e. the ability to just <code>open()</code> whatever by a global path, to reference a process by PID, that kind of thing.
The idea is that you open the directories and other resources you anticipate working with, and then sandbox yourself to the resources you have and those derived from them.
For example, roughly like this:</p>
<pre><code class="language-c">int web_root = open("/var/www/site", O_DIRECTORY);
int tcp_sock = socket(PF_INET6, SOCK_STREAM, 0);
// bind, listen etc. omitted
cap_enter(); // also check the return code lol
// here we can accept client connections and talk to them,
// access files under /var/www/site, and nothing more
while (fd = event_loop_thing_poll(pfds)) {
if (fd == tcp_sock) {
event_loop_thing_add(pfds, accept(tcp_sock, NULL, NULL));
} else {
char *requested_path = read_request(fd);
int ffd = openat(web_root, requested_path, O_RDONLY);
sendfile(ffd, fd, /* … */);
close(ffd);
}
}
</code></pre>
<p>There are additional features like fine-grained limits <a href="https://man.freebsd.org/cgi/man.cgi?cap_rights_limit">called “rights”</a> that are inherited by everything derived from a capability and can only be reduced, never expanded—so like, you can have a directory handle so damn read-only that files opened below it using <code>openat</code> could never be read-write—but that is the core idea.
As a result, we have one of the strongest process sandboxes out there, based on a principled thought-out model instead of arbitrary deny/allow lists.
The tradeoff is however that it’s not as easy to sandbox existing software into such a paradigm-shifting system, so e.g. OpenBSD gets to tout a bigger quick practical impact of their pledge+unveil system :)
However some <a href="https://www.youtube.com/watch?v=ErXtGMmRzJs">great</a> <a href="https://www.youtube.com/watch?v=TGA4wbjbqXc">research</a> has been done since into mechanisms for retrofitting existing applications, and I’m sometimes trying to <a href="https://reviews.freebsd.org/D38351">continue</a> that line of reseach myself.</p>
<h2 id="to-the-mainstream" tabindex="-1"><a class="header-anchor" href="#to-the-mainstream"><span>…to the mainstream…</span></a></h2>
<p>So quite a few years later, during the explosion of various “cloud services” (advanced ways of running your code on someone else’s computer), this idea was extended further.
<a href="https://lwn.net/Articles/674770/">CloudABI</a> (2016) was an attempt to define a sandboxed application format for “the cloud” based on a simple idea:
what if we just have a new OS-neutral ABI that already starts in Capsicum capability mode and doesn’t include any syscalls inappropriate for that, i.e. ones that use global namespaces?</p>
<p>The design was very clever.
Resources would be injected before program start by a launcher that would make that easy.
The system call interface was entirely <a href="https://en.wikipedia.org/wiki/VDSO">vDSO</a>-based, one of the implications of which was that it was possible to run CloudABI binaries—without the sandboxing—on unmodified Linux and macOS using the launcher, allowing for developers to get on board easily.</p>
<p>Sadly, native (secure) support only got upstreamed to FreeBSD and nowhere else, and the project fizzled out due to lack of industry interest.
Not without leaving a huge influence behind though!</p>
<p>At the same time, <a href="https://webassembly.org/">WebAssembly</a> was taking off as a solution for another level of sandboxing and abstraction, the machine level.
Born out of <a href="https://en.wikipedia.org/wiki/Asm.js">previous efforts</a> to compile existing C/C++ projects (as big as game engines) to the browser, Wasm has quickly expanded into a huge array of other use cases as well, because of what it actually ended up being: the most lightweight and neutral low-level abstract machine.</p>
<p>Currently it seems to be catching on in the “cloud” industry, which has been looking for alternatives to heavyweight full-OS virtualization as the security boundary.
That was helped by the “next big thing” being “edge” instead of “cloud”, which mostly seems to mean running customer code all across a worldwide CDN instead of in just a few huge datacenters.
But the hype was all around a secure abstract machine when in reality the need for a common secure ABI was arguably even bigger there – assuming your goal isn’t explicitly doing vendor lock-in by providing a custom interface instead of a standard one :D
(Hm, would it have helped if CloudABI was instead named EdgeABI?)</p>
<p>Anyway, thankfully, what’s fulfilling the need for an ABI is <a href="https://wasi.dev/">WASI, the WebAssembly System Interface</a>, which is… basically kinda sorta just a <code>wasm32-cloudabi</code> target if you look at it! (Well, with the whole <a href="https://component-model.bytecodealliance.org/introduction.html">Component Model</a> thing that’s only going to be one aspect of it but still.)
The <a href="https://github.com/bytecodealliance/wasmtime/blob/4ba8b6c0d99d258aa0a4ed4ee7c687fcddae6c8e/docs/WASI-overview.md">WASI overview</a> explicitly references CloudABI and Capsicum.
And even the aforementioned research into Capsicumizing existing software in the form of <a href="https://github.com/musec/libpreopen">libpreopen</a>.
In a way, we have won after all! :)
The industry-hyped, Wasm-workgroup-blessed, by-all-compilers-supported ABI for POSIX-y applications is based on exactly these ideas.</p>
<h2 id="and-back-to-the-regular-syscall-interfaces" tabindex="-1"><a class="header-anchor" href="#and-back-to-the-regular-syscall-interfaces"><span>…and back to the regular syscall interfaces…</span></a></h2>
<p>Meanwhile, other Things have been Happening with the system call API design all this time, right in the normal Unix-like kernels of “the present” that we actually run on hardware.</p>
<p>Way back in 2011, <a href="https://kernelnewbies.org/Linux_2_6_39">Linux 2.6.39</a> introduced the <code>O_PATH</code> flag which allows opening, well, “only the path”—getting a stable reference to an inode—without actually opening the <em>contents</em>. These descriptors can only be used with operations that don’t care about the contents, like the <code>*at()</code> calls we’re discussing. In most cases, this is just an optimization: not opening what we don’t <em>need</em> opened. It’s fine to “fully” open a directory and then <code>openat()</code> below. However the difference in semantics comes into play especially <a href="https://github.com/flatpak/xdg-desktop-portal/pull/532#issuecomment-734305639">with symbolic links</a>. To support these semantics as required by xdg-document-portal, FreeBSD got <code>O_PATH</code> merged <a href="https://reviews.freebsd.org/rG8d9ed174f3afba5f114742447e622fc1173d4774">in 2021</a>, a whole decade later.</p>
<p>But the most important feature making directory descriptors attractive? The ability to do strictly-beneath lookups, like all lookups are under Capsicum, but no matter if your whole process is in a sandbox mode or not. Having an explicit flag in the API enables the developer to just pass a reference to a directory and tell the kernel to open a path <em>definitely under</em> that subtree in the file system, without worrying about processing the path carefully to avoid escapes.</p>
<p>This was first proposed for FreeBSD <a href="https://reviews.freebsd.org/D2808">back in 2015</a> as <code>O_BENEATH</code> and <a href="https://reviews.freebsd.org/D17547">landed in late 2018</a> in the development branch. Then <a href="https://kernelnewbies.org/Linux_5.6">Linux 5.6</a> in 2020 shipped with the <a href="https://www.man7.org/linux/man-pages/man2/openat2.2.html">openat2()</a> syscall that adds a bunch of controls over path resolution behavior, including the <code>RESOLVE_BENEATH</code> flag implementing the Capsicum-style behavior. Almost immediately after FreeBSD has <a href="https://reviews.freebsd.org/D25886">added <code>O_RESOLVE_BENEATH</code></a> with the correct behavior since it was discovered that the original <code>O_BENEATH</code> one actually wasn’t working as intended (oops); then <a href="https://reviews.freebsd.org/D28907">the original <code>O_BENEATH</code> was removed</a>. For FreeBSD 14 there was also a <a href="https://reviews.freebsd.org/D39773">fix landed</a> to avoid the “<code>/..</code> is <code>/</code>” behavior when opening beneath a descriptor pointing to the root directory, to make the behavior equivalent to Linux, at the request of the author of a library that I’ve <a href="https://github.com/bytecodealliance/cap-std/pull/296">added FreeBSD support to</a> which finally landed just recently, speaking of which…</p>
<h2 id="and-that-s-where-you-come-in" tabindex="-1"><a class="header-anchor" href="#and-that-s-where-you-come-in"><span>…and that’s where you come in!</span></a></h2>
<p>So. We have arrived at the current point in time, where we have decent kernel support for holding references to directories in the file system and doing useful things with them, like safely opening files strictly beneath the directory. Now what’s needed the most is: adoption, adoption, adoption!</p>
<p><strong>In Rust</strong>, use the <a href="https://github.com/bytecodealliance/cap-std#cap-std">cap-std crate</a>! It’s an excellent library that provides an appropriate high-level API for directory references:</p>
<pre><code class="language-rust">// somewhere in initialization
let mut root = Dir::open_ambient_dir("/var/www/memes", ambient_authority()).unwrap();
// fn do_the_work(/* user input */ name: &str)
let img = root.open(format!("out-{}.avif.tmp", name))?;
let log = root.open_with(
format!("log-{}.txt", name),
OpenOptions::new().create(true).write(true)
)?;
root.rename(
format!("out-{}.avif.tmp", name),
root,
format!("out-{}.avif", name)
)?; // etc.
</code></pre>
<p>And it takes advantage of modern Linux and FreeBSD system call API features to make this fast, while still supporting other platforms with a fallback method (essentially doing a system call per path component which should honestly be <em>fine</em> in most cases).</p>
<p><strong>In other languages</strong>, I’m not yet aware of cap-std equivalents; please do start working on your own! Or sponsor me to work on one for your favorite language I guess! :)</p>
<p>The basic idea of what such a library would be is this:</p>
<pre><code class="language-c">int open_beneath(int dirfd, const char *pathname, int flags, mode_t mode) {
#ifdef __FreeBSD__
// TODO: validate flags/mode to match Linux behavior
return openat(dirfd, pathname, flags | O_RESOLVE_BENEATH, mode);
#elif defined(__linux__)
struct open_how how = {
.flags = flags,
.mode = mode,
.resolve = RESOLVE_BENEATH,
};
return (int)syscall(SYS_openat2, dirfd, pathname, &how, sizeof(how));
#else
#error "TODO: the whole fallback algorithm from cap-std"
#endif
}
// same with other operations
</code></pre>
<p><strong>For young or evolving languages</strong>, advocate for this paradigm to be incorporated into the standard library or “blessed” external libraries! I’ve noticed some awareness of the aforementioned ideas in this space:</p>
<ul>
<li>in the Zig standard library, <a href="https://github.com/ziglang/zig/blob/804cee3b93cb7084c16ee61d3bcb57f7d3c9f0bc/lib/std/fs/Dir.zig#L1">the <code>Dir</code> type is an fd wrapper with <code>*at</code> ops</a> but sadly as of right now there’s no attempt at providing the <code>RESOLVE_BENEATH</code> behavior;</li>
<li>the new OCaml I/O library <code>eio</code> has a <a href="https://github.com/ocaml-multicore/eio/blob/4856fc430db261eb4c9e90107f0f629e1284d945/lib_eio_posix/fs.ml#L19-L23">sandboxed dir type</a> that for now uses <code>realpath</code> + string trickery and doesn’t hold an fd, but the authors are aware that it probably should and <code>RESOLVE_BENEATH</code> exists.</li>
</ul>
<p>It would be really awesome if we could push for a cap-std-like API to be the recommended default one everywhere.</p>Introducing TiddlyPWA: putting TiddlyWiki on modern web app steroids2023-07-26T00:00:00Zhttps://val.packett.cool/blog/tiddlypwa/Oops, I think I just turned the legendary personal wiki system into an offline-first, installable, encrypted, synchronized Progressive Web App<p><em>tl;dr: go play with the new thing <a href="https://tiddly.packett.cool/">there</a>, report issues <a href="https://codeberg.org/valpackett/tiddlypwa/issues">there</a> and support me <a href="https://www.patreon.com/valpackett">there</a> :3 but please do read the full story below!</em></p>
<p><a href="https://tiddlywiki.com/">TiddlyWiki</a> is quite a famous note-taking system.
Describing itself as “a non-linear personal web notebook”, it’s actually quite a bit more than that.
Thanks to a plugin system, it’s basically an app platform, and plenty of <a href="https://thaddeusjiang.github.io/Projectify/">cool stuff</a> has been made for it.</p>
<p>And it exists as a unique thing: a self-contained web application—a single HTML file with both the app and the content—that you can take around on a USB flash drive.
Oh wait… who even uses those anymore for anything other than OS installers and bringing files to a print shop?</p>
<p>So of course, a bunch of storage solutions have sprung up.
There’s an official node.js based server mode where it becomes more like a traditional web app,
there are browser extensions and even modules inside of TW itself for saving the updated single file to a server,
there are mobile and desktop apps that try to combine that saving with some kind of offline support,
there are people using a file sync solution like Syncthing – I’ve even heard of some people using Syncthing with the server version,
syncing the <code>.tid</code> files between devices and <em>running the server in <a href="https://termux.dev/en/">Termux</a> on their phones</em> to access them on mobile.
Oof. While I’m nerdy enough to <em>be able</em> to do that, I’m also enough of a spoiled basic consumer to know that that’s not the user experience <em>I want</em>.</p>
<p>What I want is something that works like <em>an app</em>. Fully working offline, syncing <em>efficiently</em> (not by POSTing a multi-megabyte HTML file), quickly and reliably when online.
And with client-side encryption, because there’s just <em>no reason</em> to let the server side read the contents here.</p>
<p>There has actually been one good attempt at bringing this kind of sync to TiddlyWiki: <a href="https://noteself.org/">NoteSelf</a>, which integrated <a href="https://pouchdb.com/">PouchDB</a> for storage.
I liked the sound of it, but in practice it wasn’t up to my standards.
It’s a heavyweight modification of TiddlyWiki that doesn’t keep up with latest core updates, PouchDB/CouchDB feel a bit heavy themselves, there’s no encryption, and the offline support is just “run it from your hard drive”.</p>
<p>So… this is where I come in.
Last year, looking to try TiddlyWiki once again, looking through the options and getting frustrated with the aforementioned everything, one thing inspired me.
I stumbled upon <a href="https://github.com/Jermolene/TiddlyWiki5/issues/3420">a basic IndexedDB plugin</a> which made me realize that there is an API for storage backends inside TiddlyWiki.
I realized that there’s no need for core modifications, that I could just start with this and—knowing what I know about the web platform—take it to the next level.
Make a plugin that combines IndexedDB, encryption, a sync engine (with a custom super-lightweight server counterpart for it), and adds a Service Worker and a Web Manifest
to turn TW into a <a href="https://web.dev/what-are-pwas/">Progressive Web App</a> that works offline and installs on mobile with “add to home screen”.
That was a whole Vision. And I knew it had to be done. It just had to exist. So I got to work, putting aside my typical FreeBSD system-level work to get back to good old web dev.</p>
<p>Now, a whole year after that inspiring moment, having gone through huge personal life changes and a reverse putting-aside in favor of my other work again…
it’s finally ready for the public to see.</p>
<p>So.</p>
<p><a href="https://tiddly.packett.cool/">Here it is.</a>.</p>
<figure>
<video controls muted preload="none" width="1920" height="1080" poster="data:image/avif;base64,AAAAIGZ0eXBhdmlmAAAAAGF2aWZtaWYxbWlhZk1BMUIAAAGNbWV0YQAAAAAAAAAoaGRscgAAAAAAAAAAcGljdAAAAAAAAAAAAAAAAGxpYmF2aWYAAAAADnBpdG0AAAAAAAEAAAAsaWxvYwAAAABEAAACAAEAAAABAAACLQAAEIgAAgAAAAEAAAG1AAAAeAAAAEJpaW5mAAAAAAACAAAAGmluZmUCAAAAAAEAAGF2MDFDb2xvcgAAAAAaaW5mZQIAAAAAAgAAYXYwMUFscGhhAAAAABppcmVmAAAAAAAAAA5hdXhsAAIAAQABAAAAw2lwcnAAAACdaXBjbwAAABRpc3BlAAAAAAAABQAAAALQAAAAEHBpeGkAAAAAAwgICAAAAAxhdjFDgQUMAAAAABNjb2xybmNseAACAAIABoAAAAAOcGl4aQAAAAABCAAAAAxhdjFDgQUcAAAAADhhdXhDAAAAAHVybjptcGVnOm1wZWdCOmNpY3A6c3lzdGVtczphdXhpbGlhcnk6YWxwaGEAAAAAHmlwbWEAAAAAAAAAAgABBAECgwQAAgQBBYYHAAARCG1kYXQSAAoHGWpn/Z/KoDJrE/wEIFYAWRoZFFM8pObBcj0LotyhWLs+CPamzgDO3Y2pbLeJPG5YJuBoUVuN1OPoVzJOLTUhzPg2C6Nom6DlJb0gcr14ky8f3wG5kwngoioo6NiZj3NEgUXgeUXVzu3zvkinGLoHG3/rIaASAAoKGWpn/Z/JAgIGhDL3IEzUQUEI8lJy/UeEuJt/8SpGcGqR8FEt5otYnI/Zchm2rtKmQUeLToLU9FD5KOjOZTeuGpMH6B8prGVKcgeOqAKXt0AaEuSjju3graECLqdKyBoj2nB2fOxNdvALNeKU3R29YvvaEAzBhFafzOxdKBaIYgFRGctEFDozLkqgQ1/++6r6jEqxX///nys5GURrp/wfe1BsRjzEkZEf/vnJYIPmRv//6b7w6LxlTuaviMZaW7hJs/dXIoOHEoC65jQ2qfNrkpX0QHR/7VKYijqS/fL6Q+zBPjh3Rtl0UaZU2ccwCjiaf6L1gGGzMzmeUAAFxO3X9d21wFVJXqw15C2316+rsNpMQBwzf7dZ04sqWFY6zZ+xBxB22NH5yvJw6OksYiZa4P5X0LCQWLzOe3ffedy6GCk6n5tiaX0oYOjhb/orCApA41xY0VHjmLxRsGVd/2Z7lJggyG8BovOv7OxFaf6um2hKSniQSKkRdArVm6dkN/y2qzzI+As5aS4vl1sNJvrYAz/Lh6rar1YYNFBl0iQSEeUWeaw31uSCxuxFxJBypVqID2zLxyZJyGD197C258gt+f1jz/3qD9ES+xu92q/AKvoEx5q5D7id6/4v+/pg8RlU/nC84IPbqhhUn/W28/Ho0AIbplOlQDUzdzr7jE41/U5g+t7HPkV3t/a/Jp0nMHIFUSx0456WaImpCq2+CKSFtUjbSZU7ETwvMV8uY8RnIR0zo2IsKZJtkBI7Ew/U+KJ+WxFcwO2KjTY/DqziV06GWvhoxfJCT5wej9g7I8BSXIYbzvt7TEnXN/7Jxgjdh75p9L1wbh/Zraot+WcbUP58Gz7tT8NjTwTSg3UqFGaRdvTnW7D7ait8KlC5yG/4ZvEB8s4umT0WAKoCPRb359686IJlB/jx8GVra9lrDfx+A0mpL5P1FxemmDNapxGI1IxOXUEUuZobVKB6RWzKOT8k59AOuZZ1p4aRAsI/WxRWfmfs5N9PyupxVJfe5SeFVUiOX4sDks3rFV1X617CGhM7xWkqIiXBqvmir0tP9D4ASrry2tUHZzHkOJz5Mf6POcfnVnWb2J33LCUd1zCtYUD+s45bM1ucLb28Y6WDAifn7IPriT3ecwIeFVXEcl176osST17bjroWz0kWBegbbjXt1bcMxLs4MWFpjOcuT0IP/3IXIZ/ZlGK4p4F8z0jQn+5rOQGnhuMCGRFnfzAYAtfqYX0kFBWRmPKEjRimtIr7TmsRdYUuva6csAwbxUg7/z+RA5SMY32q0lmdPZ5XvQJxy7GkGwmVMCTnh6e+t8ZgX+/tk2V2VsvhY3jv6nrTaxCbmMuiH4hG2H8Jj2FBaDaNEJE5W2rTtChVSfulKXzZc5MzaS3leOr4L5Rzm6+aWy50iB476ffzgqdh4Ls30L5ywWDkr3Kx5HAgcUeppb+DliTNDrtp4Ze+m4lorbAQ6qzET2WJLNiJYZ+pJt8wWJe/qpWVTnCZx4MVlqzNyLedNNzpbPjnuzVbYkog2vNeXBvaUOrSd3qG7Jc8nAExzu53SSMeg03pf/u8xEPVp0Svo5sQspDjHY2woDbxyFcDisav7p9cwiELCv+9SK5lEOSJmCDEqJajeR9q56BJOnOnyDmjWX6wp7CskETN0Rs16AajjnF0wAfYx88RvdwfKu6j+/WWNr+nN1REoPFGg3DiahMtt01jskZsyH0WKd0bwkc6XutWPmd6przV63wZiIiLVbebRNAPfX8z5eXQZ8R/7rWx5Xc9xcVJOaoCG85jQigwGYvdWEt1L5d9aBcdfSmoKwkVaXRE+MQ3IU7CT/1EI6l8kHDACmA3ga7mnPPomFPSK3mKL4ORF7KRfGAR+miQviIrgC8mG1UzUZDMjWlXrmQSOlvk6fkEgFaW5geWd+lWnYzxRoIaGjrfZUq8LdbO19w8ngpKw2Bkt0VxS8aKNY/NT+mQqgGpabzgllfV0hdKZGiiBIZiUii5pMxLuoTU+Ms0AS/EYg/u1pU0T8MV/zi6SRjwfik+/68fi0dU032CCJoKSgMOae/ibxNgU58raDkTAZ56lteB2sogxeu2Sfrhh+sezNj08KVZARqTR9dp31PIQ6LsT+neLefoCFrlLM+c+BKxsJWXYh/Ab5wlyP86cf/evWrfPzqDgSBGB2cTGJ9VTqs1pljkAa0X3QeWSB5zBV4VgLELTna03U70szpuR4vdFxrcWlhhaXIv/o5itl5Pp/Vojo/nLKcbvJOphhZjRDMWieFZ9iw0oIJz40KJilINJbmprVY4n1jbFtXWLITZs9WzbEhHNyavKphkB5SEp7sbAXqfC/aQqO1/KWK9kLIIOHr7l+u5KJVLoo+fzs08NxfJYjVQBkPMnIwTERPGYRrkVeeS450WmSMCFHijjB7JOz7jcm2aSxuSmRWq1dEjhSumFegOHh8avmMm2sVDeKrSb2gTvwoWoFKTcdBtAiczF6rR9EP9tFWfIv/54/QES7/iGWIRTX4hPWSDF8EJZfRJoZ9/XcZeh5zvShw0CR48di3pFnm1JBG/+ZsvxPqt6S1G0MV2cDmLBpTnPvK6R2pzEwqV9LLN3jTY/vti37uqSwQTgiUEqkQvVo+a3YeUQPjlBfCuYJSdN801hQn223tiIAmN0xAM/P5AyrzUHAYYxdT9Nt/skfrdM/1Tze7P+JNpQDcH0k8hssEJEl4SQ0M3+I9PIvX+k/7xfqiX5ocra+dIX1xYbnHUvTOQorjl+0xO9Eor2shn3pD/hGTYPopGZbTcPIfBZ87upXOIw2AE6+KweDA7/Ld7EwPZ4TartCiYOT4ZNq8zBz+SVRmtK0RfJb8uY7KDfckU+Jy6OjOU9qkcK/xpJKGJv0IOVLVaeDb1u0LCtpCqvZvJwWj/XyCS+O8mqcWMHesAPrnFGzJm7X0hz0ZAWUIFZU38E2id/4RGS7uSWtfq9/79IMzeCKji5h8rvhwxNIUdvX2JPSMJQaqao1LaxcXfT4K0+QtJKB8CiTIjd0P+5lOyKLQo169s//jhh4hTIrvuJQdu5loWbx7WMCvjpPLj7GCaTMiiZ72wkp7urChQPRIwCVSm653f/8O8W3Htm+xYtgRYloQSdHMfb55Z1fSJzwwi7H5SmK/9XRW39QZrNvg0ATPEdVxn3GSifoVJUjDroqcYYGxHWHUQKs0zzo6v/FSfO/UzGXRFyIp2xL6BhknxeFBkfIIdxZBlKahKQStWxn7nB5HeB/WAAiaGBJW2BKr5UqoFUgolAOut57R3sgJjaDyBvlEXYCQfNzM4M5iLGaGnoN6Kd2yxCEcMoaaEO94B1BeY+tSpqyDPE79X3O0Et3tqkgCEI1ouJohZevS7DurrHQ8MjY0v5g2OJ63Ydla1oETQFn0lwS0pqmDBQCCg03Na7nB0T3z2xSJfe/Is42nlzAtr8JMamhYHMc/5zYfZo31KBVnG2/VKKZoZ25tFQLxm7vOjlJCTRWvensI7037MvEmgnVNhZLyzIngf9nhit+UORKcCu6ZWt0+RkmFHWYl71fyBqUbakh6DVGNXTZEs/25jllG+JnITM3u6H/+I8rWXGVVEfP2GuopYXRBHvH31C7+EGctYRfflDGYGy0+zJyQXBgZmZlnTCEoiftl8sJzqU/vRXexGetjwCn+m9Qq7K37ZObpnGiC97ifMNo/4jAujbZwkxuS//88wXZPiPh8lvyOC1ggJh/rfkDWjMvdQsfLP51Ah1bmZeA3hVcr+jO3uNUVIXcZgZZM6zaFIPpe7fY+JuRdzu7S6IIw3oUYTrTd9iN4KaQmFUgL7M7ECehtYuh0bRiFlLR+kQbSq+K7uThs4Cx8sbQ11aCM0617CFiX0DmBl/uEC6esdRWqRZWTIoru66WArhmvwKKXTxKB7NQFMHgZCDPx72/yVrUzQWbEpmXuiEMeynvoHYc9Bs5kj/QRHaw1MdJq5S4StzvSdHTfWEewx6odtTfZgfPAnn9FjNzEAAh67zfjr4Ly9NUI7ObjAsukhLfvmek3WizDjHGzAFyzQVyzHG3jX2Jbi/5+sCukZm46M7TFSleiwfsHhdQdyQEu02Xb+IypOrY565KUN/pW7Y5UJbs9iTlv26tmzgn2n0hCQjnTDC3ngnMOx7B3BkKrFFyC7lvhyaWtPyVxhvtPHJmK2III9SF1rT4T9nkxs3ba4QGCU7tjPw0Dq0h5kVUpLQhb314J1LCrjqC4wDgdJw5GksfcafVkH9KMrPQZYGOlGYg1a6VV5d3mgSzad+rhXjks/BKTncKAZV06Nc89LVQW83jK5tobXCAw/xuRB2WDmGF+wVWPCmK4E2lroXMO43j7sE4QEmHu5z0o5U/xPwh1iLC48c0WLFRYymN2H6EpbFbIwFcwo1VJ6PCHd47ykH7sH5blaSNoQP7HHbBVhDEmOffmO9ZW3+sHXkGJ9lwXpmyLpwDDhTwj+mbQBBa+qCjDgatWQm/l+HhUKeT4C4gEQ81DkPH1lIrFFNp2WRv9x2O5NMxdGyLRCBPva19qdYhbw0am9sR+MX4GOvn9BTky7hF0BtXUSzDrSDJCxpIS9L/qlSXGprVFi+2UKfHEj2B+HZ3wSprkK/lDuWWfnJxql162hulGeh6tbb2A1mVjucGHwV/rgPXzU+GhnaiVv2jhVZnPs3gufvRkApweUAUctJVPTmDAoik/05NTbQj5Y9KPcEHfyEpuAy6TDZ6pBxOXc2QlgYf6tE1aOZt+xy9K9MmriMFTDdGNmGt7x3hO16qBEb8lWhzbH7D05hwsPTQTnbk37lG81wc05zx1MUKLwOW1g6oozfI1A8bFqpsJQfsS+ZXKqyTeKrlqFg2n3V0JsYqtwg1po4J0RTxakGdz9WuduRlTq4MSt6+9Jt6DNhAoQ1WdTAXv8GVslVz7/OkzWt2XdN01QhKhYU9AOQHzxcEbH3QM7zrbWv5T+ndIfCmSo29XDuxW9GvAaXvlutSH1w1aZNzjNy6ZSrqn+ETgsT2vwiRxOGk26Lo31Vpp4AoXKxEW3/Ty9NaTTmKnMJ4WIfqQfWeLR4hXysptmN2IoDwPwLfAE77xKQWpbHBgBxMTZcxtE4v5tgmdiu0m/hRTlwuDgTO1O/f/q595ty9wsF/vWPt6Q84tpbRB6PvaBzoMaYaj5HtGGIYffmCPFfHlbF4ljAdFzcS9hsB718gTN2obtX98SnPl/PuvzQjD55DFVIBBovz1TXmxArF9gGRGlcmzIcaJ3aqP71zBcUhrKrmvgSK9RcC00WBhrmEaDNoxHm0gn1h26mi+7a0i7ApGzH3tzNoVMZARJEpTL3z1ZNMTMQnfVWCgKNuuqsCOgdxVTrriN9uHBm1q6qv3TsVqdiCnhnq0wf2Ct5RK/+64y8AI8ByqT8dwJT2DYPTJjAzfq34jjXACDCyxjtpqN6DPr9dF7gmy0HCEmwby5UqL9+uYBTAxerwzn58ilEGZWgAEgR5d42Y/94x8qge7Pbv4gUpaySgZkRBL+ku+h//k5YfVZ4fwNA/Mf4fRdcmuiG4xnXdKXzXomrKddiQp7u5dInBmUp2P+F2DxgVQY21qQDtYDbB4KWNSR39+RJUEjFoMbHyAYUT/nMaQWjKctHJxCoAz2jCo8gA==">
<source type="video/webm" src="https://val.packett.cool/tiddlypwa-demo.webm">
</video>
</figure>
<p><a href="https://tiddly.packett.cool/">TiddlyPWA</a>.</p>
<p>It’s still rough around the edges, but I’ve put <em>a lot</em> of effort into the setup experience, hiding all the complexity of the flexibility available with how the system is structured
and presenting a very simple “happy path” where if you open the app from a sync server it’s already preconfigured to sync with that server and so on.</p>
<p>I’ve also tried to document it, though I have to say that detailing the security model was a lot easier than trying to explain the whole synchronization/server/content-versus-app thing.
Hopefully at least for those familiar with TiddlyWiki it’s not too hard to understand that there’s the “app wiki” with the core/plugins/themes and the separate encrypted content,
and that the sync server can work with both (differently).</p>
<p>Now, to justify the whole existence of this blog post, let me also list some random fun facts and whatnot – everything that didn’t fit in the documentation itself :)</p>
<h2 id="random-development-tidbits-ha" tabindex="-1"><a class="header-anchor" href="#random-development-tidbits-ha"><span>Random Development Tidbits (ha)</span></a></h2>
<p>Because of how browsers work, I’ve had to take extra care to make the storage work across multiple running tabs without corrupting anything.
Moreover, I made it all explicitly work together well and it’s kind of a hidden feature now.
Thanks to a combination of BroadcastChannels and Web Locks, you can enjoy a multi-tab experience.
Browse the same wiki simultaneously in two tabs, the navigation will be independent but any changes will be visible everywhere.</p>
<p>This whole <a href="https://codeberg.org/valpackett/argon2ian">argon2ian</a> thing you might’ve seen was indeed created for TiddlyPWA!
I’ve started out using PBKDF2 because it’s available in the Web Crypto API, but eventually decided to upgrade to a modern password hash rather than cranking out iteration counts.
I wasn’t satisfied with the state of existing Argon2 WASM+JS wrappers, so I set out to make my own, code-golfed to the tiniest size possible.
The <a href="https://blahaj.zone/notes/9g8r2ffu4o">yak shaving stack</a> got even deeper during that subproject.
Also, I have used the very new <a href="https://developer.mozilla.org/en-US/docs/Web/API/Compression_Streams_API">Compression Streams API</a> to be able to bundle the Wasm binary as compressed while not having to bunlde a decompressor.
And this has led to the creation of a very funny bug…</p>
<p>…so since that API was already used there I started using it to compress long-enough tiddler contents as well.
Stream APIs are kind of annoying when you don’t want to stream, so I went with the whole “a stream is an async iterator” thing.
When I first gave TiddlyPWA to an external tester I got a report of not being able to save a particular tiddler.
Turns out, that iterator thing is only available in Firefox as of right now.
I accidentally made it so that <a href="https://blahaj.zone/notes/9gv2anuoss"><em>tiddlers longer than 240 bytes would only work in Firefox</em></a>. :D</p>
<p>Another <a href="https://blahaj.zone/notes/9gjs72259o">really funny bug</a> is something I <a href="https://bugzilla.mozilla.org/show_bug.cgi?id=1840934">bugzilled here</a> for Firefox Android.
When using <a href="https://github.com/florisboard/florisboard">FlorisBoard</a>, pressing “enter” would result in the password mask (bullet ••• characters) becoming the actual value of the input.
This is something that I have discovered while typing passwords into my TiddlyPWA password prompt!
I also ran into an <a href="https://github.com/mozilla-mobile/firefox-ios/issues/15244">already reported</a> <code>prompt()</code> bug in Firefox on iOS.</p>
<p>The server was initially written with SQLite as it is now, but when Deno KV was announced I got excited and rewrote it to use KV, hoping to leverage Deno Deploy for an easy server hosting experience.
I quickly ran into a 64KiB limit per entry when trying to do the whole “save the HTML page” thing, but I found <a href="https://github.com/kitsonk/kv-toolbox">kv-toolbox</a>, a module that would do chunking for me.
Then, after everything was ready, already with the aforementioned other person testing it, I discovered (and <a href="https://blahaj.zone/notes/9h1dr9g4jq">ranted about</a>) the undocumented <a href="https://github.com/denoland/deno/issues/19284">10 mutations in a transaction</a> limitation.
I wasn’t about to give up on atomicity, so I just rewrote everything back to SQLite in a burst of anger and now I hold a bit of a grudge against Deno KV >_<
Even though I recognize that, obviously, it’s just that <em>my use case is absolutely not the intended one</em>.</p>
<p><a href="https://stackoverflow.com/a/68695508">This TypeScript thing</a> that I <a href="https://blahaj.zone/notes/9h1fdi76dt">referenced on fedi</a> was actually for the SQLite part of the TiddlyPWA server.</p>
<p>There was a <a href="https://blahaj.zone/notes/9hlma651nz">very last minute</a> backwards-incompatible change I suddenly managed to do right before release.</p>
<h2 id="so-tiddlypwa-needs-you" tabindex="-1"><a class="header-anchor" href="#so-tiddlypwa-needs-you"><span>So, TiddlyPWA Needs YOU!</span></a></h2>
<p>I probably could’ve tried to make it as a commercial product with a centrally hosted server, but that’s not what I made.
TiddlyPWA is fully free (both “as in beer” and “as in freedom”) because I believe in software that empowers everyone to own their data <em>and the software itself</em>, that <s>is never gonna give you up</s> doesn’t die for business reasons.</p>
<p>Right now TiddlyPWA is still early in its development, still missing a lot of things, but hopefully is solid enough to make you check it out.
If you like what you see, please consider helping me continue working on it:</p>
<ul>
<li>go ahead and <a href="https://tiddly.packett.cool/">try TiddlyPWA</a> first of course!</li>
<li>report issues, read the code and suggest code changes <a href="https://codeberg.org/valpackett/tiddlypwa">on Codeberg</a>;</li>
<li>support me <a href="https://www.patreon.com/valpackett">on Patreon</a> :3 I’m trying to do this open source thing full time, which is only possible with your support!</li>
<li>follow me <a href="https://blahaj.zone/@valpackett">on the fediverse</a>;</li>
<li>share TiddlyPWA with anyone who might be interested.</li>
</ul>
<p>Thanks!!</p>Yet another keyboard post, or, introducing ErgoNICE2023-01-10T00:00:00Zhttps://val.packett.cool/blog/ergonice/I made a custom split mechanical keyboard! Because it's me, this involved things like contributing to a PCB design tool and discovering a ridiculously optimized way to read a keyboard matrix.<p>A whole decade ago (wow!) I read a blog post called “<a href="https://stevelosh.com/blog/2012/10/a-modern-space-cadet/">A Modern Space Cadet</a>”
and… well, started caring about keyboards.
I’ve adopted quite a few of the software tricks right there and then: shifts-as-parens and
capslock-as-control-and-escape.</p>
<p>The hardware part—the whole mechanical keyboard thing—took me a couple years to get to.
I was only using a MacBook Air when the post came out, but eventually as I got a desktop,
I had purchased my first mech which was a Leopold with original Cherry MX Brown switches.
I’ve used it for several years, but eventually I found myself wanting more.
Specifically, clicky switches and the ergonomics of the split form factor.
In 2020 I purchased <a href="https://a.aliexpress.com/_dVJsSpR">this AliExpress split keyboard</a> from a manufacturer
called “Spider Island”. I’ve <a href="https://github.com/qmk/qmk_firmware/pull/9900">ported QMK to it</a>, because of course.
It felt like a nice upgrade at the time, though the clone MX Blue switches weren’t the highest quality (as in, a couple
started having weird phantom actuations) and the SATA cable connection between the halves being flaky was annoying.</p>
<p>Around the end of 2021 I was looking at higher quality split options like the <a href="https://dygma.com/products/dygma-raise">Dygma Raise</a>
and the <a href="https://www.zsa.io/moonlander/">ZSA Moonlander</a> and… decided that I was too weird for either of them
and would rather spend money on fabrication and parts for something fully custom.
I think <a href="https://www.youtube.com/watch?v=UKfeJrRIcxw">this video</a> might have been the final inspiration.</p>
<p>So, I started thinking about what I actually wanted and came up with this list of requirements:</p>
<ul>
<li>column-staggered layout like the famous <a href="https://www.ergodox.io/">ErgoDox</a></li>
<li>nice clicky switches (after some YouTube-watching the choice was clear: <a href="https://www.youtube.com/watch?v=JaU3GkBKyNQ">Kailh Box Jade</a>!)</li>
<li>“floating key” enclosure design (why is that the common term anyway? I always wanted to call it “borderless”! because it doesn’t have a border, come on!)</li>
<li>for connections: USB Type-C to the host, anything reliable between the halves</li>
</ul>
<p>And a list of extra “wants”:</p>
<ul>
<li>volume control knob (easy)</li>
<li>pins for connecting external buttons like a big red panic button or a <a href="https://hackaday.com/2012/06/21/building-a-clutch-for-vim/">vim pedal</a></li>
<li>a magnetic connector for add-on modules like trackballs, like on the <a href="https://ultimatehackingkeyboard.com/">UHKB</a>?</li>
<li>a 3.3V TTL output-only serial port header, for a “teletype mode” i.e. directly typing into the UART of something like an RPi?</li>
<li>maybe analog input on WASD keys for gaming too?</li>
</ul>
<p>Spoiler alert: I didn’t successfully implement all the experimental stuff :)
But I’m really happy with the result!</p>
<figure class="lqip" style="--ratio:5643/2665;background:#afbeb9 url(data:image/webp;base64,UklGRjYAAABXRUJQVlA4ICoAAABwAQCdASoMAAoACQCsJYwAAk4vGAD+g9kFNR9a9sT2HJgubCeB8scAAAA=)">
<picture>
<source type="image/avif" srcset="ergonice-result.avif">
<img alt="End result, the keyboard in its full glory" src="https://val.packett.cool/ergonice-result.jpg" width="5643" height="2665"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>So, here’s how we got there…</p>
<h2 id="learning-pcb-design" tabindex="-1"><a class="header-anchor" href="#learning-pcb-design"><span>Learning PCB design</span></a></h2>
<p>The first thing you need to make a PCB-based keyboard is, well, being able to design PCBs!
So I watched <a href="https://www.youtube.com/watch?v=C7-8nUU6e3E">a KiCad tutorial on YouTube</a> to learn and…
did not start using KiCad. Because you see, I have a terminal case of hipster-ism :)
So I started to play around with the far less popular <a href="https://horizon-eda.org/">Horizon EDA</a> instead and it was an absolute dream.
What a beautiful, coherent, smart piece of software.</p>
<p>Before actually going for the full keyboard, I’ve decided to make a proof-of-concept: a little dev board
basically containing what would be the brain of the keyboard (I picked the STM32L1 family because it was
less affected by the chip shortage and, you know, power-saving sounds good) and pins.
Tiny boards are a lot cheaper to manufacture, so if something went wrong, it wouldn’t be <em>that</em> much of a loss.
So I designed it, uploaded it to JLCPCB, paid some money and… success! It worked perfectly!</p>
<div class="pic-row">
<figure class="lqip" style="--ratio:2400/1536;background:#3d6654 url(data:image/webp;base64,UklGRjAAAABXRUJQVlA4ICQAAACQAQCdASoMAAoACQCsJYwAAp/SesAA/qh979bT2VM3DShAAAA=)">
<picture>
<source type="image/avif" srcset="ferrispark-eda.avif">
<img alt="Ferrispark PCB loaded in Horizon EDA" src="https://val.packett.cool/ferrispark-eda.jpg" width="2400" height="1536"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<figure class="lqip" style="--ratio:1214/1035;background:#3d6654 url(data:image/webp;base64,UklGRjoAAABXRUJQVlA4IC4AAACQAQCdASoMAAoACQCsJZwAAkpdxAAA/f9jvL2DQUy8WEMLHkzc/41IiWRnXAAA)">
<picture>
<img alt="Two Ferrispark boards fully assembled" src="https://val.packett.cool/ferrispark-live.jpg" width="1214" height="1035"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
</div>
<p>I called this board <a href="https://codeberg.org/valpackett/ferrispark">Ferrispark</a> as a reference to Ferris the Rust mascot
and Digispark, the board whose form factor (18×29 mm) this one fits in.
With that little thingy in hand, it was possible to experiment with firmware stuff while waiting for the keyboard PCB to be manufactured and shipped.
But first…</p>
<h2 id="layout" tabindex="-1"><a class="header-anchor" href="#layout"><span>Layout</span></a></h2>
<p>Unlike “normal” keyboards that come with boring standard ANSI/ISO layouts, enthusiast keyboards are very diverse,
and with a fully custom build possibilities are truly endless. One can <a href="https://www.youtube.com/watch?v=5RN_4PQ0j1A">try to get away with very few keys</a>
or <a href="https://twitter.com/Foone/status/1489430212768985089">make an unhinged meme board</a> or whatever.
But I was after something practical, tailored to my habits, based on ergonomic innovations but not <em>too</em> different from standard layouts.</p>
<p>So I opened <a href="http://www.keyboard-layout-editor.com/">everyone’s favorite non-FOSS (boo) layout editor web app</a>, loaded the Ergodox preset,
removed all the unassigned keys surrounding the core clusters, and started adding keys that made sense to me.</p>
<p>The first thing I added was <em>two</em> columns on the right side that kinda just bring back the punctuation as it is on ANSI.
This felt really important to me because it’s not just about keeping the habits regarding the <code>{[<:;'">]}</code> stuff.
Some non-Latin scripts such as <a href="https://en.wikipedia.org/wiki/JCUKEN">Cyrillic</a> have a lot of letters, so those two columns have actual letters on them when typing those.
Coming up with alternatives (like chords) for <em>that</em> seemed like more of a nightmare than just for punctuation.
By the way, I added these columns without shifting them down, so they ended up as a 3-wide ortholinear grid cluster on the right end of the layout —
I quickly realized that it would be a good fit for a numpad on an alternate layer too!</p>
<p>Then I started adding modifiers and other miscellaneous keys. I’ve added shifts where my fingers expect them to be
(since I use shifts-as-parentheses, the “just one shift key” idea from minimalist boards is really not for me).
For the thumb clusters, I’ve added the most important text actions—Space, Tab, Backspace, Return—as large keys.
I’ve added a dedicated actual Compose key because I use one. Then I started noticing that the key count was climbing up.
Dangerously close to a funny number, even. So I’ve added four extra “macro” keys, some really extra stuff like “menu”,
and ended up with exactly 69 keys. That instantly solved the hard problem of naming things: the board was called “ErgoNICE” from that point on! :)
Here’s how the layout looks:</p>
<figure class="lqip" style="--ratio:941/394.83;">
<picture>
<img alt="ErgoNICE layout illustration" src="https://val.packett.cool/layout.svg" width="941" height="394.83"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>(This is the <em>physical</em> layout, so you see QWERTY here. I actually type on <a href="https://colemak.com/">Colemak</a> though!)</p>
<h2 id="pcbs" tabindex="-1"><a class="header-anchor" href="#pcbs"><span>PCBs</span></a></h2>
<p>Before actually designing the PCB, I started looking at existing keyboard designs (nearly all in KiCad), importing various parts
into Horizon, purchasing extra components like the rotary volume knob and headphone jacks on AliExpress and modeling them in Horizon,
and otherwise doing various preparations.</p>
<p>Keyboard PCBs aren’t rocket science. There are basically two ways to connect the keyswitches to the MCU: <a href="http://blog.komar.be/how-to-make-a-keyboard-the-matrix/">in a matrix</a>
and directly. With a compact split keyboard, direct is more feasible than ever, you don’t even need <em>that</em> huge of a microcontroller,
so I actually saw that solution in one of the designs I was looking at. But my choice of MCU was somewhat limited by the chip shortage,
and my board wasn’t <em>that</em> compact, and I wasn’t looking forward to routing all the direct connections with just 2 layers
(which was the limit for non-green boards on JLCPCB and I just <em>waaaanted</em> a black one even though it won’t be visible),
so the obvious decision was to go with a matrix with diodes on every switch.</p>
<p>The other decision I made early on was that the left half would contain the microcontroller and would be assembled by JLCPCB with a ton of SMD parts,
while the right one I would entirely hand-solder at home, using parts I already owned when possible.
And that would be… through-hole diodes and the <a href="https://www.digikey.com/en/products/detail/microchip-technology/MCP23017-E-SP/894272">MCP23017-E/SP</a>
input/output expander. Yeah, <em>the DIP package variant</em>. Because that’s what I purchased about 10 years ago (!) from
an Arduino stuff store when I was first experimenting with electronics. How convenient!</p>
<p>Speaking of diodes, something I saw and really liked was this <a href="https://github.com/keebio/Keebio-Parts.pretty/blob/c7ae3b44674679f4d767767c002fed1eacd414a1/Diode-Hybrid-Back.kicad_mod">footprint for either a surface-mount or through-hole diode</a>.
However you can’t just import that straightforwardly into Horizon. KiCad is fine with multiple pads/holes sharing a name, and
will just collapse them into one pad. Horizon is a lot more strict: each pad must have its own unique name, and if you want to construct something fancy like that,
you need to do it inside of a padstack. Thankfully, Horizon’s parametric padstack system is extremely capable. It’s based on a little
stack-based scripting language for repositioning everything based on whatever logic you want. However it was missing the ability
to reposition holes from the script, so I’ve had to <a href="https://github.com/horizon-eda/horizon/pull/644">add it</a>, and then:</p>
<figure class="lqip" style="--ratio:2400/1536;background:#2c2c2c url(data:image/webp;base64,UklGRiYAAABXRUJQVlA4IBoAAAAwAQCdASoMAAoACQCsJaQAA3AA/umILaAAAA==)">
<picture>
<source type="image/avif" srcset="padstack.avif">
<img alt="TH-and-SMD padstack loaded in Horizon EDA" src="https://val.packett.cool/padstack.jpg" width="2400" height="1536"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>So. Anyway. Actual keyboard design time. You can view <a href="https://val.packett.cool/ergonice-sch.pdf">the full schematic as a PDF here</a>.
There’s not that much to say: it contains the aforementioned matrix (well, two of them, left and right), the microcontroller and everything it requires,
the USB-C connector with <a href="https://hackaday.com/2019/07/16/exploring-the-raspberry-pi-4-usb-c-issue-in-depth/">correctly separated</a> CC resistors,
TRRS audio connectors, various pin connectors (debug header, extras like external buttons),
the rotary knob with required circuitry… oh, and a bunch of various protection. Even though I’ve noticed that various
DIY keyboard designs don’t seem to do much of it, I was really into the idea of extra safety, so I put resistors on all the external-facing pins,
transistors for reverse polarity protection on power inputs, a USBLC6 IC on the USB lines, and so on.</p>
<figure class="lqip" style="--ratio:2400/1536;background:#393939 url(data:image/webp;base64,UklGRiQAAABXRUJQVlA4IBgAAAAwAQCdASoMAAoACQCsJZwAA3AA/ud1AAA=)">
<picture>
<img alt="ErgoNICE schematic loaded in Horizon EDA" src="https://val.packett.cool/ergonice-eda-sch.png" width="2400" height="1536"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>Layout started with feeding the JSON output of the keyboard-layout-editor website to another website, <a href="http://www.keyboardcad.com/">Keyboard CAD Assistant</a>.
It produces DXF files that are supposed to be used for cutting a plate on a CNC router, but I actually needed it for the PCB.
I’ve imported the DXF into Horizon EDA, drew polygons with diagonals over each key square, and got precise centers of each key —
exactly what’s necessary to position the keyswitches!</p>
<p>Then I drew the outlines of the halves, positioned all the other components, routed the tracks… everything as expected.
Routing can be quite fun, especially when the EDA tool looks this nice (this is the “Rust” color scheme, ha):</p>
<figure class="lqip" style="--ratio:2400/1536;background:#8d604f url(data:image/webp;base64,UklGRjAAAABXRUJQVlA4ICQAAACQAQCdASoMAAoACQCsJYwCdAD1JzAA/iHYl5KgcP/lu92IAAA=)">
<picture>
<source type="image/avif" srcset="ergonice-eda-pcb.avif">
<img alt="ErgoNICE board loaded in Horizon EDA" src="https://val.packett.cool/ergonice-eda-pcb.png" width="2400" height="1536"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>After getting pretty confident that the board was correct (the design rules check in Horizon is pretty helpful!) I’ve sent it off to manufacturing.
A couple weeks later, I received the exciting notification from the post office. The long-awaited package from China!</p>
<div class="pic-row">
<figure class="lqip" style="--ratio:2848/1602;background:#2d3035 url(data:image/webp;base64,UklGRjYAAABXRUJQVlA4ICoAAACwAQCdASoMAAoACQCsJaQAAlxbHelwAP7LX2hEBtZIP5JMX24bFMQQAAA=)">
<picture>
<source type="image/avif" srcset="ergonice-pcb-close-two.avif">
<img alt="ErgoNICE boards close-up" src="https://val.packett.cool/ergonice-pcb-close-two.jpg" width="2848" height="1602"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<figure class="lqip" style="--ratio:2925/1253;background:#2d3035 url(data:image/webp;base64,UklGRjYAAABXRUJQVlA4ICoAAADwAQCdASoMAAoACQCsJaQAAq9hLuLi4AAA/rNbiXGnDHEO6nCvJbigAAA=)">
<picture>
<source type="image/avif" srcset="ergonice-pcb-close.avif">
<img alt="ErgoNICE left side board close-up, microcontroller section" src="https://val.packett.cool/ergonice-pcb-close.jpg" width="2925" height="1253"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
</div>
<p>Here’s a comparison with the 3D preview in Horizon. The real thing always looks amazing!</p>
<div class="pic-row">
<figure class="lqip" style="--ratio:2400/1536;background:#9089d5 url(data:image/webp;base64,UklGRjgAAABXRUJQVlA4ICwAAACwAQCdASoMAAoACQCsJYgAAtzSQeAAAP5yaHlEdtKxghft/0iI7z4ArAAAAA==)">
<picture>
<source type="image/avif" srcset="ergonice-eda-3d.avif">
<img alt="ErgoNICE board 3D preview" src="https://val.packett.cool/ergonice-eda-3d.jpg" width="2400" height="1536"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<figure class="lqip" style="--ratio:2726/1533;background:#2d3035 url(data:image/webp;base64,UklGRjoAAABXRUJQVlA4IC4AAADQAQCdASoMAAoACQCsJaQAAp05LvzAAADykenChOnq6PtY7q9Jxl2CXo12wAAA)">
<picture>
<source type="image/avif" srcset="ergonice-pcb-wide.avif">
<img alt="ErgoNICE boards, both halves" src="https://val.packett.cool/ergonice-pcb-wide.jpg" width="2726" height="1533"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
</div>
<p>If you want to play around with the PCB design files in Horizon EDA yourself, it’s in <a href="https://codeberg.org/valpackett/ergonice/src/branch/trunk/pcb-ergonice">the <code>pcb-ergonice</code> directory in the repo</a>.
And in the <a href="https://codeberg.org/valpackett/ergonice/releases/tag/r1">release downloads</a>, there is an export with Gerber files and BOM/CPL for assembly.
Disclaimer: the revision 1 which is published there is slightly different from the revision 0 which I physically have built. Keep reading to see the bug that I fixed there!</p>
<h3 id="analog-input" tabindex="-1"><a class="header-anchor" href="#analog-input"><span>Analog input?</span></a></h3>
<p>Analog keys are a pretty fun thing for gaming, allowing you to <a href="https://www.youtube.com/watch?v=1WNI-f6QDPQ">move slowly in CS for example</a>.
I stumbled upon <a href="https://alltrons.com/analog-keyboard-technology/">these people here</a> that were trying to commercially sell add-on flex PCBs
for adding the capability to an otherwise normal keyboard and began wondering if it’s possible to just DIY it.
It seems to be a bit of a scary topic because they have a patent in some jurisdictions, but who would go after a non-commercial personal project?
So, their technique is just using a <a href="https://www.ti.com/product/LDC1614">Texas Instruments inductance to digital converter</a> with a PCB coil.
Luckily, these chips were easily available on JLCPCB’s assembly service (though not cheap), so I just went ahead with the experiment.
That is, I designed a little “evaluation board”, suspiciously shaped to fit under the keyboard PCB’s WASD cluster and connect to it using 2.54mm headers :)</p>
<p>The interesting part of the design process is of course the PCB coils. TI provides <a href="https://webench.ti.com/wb5/LDC/#/spirals">an online tool</a> for generating them.
It supports some export… into a couple commercial EDA tools. But I found a way get the results into Horizon. Get this:
export as an EAGLE project, open that in KiCad, export the coil as SVG, clean it up in Inkscape (merge into one SVG path), export as DXF R12,
and finally import into Horizon with a downscaling factor of 10000 because reasons. Oof, it’s there! But we can’t connect anything to it,
because it’s just lines, not tracks. And somehow they’re not even all connected.</p>
<p>Naturally, this was an opportunity to dig into Horizon EDA’s codebase and <a href="https://github.com/horizon-eda/horizon/pull/639">add some new tools</a>!
This was a very enjoyable experience, and with some quick feedback from the author of Horizon I split one of the tools into two, and here they are:
“select connected lines”, “merge duplicate junctions”, and “lines to tracks”. With Horizon becoming this much better, the board was easy to make.
This is how it looked:</p>
<div class="pic-row">
<figure class="lqip" style="--ratio:1303/1199;background:#2ec47a url(data:image/webp;base64,UklGRjYAAABXRUJQVlA4ICoAAABwAQCdASoMAAoACQCsJbAAAqu6gAD9RbAlls++3C41aChu5Tb/t7VAAAA=)">
<picture>
<source type="image/avif" srcset="ldc-board.avif">
<img alt="TI LDC test board flat on the desk" src="https://val.packett.cool/ldc-board.jpg" width="1303" height="1199"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<figure class="lqip" style="--ratio:3260/1822;background:#303030 url(data:image/webp;base64,UklGRi4AAABXRUJQVlA4ICIAAACQAQCdASoMAAoACQCsJZQAApZCS4AA/Y/V9/qMVolQAAAA)">
<picture>
<source type="image/avif" srcset="ldc-testing.avif">
<img alt="TI LDC test board in a testing setup" src="https://val.packett.cool/ldc-testing.jpg" width="3260" height="1822"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
</div>
<p>To try it out, I wrote a <a href="https://codeberg.org/valpackett/ldc1x1x">Rust embedded-hal driver for the LDC1x1x chips</a> and
an <a href="https://codeberg.org/valpackett/freebsd-embedded-hal">embedded-hal implementation for FreeBSD userspace APIs</a>
so that I could test it directly on my PC with a CP2112 USB-to-I²C adapter, just piping the output from a demo program into
<a href="https://github.com/mogenson/ploot">ploot</a>.</p>
<p>What I’ve found is that while the stream of numbers was indeed correlated with how far the key was pushed down,
it was not good. Whether I was catching the movement of the finger or the spring was pretty confusing, especially
when under the actual keyboard PCB. The fact that the switches of my choice have a click bar might’ve been a negative impact,
the distance to the switch from behind of the PCB was probably a problem, and the large 4-layer coil probably wasn’t quite compensating
for that (or was it actually just bad?).</p>
<p>Either way, I couldn’t attach it to the keyboard because it turns out I’ve made a silly mistake in the schematic:
I forgot to connect one of the columns to the microcontroller :D
So I ended up running a bodge wire to one of the holes intended for the LDC board:</p>
<figure class="lqip" style="--ratio:1269/685;background:#303030 url(data:image/webp;base64,UklGRigAAABXRUJQVlA4IBwAAAAwAQCdASoMAAoACQCsJQAAhnAA/umIFOcEoOAA)">
<picture>
<img alt="ErgoNICE left side PCB with bodge wire" src="https://val.packett.cool/ergonice-pcb-bodge.jpg" width="1269" height="685"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<h2 id="case" tabindex="-1"><a class="header-anchor" href="#case"><span>Case</span></a></h2>
<p>There are many ways to make a keyboard enclosure, but as I was into 3D printing, that question was already answered.
How convenient that a split keyboard fits well into the dimentions of an Ender 3 build plate!
I’ve been using <a href="https://github.com/realthunder/FreeCAD">realthunder’s FreeCAD fork</a> for modeling 3D printed parts,
so that’s what I used for this one as well.
Of course FreeCAD’s UI is clunky and its core is <a href="https://github.com/realthunder/FreeCAD/issues?q=fillet+crash">crashy</a>,
but I’d rather not involve proprietary software in this project.
Having CAD files in an open format is that valuable to me.
And before someone starts preaching code-CAD like cadquery to me:
sorry, I love sketching with a mouse and hate school-style math too much :D</p>
<p>I started out with exporting a 3D model of the boards and components on them as a STEP file from Horizon and importing it into FreeCAD.
In the file, everything was cleanly separated out, i.e. each component was its own body. However due to format limitations
every instance of the same part (e.g. every switch) is its own independent body, which takes up a lot of space on disk.
I wrote a Python script in FreeCAD that would take the currently selected bodies and replace all of them except the first one
with a link to the first one, removing the duplication of actual 3D model data. (Sadly I lost that script by now, but it was tiny.)</p>
<p>With a model of the boards, it wasn’t too hard to make an enclosure around them. The enclosure has two main parts.
The plate is a flat extrusion that gets permanently attached between top of the PCB and the bottom halves of the switches.
The tray is the rest of the enclosure, attached to the plate with screws.
In addition there are tenting wedges that attach to the bottom of the tray, and the actual knob that goes on the rotary encoder.
This is how it all looks in FreeCAD:</p>
<div class="pic-row">
<figure class="lqip" style="--ratio:2391/1245;background:#838586 url(data:image/webp;base64,UklGRioAAABXRUJQVlA4IB4AAACQAQCdASoMAAoACQCsJZQAAVTPJAAAU2itLPccAAA=)">
<picture>
<img alt="ErgoNICE case overall look in FreeCAD" src="https://val.packett.cool/ergonice-case-cad-overall.png" width="2391" height="1245"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<figure class="lqip" style="--ratio:2391/1245;background:#838586 url(data:image/webp;base64,UklGRjQAAABXRUJQVlA4ICgAAADQAQCdASoMAAoACQCsJZQAApEG15B0gACfYjCV2Aj3pbyPx40fRxQA)">
<picture>
<source type="image/avif" srcset="ergonice-case-cad-cross.avif">
<img alt="ErgoNICE case cross-section in FreeCAD" src="https://val.packett.cool/ergonice-case-cad-cross.png" width="2391" height="1245"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
</div>
<p>Here’s a look into the tray from the above. I’ve added these supports underneath each key for extra rigidity.
Otherwise it’s… not that remarkable?</p>
<figure class="lqip" style="--ratio:2391/1245;background:#838586 url(data:image/webp;base64,UklGRjQAAABXRUJQVlA4ICgAAADwAQCdASoMAAoACQCsJZACsAD7AQhY8AAA/IlGCE0GihYfqGcHw4AA)">
<picture>
<source type="image/avif" srcset="ergonice-case-cad-tray.avif">
<img alt="ErgoNICE tray in FreeCAD" src="https://val.packett.cool/ergonice-case-cad-tray.png" width="2391" height="1245"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>I printed the parts on a heavily modified Ender 3 Pro out of dark gray sparkly PETG.
It turned out pretty well! Everything fit together easily, the tolerances for things like the space for the key switches
were exactly right, it looks very decent for a DIY object.
Again, this is how it looks:</p>
<figure class="lqip" style="--ratio:5643/2665;background:#afbeb9 url(data:image/webp;base64,UklGRjYAAABXRUJQVlA4ICoAAABwAQCdASoMAAoACQCsJYwAAk4vGAD+g9kFNR9a9sT2HJgubCeB8scAAAA=)">
<picture>
<source type="image/avif" srcset="ergonice-result.avif">
<img alt="End result, the keyboard in its full glory" src="https://val.packett.cool/ergonice-result.jpg" width="5643" height="2665"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>If you want to play around with the source model, it’s <a href="https://codeberg.org/valpackett/ergonice/src/branch/trunk/case">all of that XML in the <code>case</code> directory</a>
stored in the repo, saved using the version-control-friendly “save as directory” functionality of realthunder’s FreeCAD.
And in the <a href="https://codeberg.org/valpackett/ergonice/releases/tag/r1">release downloads</a>, there’s an archive with STL and STEP exports.</p>
<h2 id="firmware" tabindex="-1"><a class="header-anchor" href="#firmware"><span>Firmware</span></a></h2>
<p>Now that everything is put together physically, we need to put software on the tiny little computer that runs the keyboard
(it’s computers all the way down!).</p>
<p>The most common way to get some keyboard firmware going is to use a popular project like <a href="https://qmk.fm/">QMK</a> (a classic
that started on AVR and added Arm later) or <a href="https://zmk.dev/">ZMK</a> (popular with Bluetooth, based on a whole RTOS called Zephyr and
configured with <a href="https://www.devicetree.org/">flattened device trees</a>).
But of course, “common way” means it’s not what I’m going to do.</p>
<h3 id="rust-on-stm32l1" tabindex="-1"><a class="header-anchor" href="#rust-on-stm32l1"><span>Rust on STM32L1</span></a></h3>
<p>I like <a href="https://www.rust-lang.org/">the Rust programming language</a> quite a lot and it’s, like, good for embedded so <em>of course</em> I’m going to use it here.
Naturally, I’m not the only one doing so: I was quite happy to discover that there was already a library called
<a href="https://github.com/TeXitoi/keyberon">Keyberon</a> for handling all the uhh… keyboarding.</p>
<p>Now, how does the Rust ecosystem for STM32 look like?
Unlike the C world where you interact with one monolithic SDK (either the vendor-provided one or <a href="https://libopencm3.org/">libopencm3</a>)
there is a lot more code sharing and integration due to the magic of package management.
The “libopencm3” of Rust is spread over a variety of crates: the center of that vague “SDK” is
interface crates like <a href="https://github.com/rust-embedded/embedded-hal">embedded-hal</a> and <a href="https://github.com/rust-embedded-community/usb-device">usb-device</a>,
and there are both drivers that <em>use</em> those interfaces and microcontroller support crates such as the <a href="https://github.com/stm32-rs">stm32-rs</a> ones that <em>implement</em> them.
There is however the “parallel ecosystems” thing too, but not because of vendors: the rather standalone world here is
<a href="https://embassy.dev/">Embassy</a>, an <code>async</code> embedded framework.</p>
<p>And in fact, because stm32-rs’s <code>stm32l1xx-hal</code> is not actively maintained, I’ve considered using Embassy for the project.
I played around with it, <a href="https://github.com/embassy-rs/embassy/pull/579">fixing some STM32L1 initialization code</a> and stuff,
but ultimately ended up just <a href="https://github.com/stm32-rs/stm32l1xx-hal/pull/14">forking <code>stm32l1xx-hal</code></a>, adding USB support and I²C timeouts and of course fixing bugs.</p>
<p>With a reasonable HAL crate and Keyberon in hand, it’s pretty straightforward to put the pieces together.
But I didn’t do anything in the straightforward way because I can get kinda obsessed with efficiency :)</p>
<h3 id="stm32-hardware-magic" tabindex="-1"><a class="header-anchor" href="#stm32-hardware-magic"><span>STM32 Hardware Magic</span></a></h3>
<p>The cool thing to do for efficiency in embedded development is to let hardware do things as much as possible instead of software.</p>
<p>The first opportunity to leverage nice STM32 peripherals is of course the volume knob: the hardware timers have the
ability to read a rotary encoder instead of, well, counting time.
As long as you connect the encoder to a pair of pins that do correspond to a timer, which I did as
I knew this beforehand, at the PCB design stage.
So in the code, grabbing the timer for this purpose is as simple as:</p>
<pre><code class="language-rust">let knob_cnt = cx.device.TIM2.qei((gpioa.pa0, gpioa.pa1), &mut rcc);
</code></pre>
<p>And when polling all the things, we simply check if the count has increased or decreased since the previous
time we did that, and by how much.
Depending on that, we press-and-release a designated key position in the layout,
where we place volume up / volume down keys on the main layer and other fun stuff on other layers
(e.g. I made it so that Fn+knob is scroll up / scroll down, just for fun).</p>
<pre><code class="language-rust">let knob_now = cx.local.knob_cnt.count() as i16;
let knob_row = if knob_now > *cx.local.knob_last { 1 } else { 0 };
for _ in 0..(knob_now - *cx.local.knob_last).abs() / 2 {
handle_event::spawn(layout::Event::Press(knob_row, (LCOLS + RCOLS) as u8)).unwrap();
handle_event::spawn(layout::Event::Release(knob_row, (LCOLS + RCOLS) as u8)).unwrap();
}
*cx.local.knob_last = knob_now;
</code></pre>
<p>Now, how about something more advanced? But what is left there to automate?
Well, of course, the actual reading of the keyboard matrix! It’s a common thing, there should be hardware solutions!
And there are. There are ICs that do keyboard matrix scanning (usually I/O expanders with that functionality),
the Programmable I/O peripheral of the RP2040 might be promising for this application,
of course cool things can be done with FPGAs (you can make a microcontroller with your own key scanner peripheral
like in the <a href="https://github.com/esden/icekeeb">icekeeb</a> project).
But it turns out our little STM32 already has a great tool for the job!</p>
<p>Because… it can DMA between memory and banks of GPIO pins, triggered by a timer.
When I was thinking about the DMA capabilities, I was wondering if someone has already done what I wanted to do and yes!
<a href="https://summivox.wordpress.com/2016/06/03/keyboard-matrix-scanning-and-debouncing/">This 2016 blog post</a> describes
exactly how to do it. (With some bonus big-brain thoughts on key debouncing.)
This part of the post sounded somewhat worrying:</p>
<blockquote>
<p>Allocate all row output pins on one GPIO port and all column input pins on another GPIO port</p>
</blockquote>
<p>as I have <em>not</em> done that on my PCB. But well, I quickly realized that I could extend this idea to work
with pins arbitrarily scattered across both of the two pin banks, just by using more timers.
But my job was slightly easier as only columns (outputs in my design) were in both banks, A and B,
while rows (inputs) only were in bank B.</p>
<details>
<summary class="open-more">Here's how the Rust magic for that looks</summary>
<p>To generate the bit patterns, I used a <code>const fn</code>:</p>
<pre><code class="language-rust">const fn gen_gpio<const N: usize>(pins: [i32; N]) -> [u32; N] {
let mut result: [u32; N] = [0; N];
let mut p = 0;
while p < N {
if pins[p] >= 0 { result[p] = 1 << pins[p]; }
p += 1;
}
result
}
static mut IN_GPIOB: [[u32; LCOLS]; 2 * BOUNCES] = [[69; LCOLS]; 2 * BOUNCES];
static OUT_GPIOA: [u32; LCOLS] = gen_gpio([-1, -1, -1, -1, -1, 8, -1]);
static OUT_GPIOB: [u32; LCOLS] = gen_gpio([ 1, 12, 13, 14, 15, -1, 0]);
</code></pre>
<p>Okay, not that cool, it’s a rather verbose way to avoid writing <code>1 <<</code> everywhere, but I’m a big fan of
compile-time function execution.
But more to the point, the memory layout already explains how the DMA setup works.
On every tick, the next output configuration will be selected across both GPIO ports (activating the next column)
and the next readout of the inputs (well, of all the inputs on bank B) will be put into memory.</p>
<p>For configuring the DMA engines, which don’t have friendly wrappers in the HAL crate,
I wrote <a href="https://codeberg.org/valpackett/ergonice/src/commit/e3c0f93648354c8abb4ec118c95ed4e913308600/fw/src/dma_scan.rs">a macro</a>
wrapping the raw hardware register writes to make it look decent like so:</p>
<pre><code class="language-rust">dma_chan! { dma1: // TIM4_CH1 -> DMA1_CH1
cndtr1 [LCOLS] cmar1 [&OUT_GPIOB as *const _]
cpar1 [gpiob_odr] ccr1 [mem2per nointr]
};
dma_chan! { dma1: // TIM3_CH3 -> DMA1_CH2
cndtr2 [LCOLS] cmar2 [&OUT_GPIOA as *const _]
cpar2 [gpioa_odr] ccr2 [mem2per nointr]
};
dma_chan! { dma1: // TIM3_UP -> DMA1_CH3
cndtr3 [LCOLS * BOUNCES * 2] cmar3 [&mut IN_GPIOB as *mut _]
cpar3 [gpiob_idr] ccr3 [per2mem intr]
};
</code></pre>
<p>So, the output channels just cycle through the very short 7-element arrays, enabling only one column at a time.
The input reading channel is the interesting one.
The <code>* 2</code> in the array length is related to DMA double buffering.
See, the engine can interrupt the CPU both when it goes through the entire array and wraps around,
and right in the middle of the array. This allows the CPU and the DMA engine to avoid stepping on each other’s feet:
on the half interrupt we read the first half of the memory, and on the full interrupt we read the second one.
And yeah, each half contains a few readouts in a row, to be fed into a debouncer.
This is how it’s done, converting the raw GPIO readout bytes into a matrix for Keyberon:</p>
<pre><code class="language-rust">static ROW_GPIOS: [u32; ROWS] = gen_gpio([4, 5, 6, 7, 2]);
let is_half = dma1.isr.read().htif3().bit();
let scans = unsafe {
&IN_GPIOB[if is_half {
0..BOUNCES
} else {
BOUNCES..2 * BOUNCES
}]
};
for scan in scans {
let mut mat = [[false; LCOLS]; ROWS];
for (c, colscan) in scan.iter().enumerate() {
for (r, rowmask) in ROW_GPIOS.iter().enumerate() {
mat[r][c] = (colscan & rowmask) != 0;
}
}
for ev in cx.local.debouncer_l.events(matrix::PressedKeys(mat)) {
handle_event::spawn(ev.transform(left2global)).unwrap();
}
}
</code></pre>
</details>
<p>And… it works! Now, of course I haven’t measured any efficiency gains versus doing everything the basic software way :D
But having fun with the implementation was the point more than anything.</p>
<p>The entire source of the firmware is in <a href="https://codeberg.org/valpackett/ergonice/src/branch/trunk/fw">the <code>fw</code> directory in the repo</a>.</p>
<h3 id="things-not-done" tabindex="-1"><a class="header-anchor" href="#things-not-done"><span>Things not done</span></a></h3>
<p>Of course the end result ended up less ambitious than the initial project.
Out of the initial list of ideas, the analog input fell out at the PCB stage, while
the extra peripheral (trackball/etc.) support kinda fell out at the case design stage.
The “teletype mode” via the output-only serial port header only fell out here at the firmware stage though,
with potential to come back and add it someday.</p>
<p>However there’s another thing I was rather interested in but didn’t get to try in the firmware, and that is
<a href="https://learn.microsoft.com/en-us/windows-hardware/drivers/hid/selective-suspend-for-hid-over-usb-devices">USB Selective Suspend</a>
which should allow the microcontroller to go to sleep for idle periods.
This isn’t easy to accomplish, but it seems like resuming the USB connection after sleep on the STM32L1 series might be possible.</p>FreeBSD and custom firmware on the Google Pixelbook2019-10-06T00:00:00Zhttps://val.packett.cool/blog/pixelbook/A search for a new thin-and-light laptop, a journey through the Chromebook firmware trust architecture, some FreeBSD kernel development, and finally, something about actually customizing open source firmware.<p><a href="https://val.packett.cool/blog/thinkpad-x240/">Back in 2015</a>, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on.
Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn’t use the trackpoint much.
But this summer I’ve decided that it was time for something newer.
I wanted something…</p>
<ul>
<li>lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);</li>
<li>with a 3:2 display (why is Lenovo making these Serious Work™ laptops 16:9 in the first place??
16:9 is <strong>awful</strong> in below-13-inch sizes especially);</li>
<li>with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);</li>
<li>with USB-C ports;</li>
<li>without a dGPU, <em>especially</em> without an NVIDIA GPU;</li>
<li>assembled with screws and not glue (I don’t necessarily need expansion and stuff in a laptop all that much,
but being able to replace the battery without dealing with a glued chassis is good);</li>
<li>supported by FreeBSD of course (“some development required” is okay but I’m not going to write <em>big</em> drivers);</li>
<li>how about something with open source firmware, that would be fun.</li>
</ul>
<p>The <a href="https://github.com/aarch64-laptops">Qualcomm aarch64 laptops</a> were out because embedded GPU drivers like freedreno
(and UFS storage drivers, and Qualcomm Wi-Fi…) are not ported to FreeBSD.
And because Qualcomm firmware is very cursed.</p>
<p><a href="https://rosenzweig.io/blog/panfrost-on-the-rk3399-meow.html">Samsung’s RK3399 Chromebook</a>
or the new <a href="https://www.pine64.org/pinebook-pro/">Pinebook Pro</a> would’ve been awesome… if I were a Linux user.
No embedded GPU drivers on FreeBSD, again. No one has added the stuff needed for FDT/OFW attachment to LinuxKPI.
It’s rather tedious work, so we only support PCIe right now.
(Can someone please make an ARM laptop with a PCIe GPU, say with an <a href="https://en.wikipedia.org/wiki/Mobile_PCI_Express_Module">MXM slot</a>?)</p>
<p>So it’s still gonna be amd64 (x86) then.</p>
<p>I really liked the design of the Microsoft Surface Book, but the
<a href="https://www.ifixit.com/Teardown/Microsoft+Surface+Book+Teardown/51972#s113615">iFixit score of 1 (one)</a>
and especially the Marvell Wi-Fi chip that doesn’t have a driver in FreeBSD are dealbreakers.</p>
<p>I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that’s fun.
But going <em>back</em> in processor generations just doesn’t feel great.
I want something more efficient, not less!</p>
<p>And then I discovered the <a href="https://en.wikipedia.org/wiki/Google_Pixelbook">Pixelbook</a>.
Other than the big huge large bezels around the screen, I liked everything about it.
Thin aluminum design, a 3:2 HiDPI screen, <strong>rubber palm rests</strong> (why isn’t every laptop ever doing that?!),
the “convertibleness” (flip the screen around to turn it into… something rather big for a tablet, but it is useful actually),
a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi),
and that famous coreboot support (Chromebooks’ stock firmware <em>is</em> coreboot + depthcharge).</p>
<p>So here it is, my new laptop, a Google Pixelbook.</p>
<h2 id="what-is-a-chromebook-even" tabindex="-1"><a class="header-anchor" href="#what-is-a-chromebook-even"><span>What is a Chromebook, even</span></a></h2>
<p>The write protect screw is kind of a meme.
All these years later, it’s That Thing everyone on various developer forums associates with Chromebooks.
But times have moved on.
As a reaction to glued devices and stuff, the Chrome firmware team has discovered a new innovative way of asserting
physical presence: sitting around for a few minutes, pressing the power button when asked.
Is actually pretty clever though, it <em>is</em> more secure than… not doing that.</p>
<p>Wait, what was that about?</p>
<p>Let’s go back to the beginning and look at firmware security in Chromebooks and other laptops.</p>
<p>These devices are designed for the mass market first.
Your average consumer trusts the vendor and (because they’ve read a lot of scary news) might be afraid of scary attackers
out to install a stealthy rootkit right into their firmware.
Businesses are even more afraid of that, and they push for boot security on company laptops even more.
This is why <a href="https://github.com/corna/me_cleaner/wiki/Intel-Boot-Guard">Intel Boot Guard</a> is a thing
that the vast majority of laptops have these days.
It’s a thing that makes sure only the vendor can update firmware.
Evil rootkits are out. Unfortunately, the user is also out.</p>
<p>Google is not like most laptop vendors.</p>
<p>Yes, Google is kind of a surveillance capitalism / advertising monster, but that’s not what I’m talking about here.
Large parts of Google are very much driven by FOSS enthusiasts.
Or something. Anyway, the point is that Chromebooks are based on FOSS firmware and support
user control as much as possible.
(Without compromising regular-user security, but turns out these are not conflicting goals and we can all be happy.)</p>
<p>Instead of Boot Guard, Google has its own way of securing the boot process.
The root of trust in modern (>=2017) devices is a special
<a href="https://www.youtube.com/watch?v=gC-lbMNmIsg">Google Security Chip</a>, which in normal circumstances
also ensures that only Google firmware runs on the machine, but:</p>
<ul>
<li>if you sit through the aforementioned power-button-clicking procedure,
you get into Developer Mode: OS verification is off, you have a warning screen at boot,
and you can press Ctrl-D to boot into Chrome OS, or (if you enabled this via a command run as root)
Ctrl-L to open SeaBIOS.
<ul>
<li>Here’s the fun part… it doesn’t have to be SeaBIOS.
You can flash any Coreboot payload into the <code>RW_LEGACY</code> slot right from Chrome OS, reboot, press a key
and you’re booting that payload!</li>
</ul>
</li>
<li>if you also <a href="https://www.sparkfun.com/products/14746">buy</a> or
<a href="https://chromium.googlesource.com/chromiumos/third_party/hdctools/+/master/docs/ccd.md#making-your-own-suzyq">solder</a>
a special cable (“SuzyQable”) and do the procedure a couple times more, your laptop turns into the
Ultimate Open Intel Firmware Development Machine. Seriously.
<ul>
<li>the security chip is a debug chip too!
<a href="https://chromium.googlesource.com/chromiumos/platform/ec/+/master/docs/case_closed_debugging_cr50.md">Case Closed Debugging</a>
gives you serial consoles for the security chip itself, the embedded controller (EC) and the application processor (AP, i.e. your main CPU),
and it also gives you a flasher
(via <a href="https://aur.archlinux.org/packages/flashrom-chromeos/">special flashrom</a> for now, but I’m told there’s plans to upstream)
that allows you to write AP and EC firmware;</li>
<li>some security is still preserved with all the debugging: you can (and should) set a CCD password, which lets you
lock-unlock the debug capabilities and change write-protect whenever you want, so that only you can flash firmware
(at least without opening the case and doing <strong>very</strong> invasive things, the flash chip is not even SOIC anymore I think);</li>
<li>and you can hack without fear: the security chip is not brickable! Yes, yes, that means the chip is only a
“look but don’t touch” kind of open source, it will only boot Google-signed firmware.
Some especially paranoid people think this is An NSA Backdoor™.
I think this is an awesome way to allow FULL control of the main processor and the EC,
over just a cable, with <strong>no way of bricking the device</strong>!
And to solve the paranoia, <a href="https://reproducible-builds.org/">reproducible builds</a> would be great.</li>
</ul>
</li>
</ul>
<h2 id="you-mentioned-something-about-freebsd-in-the-title" tabindex="-1"><a class="header-anchor" href="#you-mentioned-something-about-freebsd-in-the-title"><span>You mentioned something about FreeBSD in the title?</span></a></h2>
<p>Okay, okay, let’s go.
I didn’t even <em>want</em> to write an introduction to Chromebooks but here we are.
Anyway, while waiting for the debug cable to arrive, I’ve done a lot of work on FreeBSD,
using the first method above (<code>RW_LEGACY</code>).</p>
<p>SeaBIOS does not have display output working in OSes that don’t specifically support the Coreboot framebuffer
(OpenBSD does, FreeBSD doesn’t), and I really just hate legacy BIOS, so
I’ve had to install a UEFI implementation into <code>RW_LEGACY</code> since I didn’t have the cable yet.
My own EDK2 build did not work (now I see that it’s probably because it was a debug build and that has failing assertions).
So I’ve downloaded <a href="https://mrchromebox.tech/">MrChromebox’s full ROM image</a>, extracted the payload using <code>cbfstool</code>
and flashed that. Boom. Here we go, press Ctrl-L for UEFI. Nice. Let’s install FreeBSD.</p>
<p>The live USB booted fine. With the EFI framebuffer, an NVMe SSD and a PS/2 keyboard it was a working basic system.
I’ve resized the Chrome OS data partition (Chrome OS recovers from that fine, without touching custom partitions),
found that there’s already an EFI system partition (with a GRUB2 setup to boot Chrome OS, which didn’t boot like that o_0),
installed everything and went on with configuration and developing support for more hardware.</p>
<p>(note: I’m leaving out the desktop configuration part here, it’s mostly a development post; I use <a href="https://github.com/WayfireWM/wayfire">Wayfire</a> as my display server if you’re curious.)</p>
<p>So how’s the hardware?</p>
<h3 id="wi-fi-and-bluetooth" tabindex="-1"><a class="header-anchor" href="#wi-fi-and-bluetooth"><span>Wi-Fi and Bluetooth</span></a></h3>
<p>Well, that was easy.
The Pixelbook has an Intel 7265.
The exact same wireless chip that was in my ThinkPad.
So, Wi-Fi works great with <code>iwm</code>.</p>
<p>Bluetooth… if this was the newer 8265, would’ve already just worked :D</p>
<p>These Intel devices present a “normal” <code>ubt</code> USB Bluetooth adapter, except it only becomes normal if you upload firmware
into it, otherwise it’s kinda dead.
(And in that dead state, it spews interrupts, raising the idle power consumption by preventing the system
from going into package C7 state! So <code>usbconfig -d 0.3 power_off</code> that stuff.)
FreeBSD now has a firmware uploader for the 8260/8265, but it does not support the older protocol used by the 7260/7265.
It wouldn’t be that hard to add that, but no one has done it yet.</p>
<h3 id="input-devices" tabindex="-1"><a class="header-anchor" href="#input-devices"><span>Input devices</span></a></h3>
<h4 id="keyboard" tabindex="-1"><a class="header-anchor" href="#keyboard"><span>Keyboard</span></a></h4>
<p>Google kept the keyboard as good old PS/2, which is great for ensuring that you can get started with a custom OS
with a for-sure working keyboard.</p>
<p>About the only interesting thing with the keyboard was the Google Assistant key, where the Win key usually is.
It was not recognized as anything at all.
I used DTrace to detect the scancode without adding prints into the kernel and rebooting:</p>
<pre><code>dtrace -n 'fbt::*scancode2key:entry { printf("[st %x] %x?\n", *(int*)arg0, arg1); } \
fbt::*scancode2key:return { printf("%x\n", arg1); }'
</code></pre>
<p>And wrote <a href="https://reviews.freebsd.org/D21565">a patch</a> to interpret it as a useful key
(right meta, couldn’t think of anything better).</p>
<h4 id="touch" tabindex="-1"><a class="header-anchor" href="#touch"><span>Touch*</span></a></h4>
<p>The touchpad and touchscreen are HID-over-I²C, like on many other modern laptops.
I don’t know why this cursed bus from the 80s is <em>gaining</em> popularity, but it is.
At least FreeBSD has a driver for Intel (Synopsys DesignWare really) I²C controllers.</p>
<p>(Meanwhile Apple MacBooks now use <a href="https://github.com/roadrunner2/macbook12-spi-driver"><em>SPI for even the keyboard</em></a>.
FreeBSD has an Intel SPI driver but right now it only supports ACPI attachment for Atoms and such, not PCIe yet.)</p>
<p>The even better news is that <a href="https://github.com/wulf7/iichid">there is a nice HID-over-I²C driver in development as well</a>.
(note: the <a href="https://people.freebsd.org/~wulf/acpi_iicbus.patch">corresponding patch for configuring the devices via ACPI</a>
is pretty much a requirement, uncomment <code>-DHAVE_ACPI_IICBUS</code> in the iichid makefile too to get that to work.
Also, <a href="https://bugs.freebsd.org/bugzilla/attachment.cgi?id=207356&action=diff">upcoming Intel I²C improvement patch</a>.)</p>
<p>The touchscreen started working with that driver instantly.</p>
<p>The touchpad was… a lot more “fun”.
The I²C bus it was on would just appear dead.
After some <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=240339">debugging</a>, it turned out that the in-progress
iichid driver was sending a wrong extra out-of-spec command, which was causing Google’s touchpad firmware
to throw up and lock up the whole bus.</p>
<p>But hey, nice bug discovery, if any other device turns out to be as strict in accepting input, no one else would have that problem.</p>
<p>Another touchpad thing: by default, you have to touch it with a lot of pressure.
Easily fixed in libinput:</p>
<pre><code>% cat /usr/local/etc/libinput/local-overrides.quirks
[Eve touchpad]
MatchUdevType=touchpad
AttrPressureRange=12:6
</code></pre>
<h4 id="upd-2019-10-24-pixelbook-pen" tabindex="-1"><a class="header-anchor" href="#upd-2019-10-24-pixelbook-pen"><span><strong>UPD 2019-10-24</strong> Pixelbook Pen</span></a></h4>
<p>The touchscreen in the Pixelbook is made by Wacom, and supports stylus input like the usual Wacom tablets.
For USB ones, on FreeBSD you can just use <code>webcamd</code> to run the Linux driver in userspace.
Can’t exactly do that with I²C.</p>
<p>But! <a href="https://val.packett.cool/notes/2019-10-21-20-12-26">Thankfully, it exposes generic HID stylus reports</a>, zero Wacom specifics required.
I’ve been able to <a href="https://github.com/wulf7/iichid/pull/5">write a driver</a> for that quite easily.
Now it works. With pressure, tilt, the button, all the things :)</p>
<h3 id="display-backlight-brightness" tabindex="-1"><a class="header-anchor" href="#display-backlight-brightness"><span>Display backlight brightness</span></a></h3>
<p>This was another “fun” debugging experience.
The <code>intel_backlight</code> console utility (which was still the thing to use on FreeBSD) did nothing.</p>
<p>I knew that the i915 driver on Chrome OS could adjust the brightness, so I made it work here too, and all it took is:</p>
<ul>
<li><a href="https://reviews.freebsd.org/D21564">adding more things to LinuxKPI</a> to allow uncommenting the
brightness controls in i915kms;</li>
<li>(and naturally, uncommenting them);</li>
<li>finding out that this panel uses native DisplayPort brightness configured via DPCD (DisplayPort Configuration Data),
enabling <code>compat.linuxkpi.i915_enable_dpcd_backlight="1"</code> in <code>/boot/loader.conf</code>;</li>
<li>finding out that there’s a fun bug in the… hardware, sort of:
<ul>
<li>the panel reports that it supports both DPCD backlight and a direct PWM line (which is true);</li>
<li>Google/Quanta/whoever <em>did not connect</em> the PWM line;</li>
<li>(the panel is not aware of that);</li>
<li>the i915 driver prefers the PWM line when it’s reported as available.</li>
</ul>
</li>
</ul>
<p>Turns out there was <a href="https://patchwork.kernel.org/patch/9640237/">a patch sent to Linux to add a “prefer DPCD” toggle</a>,
but for some reason it was not merged.
The patch does not apply cleanly so I just did a simpler hack version:</p>
<pre><code class="language-diff">--- i/drivers/gpu/drm/i915/intel_dp_aux_backlight.c
+++ w/drivers/gpu/drm/i915/intel_dp_aux_backlight.c
@@ -252,8 +252,12 @@ intel_dp_aux_display_control_capable(struct intel_connector *connector)
* the panel can support backlight control over the aux channel
*/
if (intel_dp->edp_dpcd[1] & DP_EDP_TCON_BACKLIGHT_ADJUSTMENT_CAP &&
- (intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP) &&
- !(intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_PWM_PIN_CAP)) {
+ (intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_AUX_SET_CAP)
+/* for Pixelbook (eve), simpler version of https://patchwork.kernel.org/patch/9618065/ */
+#if 0
+ && !(intel_dp->edp_dpcd[2] & DP_EDP_BACKLIGHT_BRIGHTNESS_PWM_PIN_CAP)
+#endif
+ ) {
DRM_DEBUG_KMS("AUX Backlight Control Supported!\n");
return true;
}
</code></pre>
<p>And with that, it works, with 65536 steps of brightness adjustment even.</p>
<h3 id="suspend/resume" tabindex="-1"><a class="header-anchor" href="#suspend/resume"><span>Suspend/resume</span></a></h3>
<p>The Pixelbook uses regular old ACPI S3 sleep, not the <a href="https://reviews.freebsd.org/D17676">fancy new S0ix thing</a>,
so that’s good.</p>
<p>On every machine with a <a href="https://en.wikipedia.org/wiki/Trusted_Platform_Module">TPM</a> though,
you have to tell the TPM to save state before suspending, otherwise you get a reset on resume.
I already knew this because I’ve <a href="https://val.packett.cool/notes/2017-11-22-15-11-44">experienced that on the ThinkPad</a>.</p>
<p>The Google Security Chip runs an open-source TPM 2.0 implementation (fun fact, written by Microsoft)
and it’s connected via… *drum roll* I²C. Big surprise (not).</p>
<p>FreeBSD already has TPM 2.0 support in the kernel, the
<a href="https://github.com/tpm2-software/tpm2-tools">userspace tool stack</a> was recently added to Ports as well.
But of course there was no support for connecting to the TPM over I²C, and especially not to the
Cr50 (GSC) TPM specifically. (it has quirks!)</p>
<p>I <a href="https://github.com/myfreeweb/freebsd/commit/600e6006c1b969b73d3fec86208f37011884f1e1">wrote a driver (WIP)</a>
hooking up the I²C transport (relies on the aforementioned ACPI-discovery-of-I²C patch).
It does not use the interrupt (I found it buggy: at first attachment, it fires continuously,
and after a reattach it stops completely) and after attach (or after system resume) the <em>first</em> command
errors out, but that can be fixed and other than that, it works.
Resume is fixed, entropy can be harvested, it <a href="https://github.com/tpm2-software/tpm2-pkcs11">could be used for SSH keys</a> too.</p>
<p>Another thing with resume: I’ve had to build the kernel with <code>nodevice sdhci</code> to prevent the
Intel SD/MMC controller (which is not attached to anything here - I’ve heard that the 128GB model might be using eMMC
instead of NVMe but that’s unclear) from <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211705">hanging for a couple minutes on resume</a>.</p>
<h3 id="dynamic-cpu-frequency" tabindex="-1"><a class="header-anchor" href="#dynamic-cpu-frequency"><span>Dynamic CPU frequency</span></a></h3>
<p>At least on the stock firmware, the old-school Intel SpeedStep did not work because the driver could not
find some required ACPI nodes (perf or something).</p>
<p>Forget that, the new <a href="https://reviews.freebsd.org/D18028">Intel Speed Shift</a> (which lets the CPU adjust frequency on its own) works nicely with the linked patch.</p>
<h3 id="tablet-mode-switch" tabindex="-1"><a class="header-anchor" href="#tablet-mode-switch"><span>Tablet mode switch</span></a></h3>
<p>When the lid is flipped around, the keyboard is disabled (unless you turn the display brightness to zero,
I’ve heard - which is fun because that means you can <a href="https://www.reddit.com/r/PixelBook/comments/dalk1l/my_pixelbook_desktop_setup/f1u4nwi/">connect a montior and have a sort-of
computer-in-a-keyboard look</a>,
like retro computers) and the system gets a notification (Chrome OS reacts to that by enabling tablet mode).</p>
<p>Looking at the DSDT table in ACPI, it was quite obvious how to support that notification:</p>
<pre><code>Device (TBMC) {
Name (_HID, "GOOG0006") // _HID: Hardware ID
Name (_UID, One) // _UID: Unique ID
Name (_DDN, "Tablet Motion Control") // _DDN: DOS Device Name
Method (TBMC, 0, NotSerialized) {
If ((RCTM () == One)) { Return (One) }
Else { Return (Zero) }
}
}
</code></pre>
<p>On Linux, this is exposed as an evdev device with switch events.
I was able to <a href="https://reviews.freebsd.org/D21612">replicate that</a> quite easily.
My display server does not support doing anything with that yet, but I’d like to do something like
enabling <a href="https://source.puri.sm/Librem5/squeekboard">an on-screen keyboard</a> to pop up automatically
when tablet mode is active.</p>
<h3 id="keyboard-backlight-brightness" tabindex="-1"><a class="header-anchor" href="#keyboard-backlight-brightness"><span>Keyboard backlight brightness</span></a></h3>
<p>I generally leave it off because I don’t look at the keyboard, but this was
<a href="https://reviews.freebsd.org/D21746">a fun and easy driver to write</a>.</p>
<p>Also obvious how it works when looking at ACPI:</p>
<pre><code>Device (KBLT) {
Name (_HID, "GOOG0002") // _HID: Hardware ID
Name (_UID, One) // _UID: Unique ID
Method (KBQC, 0, NotSerialized) {
Return (^^PCI0.LPCB.EC0.KBLV) /* \_SB_.PCI0.LPCB.EC0_.KBLV */
}
Method (KBCM, 1, NotSerialized) {
^^PCI0.LPCB.EC0.KBLV = Arg0
}
}
</code></pre>
<h3 id="using-the-debug-cable-on-freebsd" tabindex="-1"><a class="header-anchor" href="#using-the-debug-cable-on-freebsd"><span>Using the debug cable <strong>on</strong> FreeBSD</span></a></h3>
<p>The debug cable presents serial consoles as bulk endpoints without any configuration capabilities.
On Linux, they are supported by <a href="https://github.com/torvalds/linux/blob/ba606975938179b1e893e44e190d0001de8e5262/drivers/usb/serial/usb-serial-simple.c">the “simple” USB serial driver</a>.</p>
<p>Adding the device to the “simple” FreeBSD driver <code>ugensa</code> <a href="https://reviews.freebsd.org/D21863">took some debugging</a>.
The driver was clearing USB stalls when the port is opened.
That’s allowed by the USB spec and quite necessary on some devices.
Unfortunately, the debug interface throws up when it sees that request.
The responsible code in the device <a href="https://chromium.googlesource.com/chromiumos/platform/ec/+/refs/tags/cr50_v4.5/chip/g/usb.c#835">has a <code>/* Something we need to add support for? */</code> comment</a> :D</p>
<p><ins><strong>UPD</strong>: A Chrome OS engineer has notified me that the workaround is no longer necessary since Cr50 firmware version 0.3.24!</ins></p>
<h3 id="audio" tabindex="-1"><a class="header-anchor" href="#audio"><span>Audio?</span></a></h3>
<p>The only thing that’s unsupported is onboard audio.
The usual HDA controller only exposes the DisplayPort audio-through-the-monitor thing.
The speakers, mic and headphone jack are all connected to various codecs exposed via… yet again, I²C.
I am not about to write the drivers for these codecs, since I’m not really interested in audio on laptops.</p>
<h2 id="firmware-is-fun" tabindex="-1"><a class="header-anchor" href="#firmware-is-fun"><span>Firmware is Fun</span></a></h2>
<p>After the debug cable arrived, I’ve spent some time debugging the console-on-FreeBSD thing mentioned above,
and then started messing with coreboot and TianoCore EDK2.</p>
<p>My discoveries so far:</p>
<ul>
<li>there’s nothing on the AP console on stock firmware because Google compiles release FW with serial output off,
I think to save on power or something;</li>
<li><code>me_cleaner</code> <a href="https://github.com/corna/me_cleaner/issues/300">needs to be run with <code>-S -w MFS</code></a>.
As mentioned in the <code>--help</code>, the <code>MFS</code> partition contains PCIe related stuff.
Removing it causes the NVMe drive to detach soon after boot;</li>
<li>upstream Coreboot (including MrChromebox’s builds) fails to initialize the TPM, just gets zero
in response to the vendor ID request. Funnily enough, that would’ve solved the resume problem
without me having to write the I²C TPM driver for FreeBSD - but now that I’ve written it, I’d prefer
to actually have the ability to use the TPM;</li>
<li>EDK2’s recent <code>UefiPayloadPkg</code> doesn’t support <a href="https://bugzilla.tianocore.org/show_bug.cgi?id=2241">PS/2 keyboard</a>
and <a href="https://bugzilla.tianocore.org/show_bug.cgi?id=2240">NVMe</a> out of the box, but they’re very easy to add
(hopefully someone would add them upstream after seeing my bug reports);</li>
<li><code>UefiPayloadPkg</code> supports getting the framebuffer from coreboot very well;</li>
<li>coreboot can run Intel’s GOP driver before the payload (it’s funny that we’re running a UEFI module before
running the UEFI implementation) and that works well;</li>
<li>but <code>libgfxinit</code> - the nice FOSS, written-in-Ada, verified-with-SPARK implementation of Intel GPU
initialization and framebuffer configuration - supports Kaby Lake now!
<ul>
<li>however, we have a DPCD thing again with this display panel here - it reports
max lane bandwidth as <code>0x00</code>, <code>libgfxinit</code> interprets that as the slowest speed and
we end up not having enough bandwidth for the high-res screen;</li>
<li>I’ve been told that this is because there’s a new way of conveying this information that’s unsupported. I’ll dig
around in the Linux i915 code and try to implement it properly here but for now, I just did a quick hack,
hardcoding the faster bandwidth. Ta-da! My display is initialized with formally verified open source code!
Minus one blob running at boot!</li>
</ul>
</li>
<li>persistent storage of EFI variables needs some SMM magic. There’s
<a href="https://github.com/MrChromebox/edk2/commit/c792dbf1bd692bb95781a79d5d2d886a4933bc77">a quick patch</a> that
changes EDK2’s emulated variable store to use coreboot’s SMM store.
EDK2 has a proper SMM store of its own, I’d like to look into making that coreboot-compatible or at least
just writing a separate coreboot-compatible store module.</li>
<li><strong>UPD 2019-10-24</strong> for external displays, DisplayPort alt mode on USB-C can be used. Things to note:
<ul>
<li>DP++ (DP cables literally becoming HDMI cables) can’t work over USB Type C,
which is why there are no <code>HDMI-A-n</code> connectors on the GPU, so a passive HDMI-mDP dongle plugged into a mDP-TypeC dongle won’t work;</li>
<li>the Chrome EC runs the alt mode negotiation, the OS doesn’t need any special support;</li>
<li>for DP dongles to work at all, the EC <strong>must run RW firmware</strong> and that doesn’t happen as-is
with upstream coreboot. There is a jump command on the EC console.
Also <a href="https://github.com/MrChromebox/coreboot/commit/3f6e950980e65980476808938f0395f187f36b3c">this patch</a>
should help?? (+ <a href="https://github.com/MrChromebox/coreboot/commit/23e81ecad1e20cd7bfbeb59e06804a5d5ba2f3bb">this</a>)</li>
</ul>
</li>
</ul>
<h3 id="an-aside-why-mess-with-firmware" tabindex="-1"><a class="header-anchor" href="#an-aside-why-mess-with-firmware"><span>An aside: why mess with firmware?</span></a></h3>
<p>If you’re not the kind of person who’s made happy by just the fact that some more code during the boot process
of their laptop is now open and verified, and you just want things to work, you might not be as excited
about open source firmware development as I am.</p>
<p>But you can do cool things with firmware that give you <strong>practical</strong> benefit.
The best example I’m seeing is better Hackintosh support.
Instead of patching macOS to work on your machine, you could patch your machine to pretend to almost be a Mac:</p>
<ul>
<li><a href="https://mjg59.dreamwidth.org/52149.html">Creating hardware where no hardware exists</a> (using SMM
to emulate Apple’s System Management Controller in coreboot);</li>
<li><a href="https://github.com/acidanthera/AppleSupportPkg">AppleSupportPkg</a> (making EDK2 more compatible with Apple things).</li>
</ul>
<p>Is this cool or what?</p>
<h2 id="conclusion" tabindex="-1"><a class="header-anchor" href="#conclusion"><span>Conclusion</span></a></h2>
<p>Pixelbook, FreeBSD, coreboot, EDK2 good.</p>
<p>Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)</p>Moving VMs from VirtualBox to Client Hyper-V2017-07-20T00:00:00Zhttps://val.packett.cool/blog/vbox-to-hyper-v/The year of Windows on the desktop? Taking Microsoft's hypervisor for a spin by setting up a modern FreeBSD VM.<p>I’ve decided to move the VMs on my desktop from VirtualBox to Microsoft Hyper-V.
Because reasons.</p>
<p>Actually because I’ve upgraded my desktop to an AMD Ryzen CPU: first, AMD-V/SVM is not supported by the <em>Intel</em> HAXM thing from the Android SDK, so I wanted to try out Microsoft’s Hyper-V based Android “emulator” (VM configurator/runner thingy) instead.
Second, giving 16 virtual CPUs on an SMT 8-core to a FreeBSD guest in VirtualBox results in a weird performance issue.
(Though giving 4 vCPUs to <em>multiple</em> VMs on a 4-core CPU worked fine.)
Third, it’s <em>Oracle</em> VM VirtualBox and no one likes Oracle.</p>
<figure class="lqip" style="--ratio:1981/1361;background:rgb(230,232,233) url(data:image/webp;base64,UklGRmIAAABXRUJQVlA4IFYAAABQBQCdASowACAAP+Hq6Gy/vLAptVv4A/A8CWUAAFvoq1QAz3i1Z8XiXfRZhGz5R7FJeYAA/lHT2bCUVBNGZjAIYCm9raJANh9WUjtiCqakB89wXAwAAA==)">
<picture>
<img alt="Screenshot" src="https://val.packett.cool/scr.png" width="1981" height="1361"
class="u-photo" loading="lazy" decoding="async">
</picture>
</figure>
<p>So, here’s how you can do it as well.</p>
<h2 id="how-to-get-hyper-v" tabindex="-1"><a class="header-anchor" href="#how-to-get-hyper-v"><span>How to get Hyper-V</span></a></h2>
<p>You need Windows 10 Pro, Enterprise or Education.
(Or Windows Server, obviously.)
Just enable it as a feature and restart.</p>
<p>Alternatively, installing the <a href="https://www.visualstudio.com/vs/msft-android-emulator/">MS Android “emulator”</a> automatically enables it.</p>
<h2 id="how-to-migrate-a-vm-freebsd-linux-or-windows-with-efi" tabindex="-1"><a class="header-anchor" href="#how-to-migrate-a-vm-freebsd-linux-or-windows-with-efi"><span>How to migrate a VM (FreeBSD, Linux or Windows with EFI)</span></a></h2>
<p>(NOTE: older versions of FreeBSD apparently had some loader issue that prevented EFI boot in Hyper-V. Everything works for me on a recent build of 11-STABLE.)</p>
<p>In VirtualBox, go to the Virtual Media Manager (Ctrl+D) and copy your disk as <code>VHD</code>.
In the Hyper-V Manager, use the Edit Disk dialog to convert the <code>VHD</code> to <code>VHDX</code>.</p>
<p>If you haven’t done that yet, go to the Virtual Switch Manager and make a virtual switch (“External” is like bridge mode in VBox).</p>
<p>Now make a virtual machine.
Generation 2, no dynamic memory (FreeBSD doesn’t support that), select the virtual switch and the VHDX disk.</p>
<p>Click Connect and it should just work.</p>
<p>By the way, it’s nice that you can always close the console window without powering off the VM, unlike in VirtualBox where you need a special “Detachable start”.</p>
<p>Interestingly, if you create the VM without a disk and attach the disk later, you won’t see “boot from hard drive” in the firmware / boot order settings.
And there’s no add button! (WTF?)
The fix is to use PowerShell:</p>
<pre><code class="language-powershell">$vm = Get-VM "YOUR VM NAME"
Set-VMFirmware $vm -FirstBootDevice (Get-VMHardDiskDrive $vm)
</code></pre>
<p>Speaking of which, it’s nice to have a directly integrated PowerShell interface to all the things.
My little <a href="https://github.com/myfreeweb/xvmmgr">xvmmgr</a> script was initially written for VirtualBox, and that required <em>COM</em>.</p>
<h2 id="how-to-migrate-a-vm-other-os" tabindex="-1"><a class="header-anchor" href="#how-to-migrate-a-vm-other-os"><span>How to migrate a VM (other OS)</span></a></h2>
<p>Well, a similar process, but use Generation 1.</p>
<h2 id="my-experience-so-far" tabindex="-1"><a class="header-anchor" href="#my-experience-so-far"><span>My experience so far</span></a></h2>
<p>Client Hyper-V has pleasantly surprised me.
It’s a very smooth experience: it <em>looks</em> like a Type 2 hypervisor even though it’s actually Type 1, it runs VMs without any performance issues… what else could you ask for?</p>
<p>Well, the downside is its lack of flexibility in terms of paravirtualized (MS calls them “synthetic” or something) vs emulated devices.</p>
<p>All you get is the choice between two generations.
Generation 1 means legacy BIOS boot from an emulated <em>IDE</em> drive with emulated all the things plus optionally some paravirtualized devices like the NIC.
Generation 2 means EFI boot from a SCSI drive with paravirtualized <em>everything</em>.
Oh and the SCSI controller is also on the <code>vmbus</code>.
So there’s no way to use EFI and SCSI with e.g. OpenBSD, you need full Hyper-V support for at least the disk and network to do that.
Thankfully Microsoft contributed that support to FreeBSD! :)</p>FreeBSD on the ThinkPad X2402016-01-03T00:00:00Zhttps://val.packett.cool/blog/thinkpad-x240/The beginning of my FreeBSD laptop adventure. 2016 was an awkward time for this but I have contributed to fixing that.<p>So, I bought a laptop to run FreeBSD.</p>
<p>I was going to get a <a href="http://blog.grem.de/pages/c720.html">C720 Chromebook</a>, but I got a good deal for an X240. Yeah, yeah, a laptop from the preinstalled-insecure-adware company, whatever. Anyway, it’s a ThinkPad, so it feels very solid, has an excellent keyboard and good free software support.</p>
<p>So, let’s get FreeBSD running!</p>
<h2 id="installation" tabindex="-1"><a class="header-anchor" href="#installation"><span>Installation</span></a></h2>
<p>I’ve replaced the stock HDD with an SSD, compiled the <a href="https://github.com/freebsd/freebsd-base-graphics/tree/drm-i915-update-38">drm-i915-update-38</a> branch of FreeBSD on a different machine, wrote the memstick image to an old USB flash drive, booted it and installed FreeBSD on the ThinkPad.</p>
<p><ins><strong>UPDATE</strong>: that landed in head a long time ago, I think you can just pick up the latest release now.</ins></p>
<p>The first installation, with ZFS root + UFS <code>/boot</code>, did not work because the EFI loader couldn’t load <code>zfs.ko</code>. After reinstalling on UFS, the loader does load <code>zfs.ko</code>… Oh well.</p>
<p><ins><strong>UPDATE</strong>: now ZFS root works out of the box.</ins></p>
<p>GRUB 2 is also an option (and <em>the</em> option for using <a href="https://www.freshports.org/sysutils/beadm">sysutils/beadm</a>), but <a href="https://unix.stackexchange.com/questions/250028/grub2-security-vulnerability-pressing-backspace-28-times-what-are-my-risks-wh/250079">the recent “backspace 28 times to bypass boot passphrase” vulnerability</a> really discouraged me from installing it. Of course, <a href="https://twitter.com/rootkovska/status/677780387804901377">what are you even trying to protect with that passphrase</a>, but ugh, GNU code “quality”.</p>
<h2 id="power-management" tabindex="-1"><a class="header-anchor" href="#power-management"><span>Power management</span></a></h2>
<p>The usual laptop settings for <code>/etc/rc.conf</code>:</p>
<pre><code>powerd_enable="YES"
powerd_flags="-a hiadaptive -b adaptive -i 75 -r 85 -p 500"
performance_cx_lowest="Cmax"
economy_cx_lowest="Cmax"
</code></pre>
<p><ins><strong>UPDATE</strong>: <a href="https://github.com/lonkamikaze/powerdxx">powerd++</a> is a better powerd!</ins></p>
<p>And for <code>/boot/loader.conf</code>:</p>
<pre><code>hw.pci.do_power_nodriver=3
drm.i915.enable_rc6=7
hw.snd.latency=7
hint.pcm.0.buffersize=65536
hint.pcm.1.buffersize=65536
hint.pcm.2.buffersize=65536
hw.snd.feeder_buffersize=65536
</code></pre>
<p>Battery life with the internal + big external battery: ~8 - 8.5 hours of mostly surfing the web with Firefox on Wi-Fi with 50% screen brightness. (Obviously, more hours without Firefox :D) I don’t know how some reviewers got 20 hours of Wi-Fi browsing on Windows. Linux users say it’s <a href="https://www.reddit.com/r/Ubuntu/comments/2oheiu/linux_os_suggestion_for_lenovo_x240/cmn7n0o">6-7 hours</a> or <a href="http://www.splitbrain.org/blog/2014-03/08-lenovo_thinkpad_x240_xubuntu">above 8 hours</a>, so FreeBSD is not worse than Linux there. That’s good :-)</p>
<p>I couldn’t get suspend/resume to work. It does suspend but doesn’t resume (pressing the power button makes the fans spin, but the power button is still blinking).</p>
<p>But putting the X240 into sleep mode for short breaks is not really necessary. With the huge battery and the ultra-low-power processor, just leaving it running for 15-30 minutes won’t drain the battery much.</p>
<p>Oh, and the power consumption can be measured with Intel’s performance counters. Install <a href="https://freshports.org/sysutils/intel-pcm/">sysutils/intel-pcm</a> and run:</p>
<pre><code>$ sudo kldload cpuctl
$ sudo pcm.x
</code></pre>
<p>Power consumption of the CPU (and GPU, and everything else on the chip) when idle and running Xorg is around 3 Watt.</p>
<h2 id="ethernet-and-wi-fi" tabindex="-1"><a class="header-anchor" href="#ethernet-and-wi-fi"><span>Ethernet and Wi-Fi</span></a></h2>
<p>Works. This laptop has Intel’s networking hardware, which is great news for free operating systems. Not that I like Intel (super evil Management Engine!!) but they do write open source drivers for Linux, and BSD developers port them to the BSDs.</p>
<p>The Intel PRO/1000 Ethernet card is supported by the <code>em</code> driver.</p>
<p>The Intel 7260 wireless card is supported by the <code>iwm</code> driver.</p>
<p>Only <code>802.11a/b/g</code> is supported in <code>iwm</code> for now (IIRC because the driver is imported from OpenBSD, and they’re still working on <code>802.11n</code> support).</p>
<h2 id="bluetooth" tabindex="-1"><a class="header-anchor" href="#bluetooth"><span>Bluetooth</span></a></h2>
<p>Doesn’t work.</p>
<p>Apparently, it’s <a href="http://www.ubuntu.com/certification/catalog/component/usb/4103/8087%3A07dc/">this one</a>.</p>
<p>It’s not even connecting as a USB device:</p>
<pre><code>usbd_req_re_enumerate: addr=1, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 1 failed, USB_ERR_TIMEOUT
ugen0.2: <Unknown> at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
</code></pre>
<p>I never use Bluetooth on laptops, anyway.</p>
<h2 id="graphics-intel-hd-graphics-on-haswell" tabindex="-1"><a class="header-anchor" href="#graphics-intel-hd-graphics-on-haswell"><span>Graphics (Intel HD Graphics on Haswell!)</span></a></h2>
<p>Works. <s>Well, there’s a reason I’m using the <code>drm-i915-update-38</code> branch ;-) This is not in a release yet — it’s not even in <code>-CURRENT</code>! — so I’m not expecting perfect quality.</s></p>
<p><ins><strong>UPDATE</strong>: this was merged a long time ago. There’s a new <code>drm-next</code> in <a href="https://github.com/FreeBSDDesktop/freebsd-base-graphics">the graphics team’s fork</a> though, and it brings Skylake support, Wayland…</ins></p>
<p>But it works fine with correct settings.</p>
<p>Do not load <code>i915kms</code> in the boot loader!! The system won’t boot. Instead, use the <code>kld_list</code> setting in <code>/etc/rc.conf</code> to load the module later in the boot process.</p>
<p>When you load <code>i915kms</code>, it will repeat this error for less than a second:</p>
<pre><code>error: [drm:pid51453:intel_sbi_read] *ERROR* timeout waiting for SBI to complete read transaction
error: [drm:pid51453:intel_sbi_write] *ERROR* timeout waiting for SBI to complete write transaction
</code></pre>
<p>That’s okay, it works anyway. Looks like <a href="https://github.com/freebsd/freebsd/commit/4ef184b756b083683d4bac92ab02330aa08c4427">this is not even Haswell specific</a>.</p>
<p>So, here’s the <code>xorg.conf</code> part:</p>
<pre><code>Section "Device"
Option "AccelMethod" "sna"
Option "TripleBuffer" "true"
Option "HotPlug" "true"
Option "TearFree" "false"
Identifier "Card0"
Driver "intel"
BusID "PCI:0:2:0"
EndSection
</code></pre>
<p><ins><strong>UPDATE</strong>: with <code>drm-next</code>, the <code>modesetting</code> driver with <code>glamor</code> acceleration works!</ins></p>
<p>Brightness adjustment works via <s>both</s> <a href="https://www.freshports.org/graphics/intel-backlight/">graphics/intel-backlight</a> <s>and <code>acpi_video</code> (<code>sysctl hw.acpi.video.lcd0.brightness</code>)</s>. <s>The brightness keys on the keyboard don’t work properly though. The fn key on F5 (lower brightness) just sets the brightness to maximum, F6 (raise brightness) does nothing. Here’s the error that’s shown when pressing the lower brightness key with <code>drm.debug=3</code> in <code>/boot/loader.conf</code>:</s></p>
<pre><code>[drm:KMS:pid12:intel_panel_get_max_backlight] max backlight PWM = 852
[drm:KMS:pid12:intel_panel_actually_set_backlight] set backlight PWM = 841
[drm:pid12:intel_opregion_gse_intr] PWM freq is not supported
</code></pre>
<p><s>So I’ve configured F5 and F6 (the real function keys, FnLock mode) to call <code>intel_backlight</code>.</s></p>
<p><ins><strong>UPDATE</strong>: <code>acpi_video</code> is the one incorrectly changing the brightness to max! Don’t load it. <code>acpi_ibm</code> changes the brightness correctly!</ins></p>
<p>HDMI output works with a Mini DisplayPort adapter. 1080p video playback on an HDMI TV using <code>mpv</code> is smooth.</p>
<p>VAAPI video output and hardware accelerated decoding works. With <code>mpv --vo=vaapi --hwdec=vaapi</code>, CPU usage is around 20% for a 1080p H.264 video (vs. 60% with software decoding), the fans stay silent. You’ll need to install <a href="https://www.freshports.org/multimedia/libva-intel-driver/">multimedia/libva-intel-driver</a> and <a href="https://www.freshports.org/multimedia/mpv">multimedia/mpv</a> from pkg, and rebuild <a href="https://www.freshports.org/multimedia/ffmpeg">multimedia/ffmpeg</a> with the VAAPI option.</p>
<p><s>OpenCL on the Haswell GPU (powered by Beignet) doesn’t work yet. <code>clinfo</code> shows:</s></p>
<pre><code>Beignet: self-test failed: (3, 7, 5) + (5, 7, 3) returned (3, 7, 5)
</code></pre>
<p><ins><strong>UPDATE</strong>: OpenCL was fixed a long time ago.</ins></p>
<h2 id="audio" tabindex="-1"><a class="header-anchor" href="#audio"><span>Audio</span></a></h2>
<p>Works. The built-in Realtek ALC292 sound card just works. FreeBSD’s audio support is good.</p>
<p>The internal microphone is recognized as a separate device:</p>
<pre><code>$ cat /dev/sndstat
Installed devices:
pcm0: <Intel Haswell (HDMI/DP 8ch)> (play)
pcm1: <Realtek ALC292 (Analog 2.0+HP/2.0)> (play/rec) default
pcm2: <Realtek ALC292 (Internal Analog Mic)> (rec)
</code></pre>
<p>HDMI audio works too (<code>sysctl hw.snd.default_unit</code> to switch the sound card; applications that play sound have to be restarted.)</p>
<h2 id="webcam" tabindex="-1"><a class="header-anchor" href="#webcam"><span>Webcam</span></a></h2>
<p>Works. With <code>webcamd</code>, of course. But I don’t need it, so I’ve disabled it in the BIOS Setup.</p>
<h2 id="sd-card-reader" tabindex="-1"><a class="header-anchor" href="#sd-card-reader"><span>SD card reader</span></a></h2>
<p>Doesn’t work.</p>
<p><code>pciconf</code> detects it as:</p>
<pre><code>none2@pci0:2:0:0: class=0xff0000 card=0x221417aa chip=0x522710ec rev=0x01 hdr=0x00
vendor = 'Realtek Semiconductor Co., Ltd.'
device = 'RTS5227 PCI Express Card Reader'
</code></pre>
<p>It’s supported in OpenBSD with <a href="http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man4/rtsx.4?query=rtsx&sec=4">rtsx(4)</a>. FreeBSD bugs for this: <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=161719">161719</a>, <a href="https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204521">204521</a>.</p>
<p>It should be possible to use it with OpenBSD/NetBSD/Linux in a bhyve VM with PCI passthrough, like <a href="http://0xfeedface.org/2014/12/11/FreeBSD-Intel-wifi-via-bhyve.html">the Wi-Fi card before iwm was added</a>. That would also be more secure (that’s what <a href="https://www.qubes-os.org/">Qubes</a> does for all the hardware.) But I don’t need to use SD cards on this laptop.</p>
<h2 id="trackpad-and-trackpoint" tabindex="-1"><a class="header-anchor" href="#trackpad-and-trackpoint"><span>Trackpad and TrackPoint</span></a></h2>
<p><s>Oh, this is the most interesting part. Well, it works, sure. But there are at least three ways of using them, none of which is perfect.</s></p>
<p><ins><strong>UPDATE</strong>: <a href="https://github.com/freebsd/freebsd/commit/bc8448b7c73bbbcaf15b375eb9648e963bcc8334">evdev synaptics landed in current</a>!</ins></p>
<h3 id="moused" tabindex="-1"><a class="header-anchor" href="#moused"><span>moused</span></a></h3>
<p>FreeBSD includes <code>moused</code>, a little daemon that watches the mouse device you tell it to watch and forwards all events to a virtualized mouse, which is accessible to Xorg at <code>/dev/sysmouse</code> and also works on the text console. It has advanced support (like sensitivity settings) for Synaptics touchpads and ThinkPad TrackPoints (set <code>hw.psm.synaptics_support=1</code> and <code>hw.psm.trackpoint_support=1</code> in <code>/boot/loader.conf</code> to enable).</p>
<p>Sadly, it forwards all events to a virtualized <em>mouse</em>. Not a trackpad, just a mouse, so the experience is not as good.</p>
<p><code>xorg.conf</code>:</p>
<pre><code>Section "InputDevice"
Identifier "SysMouse"
Driver "mouse"
Option "Device" "/dev/sysmouse"
EndSection
</code></pre>
<h3 id="xf86-input-synaptics" tabindex="-1"><a class="header-anchor" href="#xf86-input-synaptics"><span>xf86-input-synaptics</span></a></h3>
<p>This is the Synaptics driver that provides a great trackpad experience. Inertial scrolling, horizontal scrolling, natural scrolling, perfectly smooth cursor movement… Everything is as good as in OS X on a MacBook.</p>
<p><s>But the TrackPoint doesn’t work. On the X240, it’s attached to the trackpad as a guest mouse. The trackpad forwards the TrackPoint’s PS/2 events with a special mark (IIRC, it’s W = 3).</s></p>
<p>And the Synaptics driver <a href="http://cgit.freedesktop.org/xorg/driver/xf86-input-synaptics/commit/?id=b19e3782a77c171ca20fc962f95923495fdb7978">stopped supporting guest devices in 2010</a>.</p>
<p><ins><strong>UPDATE</strong>: <a href="https://github.com/myfreeweb/xf86-input-synaptics">added the guest mouse support back to xf86-input-synaptics</a>! Also adds ClickPad support to the raw PS/2 protocol used by the driver on the BSDs (on Linux, it uses evdev, and they only added clickpad support there).</ins></p>
<p><s>Also, clicking doesn’t work, but I don’t care. I’m used to tapping.</s></p>
<p><code>xorg.conf</code>:</p>
<pre><code>Section "InputDevice"
Identifier "Touchpad"
Driver "synaptics"
Option "Protocol" "psm"
Option "Device" "/dev/psm0"
Option "VertEdgeScroll" "off"
Option "VertTwoFingerScroll" "on"
Option "HorizEdgeScroll" "off"
Option "HorizTwoFingerScroll" "on"
Option "VertScrollDelta" "-111"
Option "HorizScrollDelta" "-111"
Option "ClickPad" "on"
Option "SoftButtonAreas" "4201 0 0 1950 2710 4200 0 1950"
Option "AreaTopEdge" "5%"
EndSection
</code></pre>
<h3 id="evdevfbsd" tabindex="-1"><a class="header-anchor" href="#evdevfbsd"><span>evdevfbsd</span></a></h3>
<p>I accidentally found <a href="https://github.com/jiixyj/evdevfbsd">evdevfbsd</a>, a little program that exposes PS/2 devices as <code>evdev</code> devices via CUSE (<strong>C</strong>haracter Device in <strong>Use</strong>rspace).</p>
<p><code>evdev</code> is a protocol that comes from Linux. It allows kernel (or CUSE) drivers to provide a standardized interface for devices so that Xorg wouldn’t care about any particular vendor.</p>
<p><code>evdevfbsd</code> correctly separates the trackpad and the TrackPoint. Cursor movement works. But only cursor movement. No touch scrolling, no tapping, no clicking. And it looks like it might be <code>xf86-input-evdev</code>’s fault, because <a href="https://github.com/gvalkov/python-evdev/blob/master/bin/evtest.py">evtest.py</a> shows tap events when tapping!</p>
<h3 id="something-else" tabindex="-1"><a class="header-anchor" href="#something-else"><span><s>Something else?</s></span></a></h3>
<p><s>I’ve tried to write a CUSE program that works as a proxy between <code>/dev/psm0</code> and the Synaptics driver, extracting guest (TrackPoint) events in the process.</s></p>
<p><s>It almost works… the only problem is that the Synaptics driver locks the whole X server while reading from my proxy, so only mouse movement works and nothing else, not even Ctrl+Alt+F1 to switch to a console. Well, the power button works. And SSHing into the laptop.</s></p>
<p><s>Seems like the problem with CUSE is that there’s no way to find out, in the <code>poll</code> method, that the process that polls your device wants to stop. So, when <code>moused</code> reads from the proxy, it works, but when you stop <code>moused</code> with Ctrl-C, it doesn’t stop until you touch the TrackPoint or the trackpad a little to send an event.</s></p>
<p><ins><strong>UPDATE</strong>: <a href="https://github.com/myfreeweb/xf86-input-synaptics">added the guest mouse support back to xf86-input-synaptics</a>!</ins></p>
<h2 id="touchscreen" tabindex="-1"><a class="header-anchor" href="#touchscreen"><span>Touchscreen</span></a></h2>
<p>Works. It’s recognized as a USB HID device at <code>/dev/uhid0</code>. There are two ways to use it in Xorg.</p>
<h3 id="mouse-emulation" tabindex="-1"><a class="header-anchor" href="#mouse-emulation"><span>Mouse emulation</span></a></h3>
<p>The simple way: you can use it with the <code>mouse</code> driver, as a regular mouse. Obviously, this does not provide multi-touch.</p>
<p><code>xorg.conf</code>:</p>
<pre><code>Section "InputDevice"
Identifier "Touchscreen"
Driver "mouse"
Option "Protocol" "usb"
Option "Device" "/dev/uhid0"
EndSection
</code></pre>
<h3 id="multi-touch" tabindex="-1"><a class="header-anchor" href="#multi-touch"><span>Multi-touch</span></a></h3>
<p>The other way: you can use it with <code>webcamd</code> and the <code>evdev</code> driver. This will actually support multi-touch.</p>
<p>Recompile <a href="https://freshports.org/x11-drivers/xf86-input-evdev">x11-drivers/xf86-input-evdev</a> from ports with the <code>MULTITOUCH</code> option, start <code>webcamd</code> like this (note that CUSE is part of the base system on <code>11-CURRENT</code>, so it’s not called <code>cuse4bsd</code> anymore):</p>
<pre><code>$ sudo make -C /usr/ports/x11-drivers/xf86-input-evdev config deinstall install clean
$ sudo kldload cuse
$ sudo webcamd -d ugen1.3 -N Touchscreen-ELAN -M 0
</code></pre>
<p><code>xorg.conf</code>:</p>
<pre><code>Section "InputDevice"
Identifier "Touchscreen"
Driver "evdev"
Option "Device" "/dev/input/event0"
EndSection
</code></pre>
<p>Start <code>chrome --touch-events</code> and visit the <a href="http://www.snappymaria.com/canvas/TouchEventTest.html">touch event test</a>! Also, you can scroll in GTK+ 3 applications like <a href="http://corebird.baedert.org/">Corebird</a>.</p>
<p>Unfortunately, it’s really bad at detecting when a touch <em>ends</em>. This means that scrolling and tapping <a href="https://www.youtube.com/watch?v=nMR1o5cc2nc">will get stuck</a>. So I’m using the <code>mouse</code> driver for now.</p>
<p><strong>UPDATE:</strong> the new <code>wmt(4)</code> kernel driver supports the touchscreen perfectly, without that issue!! Also, <code>libinput</code> is better than <code>evdev</code> in Xorg.</p>
<h2 id="tpm-trusted-platform-module" tabindex="-1"><a class="header-anchor" href="#tpm-trusted-platform-module"><span>TPM (Trusted Platform Module)</span></a></h2>
<p>Works. (With the dedicated TPM 1.2 module. Haven’t tried Intel’s built-in TPM 2.0 support. The choice between them is in the BIOS/UEFI settings.)</p>
<p>OpenSSH works with a TPM key through <a href="https://github.com/ThomasHabets/simple-tpm-pk11">simple-tpm-pk11</a>.</p>
<p><strong>UPDATE:</strong> turns out the TPM was preventing the laptop from waking up from suspend! (And I did unload the tpm module before suspend.) Disabled it in firmware settings.</p>
<h2 id="conclusion" tabindex="-1"><a class="header-anchor" href="#conclusion"><span>Conclusion</span></a></h2>
<p>It’s possible to use a Haswell ThinkPad with FreeBSD right now :-) Everything except Bluetooth, SD cards and <s>waking up from sleep</s> works.</p>
<p>OpenBSD would be better though. They have excellent ThinkPad support, because OpenBSD developers use OpenBSD on ThinkPads. But I’m working on software that uses FreeBSD jails, and I just prefer FreeBSD.</p>