A completely-from-scratch hobby operating system: bootloader, kernel, drivers, C library, and userspace including a composited graphical UI, dynamic linker, syntax-highlighting text editor, network stack, etc.
ToaruOS has consumed the greater part of my life for the last eleven years. This release has been a long time coming: My first plans for a 64-bit, SMP-capable port of the OS date back to before the 1.0 release. The repository in which I built the new kernel was something I had already set up as far back as 2015. And now, finally, I think it is ready.
What is ToaruOS?
ToaruOS is a complete hobby operating system for x86-64 PCs. ToaruOS is made up of several components:
Misaka, an SMP-enabled kernel for x86-64. Misaka is the main focus of this release, as it replaces the older "toaru32" kernel from 1.x.
Yutani, a software-compositing windowing system. Yutani has been a core part of ToaruOS since very early in its development, and this release brings a number of improvements to the windowing protocol and compositor implementation.
Kuroko, a bytecode-compiled interpreted programming language. Kuroko first appeared in ToaruOS 1.14 as a replacement for Python, and has improved greatly since it made its debut.
Bim, a Vim-inspired code editor with syntax highlighting and plugins powered by Kuroko. All code written for ToaruOS in the last few years has been written in Bim.
Graphical applications for Yutani, including a file browser, terminal emulator, widget-powered panel, and a package manager.
A vast collection of terminal utilities, including a large subset of standard POSIX utilities.
The ToaruOS libc, a cleanroom implementation of the C standard library. ToaruOS's libc has proven itself by supporting a considerable set of complex ports.
A full suite of userspace libraries to provide support for PNG and JPEG images, TrueType fonts, and more.
A dynamic linker (ld.so).
A set of bootloaders for BIOS and EFI systems.
All software components of ToaruOS are original to the project.
What's New in ToaruOS 2.0?
There are two ways to look at 2.0: First, as a follow up to 1.14.1. Compared to the last "stable" release, the changes in 2.0 are extensive but iterative. Alternatively, 2.0 can be compared to 1.0. That release was built on a number of third-party elements, and 2.0 demonstrates how those elements have been successfully replaced.
Features of Misaka
x86-64 architecture support and a far more portable base than its predecessor.
Support for symmetric multiprocessing (SMP).
VFS and process support for filesystem groups and supplementary group lists.
Per-thread and per-core CPU usage tracking.
Support for gzip-compressed ramdisks.
Loadable driver modules, which are now loaded from the ramdisk.
Process tracing with ptrace, providing visibility into system calls and signal delivery, single-step debugging, and cross-process memory access.
A newly-rewritten network stack providing a BSD-style sockets API, support for multiple interfaces, a loopback interface, and listening sockets.
Changes Since 1.14.1
New userspace utilities exposing functionality added in Misaka: top, strace, dbg, ping, cpuwidget.
Several improvements have been made to the graphics library, including the addition of affine transformations.
Several improvements have been made to the window compositing framework.
A new TrueType text rasterizer has been added.
A new marked-up text formatting library has been added.
The BIOS loader has been improved and should support a wider range of hardware configurations (including hard disk boot), the EFI loader has been rewritten, and both now support editing the kernel command line.
The panel has been redesigned, and widgets have been improved with loaded libraries, dynamic layout, and new popups.
The Julia set viewer has been rewritten with new palettes and exploration features.
A new calculator app has been added, powered by Kuroko.
The libc now supports basic timezone configuration. Please set your system RTC to UTC for accurate time.
The timezone offset and panel weather widget are automatically configured by location services, when available.
Driver support has been added for the Ensoniq ES1371 chipset emulated by VMware Workstation.
Many more things I lost track of.
Known Issues
The new network stack remains experimental and the TCP implementation is missing functionality I hope to deliver in a future update.
As in previous releases, ToaruOS's libc remains incomplete. Some functions may be provided as stubs, or may not accept all expected options.
There are known issues with the ext2 implementation and, while possible, disk installations are not recommended.
Please see the attached discussion for additional issues found in this release.
What's Next?
Several features were deferred to future releases in order to avoid scope creep and get 2.0 finalized. My roadmap for the future looks something like this:
2.0.x: Various bug fixes and functionality that was almost ready for 2.0, such as TCP stack improvements.
2.1: AHCI, xHCI, USB HID devices, and other improved hardware support that was specifically deferred.
2.2: AArch64 port.
Running ToaruOS
About the Live CD
ToaruOS is distributed as a "live CD". The default configuration will boot into a user session for local. The password for this account is also local, and it is permitted to use the sudo command. When prompted for a password by either the graphical or command-line sudo utilities, enter local. The live CD is configured to provide an in-memory read-write filesystem and packages can be installed up to the limits of system memory, but no data will persist between boots. As in previous releases of ToaruOS, an experimental read-write ext2 implementation is available (insmod /mod/ext2.ko) and drivers for IDE hard disks are provided (insmod /mod/ata.ko). While these drivers have seen significant improvements since previous releases, their use on real systems is not recommended.
Virtual Machines
I generally recommend using a virtual machine such as QEMU, VirtualBox or VMware Workstation to run ToaruOS, as driver support for real hardware configurations is limited.
You may also wish to use multiple virtualized CPUs with -smp 2. For platforms other than Linux, substitute an appropriate command line flag for -enable-kvm to enable hardware virtualization.
For VirtualBox, be sure to provide 32MB of display memory and configure an Intel NIC with NAT, and AC97 audio for best support.
ToaruOS 2.0 has also been tested in VMware Player.
Real Hardware
Users wishing to try ToaruOS 2.0 on real hardware are recommended to use GRUB. Booting ToaruOS 2.0 is notably simplified over 1.x, as modules no longer need to be loaded by the bootloader to achieve a functioning system. Extract the kernel and ramdisk files from the release CD and place them in a path accessible from your GRUB loader, then add a menu entry with the following commands:
multiboot2 /kernel root=/dev/ram0 vid=auto migrate
module2 /ramdisk.igz
set gfxpayload=keep # You may need to specify an appropriate mode here.
Credits and Licenses
ToaruOS itself is made available under the following terms, commonly known as the NCSA / University of Illinois License:
Copyright (c) 2011-2021 K Lange, et al. (hereafter [fullname]). All rights reserved.
Developed by: ToaruOS (hereafter [project])
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation files
(the "Software"), to deal with the Software without restriction,
including without limitation the rights to use, copy, modify, merge,
publish, distribute, sublicense, and/or sell copies of the Software,
and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimers.
* Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimers in the
documentation and/or other materials provided with the distribution.
* Neither the names of [fullname], [project] nor the names of its
contributors may be used to endorse or promote products derived from
this Software without specific prior written permission.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
CONTRIBUTORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS WITH
THE SOFTWARE.
ToaruOS integrates components separately licensed under the ISC License (apps/bim.c and related files):
Copyright (C) 2012-2021 K. Lange
Permission to use, copy, modify, and/or distribute this software for any
purpose with or without fee is hereby granted, provided that the above
copyright notice and this permission notice appear in all copies.
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
Kuroko, ToaruOS's bytecode-compiled interpreted language, is based on prior work by Robert Nystrom and is distributed under the MIT License:
Copyright (c) 2020-2021 K. Lange <klange@toaruos.org>
Copyright (c) 2015 Robert Nystrom
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
ToaruOS is built with GCC and its runtime support library (libgcc) is provided on the CD as a dynamic shared object. This file carries a GPLv3 license with a linking exception, which you may find here alongside the requisite sources: https://github.com/toaruos/gcc/
Finally, ToaruOS includes the DejaVu family of TrueType fonts, license details for which may be found here: https://dejavu-fonts.github.io/License.html (embedded license text for fonts can be viewed in the OS with the font-tool utility using the -s flag)
All other assets, including icons, wallpapers, mouse cursors, and so on, are original works by the ToaruOS team and released under the same NCSA license as the rest of the OS. ToaruOS design assets may be made available under alternative Creative Commons terms by request.
Changelog
about: accept resize messages
ahci: more testing
ahci: wip module
assert: use dprintf, not printf, when printing assertion message
ata: deal in pages, not sectors
ata: Don't make device nodes for disks that claim to be empty
ata: drop the disk scheduler task and just use a proper sleepable mutex
ata: Ensure we lock around submitting ATAPI command so we're woken up correctly
ata: fix badly allocated prdt
ata: fix bad timeout on ATAPI initialization, fixes vbox seeing the CD
ata: identify disk before checking its size...
ata: New disk scheduler; less terrible but very slow
ata: remove partition info which belongs to dospart module
ata: Rudimentary recovery of the legacy ATA driver
ata: slap a warning at the top of that
base: Accept QEMU IDE controller for ata driver autoload
base: Add 'guest' user with no groups, not in sudoers
base: add removed shell scripts from /bin
base: Add /src/boot and /src/modules
base: Cleaned up volume icons
base: Cleanup local's rc files
base: define some byte order macros
base: Don't wait for msk on boot
base: Ensure we're still providing the fake libm.so
base: Install auto-dep.krk to /bin instead of /usr/bin
base: Land new assets for 2.0
base: Load ATA drivers on vbox if IDE is detected
base: Load iso9660 and add a desktop icon when mounting CD at boot
base: Load the piix4 remapper if we see the ISA bridge; fixes OVFM under qemu without -M q35
base: make sure dir exists to build host Kuroko
base: minor cleanup of /usr/share
base: Only show tutorial once on first graphical login for 'local'
base: Prepare for 2.0.0
base: Print message when loading drivers on startup
base: Remove old qemu thing from startup
base: remove unsupported startup script
base: Show some toasts on startup.
base: Slightly less useless help files
base: update /etc/os-release
base: update package repository on startup
base: Update README.md
base: Wait a bit before starting the wizard
bim: fix accidental wraparound when scrolling to '-1'
bim: fix argument parsing derp
bim: Report errors when opening files
bim: sync with upstream
bim: sync with upstream
bim: syntax highlighter for /etc/issue files
bim: Update gas highlighter
bim: upstream bug fix
bim: Use ansi theme on vga terminal
bim: version 3.0.0
block-dev-stats: Test tool for ongoing ata rewrite
boot: Add 1280x800 as a preferred resolution
boot: Add a shortcut to toggle video/text mode
boot: Add a status indicator when loading
boot: add command line editor
boot: add menu timeout
boot: adjust timeout a bit
boot: alternate help message when highlighting a toggle
boot: check for long mode support so we can complain if someone runs the wrong qemu
boot: Clean up editor redraw, shift-arrow input?
boot: Cleanup strings; spot-check memory on startup
boot: Don't clear lines before printing on them when drawing banners
boot: enable SMP by default
boot: enable vmware resize by default? it seems stable
boot: Fall back to 24bpp modes
boot: first pass at cleanup, just make sure everything has lead comments
boot: fixup returning from editor to menu
boot: generalize video mode selection to work on BIOS
boot: Jump back to real mode to load sectors?
boot: less debug text
boot: Make sure menu displays if an unrecognized key is pressed
boot: make this bootload v4.0?
boot: MBR / disk boot stub
boot: merge retooled EFI loader
boot: more editor key binds
boot: Prettify
boot: recommend 1GiB of RAM
boot: Recover BIOS loader.
boot: retain last mode when toggle to text mode
boot: Rewrite README
boot: Rewrite the legacy BIOS loader
boot: Select current mode on video menu; esc to go back without changing
boot: Set a kcmdline flag when debug is set in menu
boot: Support AOUT_KLUDGE kernels
boot: Use CD for QEMU
boot: VBE modesetting
build: Remove C++ support, stop including libstdcxx on the ramdisk
calculator: Don't let the window get too tiny
calculator: screwy, kuroko-backed calculator
calculator: set binpath so we can at least import 'math'
calculator: simple text input
chown: fixup semantics, support names in command
color-picker: New widget demo
compositor: Add a new OVERLAY layer, specifically for toasts
compositor: Add another layer above overlays for menus so they show in front of toasts
compositor: Add I-beam cursor; logic in terminal, file-browser
compositor: Animate bottom and top windows normally if they aren't marked ALT_ANIMATION
compositor: Avoid more transformations.
compositor: avoid transformations if we're just transparent
compositor: Close stale windows that were lost in the stack
compositor: correctly set next focused window when a window closes
compositor: don't animate regular dropdown menus
compositor: drop support for replacement renderers