Untitled Blog.

“Pseudoterminals”, usually known as just PTYs, are important building blocks in UNIX and UNIX-like operating systems. The story of how PTYs became what they are today is interesting, but well-covered on the Internet by multiple others. The gist of it is that PTYs act as virtual stand-in devices for true TTYs.

If you wanted to write a terminal emulator on Linux (or any other OS providing the so-called UNIX 98 PTY interfaces), you would start by opening the /dev/ptmx device; the opened file descriptor will act as the master, and a corresponding slave device is created (on Linux, this is in /dev/pts). This slave device is what the applications running under the terminal have connected as their standard input, output and error devices. (You should be able to observe this on Linux using ls -lah /proc/self/fd.)

Once you‘ve successfully opened /dev/ptmx, you must call grantpt() and unlockpt() in order to set up the slave PTY. What do they do? Here is what grantpt() does according to the GNU C Library documentation:

The grantpt() function changes the ownership and access permission of the slave pseudo-terminal device corresponding to the master pseudo-terminal device associated with the file descriptor filedes. The owner is set from the real user ID of the calling process, and the group is set to a special group (typically tty) or from the real group ID of the calling process. The access permission is set such that the file is both readable and writable by the owner and only writable by the group.

On some systems this function is implemented by invoking a special setuid root program. As a consequence, installing a signal handler for the SIGCHLD signal may interfere with a call to grantpt().

This seems pretty straight-forward, but it isn‘t necessarily true. What exactly do I mean? Well, let‘s check the musl implementation of grantpt() to make it clearer:

int grantpt(int fd)
{
	return 0;
}

Yes, that‘s right. It does nothing. On Linux, the kernel takes care of ensuring the file permissions are set properly right off the bat.

The story in glibc is a little more complicated, for two reasons:

1. It supports the Hurd, which indeed does require a setuid helper known as pt_chown.

2. It tries to return EINVAL if you pass a file descriptor that isn‘t a PTY master device. Presumably this isn‘t actually required by the POSIX specification, which only uses the word “may” for this behavior, but nonetheless seems like a good way to catch bugs.

What about unlockpt()? Surprisingly, that still seems to be necessary, as Linux does still support locking PTYs, and PTYs do start locked still.

That is surprising mainly because neither grantpt() nor unlockpt() really make that much sense for Linux anymore; it made sense in the past because PTY devices used to be re-used, so locking a PTY slave helped close the gap where a malicious process could try to open the slave before proper permissions were applied by grantpt(). With devpts on modern Linux, PTY devices are always created on demand and not persisted, so this mechanism is not very useful.

So there you have it. What does grantpt() do on Linux? Nothing. Nothing at all.