openpty | Rust implementation of openpty

That's exactly the side effects of not putting the tty in raw mode. Usually a program (like expect) which handles pty would put the outer tty in raw mode.

• Your Python script's tty (or pty) echos what you input and the new pty echos for the 2nd time. You can disable ECHO on the new pty. For example:

Installing the JSON formatter (or other similar extension) should work to prettify it in VSCode itself.

This is a good question, and a good example of why we have pseudoterminals.

For the daemon to be able to use an ncurses interface, it requires a pseudoterminal (the slave side of a pseudoterminal pair), which is available from the point the daemon starts executing, continuously, until the daemon exits.

For a pseudoterminal to exist, there must be a process that has an open descriptor to the master side of the pseudoterminal pair. Additionally, it must consume all output from the pseudoterminal slave side (visible stuff output by ncurses). Usually, a library like vterm is used to interpret that output to "draw" the actual text framebuffer into an array (well, usually two arrays - one for the wide characters displayed in each cell (specific row and clumn), and another for the attributes like color).

For the pseudoterminal pair to work correctly, either the process at the master end is a parent or ancestor of the process running ncurses in the slave end, or the two are completely unrelated. The process running ncurses in the slave end should be in a new session, with the pseudoterminal as its controlling terminal. This is easiest to achieve, if we use a small pseudoterminal "server" that launches the daemon in a child process; and indeed, this is the pattern that is typically used with pseudoterminals.

The first scenario is not really feasible, because there is no parent/master process maintaining the pseudoterminal.

We can provide the behaviour of the first scenario, by adding a small pseudoterminal-providing "janitor" process, whose task is to maintain the pseudoterminal pair in existence, and to consume any ncurses output generated by the process running in the pseudoterminal pair.

However, that behavour also matches the second scenario.

Put another way, here is what would work:

1. Instead of launching the daemon directly, we use a custom program, say 'janitor', that creates a pseudoterminal and runs the daemon inside that pseudoterminal.

2. Janitor will stay running for as long as the daemon runs.

3. Janitor provides an interface for other processes to "connect" to the master side of the pseudoterminal pair.

   This does not necessarily mean 1:1 proxying of data. Usually input (keypresses) to the daemon are provided unmodified, but how the contents of the pseudoterminal "framebuffer", the character-based virtual window contents, are transferred does vary. This is completely under our own control.

4. To connect to the janitor, we'll need a second helper program.

   In the case of 'screen', these two programs are actually the same binary; the behaviour is just controlled by command-line parameters, and keypresses "consumed" by 'screen' itself, to control 'screen' behaviour and not passed to the actual ncurses-based process running in the pseudoterminal.

Thus far, we could just examine tmux or screen sources to see how they do the above; it is very straightforward terminal multiplexing stuff.

However, here we have a very interesting bit I had not considered before; this small bit made me understand the quite important core of this question:

Multiple users can have their own instances of the UI.

A process can only have one controlling terminal. This specifies a certain relationship. For example, when the master side of the controlling terminal is closed, the pseudoterminal pair vanishes, and the descriptors open to the slave side of the pseudoterminal pair become nonfunctional (all operations yield EIO, if I recall correctly); but more than that, every process in the process group receives a HUP signal.

The ncurses newterm() function lets a process connect to an existing terminal or pseudoterminal, at run time. That terminal does not need to be the controlling terminal, nor does the ncurses-using process need to belong to that session. It is important to realize that in this case, the standard streams (standard input, output, and error) are not redirected to the terminal.

So, if there is a way to tell a daemon that it has a new pseudoterminal available, and should open that because there is a user that wants to use the interface the daemon provides, we can have the daemon open and close the pseudoterminals on demand!

Note, however, that this requires explicit co-operation between the daemon, and the processes that are used to connect to the ncurses-based UI the daemon provides. There is no standard way of doing this with arbitrary ncurses-based processes or daemons. For example, as far as I know, nano and top provide no such interface; they only use the pseudoterminal associated with the standard streams.

After posting this answer – hopefully fast enough before the question is closed because others do not see the validity of the question, and its usefulness to other server-side POSIXy developers –, I shall construct an example program pair to exemplify the above; probably using an Unix domain socket as the "new UI for this user, please" communications channel, as file descriptors can be passed as ancillary data using Unix domain sockets, and identity of the user at either end of the socket can be verified (credentials ancillary data).

However, for now, let's go back to the questions asked.

What is wrong with the above pseudo code? [Typically I either get a segfault inside fileno_unlocked() in newterm() or output on the invoking terminal rather than a new invisible terminal.]

newinfd and newoutfd should be the same (or dup()s of) the pseudoterminal slave end file descriptor, slavefd.

I think there should also be an explicit set_term() with the SCREEN pointer returned by newterm() as a parameter. (It could be that it gets automatically called for the very first terminal provided by newterm(), but I'd rather call it explicitly.)

newterm() connects to and prepares a new terminal. The two descriptors usually both refer to the same slave side of a pseudoterminal pair; infd can be some other descriptor where the user keypresses are received from.

Only one terminal can be active in ncurses at a time. You need to use set_term() to select which one will be affected by following printw() etc. calls. (It returns the terminal that was previously active, so that one can do an update to another terminal and then return back to the original terminal.)

(This also means that if a program provides multiple terminals, it must cycle between them, checking for input, and update each terminal, at a relatively high frequency, so that human users feel the UI is responsive, and not "laggy". A crafty POSIX programmer can select or poll on the underlying descriptors, though, and only cycle through terminals that have input pending.)

Do I have the master and slave ends the right way around?

Yes, I do believe you do. Slave end is the one that sees a terminal, and can use ncurses. Master end is the one that provides keypresses, and does something with the ncurses output (say, draws them to a text-based framebuffer, or proxies to a remote terminal).

What does login_tty actually do here?

There are two commonly used pseudoterminal interfaces: UNIX98 (which is standardized in POSIX), and BSD.

With the POSIX interface, posix_openpt() creates a new pseudoterminal pair, and returns the descriptor to its master side. Closing this descriptor (the last open duplicate) destroys the pair. In the POSIX model, initially the slave side is "locked", and unopenable. unlockpt() removes this lock, allowing the slave side to be opened. grantpt() updates the character device (corresponding to the slave side of the pseudoterminal pair) ownership and mode to match the current real user. unlockpt() and grantpt() can be called in either order, but it makes sense to call grantpt() first; that way the slave side cannot be opened "accidentally" by other processes, before its ownership and access mode have been set properly. POSIX provides the path to the character device corresponding to the slave side of the pseudoterminal pair via ptsname(), but Linux provides an TIOCGPTPEER ioctl (in kernels 4.13 and later) that allows opening the slave end even if the character device node is not shown in the current mount namespace.

Typically, grantpt(), unlockpt(), and opening the slave side of the pseudoterminal pair are done in a child process (that still has access to the master-side descriptor) that has started a new session using setsid(). The child process redirects standard streams (standard input, output, and error) to the slave side of the pseudoterminal, closes its copy of the master-side descriptor, and makes sure the pseudoterminal is its controlling terminal. Usually this is followed by executing the binary that will use the pseudoterminal (usually via ncurses) for its user interface.

With the BSD interface, openpty() creates the pseudoterminal pair, providing open file descriptors to both sides, and optionally sets the pseudoterminal termios settings and window size. It roughly corresponds to POSIX posix_openpt() + grantpt() + unlockpt() + opening the slave side of the pseudoterminal pair + optionally setting the termios settings and terminal window size.

With the BSD interface, login_tty is run in the child process. It runs setsid() to create a new session, makes the slave side the controlling terminal, redirects standard streams to the slave side of the controlling terminal, and closes the copy of the master side descriptor.

With the BSD interface, forkpty() combines openpty(), fork(), and login_tty(). It returns twice; once in the parent (returning the PID of the child process), and once in the child (returning zero). The child is running in a new session, with the pseudoterminal slave side as its controlling terminal, already redirected to the standard streams.

Is there any practical difference between openpty() + login_tty() vs posix_openpt() + grantpt() [ + unlockpt() + opening the slave side]?

No, not really.

Both Linux and most BSDs tend to provide both. (In Linux, when using the BSD interface, you need to link in the libutil library (-lutil gcc option), but it is provided by the same package that provides the standard C library, and can be assumed to be always available.)

I tend to prefer the POSIX interface, even though it is lots more verbose, but other than kinda preferring POSIX interfaces over BSD ones, I don't even know why I prefer it over the BSD interface. The BSD forkpty() does basically everything for the most common use cases in one call!

Also, instead of relying on ptsname() (or the GNU ptsname_r() extension), I tend to first try the Linux-specific ioctl if it looks like it is available, and fall back to ptsname() if it is not available. So, if anything, I probably should prefer the BSD interface.. but the libutil kinda sorta annoys me a little bit, I guess, so I don't.

I definitely have no objection to others preferring the BSD interface. If anything, I'm a bit puzzled as to how my preference even exists; normally I prefer the simpler, more robust interfaces over the verbose, complex ones.

Does there have to be a running process associated with or slave master tty at all times?

There has to be a process having the master side of the pseudoterminal open. When the last duplicate of the descriptor is closed, the kernel destroys the pair.

Also, if the process having the master side descriptor does not read from it, the process running in the pseudoterminal will unexpectedly block in some ncurses call. Normally, the calls do not block (or only block for very short durations, shorter than what humans notice). If the process just reads but discards the input, then we do not actually know the contents of the ncurses terminal!

So, we can say that having a process that reads from the pseudoterminal pair master side, keeping a descriptor open to the master side, is absolutely required.

(The slave side is different; because the character device node is usually visible, a process can close its connection to the pseudoterminal temporarily, and just reopen it later. In Linux, when no process has an open descriptor to the slave side, the process reading from or writing to the master side will get EIO errors (read() and write() returning -1 with errno==EIO). I'm not absolutely certain if this is guaranteed behaviour, though; haven't thus far ever relied on it, and only noticed it recently (when implementing an example) myself.

Use tigetstr for fetching strings that correspond to the keys. Most are easily related to the curses names (which are listed in the getch manpage):

You can use os.close to close master and slave.

os.close(fd) Close file descriptor fd.

Note This function is intended for low-level I/O and must be applied to a file descriptor as returned by os.open() or pipe(). To close a “file object” returned by the built-in function open() or by popen() or fdopen(), use its close() method.

For my use case I eventually came up with the following script (wrapper.py):

The difference seems to be caused by the underlying libc implementations: glibc of Debian, vs musl-libc of Alpine.

While GNU libc is the defacto standard libc implementation in Linux, musl libc is used by a handful of distributions, such as Alpine Linux and Void Linux. musl is a minimalistic strict-POSIX libc implementation, and is generally not compatible with glibc. Usually, software projects have to be ported to musl libc to be supported on Alpine, especially non trivial applications.

The compilation of syscall.c breaks on several places, the first being:

I think your mistake here is that you are trying to read the slave. If you read the master instead you should get your output.

Quote from: http://rachid.koucha.free.fr/tech_corner/pty_pdip.html

A pseudo-terminal is a pair of character mode devices also called pty. One is master and the other is slave and they are connected with a bidirectional channel. Any data written on the slave side is forwarded to the output of the master side. Conversely, any data written on the master side is forwarded to the output of the slave side as depicted in figure 2.

RPI

This is expected. std::process::Child::stdin and friends are not set for raw file handles (because Rust doesn't know what they are, the builder doesn't have the master end of your pty).

You can construct your rusty file handle for the master yourself: