libiop | easy starter library

The buffer size of pipes isn't unlimited. You're creating a deadlock, where the child process is hanging because it's trying to write to a buffer that's full, and your parent process doesn't try to read anything from the buffer until the child process has completed. To fix the problem, you need to use another thread to read from the buffer while the child process is still running. The simplest way to do this is to use readProcess or a similar function in place of createProcess. If this doesn't give you enough flexibility to do what you want, then you'll need to create and manage the other thread yourself, which you can see how to do by looking at how readProcess is implemented.

There is no way to check if a FILE* was already closed. The safest way is to check if fHandler is NULL, then set the fHandler to NULL directly after you call fclose.

Looks like you corrupted your heap by writing past the end of your heap-allocation. I think your problem is here:

Are there any print calls which are guaranteed to NOT ever call malloc()

Usually you can call fprintf(stderr, ...). This is because stderr is unbuffered by default.

However, this may not work from within malloc early in the process lifetime, before the rest of libc has initialized itself.

Your best bet is to use write(2).

You are calling printf within your malloc implementation. That is not going to end well.

In the stack trace, you can clearly see that printf itself calls malloc.

If your malloc is not prepared to to be called while in the middle of manipulating its data structures, it will crash (possibly that's what's happening here).

Alternatively, you can also end up with infinite recursion, when malloc calls printf, which calls malloc, which calls printf, etc.

TL;DR: when implementing something as low level as malloc, you must stick to either low-level functions which don't themselves allocate anything, or to direct system calls.

Why is malloc() getting called before the program starts anyway?

Because low-level functions in e.g. dynamic loader need to allocate memory during their own initialization.

Your malloc must work very early in the process lifetime; long before main.

Is printing inside malloc() forbidden?

Everything that might allocate memory is forbidden.

In practice, you need to call only async-signal safe routines, because non-async-signal safe ones may allocate, if not now then in the future.

The comment by Ctx:

This is not possible; sbrk() is used by the libc memory allocator internally, so many libc functions use it implicitly. Either do not use libc at all, or do your memory allocations with malloc() or mmap().

is correct. You can only use sbrk if you don't use malloc, and you can't know if you use malloc because unless documented otherwise by the implementation, anything in libc might do so.

If your libc supports replacing malloc (note: most do) then you can write a full malloc replacement (this includes all malloc-family functions, not just malloc itself) that either don't use sbrk, or that cooperate with your use of it, and then you're free to use it. However otherwise, you simply can't use sbrk.

Note also that in some environments, including frequently with position-independent executables (PIE), sbrk will have little or no memory to work with and will frequently fail after only a few allocations or after none at all, because of running into memory mapped for something else. The whole concept of sbrk is backwards and should not be used at all in modern code. For some allocator strategies, it may make sense as one choice of backend, with mmap or something else also used. But it should never be the sole way your allocator obtains new memory, and in my opinion it's not really useful to support it at all. mmap is better in almost all ways.