How do libraries work? Part 1.

I’ve been always wondering what happens, step by step when I use static or dynamic libraries and how can I actually use them. This very first blog post is about writing and including a dynamic library in MacOS system. Linux and Windows are coming soon. I will be mostly focusing on programming part, however, to make things clear I will mention the most important things. The detailed technical description of how operating system manages it internally can be found in additional resources at the bottom.

Dynamic libraries in MacOS

According to Apple developer’s docs:

Apps written for OS X benefit from this feature because all system libraries in OS X are dynamic libraries.

So, how is it handled by the system? When a program is launched, kernel allocates resources for it and loads code and data into its space. It also loads dynamic loader, named dyld and lets it load dependent libraries (the ones our program was compiled with and that have undefined symbols). We will give a look to those undefined symbols later. Afterwards control is passed to the program itself. At load time we can use constructors that will provide initialization of those libraries.

Let’s code!

Let’s say I want to implement some simple mechanisms I find useful in many projects. In order to achieve this, I am writing a dynamic library with functions I use. I’m going to name it mymath,:

#include <iostream>
#include "mymath.h"

#define EXPORT __attribute__((visibility("default")))

__attribute__((constructor))
static void _constructor() {
    std::cout << "Initializing mymath\n";
}

EXPORT
int mymath::add(int a, int b) {
    return a + b;
}
EXPORT
int mymath::sub(int a, int b) {
    return a - b;
}

__attribute__((destructor))
static void _destructor() {
    std::cout << "Destroying mymath\n";
}

Two functions encapsulated in a namespace, constructor and destructor. That’s it. Of course header file will look like this:

namespace mymath {
    int add(int, int);
    int sub(int, int);
}

And the Makefile:

all:
    llvm-g++ -Iinclude -dynamiclib -std=c++17 src/mymath.cpp -current_version 1.0 -compatibility_version 1.0 -fvisibility=hidden -o bin/mymath.dylib

That’s it. The structure of the project is also pretty straightforward:

$ tree
 .
 ├── Makefile
 ├── bin
 │   └── mymath.dylib
 ├── include
 │   └── mymath.h
 └── src
     └── mymath.cpp

Having achieved this, we can run make command and start using this library in a project of our own.

$ tree
 .
 ├── Makefile
 ├── bin
 │   ├── main
 │   └── mymath.dylib
 ├── include
 │   └── mymath.h
 ├── lib
 │   └── mymath.dylib
 └── src
     └── main.cpp

We copy the header file and binary library and create new main file:

#include <iostream>
#include "mymath.h"

void myFunction() {
    int a, b;
    a = 55;
    b = 66;
    std::cout << "My math adds: " << mymath::add(a, b) << std::endl;
    std::cout << "My math subtracts: " << mymath::sub(a, b) << std::endl;
}

int main() {
    std::cout << "Hello Easy C++ project!" << std::endl;
    myFunction();
    return 0;
}

with Makefile as follows (it’s 90% generated by VS Code plugin Easy C++ Project, which I use):

CXX       := clang++
CXX_FLAGS := -Wall -Wextra -std=c++17 -g

BIN     := bin
SRC     := src
INCLUDE := include
LIB     := lib

LIBRARIES   := lib/mymath.dylib
EXECUTABLE  := main


all: $(BIN)/$(EXECUTABLE)

run: clean all
    ./$(BIN)/$(EXECUTABLE)

$(BIN)/$(EXECUTABLE): $(SRC)/*.cpp
    $(CXX) $(CXX_FLAGS) -I$(INCLUDE) -L$(LIB) $^ -o $@ $(LIBRARIES)

clean:
    -rm -rf $(BIN)/*

Note that the created dynamic library is in two places: in lib and bin folders. This happens because in first case it’s needed by linker to link the created binaries and in the second case by dylib to load it into process’ memory on runtime. We would not need the second copy if we placed the library eg. in /usr/local/lib folder, or updated DYLD_LIBRARY_PATH variable.

Now. What will be printed when we compile and run our code?

$ make && ./bin/main
 clang++ -Wall -Wextra -std=c++17 -g -Iinclude -Llib src/main.cpp -o bin/main lib/mymath.dylib
 Initializing mymath
 Hello Easy C++ project!
 My math adds: 121
 My math subtracts: -11
 Destroying mymath

As expected libraries we loaded first and unloaded last meaning the code we created in constructor run even before the first line in main() function. Also, we can spot undefined symbols in newly created bin/main file using nm tool:

$ nm --demangle bin/main
(...)
                  U __Unwind_Resume
 0000000100001050 T myFunction()
                  U mymath::add(int, int)
                  U mymath::sub(int, int)

The U letter next to our function names indicates that we have an undefined symbol, which is one that is referenced outside our binary.

That’s pretty much it. My curiosity has been partially satisfied. In next part I’ll show how to transform this code to load and unload libraries in runtime and Linux and Windows will be coming soon afterwards.

Additional resources: