Stripping Rust static libraries: symbols begone!

This blog post was last updated on July 7, 2024 to fix the Android section.

One of my first tasks with the fine folks at Centricular was to sort out a problem we had with the distribution of Rust libraries. GStreamer’s Rust-based plugins are implemented as a C ABI library through the cargo-c tool. When the binary distribution is put together, these plugins are compiled into dynamic libraries by default, but there are two platforms that require static libraries: iOS and Android. For these two platforms, a dynamic library that would weigh around 10 MB gets replaced by a static library of ~550MB.

Yes, that’s a 50x increase. There has to be a way to skip that, right?

First of all, we need to identify what’s ballooning the size of the libraries. That is really easy to identify: the author of cargo-c says that all static libraries are brought in through the pkg-config output; however, since the origin language is Rust, there is one library that must be embedded before downstream consumption: Rust’s precompiled standard library, or stdlib for short. -link_wholeing this library means that all symbols we don’t use are coming along for the ride, as well as their debugging information. And that’s ignoring the ABI hygiene issues that will arise when linking more than one cargo-c generated library together.

So, how can this be fixed for the platforms we support?

Darwin

Darwin-based platforms, like iOS, are also conceptually really simple, though it wasn’t easy to figure it out. Since the MachO binary format mixes “flat” symbol exports (like ELF) and strong link-time symbol resolution (like Windows), ld64 does not allow stripping libraries except at link-time:

strip no longer removes relocation entries under any condition. Instead, it updates the external relocation entries (and indirect symbol table entries) to reflect the resulting symbol table. strip prints an error message for those symbols not in the resulting symbol table that are needed by an external relocation entry or an indirect symbol table. The link editor ld(1) is the only program that can strip relocation entries and know if it is safe to do so.

Attempting to do so will either result in strip’s call to ld complaining about undefined symbols, or a completely stubbed library.

Is there a workaround for this? A key insight here is that the Rust stdlib exports all symbols as global. So, if we want to strip symbols, we should inject the -exported_symbols_list flag into the library linking step, right?

Unfortunately, Rust does not use the linker to create the static library at all; it archives the object files using its own bespoke tools. Researching alternatives online, I could only find another blog post that demonstrates a way to achieve this with CMake; that led me to another little documented, but fully Apple-supported alternative to achieve the same: Single-Object Prelink.

From an implementation point of view, when one enables this option, Xcode will instead:

  • link world + dog into a new object file (ld -r) with all the flags necessary, just like a dynamic library
  • then archive the object file using libtool -static (with extra, Xcode-specific information)

The call to ld is where we can inject the list of symbols to be kept. And we can unpack the library to get the object files. So, what was my fix for this?

Stripping Rust stdlib symbols

The first step is to put together a list of all the symbols to be exported. Don’t try using wildcards, it must be exhaustive! If you don’t have one at hand, you can use nm to list all the symbols, filter them as needed, and then save the result to a file:

nm -gjPUA path_to_the_library

Xcode’s nm cannot be used for this step because it does not understand symbols generated with LLVM 16, like Rust’s. The toolchain supplies an optional component, llvm-tools-preview, that supplies a matching version. To invoke the supplied tool, you can

  • find the path to llvm-nm in rustc’s system root folder, for instance through find $(rustc --print sysroot) -name 'llvm-nm'
  • install the cargo-binutils crate, then replace nm with cargo nm in the command above

Second step is to extract the object files from the library:

ar xv path_to_the_library

Third, call up ld to link all the objects along with the module definition file (if not in a shell, replace *.o with the list of all the .o files extracted in the previous step):

ld -r -exported_symbols_list path_to_your_definition_file -o prelinked.o *.o

And finally, reassemble the static library with Xcode’s libtool utility:

libtool -static -o path_to_the_library prelinked.o

With this change, I shaved almost 90% of the biggest library we have (the WebRTC plugin), without losing debugging information:

  • debug: 504 MB -> 93 MB
  • release: 144 MB -> 18 MB

Linux based platforms

In the previous inception of this post, I claimed that this could be achieved with just the use of strip plus export symbols selection. This is not correct. To quote myself in the merge request fixing this:

Using strip with --keep-symbol looked sensible, but the utility did not truly parse all the symbols and constructed a dependency chain. Instead, placeholders to the next address were generated in place of all the .rodata symbols referenced in the functions to be kept. The result of this nightmare was crashes that looked completely senseless, until one checked the disassembly of the functions – the gst_plugin_xxx_register function was there, but neither the call nor the parameters referenced anywhere valid in the data sections.

Linux-based platforms, such as Android, can instead also make use of Single-Object Prelink, which in this OS is called “partial linking” or “relocatable linking”. The key difference with regards to macOS is that while the symbol stripping is performed by ld, symbol visibility (to hide the remains of the Rust stdlib) can only be changed with a separate call to objcopy:

ar xv path_to_the_library
ld --relocatable --export-dynamic-symbol=gst_plugin_* -o prelinked.o *.o
objcopy --wildcard --keep-global-symbol=gst_plugin_* prelinked.o
ar rs path_to_the_library prelinked.o

Windows

We also looked into enabling these improvements for Microsoft’s platforms. However, neither MSVC nor MinGW-w64 provide tools that allow tinkering with static libraries in the ways we need above.

Conclusion

We managed to shave off between 50% and 90% off our static libraries by stripping all unused bits of the Rust stdlib. This was possible on both macOS and Linux with reasonably up-to-date tools, but to the best of my knowledge, it’s not yet possible for the Windows ABI.

For a future piece, we’d love to fully deduplicate the stdlib from the individual plugins. Having a way to supply it externally would remove the need for tinkering with the object files. This was filed as issue #111594 in the Rust repository.

Please head over to the GStreamer repo to have a look at the gory details, and let me know if you have any questions!