Velocity/native/build-support/compile-linux-compress.sh

#!/bin/bash

if [ ! "$CC" ]; then
  # The libdeflate authors recommend that we build using GCC as it produces "slightly faster binaries":
  # https://github.com/ebiggers/libdeflate#for-unix
  export CC=gcc
fi

if [ ! -d libdeflate ]; then
  echo "Cloning libdeflate..."
  git clone --branch v1.21 --single-branch https://github.com/ebiggers/libdeflate.git
fi

echo "Compiling libdeflate..."
cd libdeflate || exit
rm -rf build && cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON -B build && cmake --build build --target libdeflate_static
cd ..

CFLAGS="-O2 -I$JAVA_HOME/include/ -I$JAVA_HOME/include/linux/ -fPIC -shared -Wl,-z,noexecstack -Wall -Werror -fomit-frame-pointer"
ARCH=$(uname -m)
mkdir -p src/main/resources/linux_$ARCH
$CC $CFLAGS -Ilibdeflate src/main/c/jni_util.c src/main/c/jni_zlib_deflate.c src/main/c/jni_zlib_inflate.c \
    libdeflate/build/libdeflate.a -o src/main/resources/linux_$ARCH/velocity-compress.so
JNI native zlib compression 🔥 2018-08-02 04:22:09 +02:00			`#!/bin/bash`

Clean up more cipher code and make it -Wall + -Werror clean 2020-09-16 06:07:52 +02:00			`if [ ! "$CC" ]; then`
			`# The libdeflate authors recommend that we build using GCC as it produces "slightly faster binaries":`
			`# https://github.com/ebiggers/libdeflate#for-unix`
			`export CC=gcc`
			`fi`

Switch out Cloudflare zlib for libdeflate. libdeflate is significantly faster than vanilla zlib, zlib-ng, and Cloudflare zlib. It is also MIT-licensed (so no licensing concerns). In addition, it simplifies a lot of the native code (something that's been tricky to get right). While we're at it, I have also taken the time to fine-time compression in Velocity in general. Thanks to this work, native compression only requires one JNI call, an improvement from the more than 2 (sometimes up to 5) that were possible before. This optimization also extends to the existing Java compressors, though they require potentially two JNI calls. 2020-05-24 16:56:26 +02:00			`if [ ! -d libdeflate ]; then`
			`echo "Cloning libdeflate..."`
Fall 2024 recompile of the natives (#1416) This time around, instead of the very ad hoc approach I have historically taken, this time we actually have a convenient script that takes care of basically all the hard work necessary, including cross-compilation. We compile natives for Ubuntu 20.04 LTS (OpenSSL 1.1.x native and libdeflate) and Ubuntu 22.04 LTS (OpenSSL 3.x.x native), for both x86_64 and aarch64. The macOS natives have also been recompiled on macOS Sonoma. 2024-09-01 02:30:47 +02:00			`git clone --branch v1.21 --single-branch https://github.com/ebiggers/libdeflate.git`
Switch to zlib-ng zlib-ng boasts higher throughput than regular zlib, by combining patches from Cloudflare, zlib, and ARM's improvements to zlib along with a more modern codebase. Profiling consistently shows that compression is the largest CPU expense by far, so even a minor speed-up here is significant. 2019-09-09 05:30:28 +02:00			`fi`

Switch out Cloudflare zlib for libdeflate. libdeflate is significantly faster than vanilla zlib, zlib-ng, and Cloudflare zlib. It is also MIT-licensed (so no licensing concerns). In addition, it simplifies a lot of the native code (something that's been tricky to get right). While we're at it, I have also taken the time to fine-time compression in Velocity in general. Thanks to this work, native compression only requires one JNI call, an improvement from the more than 2 (sometimes up to 5) that were possible before. This optimization also extends to the existing Java compressors, though they require potentially two JNI calls. 2020-05-24 16:56:26 +02:00			`echo "Compiling libdeflate..."`
			`cd libdeflate \|\| exit`
Fall 2024 recompile of the natives (#1416) This time around, instead of the very ad hoc approach I have historically taken, this time we actually have a convenient script that takes care of basically all the hard work necessary, including cross-compilation. We compile natives for Ubuntu 20.04 LTS (OpenSSL 1.1.x native and libdeflate) and Ubuntu 22.04 LTS (OpenSSL 3.x.x native), for both x86_64 and aarch64. The macOS natives have also been recompiled on macOS Sonoma. 2024-09-01 02:30:47 +02:00			`rm -rf build && cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON -B build && cmake --build build --target libdeflate_static`
Switch to zlib-ng zlib-ng boasts higher throughput than regular zlib, by combining patches from Cloudflare, zlib, and ARM's improvements to zlib along with a more modern codebase. Profiling consistently shows that compression is the largest CPU expense by far, so even a minor speed-up here is significant. 2019-09-09 05:30:28 +02:00			`cd ..`

Recompile natives for Linux x86_64 Debian 9 was used to compile the libdeflate native and OpenSSL 1.1.x crypto, CentOS for OpenSSL 1.0.x crypto only. 2020-10-27 06:44:14 +01:00			`CFLAGS="-O2 -I$JAVA_HOME/include/ -I$JAVA_HOME/include/linux/ -fPIC -shared -Wl,-z,noexecstack -Wall -Werror -fomit-frame-pointer"`
Lay down foundation for cross-architecture natives We have dropped the rarely used kqueue and replaced it with the new Netty aarch64 native. In addition, lay down the foundation for other aarch64 natives. 2020-06-04 20:56:17 +02:00			`ARCH=$(uname -m)`
			`mkdir -p src/main/resources/linux_$ARCH`
Clean up more cipher code and make it -Wall + -Werror clean 2020-09-16 06:07:52 +02:00			`$CC $CFLAGS -Ilibdeflate src/main/c/jni_util.c src/main/c/jni_zlib_deflate.c src/main/c/jni_zlib_inflate.c \`
Fall 2024 recompile of the natives (#1416) This time around, instead of the very ad hoc approach I have historically taken, this time we actually have a convenient script that takes care of basically all the hard work necessary, including cross-compilation. We compile natives for Ubuntu 20.04 LTS (OpenSSL 1.1.x native and libdeflate) and Ubuntu 22.04 LTS (OpenSSL 3.x.x native), for both x86_64 and aarch64. The macOS natives have also been recompiled on macOS Sonoma. 2024-09-01 02:30:47 +02:00			`libdeflate/build/libdeflate.a -o src/main/resources/linux_$ARCH/velocity-compress.so`