Velocity/native/compile-linux.sh

#!/bin/bash

if [ ! "$CC" ]; then
  # The libdeflate authors recommend that we build using GCC as it produces "slightly faster binaries":
  # https://github.com/ebiggers/libdeflate#for-unix
  export CC=gcc
fi

if [ ! -d libdeflate ]; then
  echo "Cloning libdeflate..."
  git clone https://github.com/ebiggers/libdeflate.git
fi

echo "Compiling libdeflate..."
cd libdeflate || exit
cmake -B build && cmake --build build --target libdeflate_static
cd ..

CFLAGS="-O2 -I$JAVA_HOME/include/ -I$JAVA_HOME/include/linux/ -fPIC -shared -Wl,-z,noexecstack -Wall -Werror -fomit-frame-pointer"
ARCH=$(uname -m)
mkdir -p src/main/resources/linux_$ARCH
$CC $CFLAGS -Ilibdeflate src/main/c/jni_util.c src/main/c/jni_zlib_deflate.c src/main/c/jni_zlib_inflate.c \
    libdeflate/build/libdeflate.a -o src/main/resources/linux_$ARCH/velocity-compress.so
$CC $CFLAGS -shared src/main/c/jni_util.c src/main/c/jni_cipher_openssl.c \
    -o src/main/resources/linux_$ARCH/velocity-cipher.so -lcrypto
JNI native zlib compression 🔥 2018-08-02 04:22:09 +02:00			`#!/bin/bash`

Clean up more cipher code and make it -Wall + -Werror clean 2020-09-16 06:07:52 +02:00			`if [ ! "$CC" ]; then`
			`# The libdeflate authors recommend that we build using GCC as it produces "slightly faster binaries":`
			`# https://github.com/ebiggers/libdeflate#for-unix`
			`export CC=gcc`
			`fi`

Switch out Cloudflare zlib for libdeflate. libdeflate is significantly faster than vanilla zlib, zlib-ng, and Cloudflare zlib. It is also MIT-licensed (so no licensing concerns). In addition, it simplifies a lot of the native code (something that's been tricky to get right). While we're at it, I have also taken the time to fine-time compression in Velocity in general. Thanks to this work, native compression only requires one JNI call, an improvement from the more than 2 (sometimes up to 5) that were possible before. This optimization also extends to the existing Java compressors, though they require potentially two JNI calls. 2020-05-24 16:56:26 +02:00			`if [ ! -d libdeflate ]; then`
			`echo "Cloning libdeflate..."`
			`git clone https://github.com/ebiggers/libdeflate.git`
Switch to zlib-ng zlib-ng boasts higher throughput than regular zlib, by combining patches from Cloudflare, zlib, and ARM's improvements to zlib along with a more modern codebase. Profiling consistently shows that compression is the largest CPU expense by far, so even a minor speed-up here is significant. 2019-09-09 05:30:28 +02:00			`fi`

Switch out Cloudflare zlib for libdeflate. libdeflate is significantly faster than vanilla zlib, zlib-ng, and Cloudflare zlib. It is also MIT-licensed (so no licensing concerns). In addition, it simplifies a lot of the native code (something that's been tricky to get right). While we're at it, I have also taken the time to fine-time compression in Velocity in general. Thanks to this work, native compression only requires one JNI call, an improvement from the more than 2 (sometimes up to 5) that were possible before. This optimization also extends to the existing Java compressors, though they require potentially two JNI calls. 2020-05-24 16:56:26 +02:00			`echo "Compiling libdeflate..."`
			`cd libdeflate \|\| exit`
Fix native compile as libdeflate moved to CMake Fixes #890 correctly. Also rebuild the compression/encryption natives on macOS and Linux ARM64. 2023-01-01 17:45:47 +01:00			`cmake -B build && cmake --build build --target libdeflate_static`
Switch to zlib-ng zlib-ng boasts higher throughput than regular zlib, by combining patches from Cloudflare, zlib, and ARM's improvements to zlib along with a more modern codebase. Profiling consistently shows that compression is the largest CPU expense by far, so even a minor speed-up here is significant. 2019-09-09 05:30:28 +02:00			`cd ..`

Recompile natives for Linux x86_64 Debian 9 was used to compile the libdeflate native and OpenSSL 1.1.x crypto, CentOS for OpenSSL 1.0.x crypto only. 2020-10-27 06:44:14 +01:00			`CFLAGS="-O2 -I$JAVA_HOME/include/ -I$JAVA_HOME/include/linux/ -fPIC -shared -Wl,-z,noexecstack -Wall -Werror -fomit-frame-pointer"`
Lay down foundation for cross-architecture natives We have dropped the rarely used kqueue and replaced it with the new Netty aarch64 native. In addition, lay down the foundation for other aarch64 natives. 2020-06-04 20:56:17 +02:00			`ARCH=$(uname -m)`
			`mkdir -p src/main/resources/linux_$ARCH`
Clean up more cipher code and make it -Wall + -Werror clean 2020-09-16 06:07:52 +02:00			`$CC $CFLAGS -Ilibdeflate src/main/c/jni_util.c src/main/c/jni_zlib_deflate.c src/main/c/jni_zlib_inflate.c \`
Fix native compile as libdeflate moved to CMake Fixes #890 correctly. Also rebuild the compression/encryption natives on macOS and Linux ARM64. 2023-01-01 17:45:47 +01:00			`libdeflate/build/libdeflate.a -o src/main/resources/linux_$ARCH/velocity-compress.so`
I have sinned but it feels good 2022-06-08 07:18:19 +02:00			`$CC $CFLAGS -shared src/main/c/jni_util.c src/main/c/jni_cipher_openssl.c \`
Rewrite native crypto to use OpenSSL OpenSSL is much more portable and optimized (important for aarch64) and most systems already have a version. Unfortunately, OpenSSL likes to break their ABI. Thankfully, Velocity's natives system is very flexible largely, so we can provide multiple versions of this crypto. Versions of the dynamically-linked crypto were compiled on CentOS 7 (still supported until 2024, uses OpenSSL 1.0.x) and Debian 9 (the oldest distro including OpenSSL 1.1.0, whose LTS supports ends in 2022). The choice of distros was intended to cover most modern distributions (2014 and afterwards). An ARM compilation (using Debian 9) will be published soon. 2020-06-23 02:14:46 +02:00			`-o src/main/resources/linux_$ARCH/velocity-cipher.so -lcrypto`