Rust FFI vs Golang FFI (cgo)
I was put to a task where I needed to use a lot of FFI; specifically, I am talking about libav. A lot of FFI means that there will be many calling between higher-level language <-> C library. The library that I needed to use is significant, and it would not be easy to rewrite it in that language. C programming language was not an option because in our company, it is considered as an unsafe language, and we did not want to start a new project in C; the same applies for C++. Our primary programming language in the company is Golang. Rust is now growing in popularity, and I would like to give it a try since Rust programming language has many benefits that Golang does not have.
In this blog post, I would like to compare the speed of FFI in each language. From the beginning, I would say that Rust will be faster than Go since Go has a runtime. Moreover, Golang’s cgo generates a lot of calling overhead. I will talk more about it later. But the question is how much it will be slower? Will it be doable to use Golang in production?
The goal will be simple; on input, there will be an FLV container that contains a video track encoded with H.264 and an audio track encoded with AAC. We will read the FLV file from higher-level language and pass the read data to the libav, aka avformat wrapper. Libav will remux this file for use to the TS file. Remuxing is a lossless process that simply takes the video and audio streams from one container and puts them into a new container. This should show us how fast it really is and how much overhead is generated. For benchmarking purposes, I will run the remuxing process 400 times, and then I will compare the results. Test file has the following properties.
Input #0, flv, from 'test.flv':
Metadata:
Duration: 00:02:17.34, start: 0.021000, bitrate: 1864 kb/s
Stream #0:0: Video: h264 (High), yuv420p(progressive), 1280x544, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc
Stream #0:1: Audio: aac (LC), 48000 Hz, stereo, fltp, 128 kb/s
avlib wrapper aka avfomat
Firstly let’s write C wrapper around avlibs. Following codec reads file from ../test.flv and outputs to ./test.ts. Beware compatible codecs when remuxing between diferent formats. The C code may not be optimal, but it works.
Golang version
Writing FFI in golang is a little bit tricky because cgo dynamically checks whenever you pass something that contains go pointer to C, as described here. So we will tweak little bit our test.c file, and our go code will look like this. This means that we cannot directly pass a pointer to a golang func. This dynamic check behavior can be disabled, but it is considered as unsafe.
Rust version
I would say that Rust version is more straightforward because it does not have a runtime. But you need to keep in mind that we can lose the benefit of memory safety because you are calling C code. Because C code does not deallocate automatically. What I like more about this version is that you can pass a function pointer directly to the C, and there is literally no overhead generated or other wrapper functions.
Results and summary
All code is hosted on GitHub. If you find any inconsistence in my blog post please let me know. I am relatively new to Rust and Golang FFI. I am writing a blog post about this problem because it helps me to more understand the problem.
+-----------+-----------+-------------+------------+
| Language | User time | System time | Total time |
+-----------+-----------+-------------+------------+
| C - gcc | 27.62s | 18.42s | 1:12.35s |
| C - clang | 29.6s | 19.28s | 1:18.4s |
| Go | 46.87s | 28.37s | 1:31.39s |
| Rust | 19.72s | 15.28s | 35.1s |
+-----------+-----------+-------------+------------+
PC:
- Intel core i7 4770k
- NVMe disk
version:
- gcc 10.2.0
- clang 11.0.1
- go 1.15.6
- rust 1.48Results are quite impressive. I expected that golang version would be slower than Rust, but I did not expect that Rust will be even almost twice as fast as the C version. I will focus on the reason why Rust is that fast. I will periodically update this blogpost as I gain better knowledge of the problem. More results coming soon.
