4 of 6 standards met
At Google we found around 3-5% of performance uplift for different kind of tests, mostly nicely compressed data. The improvement is mostly uniform across levels, x86-64 and aarch64 platforms with no change in compression ratios (variance is around 0.08%). We decided to implement a slower crc32c version for other platforms. Let us know if you want to leave multiplication hashes but this will lead to slightly different outputs across platforms. We decided to prioritize stability rather than speed but we don't have an opinion on this. This patch directly can be detrimental to 32bit platforms as well as RISC-V 64 bits. One test suddenly started not to have an error, please take a look :)
Describe the bug It looks like the zstd command-line tool has a limit on dictionary sizes of 33554432 bytes, even when using --long windows. Is that a fundamental limit of the zstd format or is it just a limitation of the CLI and it doesn't alter it based on the compression window? To Reproduce Steps to reproduce the behavior: 1. Stage 2 75 MB files, 1.txt and 2.txt in a directory. 2. 3. Get error Expected behavior Allow for using compression dictionaries up to the window size (or close to it). Desktop (please complete the following information): OS: Mac Version 1.5.7 Additional context** This is in the context of delta-compressing large WASM files for compression dictionary transport which allows for compression windows up to 128 MB.
Repository: facebook/zstd. Description: Zstandard - Fast real-time compression algorithm Stars: 26750, Forks: 2420. Primary language: C. Languages: C (76.2%), C++ (13.7%), Shell (3.1%), Python (2.5%), Makefile (1.9%). Homepage: http://www.zstd.net Latest release: v1.5.7 (1y ago). Open PRs: 39, open issues: 218. Last activity: 3d ago. Community health: 87%. Top contributors: Cyan4973, terrelln, inikep, felixhandte, stellamplau, sean-purcell, embg, daniellerozenblit, GeorgeLu97, dependabot[bot] and others.
Last 12 weeks · 36 commits
Is your feature request related to a problem? Please describe. Executable packers like UPX can't use ZSTD due to the copyright requirements of the LICENSE, specifically this clause: Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. Describe the solution you'd like Relax the requirement for this specific use case only, as this prevents using ZSTD in binary packers since there is nowhere for them to enforce copyright into the target binary, and requires redustributing as part of the decompression of the executable. Describe alternatives you've considered N/A Additional context Request for this support has been around for 7 years and is blocked by this, see https://github.com/upx/upx/issues/295
As already said in #4547, there is no RC currently, but for consistence reasons and to avoid potential RC in the future by possible changes... Possible alternative could be to lock the fresh created mutex before first worker thread gets created (and unlock it before return). Closes #4547, replaces #4558
Fixes a potential data race in where was being updated without holding the . Worker threads could access before it was fully initialized, leading to undefined behavior. The update to is now protected by the to ensure thread safety during initialization. remains unprotected, as it is only modified by the parent thread during setup. Fix for #4547