lzma_options_lzma Struct Reference

Options specific to the LZMA1 and LZMA2 filters. More...

#include <lzma.h>

Data Fields

uint32_t dict_size
 Dictionary size in bytes.
const uint8_t * preset_dict
 Pointer to an initial dictionary.
uint32_t preset_dict_size
 Size of the preset dictionary.
uint32_t lc
 Number of literal context bits.
uint32_t lp
 Number of literal position bits.
uint32_t pb
 Number of position bits.
lzma_bool persistent
 Indicate if the options structure is persistent.
lzma_mode mode
uint32_t nice_len
 Nice length of a match.
lzma_match_finder mf
uint32_t depth
 Maximum search depth in the match finder.
void * reserved_ptr1
void * reserved_ptr2
uint32_t reserved_int1
uint32_t reserved_int2
uint32_t reserved_int3
uint32_t reserved_int4
uint32_t reserved_int5
uint32_t reserved_int6
uint32_t reserved_int7
uint32_t reserved_int8
lzma_reserved_enum reserved_enum1
lzma_reserved_enum reserved_enum2
lzma_reserved_enum reserved_enum3
lzma_reserved_enum reserved_enum4

Detailed Description

Options specific to the LZMA1 and LZMA2 filters.

Since LZMA1 and LZMA2 share most of the code, it's simplest to share the options structure too. For encoding, all but the reserved variables need to be initialized unless specifically mentioned otherwise.

For raw decoding, both LZMA1 and LZMA2 need dict_size, preset_dict, and preset_dict_size (if preset_dict != NULL). LZMA1 needs also lc, lp, and pb.


Field Documentation

Dictionary size in bytes.

Dictionary size indicates how many bytes of the recently processed uncompressed data is kept in memory. One method to reduce size of the uncompressed data is to store distance-length pairs, which indicate what data to repeat from the dictionary buffer. Thus, the bigger the dictionary, the better compression ratio usually is.

Maximum size of the dictionary depends on multiple things:

  • Memory usage limit
  • Available address space (not a problem on 64-bit systems)
  • Selected match finder (encoder only)

Currently the maximum dictionary size for encoding is 1.5 GiB (i.e. (UINT32_C(1) << 30) + (UINT32_C(1) << 29)) even on 64-bit systems for certain match finder implementation reasons. In future, there may be match finders that support bigger dictionaries (3 GiB will probably be the maximum).

Decoder already supports dictionaries up to 4 GiB - 1 B (i.e. UINT32_MAX), so increasing the maximum dictionary size of the encoder won't cause problems for old decoders.

Because extremely small dictionaries sizes would have unneeded overhead in the decoder, the minimum dictionary size is 4096 bytes.

Note:
When decoding, too big dictionary does no other harm than wasting memory.

Referenced by message_filters(), and options_lzma().

Pointer to an initial dictionary.

It is possible to initialize the LZ77 history window using a preset dictionary. Here is a good quote from zlib's documentation; this applies to LZMA as is:

The dictionary should consist of strings (byte sequences) that are likely to be encountered later in the data to be compressed, with the most commonly used strings preferably put towards the end of the dictionary. Using a dictionary is most useful when the data to be compressed is short and can be predicted with good accuracy; the data can then be compressed better than with the default empty dictionary. (From deflateSetDictionary() in zlib.h of zlib version 1.2.3)

This feature should be used only in special situations. It works correctly only with raw encoding and decoding. Currently none of the container formats supported by liblzma allow preset dictionary when decoding, thus if you create a .lzma file with preset dictionary, it cannot be decoded with the regular .lzma decoder functions.

Todo:
This feature is not implemented yet.

Size of the preset dictionary.

Specifies the size of the preset dictionary. If the size is bigger than dict_size, only the last dict_size bytes are processed.

This variable is read only when preset_dict is not NULL.

Number of literal context bits.

How many of the highest bits of the previous uncompressed eight-bit byte (also known as `literal') are taken into account when predicting the bits of the next literal.

Todo:
Example

There is a limit that applies to literal context bits and literal position bits together: lc + lp <= 4. Without this limit the decoding could become very slow, which could have security related results in some cases like email servers doing virus scanning. This limit also simplifies the internal implementation in liblzma.

There may be LZMA streams that have lc + lp > 4 (maximum lc possible would be 8). It is not possible to decode such streams with liblzma.

Referenced by is_lclppb_valid(), lzma_lzma_lclppb_decode(), lzma_lzma_lclppb_encode(), message_filters(), and options_lzma().

Number of literal position bits.

How many of the lowest bits of the current position (number of bytes from the beginning of the uncompressed data) in the uncompressed data is taken into account when predicting the bits of the next literal (a single eight-bit byte).

Todo:
Example

Referenced by is_lclppb_valid(), lzma_lzma_lclppb_decode(), lzma_lzma_lclppb_encode(), message_filters(), and options_lzma().

Number of position bits.

How many of the lowest bits of the current position in the uncompressed data is taken into account when estimating probabilities of matches. A match is a sequence of bytes for which a matching sequence is found from the dictionary and thus can be stored as distance-length pair.

Example: If most of the matches occur at byte positions of 8 * n + 3, that is, 3, 11, 19, ... set pb to 3, because 2**3 == 8.

Referenced by is_lclppb_valid(), lzma_lzma_lclppb_decode(), lzma_lzma_lclppb_encode(), and message_filters().

Indicate if the options structure is persistent.

If this is true, the application must keep this options structure available after the LZMA2 encoder has been initialized. With persistent structure it is possible to change some encoder options in the middle of the encoding process without resetting the encoder.

This option is used only by LZMA2. LZMA1 ignores this and it is safe to not initialize this when encoding with LZMA1.

LZMA compression mode

Referenced by message_filters().

Nice length of a match.

This determines how many bytes the encoder compares from the match candidates when looking for the best match. Once a match of at least nice_len bytes long is found, the encoder stops looking for better condidates and encodes the match. (Naturally, if the found match is actually longer than nice_len, the actual length is encoded; it's not truncated to nice_len.)

Bigger values usually increase the compression ratio and compression time. For most files, 30 to 100 is a good value, which gives very good compression ratio at good speed.

The exact minimum value depends on the match finder. The maximum is 273, which is the maximum length of a match that LZMA can encode.

Referenced by message_filters(), and options_lzma().

Match finder ID

Referenced by message_filters(), and options_lzma().

Maximum search depth in the match finder.

For every input byte, match finder searches through the hash chain or binary tree in a loop, each iteration going one step deeper in the chain or tree. The searching stops if

  • a match of at least nice_len bytes long is found;
  • all match candidates from the hash chain or binary tree have been checked; or
  • maximum search depth is reached.

Maximum search depth is needed to prevent the match finder from wasting too much time in case there are lots of short match candidates. On the other hand, stopping the search before all candidates have been checked can reduce compression ratio.

Setting depth to zero tells liblzma to use an automatic default value, that depends on the selected match finder and nice_len. The default is in the range [10, 200] or so (it may vary between liblzma versions).

Using a bigger depth value than the default can increase compression ratio in some cases. There is no strict maximum value, but high values (thousands or millions) should be used with care: the encoder could remain fast enough with typical input, but malicious input could cause the match finder to slow down dramatically, possibly creating a denial of service attack.

Referenced by message_filters().


The documentation for this struct was generated from the following file:

Generated on Tue Oct 6 14:02:23 2009 for XZ Utils by  doxygen 1.6.1