Package org.apache.avro

Class SystemLimitException

java.lang.Object
java.lang.Throwable
java.lang.Exception
java.lang.RuntimeException
org.apache.avro.AvroRuntimeException
org.apache.avro.SystemLimitException
All Implemented Interfaces:
Serializable
public class SystemLimitException extends AvroRuntimeException
Thrown to prevent making large allocations when reading potentially pathological input data from an untrusted source.

The following system properties can be set to limit the size of bytes, strings and collection types to be allocated:

  • org.apache.avro.limits.bytes.maxLength limits the maximum size of bytes types.
  • org.apache.avro.limits.collectionItems.maxLength limits the maximum number of map and list items that can be read in a single sequence.
  • org.apache.avro.limits.string.maxLength limits the maximum size of string types.
  • org.apache.avro.limits.collectionItems.maxAllocation limits the number of array elements whose minimum encoded size is zero (such as null, a zero-length fixed, a record whose fields are all zero-byte, or a recursive schema whose cycle is conservatively broken with a 0 minimum) that may be allocated at once. Unlike other element types, these cannot be bounded by the number of bytes remaining in the stream, so the limit defaults to a fraction of the maximum heap.
The default is to permit sizes up to MAX_ARRAY_VM_LIMIT.
See Also:

  • Field Details

    • MAX_BYTES_LENGTH_PROPERTY

      public static final String MAX_BYTES_LENGTH_PROPERTY
      See Also:

    • MAX_COLLECTION_LENGTH_PROPERTY

      public static final String MAX_COLLECTION_LENGTH_PROPERTY
      See Also:

    • MAX_STRING_LENGTH_PROPERTY

      public static final String MAX_STRING_LENGTH_PROPERTY
      See Also:

    • MAX_DECOMPRESS_LENGTH_PROPERTY

      public static final String MAX_DECOMPRESS_LENGTH_PROPERTY
      System property declaring max size of any decompression stream: "org.apache.avro.limits.decompress.maxLength".
      See Also:

    • MAX_DECOMPRESS_LENGTH

      public static final long MAX_DECOMPRESS_LENGTH
      Calculated max decompress length.

    • MAX_COLLECTION_ALLOCATION_PROPERTY

      public static final String MAX_COLLECTION_ALLOCATION_PROPERTY
      System property declaring the maximum number of array elements whose minimum encoded size is zero (e.g. null, a zero-length fixed, a record whose fields are all zero-byte, or a recursive schema conservatively treated as a 0 minimum) to allocate at once: "org.apache.avro.limits.collectionItems.maxAllocation".
      See Also:

  • Constructor Details

    • SystemLimitException

      public SystemLimitException(String message)

  • Method Details

    • checkMaxBytesLength

      public static int checkMaxBytesLength(long length)
      Check to ensure that reading the bytes is within the specified limits.
      Parameters:
      length - The proposed size of the bytes to read
      Returns:
      The size of the bytes if and only if it is within the limit and non-negative.
      Throws:
      UnsupportedOperationException - if reading the datum would allocate a collection that the Java VM would be unable to handle
      SystemLimitException - if the decoding should fail because it would otherwise result in an allocation exceeding the set limit
      AvroRuntimeException - if the length is negative

    • checkMaxCollectionLength

      public static int checkMaxCollectionLength(long existing, long items)
      Check to ensure that reading the specified number of items remains within the specified limits.
      Parameters:
      existing - The number of elements items read in the collection
      items - The next number of items to read. In normal usage, this is always a positive, permitted value. Negative and zero values have a special meaning in Avro decoding.
      Returns:
      The total number of items in the collection if and only if it is within the limit and non-negative.
      Throws:
      UnsupportedOperationException - if reading the items would allocate a collection that the Java VM would be unable to handle
      SystemLimitException - if the decoding should fail because it would otherwise result in an allocation exceeding the set limit
      AvroRuntimeException - if the length is negative

    • checkMaxCollectionLength

      public static int checkMaxCollectionLength(long items)
      Check to ensure that reading the specified number of items remains within the specified limits.
      Parameters:
      items - The next number of items to read. In normal usage, this is always a positive, permitted value. Negative and zero values have a special meaning in Avro decoding.
      Returns:
      The total number of items in the collection if and only if it is within the limit and non-negative.
      Throws:
      UnsupportedOperationException - if reading the items would allocate a collection that the Java VM would be unable to handle
      SystemLimitException - if the decoding should fail because it would otherwise result in an allocation exceeding the set limit
      AvroRuntimeException - if the length is negative

    • checkMaxCollectionAllocation

      public static long checkMaxCollectionAllocation(long existing, long items)
      Check to ensure that allocating storage for the specified number of array elements whose minimum encoded size is zero remains within the heap-aware limit.

      Elements whose minimum encoded size is zero (e.g. null, a zero-length fixed, a record whose fields are all zero-byte, or a recursive schema whose cycle is conservatively broken with a 0 minimum) consume no guaranteed input bytes, so the number that may be declared is not bounded by the bytes remaining in the stream. Without a cap, a tiny payload can declare an enormous block count and drive an unbounded backing-array allocation. This limit is derived from the maximum heap (see MAX_COLLECTION_ALLOCATION_PROPERTY).

      Parameters:
      existing - The number of elements already allocated for the collection.
      items - The next number of elements to allocate.
      Returns:
      The cumulative element count if and only if it is within the limit.
      Throws:
      SystemLimitException - if the cumulative allocation would exceed the limit.
      AvroRuntimeException - if either argument is negative.

    • beginCollectionAllocationScope

      public static void beginCollectionAllocationScope()
      Begin an outermost decode scope for cumulative zero-byte collection allocation accounting. Must be paired with endCollectionAllocationScope() in a finally block. Scopes nest: only the outermost one resets the running total, so the cap applies across the whole datum rather than per collection. See checkMaxCollectionAllocation(long).

    • endCollectionAllocationScope

      public static void endCollectionAllocationScope()
      End a decode scope opened by beginCollectionAllocationScope(). When the outermost scope closes the running total is cleared so it never leaks into an unrelated later decode on the same thread.

    • checkMaxCollectionAllocation

      public static long checkMaxCollectionAllocation(long items)
      Accumulate items zero-byte-minimum collection elements into the current decode scope and verify the running total stays within the allocation limit.

      Unlike checkMaxCollectionAllocation(long, long), which bounds a single collection, this bounds the cumulative count across every collection decoded within the enclosing scope (one datum), so a record made of many small zero-byte collection fields cannot bypass the cap in aggregate. When called outside any scope it falls back to a stateless single-collection check, preserving the previous behaviour for callers that do not delimit a datum.

      Parameters:
      items - The next number of zero-byte elements to allocate.
      Returns:
      The cumulative element count if and only if it is within the limit.
      Throws:
      SystemLimitException - if the cumulative allocation would exceed the limit.
      AvroRuntimeException - if items is negative.

    • checkMaxStringLength

      public static int checkMaxStringLength(long length)
      Check to ensure that reading the string size is within the specified limits.
      Parameters:
      length - The proposed size of the string to read
      Returns:
      The size of the string if and only if it is within the limit and non-negative.
      Throws:
      UnsupportedOperationException - if reading the items would allocate a collection that the Java VM would be unable to handle
      SystemLimitException - if the decoding should fail because it would otherwise result in an allocation exceeding the set limit
      AvroRuntimeException - if the length is negative

    • checkMaxDecompressCapacity

      public static void checkMaxDecompressCapacity(long limit, long streamLength, int bytes)
      Check there is capacity to write data to a stream.
      Parameters:
      limit - total capacity limit
      streamLength - current stream size
      bytes - bytes to add to the stream
      Throws:
      SystemLimitException - if the limit is exceeded.