Module carray

Implementation of resizeable arrays of different types in Cython.

All arrays provide for the following operations:

  • access by indexing.
  • access through get/set function.
  • appending values at the end of the array.
  • reserving space for future appends.
  • access to internal data through a numpy array.

Each array also provides an interface to its data through a numpy array. This is done through the get_npy_array function. The returned numpy array can be used just like any other numpy array but for the following restrictions:

  • the array may not be resized.
  • references of this array should not be kept.
  • slices of this array may not be made.

The numpy array may however be copied and used in any manner.

class pyzoltan.core.carray.BaseArray

Bases: object

Base class for managed C-arrays.

align_array(self, LongArray new_indices)

Rearrange the array contents according to the new indices.

alloc

alloc: ‘long’

copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1)

Copy subset of values from source to self.

copy_values(self, LongArray indices, BaseArray dest)

Copy values of indexed particles from self to dest.

extend(self, ndarray in_array)

Extend the array with data from in_array.

get_c_type(self) → str

Return the c data type of this array.

get_npy_array(self) → ndarray

Returns a numpy array of the data: do not keep its reference.

length

length: ‘long’

remove(self, ndarray index_list, bool input_sorted=0)

Remove the particles with indices in index_list.

reserve(self, long size)

Resizes the internal data to required size.

reset(self)

Reset the length of the array to 0.

resize(self, long size)

Resizes the array to the new size.

set_data(self, ndarray nparr)

Set data from the given numpy array.

If the numpy array is a reference to the numpy array maintained internally by this class, nothing is done. Otherwise, if the size of nparr matches this array, values are copied into the array maintained.

squeeze(self)

Release any unused memory.

update_min_max(self)

Update the min and max values of the array.

class pyzoltan.core.carray.BaseArrayIter

Bases: object

BaseArrayIter(BaseArray arr) Iteration object to support iteration over BaseArray.

next

x.next() -> the next value, or raise StopIteration

class pyzoltan.core.carray.DoubleArray

Bases: pyzoltan.core.carray.BaseArray

Represents an array of doubles

Mallocs a memory buffer of size (n*sizeof(double)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.

n : long
Length of the array.
data: pointer
Pointer to an integer array.
length: long
Size of the array itself.
alloc: long
Size of the data buffer allocated.
>>> x = DoubleArray()
>>> x.resize(5)
>>> x.set_data(np.arange(5))
>>> x[0]
0
>>> x = DoubleArray(5)
>>> xnp = x.get_npy_array()
>>> xnp[:] = np.arange(5)
>>> x[0], x[4]
(0.0, 4.0)
append(self, double value)

Appends value to the end of the array.

copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1)

Copy a subset of values from src to self.

start_index : long
the first index in dest that corresponds to the 0th index in source
end_index : long
the first index in dest from start_index that is not copied
copy_values(self, LongArray indices, BaseArray dest)

Copies values of indices in indices from self to dest.

No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.

extend(self, ndarray in_array)

Extend the array with data from in_array.

in_array : ndarray
a numpy array with data to be added to the current array.
  • accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
get(self, long idx) → double

Gets value stored at position idx.

get_c_type(self) → str

Return the c data type for this array as a string.

index(self, double value) → long

Returns the index at which value is in self, else -1.

maximum

maximum: ‘double’

minimum

minimum: ‘double’

remove(self, ndarray index_list, bool input_sorted=0)

Remove the particles with indices in index_list.

index_list : ndarray
a list of indices which should be removed.
input_sorted : bool
indicates if the input is sorted in ascending order. if not, the array will be sorted internally.
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
reserve(self, long size)

Resizes the internal data to size*sizeof(double) bytes.

reset(self)

Reset the length of the array to 0.

resize(self, long size)

Resizes internal data to size*sizeof(double) bytes and sets the length to the new size.

set(self, long idx, double value)

Sets location idx to value.

set_view(self, DoubleArray parent, long start, long end)

Create a view of a given a parent array from start to end.

Note that this excludes the end index.

parent : DoubleArray
The parent array of which this is a view.
start : long
The starting index to start the view from.
end : long
The ending index to end the view at, excludes the end itself.
squeeze(self)

Release any unused memory.

update_min_max(self)

Updates the min and max values of the array.

class pyzoltan.core.carray.FloatArray

Bases: pyzoltan.core.carray.BaseArray

Represents an array of floats

Mallocs a memory buffer of size (n*sizeof(float)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.

n : long
Length of the array.
data: pointer
Pointer to an integer array.
length: long
Size of the array itself.
alloc: long
Size of the data buffer allocated.
>>> x = FloatArray()
>>> x.resize(5)
>>> x.set_data(np.arange(5))
>>> x[0]
0
>>> x = FloatArray(5)
>>> xnp = x.get_npy_array()
>>> xnp[:] = np.arange(5)
>>> x[0], x[4]
(0.0, 4.0)
append(self, float value)

Appends value to the end of the array.

copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1)

Copy a subset of values from src to self.

start_index : long
the first index in dest that corresponds to the 0th index in source
end_index : long
the first index in dest from start_index that is not copied
copy_values(self, LongArray indices, BaseArray dest)

Copies values of indices in indices from self to dest.

No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.

extend(self, ndarray in_array)

Extend the array with data from in_array.

in_array : ndarray
a numpy array with data to be added to the current array.
  • accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
get(self, long idx) → float

Gets value stored at position idx.

get_c_type(self) → str

Return the c data type for this array as a string.

index(self, float value) → long

Returns the index at which value is in self, else -1.

maximum

maximum: ‘float’

minimum

minimum: ‘float’

remove(self, ndarray index_list, bool input_sorted=0)

Remove the particles with indices in index_list.

index_list : ndarray
a list of indices which should be removed.
input_sorted : bool
indicates if the input is sorted in ascending order. if not, the array will be sorted internally.
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
reserve(self, long size)

Resizes the internal data to size*sizeof(float) bytes.

reset(self)

Reset the length of the array to 0.

resize(self, long size)

Resizes internal data to size*sizeof(float) bytes and sets the length to the new size.

set(self, long idx, float value)

Sets location idx to value.

set_view(self, FloatArray parent, long start, long end)

Create a view of a given a parent array from start to end.

Note that this excludes the end index.

parent : FloatArray
The parent array of which this is a view.
start : long
The starting index to start the view from.
end : long
The ending index to end the view at, excludes the end itself.
squeeze(self)

Release any unused memory.

update_min_max(self)

Updates the min and max values of the array.

class pyzoltan.core.carray.IntArray

Bases: pyzoltan.core.carray.BaseArray

Represents an array of ints

Mallocs a memory buffer of size (n*sizeof(int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.

n : long
Length of the array.
data: pointer
Pointer to an integer array.
length: long
Size of the array itself.
alloc: long
Size of the data buffer allocated.
>>> x = IntArray()
>>> x.resize(5)
>>> x.set_data(np.arange(5))
>>> x[0]
0
>>> x = IntArray(5)
>>> xnp = x.get_npy_array()
>>> xnp[:] = np.arange(5)
>>> x[0], x[4]
(0.0, 4.0)
append(self, int value)

Appends value to the end of the array.

copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1)

Copy a subset of values from src to self.

start_index : long
the first index in dest that corresponds to the 0th index in source
end_index : long
the first index in dest from start_index that is not copied
copy_values(self, LongArray indices, BaseArray dest)

Copies values of indices in indices from self to dest.

No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.

extend(self, ndarray in_array)

Extend the array with data from in_array.

in_array : ndarray
a numpy array with data to be added to the current array.
  • accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
get(self, long idx) → int

Gets value stored at position idx.

get_c_type(self) → str

Return the c data type for this array as a string.

index(self, int value) → long

Returns the index at which value is in self, else -1.

maximum

maximum: ‘int’

minimum

minimum: ‘int’

remove(self, ndarray index_list, bool input_sorted=0)

Remove the particles with indices in index_list.

index_list : ndarray
a list of indices which should be removed.
input_sorted : bool
indicates if the input is sorted in ascending order. if not, the array will be sorted internally.
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
reserve(self, long size)

Resizes the internal data to size*sizeof(int) bytes.

reset(self)

Reset the length of the array to 0.

resize(self, long size)

Resizes internal data to size*sizeof(int) bytes and sets the length to the new size.

set(self, long idx, int value)

Sets location idx to value.

set_view(self, IntArray parent, long start, long end)

Create a view of a given a parent array from start to end.

Note that this excludes the end index.

parent : IntArray
The parent array of which this is a view.
start : long
The starting index to start the view from.
end : long
The ending index to end the view at, excludes the end itself.
squeeze(self)

Release any unused memory.

update_min_max(self)

Updates the min and max values of the array.

class pyzoltan.core.carray.LongArray

Bases: pyzoltan.core.carray.BaseArray

Represents an array of longs

Mallocs a memory buffer of size (n*sizeof(long)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.

n : long
Length of the array.
data: pointer
Pointer to an integer array.
length: long
Size of the array itself.
alloc: long
Size of the data buffer allocated.
>>> x = LongArray()
>>> x.resize(5)
>>> x.set_data(np.arange(5))
>>> x[0]
0
>>> x = LongArray(5)
>>> xnp = x.get_npy_array()
>>> xnp[:] = np.arange(5)
>>> x[0], x[4]
(0.0, 4.0)
append(self, long value)

Appends value to the end of the array.

copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1)

Copy a subset of values from src to self.

start_index : long
the first index in dest that corresponds to the 0th index in source
end_index : long
the first index in dest from start_index that is not copied
copy_values(self, LongArray indices, BaseArray dest)

Copies values of indices in indices from self to dest.

No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.

extend(self, ndarray in_array)

Extend the array with data from in_array.

in_array : ndarray
a numpy array with data to be added to the current array.
  • accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
get(self, long idx) → long

Gets value stored at position idx.

get_c_type(self) → str

Return the c data type for this array as a string.

index(self, long value) → long

Returns the index at which value is in self, else -1.

maximum

maximum: ‘long’

minimum

minimum: ‘long’

remove(self, ndarray index_list, bool input_sorted=0)

Remove the particles with indices in index_list.

index_list : ndarray
a list of indices which should be removed.
input_sorted : bool
indicates if the input is sorted in ascending order. if not, the array will be sorted internally.
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
reserve(self, long size)

Resizes the internal data to size*sizeof(long) bytes.

reset(self)

Reset the length of the array to 0.

resize(self, long size)

Resizes internal data to size*sizeof(long) bytes and sets the length to the new size.

set(self, long idx, long value)

Sets location idx to value.

set_view(self, LongArray parent, long start, long end)

Create a view of a given a parent array from start to end.

Note that this excludes the end index.

parent : LongArray
The parent array of which this is a view.
start : long
The starting index to start the view from.
end : long
The ending index to end the view at, excludes the end itself.
squeeze(self)

Release any unused memory.

update_min_max(self)

Updates the min and max values of the array.

class pyzoltan.core.carray.UIntArray

Bases: pyzoltan.core.carray.BaseArray

Represents an array of unsigned ints

Mallocs a memory buffer of size (n*sizeof(unsigned int)) and sets up the numpy array. The memory is aligned to 64 byte boundaries.

n : long
Length of the array.
data: pointer
Pointer to an integer array.
length: long
Size of the array itself.
alloc: long
Size of the data buffer allocated.
>>> x = UIntArray()
>>> x.resize(5)
>>> x.set_data(np.arange(5))
>>> x[0]
0
>>> x = UIntArray(5)
>>> xnp = x.get_npy_array()
>>> xnp[:] = np.arange(5)
>>> x[0], x[4]
(0.0, 4.0)
append(self, unsigned int value)

Appends value to the end of the array.

copy_subset(self, BaseArray source, long start_index=-1, long end_index=-1)

Copy a subset of values from src to self.

start_index : long
the first index in dest that corresponds to the 0th index in source
end_index : long
the first index in dest from start_index that is not copied
copy_values(self, LongArray indices, BaseArray dest)

Copies values of indices in indices from self to dest.

No size check if performed, we assume the dest to of proper size i.e. atleast as long as indices.

extend(self, ndarray in_array)

Extend the array with data from in_array.

in_array : ndarray
a numpy array with data to be added to the current array.
  • accessing the in_array using the indexing operation seems to be costly. Look at the annotated cython html file.
get(self, long idx) → unsigned int

Gets value stored at position idx.

get_c_type(self) → str

Return the c data type for this array as a string.

index(self, unsigned int value) → long

Returns the index at which value is in self, else -1.

maximum

maximum: ‘unsigned int’

minimum

minimum: ‘unsigned int’

remove(self, ndarray index_list, bool input_sorted=0)

Remove the particles with indices in index_list.

index_list : ndarray
a list of indices which should be removed.
input_sorted : bool
indicates if the input is sorted in ascending order. if not, the array will be sorted internally.
If the input indices are not sorted, sort them in ascending order. Starting with the last element in the index list, start replacing the element at the said index with the last element in the data and update the length of the array.
reserve(self, long size)

Resizes the internal data to size*sizeof(unsigned int) bytes.

reset(self)

Reset the length of the array to 0.

resize(self, long size)

Resizes internal data to size*sizeof(unsigned int) bytes and sets the length to the new size.

set(self, long idx, unsigned int value)

Sets location idx to value.

set_view(self, UIntArray parent, long start, long end)

Create a view of a given a parent array from start to end.

Note that this excludes the end index.

parent : UIntArray
The parent array of which this is a view.
start : long
The starting index to start the view from.
end : long
The ending index to end the view at, excludes the end itself.
squeeze(self)

Release any unused memory.

update_min_max(self)

Updates the min and max values of the array.

pyzoltan.core.carray.py_aligned(long n, int item_size) → long

Align n items each having size (in bytes) item_size to 64 bits and return the appropriate number of items that would be aligned to 64 bytes.

Previous topic

Module application

Next topic

Module controller

This Page