Built-in Utilities¶
Automatic Initialization¶
The module pycuda.autoinit
, when imported, automatically performs
all the steps necessary to get CUDA ready for submission of compute kernels.
It uses pycuda.tools.make_default_context()
to create a compute context.
- pycuda.autoinit.device¶
An instance of
pycuda.driver.Device
that was used for automatic initialization.
- pycuda.autoinit.context¶
A default-constructed instance of
pycuda.driver.Context
ondevice
. This context is created by callingpycuda.tools.make_default_context()
.
The module pycuda.autoprimaryctx
is similar to pycuda.autoinit
,
except that it retains the device primary context instead of creating a new
context in pycuda.tools.make_default_context()
. Notably, it also
has device
and context
attributes.
Choice of Device¶
- pycuda.tools.make_default_context()¶
Return a
pycuda.driver.Context
instance chosen according to the following rules:If the environment variable
CUDA_DEVICE
is set, its integer value is used as the device number.If the file
.cuda-device
is present in the user’s home directory, the integer value of its contents is used as the device number.Otherwise, all available CUDA devices are tried in a round-robin fashion.
An error is raised if this does not lead to a usable context.
- pycuda.tools.get_default_device(default=0)¶
Deprecated. Use
make_default_context()
.Return a
pycuda.driver.Device
instance chosen according to the following rules:If the environment variable
CUDA_DEVICE
is set, its integer value is used as the device number.If the file
.cuda-device
is present in the user’s home directory, the integer value of its contents is used as the device number.Otherwise, default is used as the device number.
Kernel Caching¶
- pycuda.tools.context_dependent_memoize(func)¶
This decorator caches the result of the decorated function, if a subsequent occurs in the same
pycuda.driver.Context
. This is useful for caching of kernels.
- pycuda.tools.clear_context_caches()¶
Empties all context-dependent memoization caches. Also releases all held reference contexts. If it is important to you that the program detaches from its context, you might need to call this function to free all remaining references to your context.
Testing¶
- pycuda.tools.mark_cuda_test(func)¶
This function, meant for use with
py.test
, will mark func with a “cuda” tag and make sure it has a CUDA context available when invoked.
Device Metadata and Occupancy¶
- class pycuda.tools.DeviceData(dev=None)¶
Gives access to more information on a device than is available through
pycuda.driver.Device.get_attribute()
. If dev is None, it defaults to the device returned bypycuda.driver.Context.get_device()
.- max_threads¶
- warp_size¶
- warps_per_mp¶
- thread_blocks_per_mp¶
- registers¶
- smem_granularity¶
The number of threads that participate in banked, simultaneous access to shared memory.
- smem_alloc_granularity¶
The size of the smallest possible (non-empty) shared memory allocation.
- align_bytes(word_size=4)¶
The distance between global memory base addresses that allow accesses of word-size word_size bytes to get coalesced.
- align(bytes, word_size=4)¶
Round up bytes to the next alignment boundary as given by
align_bytes()
.
- align_words(word_size)¶
Return self.align_bytes(word_size)/word_size, while checking that the division did not yield a remainder.
- align_dtype(elements, dtype_size)¶
Round up elements to the next alignment boundary as given by
align_bytes()
, where each element is assumed to be dtype_size bytes large.
- static make_valid_tex_channel_count(size)¶
Round up size to a valid texture channel count.
- class pycuda.tools.OccupancyRecord(devdata, threads, shared_mem=0, registers=0)¶
Calculate occupancy for a given kernel workload characterized by
thread count of threads
shared memory use of shared_mem bytes
register use of registers 32-bit registers
- tb_per_mp¶
How many thread blocks execute on each multiprocessor.
- limited_by¶
What
tb_per_mp
is limited by. One of “device”, “warps”, “regs”, “smem”.
- warps_per_mp¶
How many warps execute on each multiprocessor.
- occupancy¶
A float value between 0 and 1 indicating how much of each multiprocessor’s scheduling capability is occupied by the kernel.
Memory Pools¶
The functions pycuda.driver.mem_alloc()
and
pycuda.driver.pagelocked_empty()
can consume a fairly large amount of
processing time if they are invoked very frequently. For example, code based on
pycuda.gpuarray.GPUArray
can easily run into this issue because a
fresh memory area is allocated for each intermediate result. Memory pools are a
remedy for this problem based on the observation that often many of the block
allocations are of the same sizes as previously used ones.
Then, instead of fully returning the memory to the system and incurring the associated reallocation overhead, the pool holds on to the memory and uses it to satisfy future allocations of similarly-sized blocks. The pool reacts appropriately to out-of-memory conditions as long as all memory allocations are made through it. Allocations performed from outside of the pool may run into spurious out-of-memory conditions due to the pool owning much or all of the available memory.
Device-based Memory Pool¶
- class pycuda.tools.PooledDeviceAllocation¶
An object representing a
DeviceMemoryPool
-based allocation of linear device memory. Once this object is deleted, its associated device memory is freed.PooledDeviceAllocation
instances can be cast toint
(andlong
), yielding the starting address of the device memory allocated.- free()¶
Explicitly return the memory held by self to the associated memory pool.
- __len__()¶
Return the size of the allocated memory in bytes.
- class pycuda.tools.DeviceMemoryPool¶
A memory pool for linear device memory as allocated using
pycuda.driver.mem_alloc()
. (see Memory Pools)- held_blocks¶
The number of unused blocks being held by this pool.
- active_blocks¶
The number of blocks in active use that have been allocated through this pool.
- managed_bytes¶
“Managed” memory is “active” and “held” memory.
- active_bytes¶
“Active” bytes are bytes under the control of the application. This may be smaller than the actual allocated size reflected in
managed_bytes
.
- allocate(size)¶
Return a
PooledDeviceAllocation
of size bytes.
- free_held()¶
Free all unused memory that the pool is currently holding.
- stop_holding()¶
Instruct the memory to start immediately freeing memory returned to it, instead of holding it for future allocations. Implicitly calls
free_held()
. This is useful as a cleanup action when a memory pool falls out of use.
Memory Pool for pagelocked memory¶
- class pycuda.tools.PooledHostAllocation¶
An object representing a
PageLockedMemoryPool
-based allocation of linear device memory. Once this object is deleted, its associated device memory is freed.- free()¶
Explicitly return the memory held by self to the associated memory pool.
- __len__()¶
Return the size of the allocated memory in bytes.
- class pycuda.tools.PageLockedAllocator(flags=0)¶
Specifies the set of
pycuda.driver.host_alloc_flags
used in its associatedPageLockedMemoryPool
.
- class pycuda.tools.PageLockedMemoryPool(allocator=PageLockedAllocator())¶
A memory pool for pagelocked host memory as allocated using
pycuda.driver.pagelocked_empty()
. (see Memory Pools)- held_blocks¶
The number of unused blocks being held by this pool.
- active_blocks¶
The number of blocks in active use that have been allocated through this pool.
- allocate(shape, dtype, order='C')¶
Return an uninitialized (“empty”)
numpy.ndarray
with the given shape, dtype, and order. This array will be backed by aPooledHostAllocation
, which can be found as the.base
attribute of the array.
- free_held()¶
Free all unused memory that the pool is currently holding.
- stop_holding()¶
Instruct the memory to start immediately freeing memory returned to it, instead of holding it for future allocations. Implicitly calls
free_held()
. This is useful as a cleanup action when a memory pool falls out of use.