当前位置: 首页 > 工具软件 > Form Slice > 使用案例 >

Memory slice解释

田志尚
2023-12-01


/**

 * SECTION:memory_slices
 * @title: Memory Slices
 * @short_description: efficient way to allocate groups of equal-sized
 *     chunks of memory
  分配大小相同的内存块组的有效方式
 *
 * Memory slices provide a space-efficient and multi-processing scalable
 * way to allocate equal-sized pieces of memory, just like the original
 * #GMemChunks (from GLib 2.8), while avoiding their excessive
 * memory-waste, scalability and performance problems.
  内存slice提供一种具有空间高效性和多处理并且可伸缩的方式来分配大小相同的块内存的方式, 就像原生的GMemChunks(从版本GLib 2.8起)一样, 同时又可以避免过度内存浪费,可伸缩性和性能问题。
 *
 * To achieve these goals, the slice allocator uses a sophisticated,
 * layered design that has been inspired by Bonwick's slab allocator
 * ([Bonwick94](http://citeseer.ist.psu.edu/bonwick94slab.html)
 * Jeff Bonwick, The slab allocator: An object-caching kernel
 * memory allocator. USENIX 1994, and
 * [Bonwick01](http://citeseer.ist.psu.edu/bonwick01magazines.html)
 * Bonwick and Jonathan Adams, Magazines and vmem: Extending the
 * slab allocator to many cpu's and arbitrary resources. USENIX 2001)
 *
  为了实现这些目标,slice分配器使用复杂的分层设计,灵感来自
  Bonwick slab分配器((Bonwick94)(http://citeseer.ist.psu.edu/bonwick94slab.html)
  Jeff Bonwick, slab allocator:一An object-caching kernel memory allocator. USENIX 1994 和(Bonwick01)(http://citeseer.ist.psu.ed bonwick01magazines.html)
  Bonwick Jonathan Adams, Magazines and vmem: Extending the slab allocator to many cpu's and arbitrary resources. USENIX 2001
*
 * It uses posix_memalign() to optimize allocations of many equally-sized
 * chunks, and has per-thread free lists (the so-called magazine layer)
 * to quickly satisfy allocation requests of already known structure sizes.
 * This is accompanied by extra caching logic to keep freed memory around
 * for some time before returning it to the system. Memory that is unused
 * due to alignment constraints is used for cache colorization (random
 * distribution of chunk addresses) to improve CPU cache utilization. The
 * caching layer of the slice allocator adapts itself to high lock contention
 * to improve scalability.
 *
  它使用posix_memalign()来优化许多个同等大小的内存块的分配, 并且具有每个线程独占的自由列表(所谓的Magazine层)以快速满足已知结构大小的分配请求。
  这个需要额外的缓存逻辑来将自由内存在交还给系统之前保存一段时间。由于缓存一致性约束而未使用的内存被用于缓存着色(随机的块分配地址)来提高CPU缓存利用率。
  slice分配器的缓存层会调整自身到适高锁争用来提高可伸缩性。


 * The slice allocator can allocate blocks as small as two pointers, and
 * unlike malloc(), it does not reserve extra space per block. For large block
 * sizes, g_slice_new() and g_slice_alloc() will automatically delegate to the
 * system malloc() implementation. For newly written code it is recommended
 * to use the new `g_slice` API instead of g_malloc() and
 * friends, as long as objects are not resized during their lifetime and the
 * object size used at allocation time is still available when freeing.
 *
  slice allocator最小可以分配两个指针大小的内存块, 并且和与malloc()不同, 它不再每一块都保留多余的空间。对于尺寸大的块,g_slice_new()和g_slice_alloc()会将自动使用系统malloc()实现。对于新写代码建议使用新的“g_slice”API而不是g_malloc()及其同类, 只要对象在其生命周期中不调整大小,并且分配时用到的对象大小在释放时依然可用。


 * Here is an example for using the slice allocator:
 * |[<!-- language="C" --> 
 * gchar *mem[10000];
 * gint i;
 *
 * // Allocate 10000 blocks.
 * for (i = 0; i < 10000; i++)
 *   {
 *     mem[i] = g_slice_alloc (50);
 *
 *     // Fill in the memory with some junk.
 *     for (j = 0; j < 50; j++)
 *       mem[i][j] = i * j;
 *   }
 *
 * // Now free all of the blocks.
 * for (i = 0; i < 10000; i++)
 *   g_slice_free1 (50, mem[i]);
 * ]|
 *
 * And here is an example for using the using the slice allocator
 * with data structures:
 * |[<!-- language="C" --> 
 * GRealArray *array;
 *
 * // Allocate one block, using the g_slice_new() macro.
 * array = g_slice_new (GRealArray);


 * // We can now use array just like a normal pointer to a structure.
 * array->data            = NULL;
 * array->len             = 0;
 * array->alloc           = 0;
 * array->zero_terminated = (zero_terminated ? 1 : 0);
 * array->clear           = (clear ? 1 : 0);
 * array->elt_size        = elt_size;
 *
 * // We can free the block, so it can be reused.
 * g_slice_free (GRealArray, array);
 * ]|
 */


/* the GSlice allocator is split up into 4 layers, roughly modelled after the slab
 * allocator and magazine extensions as outlined in:
 * + [Bonwick94] Jeff Bonwick, The slab allocator: An object-caching kernel
 *   memory allocator. USENIX 1994, http://citeseer.ist.psu.edu/bonwick94slab.html
 * + [Bonwick01] Bonwick and Jonathan Adams, Magazines and vmem: Extending the
 *   slab allocator to many cpu's and arbitrary resources.
 *   USENIX 2001, http://citeseer.ist.psu.edu/bonwick01magazines.html
  GSlice allocator分四层。
  
 * the layers are:
 * - the thread magazines. for each (aligned) chunk size, a magazine (a list)
 *   of recently freed and soon to be allocated chunks is maintained per thread.
 *   this way, most alloc/free requests can be quickly satisfied from per-thread
 *   free lists which only require one g_private_get() call to retrive the
 *   thread handle.
  线程magazine。对每一个尺寸而言,线程会单独维护刚刚释放并且可能马上会被分配的内存块。通过这种方式,大多数分配/释放(alloc/free)请求会被线程专有的列表满足,只需要通过g_private_get()来获取线程句柄。


 * - the magazine cache. allocating and freeing chunks to/from threads only
 *   occours at magazine sizes from a global depot of magazines. the depot
 *   maintaines a 15 second working set of allocated magazines, so full
 *   magazines are not allocated and released too often.
 *   the chunk size dependent magazine sizes automatically adapt (within limits,
 *   see [3]) to lock contention to properly scale performance across a variety
 *   of SMP systems.
  Magazine缓冲区层。向线程分配和释放magazine大小的内存时,只能向一个全局的magazine区进行。已分配magazine,这个区会维护15秒,这样就不需要频繁的分配和释放magazine。依赖于chunk大小的magazine大小会自己调整(有限制)


 * - the slab allocator. this allocator allocates slabs (blocks of memory) close
 *   to the system page size or multiples thereof which have to be page aligned.
 *   the blocks are divided into smaller chunks which are used to satisfy
 *   allocations from the upper layers. the space provided by the reminder of
 *   the chunk size division is used for cache colorization (random distribution
 *   of chunk addresses) to improve processor cache utilization. multiple slabs
 *   with the same chunk size are kept in a partially sorted ring to allow O(1)
 *   freeing and allocation of chunks (as long as the allocation of an entirely
 *   new slab can be avoided).
  Slab allocator层。这个allocator分配slab的时候,会以系统页大小或者其倍数,因此它肯定是页对齐的。这些内存块会根据上层的分配需求被分成小的块。尺寸间隔的大小被用在缓存着色(随机分布块地址)中以提升处理器的缓存能力。具有相同大小的slab会被放在一个部分排序环中,用来实现内存块0(1)的释放和分配。


 * - the page allocator. on most modern systems, posix_memalign(3) or
 *   memalign(3) should be available, so this is used to allocate blocks with
 *   system page size based alignments and sizes or multiples thereof.
 *   if no memalign variant is provided, valloc() is used instead and
 *   block sizes are limited to the system page size (no multiples thereof).
 *   as a fallback, on system without even valloc(), a malloc(3)-based page
 *   allocator with alloc-only behaviour is used.
  page分配器。大多数现代的系统中,posix_memalign(3)或者memalign(3)会被提供,这样就可以分配基于系统页对齐和页尺寸及其倍数大小的内存块。如果memalign类不提供,那么会使用valloc(),这样内存块的大小会被限制在系统页的大小(没有倍数)。作为后备,如果系统连valloc()也不提供,只能使用基于malloc(3)的分配器,这样就只能提供分配功能(按:没有对齐)。


 *
 * NOTES:
 * [1] some systems memalign(3) implementations may rely on boundary tagging for
 *     the handed out memory chunks. to avoid excessive page-wise fragmentation,
 *     we reserve 2 * sizeof (void*) per block size for the systems memalign(3),
 *     specified in NATIVE_MALLOC_PADDING.
  一些系统memalign(3)实现可能会依赖于边界标记发放内存块。为了避免过度page-wise碎片, 如在NATIVE_MALLOC_PADDING中指定的一样,每一块我们都为系统memalign(3)保留2 * sizeof(void *)大小的空间。




 * [2] using the slab allocator alone already provides for a fast and efficient
 *     allocator, it doesn't properly scale beyond single-threaded uses though.
 *     also, the slab allocator implements eager free(3)-ing, i.e. does not
 *     provide any form of caching or working set maintenance. so if used alone,
 *     it's vulnerable to trashing for sequences of balanced (alloc, free) pairs
 *     at certain thresholds.
    使用slab分配器就已经提供了一个快速和有效的分配器, 虽然它不适合超出单线程使用。另外,slab分配器正在实现eager free(3), 例如,不提供任何形式的缓存或工作集维护。所以,如果单独使用, 在特定的条件下,是容易破坏(alloc,free)序列对平衡。




 * [3] magazine sizes are bound by an implementation specific minimum size and
 *     a chunk size specific maximum to limit magazine storage sizes to roughly
 *     16KB.
  具体实现会限制magazine的最小尺寸,而内存块的最大尺寸又限制magazine存储最大尺寸为16 kb。


* [4] allocating ca. 8 chunks per block/page keeps a good balance between
 *     external and internal fragmentation (<= 12.5%). [Bonwick94]
  每个block/page分配8块可以在内部和外部的碎片之间取得良好的平衡(< = 12.5%)。(Bonwick94)(按:这应该是个经验值,具体含义不是很明白哎)

*/


 类似资料: