Selective collect

[tannewt]

In CircuitPython we're working on supporting our new Fruit Jam board. It uses HSTX to generate DVI video and PSRAM for a larger VM heap (via split heaps). One issue with our demos is that GC collect time gets long enough to chirp audio and stutter video. The heap grows significantly because we store bitmaps and audio buffers in the VM heap.

An idea I had to speed this up was to annotate which VM allocations need to be "collected" (aka scanning for heap pointers.) I've implemented this by adding another table like the one for finalisers. GC times dropped from 80+ms to ~25ms in our test example.

One question I struggled with is how to update the dozen or so memory allocation APIs. Some vary to allocate a struct or a variable length struct. Others vary depending on whether they should raise an error on failure, have a finaliser or zero memory. Now I'm adding a fourth boolean of whether the memory should be collected. What should the APIs be? Can we simplify things by splitting the size computation from the allocation policy?

Existing APIs

micropython/py/misc.h

Lines 70 to 115 in f498a16

	/** memory allocation ******************************************/
	// TODO make a lazy m_renew that can increase by a smaller amount than requested (but by at least 1 more element)
	#define m_new(type, num) ((type *)(m_malloc(sizeof(type) * (num))))
	#define m_new_maybe(type, num) ((type *)(m_malloc_maybe(sizeof(type) * (num))))
	#define m_new0(type, num) ((type *)(m_malloc0(sizeof(type) * (num))))
	#define m_new_obj(type) (m_new(type, 1))
	#define m_new_obj_maybe(type) (m_new_maybe(type, 1))
	#define m_new_obj_var(obj_type, var_field, var_type, var_num) ((obj_type *)m_malloc(offsetof(obj_type, var_field) + sizeof(var_type) * (var_num)))
	#define m_new_obj_var0(obj_type, var_field, var_type, var_num) ((obj_type *)m_malloc0(offsetof(obj_type, var_field) + sizeof(var_type) * (var_num)))
	#define m_new_obj_var_maybe(obj_type, var_field, var_type, var_num) ((obj_type *)m_malloc_maybe(offsetof(obj_type, var_field) + sizeof(var_type) * (var_num)))
	#if MICROPY_MALLOC_USES_ALLOCATED_SIZE
	#define m_renew(type, ptr, old_num, new_num) ((type *)(m_realloc((ptr), sizeof(type) * (old_num), sizeof(type) * (new_num))))
	#define m_renew_maybe(type, ptr, old_num, new_num, allow_move) ((type *)(m_realloc_maybe((ptr), sizeof(type) * (old_num), sizeof(type) * (new_num), (allow_move))))
	#define m_del(type, ptr, num) m_free(ptr, sizeof(type) * (num))
	#define m_del_var(obj_type, var_field, var_type, var_num, ptr) (m_free(ptr, offsetof(obj_type, var_field) + sizeof(var_type) * (var_num)))
	#else
	#define m_renew(type, ptr, old_num, new_num) ((type *)(m_realloc((ptr), sizeof(type) * (new_num))))
	#define m_renew_maybe(type, ptr, old_num, new_num, allow_move) ((type *)(m_realloc_maybe((ptr), sizeof(type) * (new_num), (allow_move))))
	#define m_del(type, ptr, num) ((void)(num), m_free(ptr))
	#define m_del_var(obj_type, var_field, var_type, var_num, ptr) ((void)(var_num), m_free(ptr))
	#endif
	#define m_del_obj(type, ptr) (m_del(type, ptr, 1))
	void *m_malloc(size_t num_bytes);
	void *m_malloc_maybe(size_t num_bytes);
	void *m_malloc_with_finaliser(size_t num_bytes);
	void *m_malloc0(size_t num_bytes);
	#if MICROPY_MALLOC_USES_ALLOCATED_SIZE
	void *m_realloc(void *ptr, size_t old_num_bytes, size_t new_num_bytes);
	void *m_realloc_maybe(void *ptr, size_t old_num_bytes, size_t new_num_bytes, bool allow_move);
	void m_free(void *ptr, size_t num_bytes);
	#else
	void *m_realloc(void *ptr, size_t new_num_bytes);
	void *m_realloc_maybe(void *ptr, size_t new_num_bytes, bool allow_move);
	void m_free(void *ptr);
	#endif
	NORETURN void m_malloc_fail(size_t num_bytes);
	#if MICROPY_TRACKED_ALLOC
	// These alloc/free functions track the pointers in a linked list so the GC does not reclaim
	// them. They can be used by code that requires traditional C malloc/free semantics.
	void *m_tracked_calloc(size_t nmemb, size_t size);
	void m_tracked_free(void *ptr_in);
	#endif

micropython/py/obj.h

Lines 922 to 936 in f498a16

	// Helper versions of m_new_obj when you need to immediately set base.type.
	// Implementing this as a call rather than inline saves 8 bytes per usage.
	#define mp_obj_malloc(struct_type, obj_type) ((struct_type *)mp_obj_malloc_helper(sizeof(struct_type), obj_type))
	#define mp_obj_malloc_var(struct_type, var_field, var_type, var_num, obj_type) ((struct_type *)mp_obj_malloc_helper(offsetof(struct_type, var_field) + sizeof(var_type) * (var_num), obj_type))
	void *mp_obj_malloc_helper(size_t num_bytes, const mp_obj_type_t *type);
	// Object allocation macros for allocating objects that have a finaliser.
	#if MICROPY_ENABLE_FINALISER
	#define mp_obj_malloc_with_finaliser(struct_type, obj_type) ((struct_type *)mp_obj_malloc_with_finaliser_helper(sizeof(struct_type), obj_type))
	#define mp_obj_malloc_var_with_finaliser(struct_type, var_type, var_num, obj_type) ((struct_type *)mp_obj_malloc_with_finaliser_helper(sizeof(struct_type) + sizeof(var_type) * (var_num), obj_type))
	void *mp_obj_malloc_with_finaliser_helper(size_t num_bytes, const mp_obj_type_t *type);
	#else
	#define mp_obj_malloc_with_finaliser(struct_type, obj_type) mp_obj_malloc(struct_type, obj_type)
	#define mp_obj_malloc_var_with_finaliser(struct_type, var_type, var_num, obj_type) mp_obj_malloc_var(struct_type, var_type, var_num, obj_type)
	#endif

Another approach would be to focus on the type of allocations being made. mp_obj_malloc is a good example of this. It makes it clear that the allocated memory is a Python object. Therefore it could have finaliser always set and always be collected.

How could we unify these allocation APIs to make them easier to maintain? Any ideas?

[jonnor]

Hi @tannewt - this is interesting work, especially as someone that does a bunch of work on C modules etc. I do not have any opinion on the API design at the moment, I would like to understand the intended usage a bit better first. What would be the typical cases where one would use this functionality, and how to decide if to use it or not. I guess any large allocation where the code that allocates knows there will not be any MP objects inside? And I guess the default behavior would have to be "do collection?", to maintain compatibility?

[tannewt]

I'm being more aggressive with it at the moment and making collection opt-in. I've made m_malloc and some m_new allocate without collection. mp_obj_malloc is collect by default because we know they are Python objects and likely have to be collected. (They are also likely small.) I've also add a m_malloc_items for things like tuple and list that have lists of Python object references.

You only need collection if you store a pointer to other VM heap allocated memory.

I mean to look at memory analysis tools (like valgrind) to see if we can use it to validate allocations.