↩ 017. malloc(0) & realloc(…, 0) ≠ 0 ↩

Thu, 17 Oct 2024 03:42:06 +0200; updated Thu, 17 Oct 2024 12:15:42 +0200, Fri, 18 Oct 2024 18:50:12 +0200, Wed, 30 Oct 2024 18:53:30 +0100

C and POSIX say that

If the size of the space requested is 0, the behavior is implementation-defined: either a null pointer shall be returned, or the behavior shall be as if the size were some non-zero value […].

one has to wonder if this is real.

# UNIX Programmer's Manual, Fifth Edition

doesn't have malloc.

# UNIX Programmer's Manual, Sixth Edition

doesn't have malloc either, but has a generic alloc(3) with alloc()/free() (generic in contrast to most programs shipping an alloc() that boils down to sbrk(1024), or just sbrking directly; this implementation is not widely used). The code and text of this implementation get expanded into the malloc(3) suite in v7, and, for the purposes of this document, they behave equivalently (except that alloc Returns −1 if there is no available core.). ./man/man3/alloc.3, ./s4/alloc.s

# UNIX™ Time-Sharing System: UNIX Programmer's Manual, Seventh Edition

has a malloc(3) with malloc and realloc:

Malloc, realloc and calloc return a null pointer (0) if there is no available memory or if the arena has been detectably corrupted […].

where malloc allocates (nbytes+WORD+WORD-1)/WORD (≥1) words, where one is eaten for the allocation size, and realloc(b, nbytes) is free(b); malloc(nbytes); memcpy() (in so many words), with memcpy() skipped if the just-freed block was malloced. usr/man/man3/malloc.3, usr/src/libc/gen/malloc.c

# The bsd

The csrg Archives, CD-ROM 1: Berkeley Systems 1978-1986, CD-ROM 2: Berkeley Systems 1987-1993, and CD-ROM 3: Final Berkeley Releases say that everything up to 4.1bsd uses the v7 allocator. 3bsd/usr/src/libc/gen/malloc.c, 4.0/usr/src/libc/gen/malloc.c.c, 4.1/usr/src/libc/gen/malloc.c 3bsd/usr/man/man3/malloc.3, 4.0/usr/man/man3/malloc.3, 4.1/usr/man/man3/malloc.3

# 4.2bsd

has a new allocator: 4.2/usr/src/lib/libc/gen/malloc.c, 4.2/usr/man/man3/malloc.3

/*
 * nextf[i] is the pointer to the next free block of size 2^(i+3).  The
 * smallest allocatable block is 8 bytes.  The overhead information
 * precedes the data area returned to the user.
 */
static union overhead *nextf[30];

but otherwise implements the same methodology by adding sizeof (union overhead) to the allocation size before rounding up and bucketing. realloc(cp, nbytes) similarly returns cp if nbytes ≤ the old allocation size, else malloc(nbytes); memcpy().

# 4.3bsd

goes about calculating it differently, but small allocations remain equivalent to allocating 8 bytes. 4.3/usr/src/lib/libc/gen/malloc.c, 4.3/usr/man/man3/malloc.3

# 4.4bsd

updates the malloc(3) RETURN VALUES wording to

The malloc() function returns a pointer to the allocated space if successful; otherwise a null pointer is returned.

and cites conformance with C89.

realloc(3) is split off with text lifted directly from C89, including

If size is zero and ptr is not a null pointer, the object it points to is freed.

This is not true and the implementation doesn't change. Thus, the claimed conformance with C89 is also not true. 4.3tahoe/usr/src/lib/libc/gen/malloc.c, 4.3reno/usr/src/lib/libc/stdlib/malloc.c, 4.4BSD-Lite1/usr/src/lib/libc/stdlib/malloc.c, 4.4BSD-Lite2/usr/src/lib/libc/stdlib/malloc.c 4.3tahoe/usr/src/man/man3/malloc.3, 4.3reno/usr/src/lib/libc/stdlib/malloc.3, 4.4BSD-Lite1/usr/src/lib/libc/stdlib/malloc.3, 4.4BSD-Lite2/usr/src/lib/libc/stdlib/malloc.3, 4.4BSD-Lite1/usr/src/lib/libc/stdlib/realloc.3, 4.4BSD-Lite2/usr/src/lib/libc/stdlib/realloc.3

# Unix User's Manual, Release 3.0

uses the v7 allocator and malloc(3C). p. 524, src/lib/libc/vax/gen/malloc.c

# Unix∗ System User's Manual, System V
UNIX™ System V—Release 2.0 Programmer Reference Manual
UNIX® System V Programmer's Reference Manual [SysVr3]

None of these make any meaningful changes to the allocator (in the domain of small allocations) or to malloc(3C). p. 585, sysv-pdp11_usr-src/lib/libc/port/gen/malloc.c p. MALLOC(3C) 1, sysv-pdp11_usr-src/lib/libc/port/gen/malloc.c p. 310 (331), 301/usr/src/lib/libc/port/gen/malloc.c, 31/usr/src/lib/libc/port/gen/malloc.c

SysVr2 and SysVr3 both additionally ship malloc(3X) (the "fast main memory allocator"), however, as -lmalloc. Within the area this document surveys, the manual is functionally equivalent, sans the WARNINGS including

Undocumented features of malloc(3C) have not been duplicated.

and, indeed, this implementation's malloc(nbytes) says if (nbytes == 0) return NULL; and realloc(ptr, size) says if(size == 0) return NULL;.

If one recalls that the DIAGNOSTICS still say that

malloc, realloc and calloc return a NULL pointer if there is not enough available memory.

and mention no other case, and the DESCRIPTION explicitly defining

The argument to free is a pointer to a block previously allocated by malloc

when free(0) doesn't work (dereferences negative pointer, so segfaults, probably?), then it's difficult to conclude that this isn't an "undocumented feature", but "just a bug in malloc(3X) actually". p. MALLOC(3X) 1, src/lib/libmalloc/malloc.c p. 469 (490), 301/usr/src/lib/libmalloc/malloc.c

This is also the birth of the DIAGNOSTICS'

When realloc returns NULL, the block pointed to by ptr is left intact.

# UNIX® System V Release 4 Programmer's Reference Manual

has a new implementation, in which malloc first tries to re-use the last-freed block, then allocations smaller than 5 ints use a special small-block queue, wherein

	/* want to return a unique pointer on malloc(0) */
	if(size == 0)
		size = WORDSIZE;

(i.e. sizeof(int)). This spills to realloc where

	/* special cases involving small blocks */
	if(size < MINSIZE || SIZE(tp) < MINSIZE)
		goto call_malloc;

where the old algorithm is reimplemented as malloc(); memcpy(); free(). p. 507, ATT-SYSVr4/lib/libc/port/gen/malloc.c, ATT-SYSVr4/lib/libc/port/gen/mallint.h

malloc(3X) is unchanged but -lmalloc free accepts NULLs now. And realloc(ptr, 0) frees ptr. As it should do, since NULL is this implementation's spelling for a zero-sized allocation. Except that it now no longer leaves ptr unchanged when returning NULL. p. 682, ATT-SYSVr4/lib/libmalloc/malloc.c

# SunOS 3.4

uses the SunOS allocator which mallocs and reallocs at least sizeof(uint) (+ once again for the accounting block). This allocator is dated 1986, so there's no reason to suspect (and no way to verify) there was another one that interceded 'twixt the v7 one. sunos_3.4_src/lib/libc/gen/malloc.h, sunos_3.4_src/lib/libc/gen/mallint.h

# SunOS 4.1.3

adjusts the minima to sizeof(double) on sparc or sizeof(uint) on m68k. SunOS 4.1.3 sunsrc.tar, SunOS 4.1.3 sunsrc/413/lib/libc/gen/common/malloc.c, SunOS 4.1.3 sunsrc/413/lib/libc/gen/common/mallint.h

It also ships libxpg which includes SysVr2-style -lmalloc SunOS 4.1.3 sunsrc.tar, SunOS 4.1.3 sunsrc/413/xpglib/malloc.c

# Solaris 8

ships both implementations functionally unchanged since SysVr4. There's no reason to suspect this has ever wavered in SysV SunOSes prior (or for some time after 2000).

# svid

has no extant copy.

# System V Interface Definition, Issue 2

includes SysVr2 malloc(3X) mostly-verbatim as malloc(BA_OS) in Volume 1. (The API is adapted minimally to match SysVr3's.) The RETURN VALUE is extended to say for the first time that

The functions malloc, realloc, and calloc return a NULL pointer if nbytes is 0 or if there is not enough available memory.

Illuminatingly, RETURN VALUE proclaims that

In System V Release 2.0, the developer can control whether the contents of the freed space are destroyed or left undisturbed […]. In System V Release 1.0, the contents are left undisturbed.

manufacturing a reality in which the undocumented behaviour of the -lmalloc opt-in extension is actually the universal baseline. p. 103 (114)

# System V Interface Definition, Third Edition

The functions malloc(), realloc(), and calloc() return a null pointer if there is not enough available memory. If the size of the space requested is zero, the behavior is implementation defined; the value returned will be either a null pointer or unique pointer. When realloc() returns NULL, the block pointed to by ptr is left intact.

This is supposed to match SysVr4, and neither of SysVr4's mallocs document which behaviour they pick. It also mandates SysVr2 (SysVr3)'s -lmalloc's leaking behaviour, which SysVr4 has fixed, making it non-compliant. p. 6-76

# X/Open Portability Guide, Part II: The X/Open System V Specification

includes SysVr2's malloc(3X) as malloc(3X) (it says it's via SVID malloc(OS) but it doesn't define malloc(0) = NULL). It notes that an older (smaller) form may also exist, but correctly recognises that it is the responsibility of the application developers to ensure that the appropriate version is linked into their applications. p. 103

# xpg4

is updated to indicate what will be returned if size is 0: NULL or a unique freeable. p. 378 (406) & 468 (495)

This is naught but a mild editorialisation of C89 wording, and later xpgssusesposixes use later Cs.

# Conclusions

Within the area this document surveys (at&t unix, the bsd, SunOS and 'til-Y2K Solaris, pre-xpg svid, xpg and its descendants):

1. every realloc(…, 0) ever behaved congruently with its malloc
2. every default malloc(0) ever behaved as-if malloc(n); n > 0
3. this is documented. just not, like, as explicitly as you'd write it today, but there hadn't been a confounding implementation that did the opposite, so it never crossed anyone's mind probably
4. this was clearly understood as relied-on behaviour, as it's retained across multiple re-implementations
5. the implementation that breaks this — SysVr2 -lmalloc — also understands this since (a) it has to implement the opposite behaviour explicitly, and (b) includes a nod to its removal in malloc(3X) (the other behaviour this could be referring to is retained — realloc can deal with twits who reallocate free blocks)
6. this difference is not documented
7. the author needs to opt into this implementation explicitly (and is — ever so tacitly — warned about this) so this is basically fine, if sub-optimal
8. AT&T issues the svid with the malloc(3X) extension presented as the universal baseline interface, with its undocumented features, incl. malloc(0) = NULL, expressly defined as standard
9. this contrapuncts the implied documentation in every other implementation, which is now awfully weak
10. the xpg manages to miss this, but it's now impossible to reconcile the behaviour mandated by the svid that no-one was actually getting with the unstated behaviour experienced universally, except by saying "yeah fuckin do whatever" a decade later

# Verdict (this section is not normative)

not real.

# Appendix — C89 realloc(…, 0)

If […] the second argument is 0, then the call frees the memory pointed to by the first argument[…]; this specification is consistent with the policy of not allowing zero-size objects.

but is not consistent with any extant implementation, obviously. p. 156 (169) p. 102 (342)

This made realloc(…, 0) unusable (since this returns NULL — an error — you're gonna free the pointer (double-free) or use it with its original size (use-after-free), like with normal realloc), and is potentially-inconsistent with malloc(0) ≠ 0, so C23 made it UB.

The rationale paper for C89 and the path realloc() takes through later standards is summarised here.

Nit-pick? Correction? Improvement? Annoying? Cute? Anything? Mail, post, or open!