The basic idea of a terminator canary is that when an attacker attempts a buffer overflow, they're forced to over-write the canary value. The program can then detect that the canary has changed value and take appropriate actions.
The value 0 is somewhat special in programming: many languages use it as an end-of-text marker. If an attacker is trying to overflow a text buffer, the use of a 0 as a terminator canary means the attack will fail: in order to keep the canary from changing, they need to include a 0 in the oversized input in a location that will cause almost all of the excess input to be ignored.
This has a problem, though: if the input is handled as binary data rather than as text, the fact that the canary has a known, fixed value means the attacker can simply over-write the canary with itself, producing an undetectable overflow.
Edit: code examples
/* This reads a length-tagged packet of up to 16 bytes length from an input stream.
*
* Note that since the programmer forgot to check the length of the input,
* a packet of more than 20 bytes (give or take alignment) will overflow onto
* sensitive parts of the stack. If bytes 17 through 20 of the outsized packet
* are 0s, this overflow won't be detected.
*/
size_t readPacket(char *stream)
{
size_t length;
char packet[16];
uint32_t canary = 0;
length = (size_t)(*stream++);
memcpy(packet, stream, length);
processPacket(packet, length);
if(canary != 0)
exit(0);
return length;
}
/* This reads a username from an input stream.
*
* Note that since the programmer used strcpy() rather than strlcpy(), a
* string of more than 20 bytes (give or take alignment) will overflow onto
* sensitive parts of the stack. However, since strcpy() stops copying once
* it encounters a byte with the value 0, in order for overflow to reach a
* sensitive part of the stack, it must change the value of the canary. If
* this happens, exit() is called and the changed stack is never used.
*/
size_t readName(char *stream)
{
char userName[16];
uint32_t canary = 0;
strcpy(userName, stream);
processUserName(userName);
if(canary != 0)
exit(0);
return strlen(userName);
}
In a real-life example, the canaries and canary-checking code may be inserted automatically by the compiler rather than manually by the programmer.
0bytes followed by the value0". Binary data is handled as "a sequence of bytes of a certain length", or for input, it's sometimes "a sequence of bytes extending to the end of the file". Binary data doesn't have a special value indicating the end of the data, while text does. – Mark Oct 03 '14 at 20:31memcpyormemset, or simply writing 'char' values at each offset of a region one by one, rather thanstr*functions which would only modify the memory region until a 0 is encountered. – Steve Dodier-Lazaro Oct 03 '14 at 20:36canarywill come afterbufferin memory and be overwritten by an overflow, but it's not guaranteed. That's one of the reasons the canaries should be inserted by the compiler rather than the programmer. – Mark Oct 04 '14 at 21:12clangordered them "canary, packet, length" (the reverse of the order they're written in), whilegccordered the variables "canary, length, packet" (a totally different order). Additionally,gccdid not include any padding, whileclangpadded "packet" out to 16 bytes. – Mark Feb 20 '17 at 19:05