Why does write continuously leave 4K bytes in the buffer?
Apr 10
I have essentially the following code:
int fileWrite(int file, void * pBuffer, size_t size)
{
size_t bytesWritten = (size_t)write( file, pBuffer, size ) ;
if (bytesWritten != size)
{
return -1;
}
return 0;
}
It works if the size is 1GB, but when the size is ~2GB, it get 4K bytes left consistently. I can fix this by wrapping write in a loop and moving the buffer up but I'm curious as to why it is always failing.
For example if size is 2147483648, write only writes 2147479552, leaving 4096 unwritten. Why would this happen and is it correct to always wrap write in a loop?
linux1 answer
Accepted answer · original discussion
Apr 10
You can find the answer in man 2 write:
It is not an error if this number is smaller than the number of bytes requested; this may happen for example because the disk device was filled.
And from the write() man page description:
ssize_t write(int fd, const void *buf, size_t count);According to POSIX.1, if
countis greater thanSSIZE_MAX, the result is implementation-defined; see NOTES for the upper limit on Linux.NOTES
On Linux,
write()(and similar system calls) will transfer at most0x7ffff000(2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.)
3 question comments
Use comments to ask for clarification. Post a solution as an answer.
Apr 11
SSIZE_MAX restriction. The write() spec says it is under no obligation to write the full buffer, even if it almost always does. The loop-less code in the question is a bug.