In 2008, I began my career as a paid writer, having written much before but never paid and rarely recognized for it. This was because I was a software engineer being measured by my coding skills, not my command of the written word (no wonder none of the other engineers were interested in writing docs). I was happy to do the writing for the team, but it slowly became clear I’d prefer to be recognized and compensated for my writing abilities.
My first gig was covering a vast swath of the Microsoft Win32 API, the largest category being the “File I/O” class of APIs, which is the largest and best example of that work. Luckily these docs still exist out there, for example the CreateFile() function:
I assume this is largely unchanged all these years because at that point in time the Win32 API was showing its age and Microsoft was moving to the then-new Azure platform for most of their new content and tech work. At that time when I was writing Win32 content it was long of tooth and becoming very much a niche technology. I notice the rendering has changed a lot, however, and not for the better, as the tables are no longer very nice to read. Kind of a sad thing to see really. Here’s the content closer to when I was working on it:
Below is another example of that period of my work, copied here in case the above links stop working (from the web archive, and the links embedded here should take you to the web archive version, all of which is my work as well):
This topic covers the various considerations for application control of file buffering, also known as unbuffered file input/output (I/O). File buffering is usually handled by the system behind the scenes and is considered part of file caching within the Windows operating system unless otherwise specified. Although the terms caching and buffering are sometimes used interchangeably, this topic uses the term buffering specifically in the context of explaining how to interact with data that is not being cached (buffered) by the system, where it is otherwise largely out of the direct control of user-mode applications.
When opening or creating a file with the CreateFile function, the FILE_FLAG_NO_BUFFERING flag can be specified to disable system caching of data being read from or written to the file. Although this gives complete and direct control over data I/O buffering, in the case of files and similar devices there are data alignment requirements that must be considered.
Note This alignment information applies to I/O on devices such as files that support seeking and the concept of file position pointers (or offsets). For nonseeking devices, such as named pipes or communications devices, turning off buffering may not require any particular alignment. Any limitations or efficiencies that may be gained by alignment in that case are dependent on the underlying technology.
In a simple example, the application would open a file for write access with the FILE_FLAG_NO_BUFFERING flag and then perform a call to the WriteFile function using a data buffer defined within the application. This local buffer is, in these circumstances, effectively the only file buffer that exists for this operation. Because of physical disk layout, file system storage layout, and system-level file pointer position tracking, this write operation will fail unless the locally-defined data buffers meet certain alignment criteria, discussed in the following section.
Note Discussion of caching does not consider any hardware caching on the physical disk itself, which is not guaranteed to be within the direct control of the system in any case. This has no effect on the requirements specified in this topic.
For more information on how FILE_FLAG_NO_BUFFERING interacts with other cache-related flags, see CreateFile.
Alignment and File Access Requirements
As previously discussed, an application must meet certain requirements when working with files opened with FILE_FLAG_NO_BUFFERING. The following specifics apply:
- File access sizes, including the optional file offset in the OVERLAPPED structure, if specified, must be for a number of bytes that is an integer multiple of the volume sector size. For example, if the sector size is 512 bytes, an application can request reads and writes of 512, 1,024, 1,536, or 2,048 bytes, but not of 335, 981, or 7,171 bytes.
- File access buffer addresses for read and write operations should be sector-aligned, which means aligned on addresses in memory that are integer multiples of the volume sector size. Depending on the disk, this requirement may not be enforced.
An application can determine a volume sector size by calling the GetDiskFreeSpace function.
Because buffer addresses for read and write operations must be sector-aligned, the application must have direct control of how these buffers are allocated. One way to sector-align buffers is to use the VirtualAlloc function to allocate the buffers. Consider the following:
- VirtualAlloc allocates memory that is aligned on addresses that are integer multiples of the system’s page size. Page size is 4,096 bytes on x64 and x86 or 8,192 bytes for the Intel Itanium processor family. For additional information, see the GetSystemInfo function.
- Sector size is typically 512 to 4,096 bytes for direct-access storage devices (hard drives) and 2,048 bytes for CD-ROMs.
- Both page and sector sizes are powers of 2.
Therefore, in most situations, page-aligned memory will also be sector-aligned, because the case where the sector size is larger than the page size is rare.Another way to obtain manually-aligned memory buffers is to use the _aligned_malloc function from the C Run-Time library. For an example of how to manually control buffer alignment, see the C++ language code example in the Example Code section of WriteFile.