How Can I Maximize Binary File Write Speed in C ?

Enhance Data Transfer Speed in Binary File Writes in C

Introduction:

Writing large buffers to binary files efficiently is often crucial for performance-sensitive applications. In this article, we will explore the question of how to optimize this process in C .

Initial Approach and Bottlenecks:

The provided code appears to underperform compared to file copy operations. Potential bottlenecks include:

• Use of fstream instead of FILE* (standard I/O) for binary writes
• Inefficient write loop with frequent system calls

Optimized Solution:

A significantly faster approach is to use FILE* and fwrite:

#include <stdio.h>

const unsigned long long size = 8ULL*1024ULL*1024ULL;
unsigned long long a[size];

int main()
{
    FILE* pFile;
    pFile = fopen("file.binary", "wb");
    for (unsigned long long j = 0; j < 1024; ++j)
    {
        // Data generation
        fwrite(a, 1, size*sizeof(unsigned long long), pFile);
    }
    fclose(pFile);
    return 0;
}

This code achieved write speeds of approximately 220MB/s, approaching the limits of the SSD.

Further Refinements:

To improve code efficiency further, we can implement the following:

• Disable stream synchronization with std::ios_base::sync_with_stdio(false)
• Vectorized data generation using std::vector and std::iota, std::shuffle, and std::random_device
• Time measurements using std::chrono

Benchmarking and Results:

Benchmarking the code on different platforms (Laptop and Desktop) with varying buffer sizes (1MB-4GB) revealed:

• On both platforms, fstream outperformed FILE* when using small buffers (1MB) but fell behind for larger buffers.
• fstream was able to fully utilize SSD bandwidth for large buffers, eliminating the performance advantage of FILE*.

Conclusion:

In summary, to efficiently write large buffers to binary files in C , consider:

• Using std::fstream for small buffers
• Using FILE* and fwrite for large buffers
• Disabling std::ios stream synchronization
• Optimizing data generation and benchmarking

The above is the detailed content of How Can I Maximize Binary File Write Speed in C ?. For more information, please follow other related articles on the PHP Chinese website!