Question: What are the advantages/disadvantages of having a block of file bigger than the size of the block in memory? What is buffering, why is it needed, and how does it work?
Answer: Blocks and buffers are related to one another but are actually different things and serve different purposes.
A block is a collection of records that are referenced by your program as separate records but which are treated by the system as being one big record. Each record that is saved in your file takes up so much space for the record data and so much space as overhead to allow the system to recognise the record as a record. By blocking a number of (logical) records together into one bigger (physical) record we reduce the space taken up by the record overhead. By selecting a block size that is as close to an exact fraction of the size of one track on a disk as we can, we reduce the wasted space within the file to a minimum and fit more data into the same sized file.
Another advantage that using larger blocks has is that the block acts in part as a buffer. All of the logical records that make up the block are all loaded into memory at the same time.
The different parts of your computer system run at different speeds. The computer's main memory is relatively fast, the disk drives are relatively slow. By using a buffer we can load larger sections of the file from the hard disk into memory in advance of when the program needs to read the records. Because the data that is to be read is in memory it can be accessed much faster than if the program had to wait for a read of the hard disk. You can also use an output buffer where written records are accumulated to be written in bulk to the hard disk at a later time. These speed up the computer because it reads and writes to the buffers at times when there is not such a high load on the processor from running your program. This helps to spread the load on the processor.
The one thing that you can't do is to make the block size used by your file bigger than the amount of main memory that you've got for your program. You can't even come close because you've got to fit not only the program and all of the data fields that it uses but in most cases each file is configured to use at least two buffers and each buffer needs to be able to hold at least one block of data. All of this needs to fit into the available space at the same time.
This article written by Stephen Chapman, Felgall Pty Ltd.