HDFS Block abstraction

HDFS Block abstraction :

 

• HDFS block size is usually 64MB-128MB and unlike other filesystems, a file smaller than the block size does not occupy the complete block size’s worth of memory.
• The block size is kept so large so that less time is made doing disk seeks as compared to the data transfer rate.
• Why do we need block abstraction :

1. Files can be bigger than individual disks.
2. Filesystem metadata does not need to be associated with each and every block.
3. Simplifies storage management - Easy to figure out the number of blocks which can be stored on each disk.
4. Fault tolerance and storage replication can be easily done on a per-block basis.