HDFS Block abstraction
HDFS Block abstraction :
- HDFS block size is usually 64MB-128MB and unlike other filesystems, a file smaller than the block size does not occupy the complete block sizeβs worth of memory.
- The block size is kept so large so that less time is made doing disk seeks as compared to the data transfer rate.
- Why do we need block abstraction :
- Files can be bigger than individual disks.
- Filesystem metadata does not need to be associated with each and every block.
- Simplifies storage management - Easy to figure out the number of blocks which can be stored on each disk.
- Fault tolerance and storage replication can be easily done on a per-block basis.
Labels: Big Data, Big Data Unit-3
Discussion & Queries (<$I18NNumComments$>):
<$CommentPager$>
- <$I18NAtCommentTimeWithPermalink$>, <$I18NCommentAuthorSaid$>
<$CommentPager$>
<$BlogCommentBody$>
<$BlogCommentDeleteIcon$>