is dfsclient caches the data block to local disk before writing?
2014-04-02 17:29
513 查看
Question:
Is staging still used in hdfs when writing the data? This doubt arose when I was going through the hdfs documents. ref :http://hadoop.apache.org/hdfs/docs/current/hdfs_design.html#Staging I believe dfsclient does not cache the datablock to local fs (as the document says) but it does streaming of 64KB packets to the datanode and caches the packets of current block only in memory via dataqueue and ackqueue.Is the document needs to be corrected or my understanding is wrong?
Answer:
DFS client does not write the data to local disk first. Instead, it streams data directly to the datanodes in the write pipeline. I will update the document.
相关文章推荐
- MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error
- MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error
- Unable to connect to data source (DSN: shangjihuiclient; Network Address: ; Port Number: 53397). Cannot connect to TimesTen Server. Verify that the TimesTen Server is running or verify that your TCP PORT is set correctly.
- #Redis Error #MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details a
- 使用OleDb写入Excel异常,"字段太小而不能接受所要添加的数据的数量" "the field is too small to accept the amount of data "
- 解决 svn: E155021: This client is too old to work with the working copy
- Can't use Subversion command line client: svn Probably the path to Subversion executable is wrong. F
- This function has none of Deterministic,no sql,or reads sql data in its declaration and binary logging is enabled(you *might* want to use the less safe log_bin_trust_function_creators variable
- Please make sure the TESSDATA_PREFIX environment variable is set to the parent directory of your "te
- Can't use Subversion command line client: svn Probably the path to Subversion executable is wrong. F
- LINQ to SQL and WCF - Sharing types, subverting the DataContext on the client side(转)
- IE 8 HTML Parsing Error: Unable to modify the parent container element before the child element is c
- the application is not licensed to create or modify schema for this type of data
- Troubleshooting the "device is busy" Error Attempting to umount a Disk
- IE 8 HTML Parsing Error: Unable to modify the parent container element before the child element is closed (KB927917)
- The application is not licensed to modify or create schema for this type of data 解决办法
- Can't use Subversion command line client***Probably the path to Subversion executable is wrong. Fix
- [WPF] How to bind to data when the DataContext is not inherited
- Please make sure the TESSDATA_PREFIX environment variable is set to the parent directory of your "te
- Access restriction: The type FtpClient is not accessible due to restriction on required library