Volley源码解析（三） 有缓存机制的情况走缓存请求的源码分析

Volley源码解析（三） 有缓存机制的情况走缓存请求的源码分析

Volley之所以高效好用，一个在于请求重试策略，一个就在于请求结果缓存。

通过上一篇文章http://www.cnblogs.com/zharma/p/8338456.html

可以看到网络请求的流程逻辑分支是如何执行的。

接下来这篇文章就从具有请求缓存的流程去分析源码是采取的何种缓存策略。

继续以最简单的例子为起点分析：

final TextView mTextView = (TextView) findViewById(R.id.text);
...

// Instantiate the RequestQueue. 初始化请求队列
RequestQueue queue = Volley.newRequestQueue(this);
String url ="http://www.google.com";

// Request a string response from the provided URL.构造请求对象
StringRequest stringRequest = new StringRequest(Request.Method.GET, url,
new Response.Listener<String>() {
@Override
public void onResponse(String response) {
// Display the first 500 characters of the response string.
mTextView.setText("Response is: "+ response.substring(0,500));
//UI线程
}
}, new Response.ErrorListener() {
@Override
public void onErrorResponse(VolleyError error) {
mTextView.setText("That didn't work!");
//UI线程
}
});
// Add the request to the RequestQueue.
queue.add(stringRequest);

初始化请求队列指的注意的是这段代码：

private static RequestQueue newRequestQueue(Context context, Network network) {
File cacheDir = new File(context.getCacheDir(), DEFAULT_CACHE_DIR);
RequestQueue queue = new RequestQueue(new DiskBasedCache(cacheDir), network);
queue.start();
return queue;
}

这里new DiskBasedCache(cacheDir)是初始化了一个基于硬盘存储的缓存。进去看一下：

/** Default maximum disk usage in bytes. */
private static final int DEFAULT_DISK_USAGE_BYTES = 5 * 1024 * 1024;

**
* Constructs an instance of the DiskBasedCache at the specified directory using
* the default maximum cache size of 5MB.
* @param rootDirectory The root directory of the cache.
*/
public DiskBasedCache(File rootDirectory) {
this(rootDirectory, DEFAULT_DISK_USAGE_BYTES);
}

可以看到默认的缓存空间是5MB的本地缓存空间。当缓存超过这个值的时候会自动清除最近最少用到的缓存文件，节约空间。当然这个空间大小事可以定制的。

接下去就是RQ（代表RequestQueue，之后用RQ代替）的start方法：

/**
* Starts the dispatchers in this queue.
*/
public void start() {
stop();  // Make sure any currently running dispatchers are stopped.
// Create the cache dispatcher and start it.
mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery);
mCacheDispatcher.start();

// Create network dispatchers (and corresponding threads) up to the pool size.
for (int i = 0; i < mDispatchers.length; i++) {
NetworkDispatcher networkDispatcher = new NetworkDispatcher(mNetworkQueue, mNetwork,
mCache, mDelivery);
mDispatchers[i] = networkDispatcher;
networkDispatcher.start();
}
}

可以看到启动了一个CachaeDispatcher线程。接下去跟进这个线程的run方法：

CacheDispatcher.java

@Override
public void run() {
if (DEBUG) VolleyLog.v("start new dispatcher");
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);

// Make a blocking call to initialize the cache.
mCache.initialize();//初始化缓存变量

while (true) {
try {
processRequest();
} catch (InterruptedException e) {
// We may have been interrupted because it was time to quit.
if (mQuit) {
return;
}
}
}
}

分析mCache.initialize();过程

DiskBasedCache.java

/**
* Initializes the DiskBasedCache by scanning for all files currently in the
* specified root directory. Creates the root directory if necessary.
*/
@Override
public synchronized void initialize() {
if (!mRootDirectory.exists()) {//通过RQ初始化的过程知道rootDirectory一定存在
if (!mRootDirectory.mkdirs()) {
VolleyLog.e("Unable to create cache dir %s", mRootDirectory.getAbsolutePath());
}
return;
}
File[] files = mRootDirectory.listFiles();//列出volley文件夹下的所有文件
if (files == null) {//没有文件表示没有缓存
return;
}
for (File file : files) {//遍历每个文件
try {
long entrySize = file.length();//获取文件的大小
CountingInputStream cis = new CountingInputStream(
new BufferedInputStream(createInputStream(file)), entrySize);//构造输入流，将文件信息读取到内存
try {
CacheHeader entry = CacheHeader.readHeader(cis);//读取缓存头的信息
// NOTE: When this entry was put, its size was recorded as data.length, but
// when the entry is initialized below, its size is recorded as file.length()
entry.size = entrySize;
putEntry(entry.key, entry);
} finally {
// Any IOException thrown here is handled by the below catch block by design.
//noinspection ThrowFromFinallyBlock
cis.close();
}
} catch (IOException e) {
//noinspection ResultOfMethodCallIgnored
file.delete();
}
}
}

下面是利用输入流度字节的操作：

/**
* Reads the header from a CountingInputStream and returns a CacheHeader object.
* @param is The InputStream to read from.
* @throws IOException if fails to read header
*/
static CacheHeader readHeader(CountingInputStream is) throws IOException {
int magic = readInt(is);
if (magic != CACHE_MAGIC) {//开始的int是一个魔数CACHE_MAGIC = 0x20150306，确保是缓存文件
// don't bother deleting, it'll get pruned eventually
throw new IOException();
}
String key = readString(is);//再读取8个字节转换出String 类型的key
String etag = readString(is);//再读取8个字节转换出String 类型的 etag
long serverDate = readLong(is);//再读取8个字节转换出long 类型的serverDate
long lastModified = readLong(is);//再读取8个字节转换出long 类型的lastModified
long ttl = readLong(is);//再读取8个字节转换出long 类型的ttl
long softTtl = readLong(is);//再读取8个字节转换出long 类型的softTtl
List<Header> allResponseHeaders = readHeaderList(is);//先读4个字节转换int表示多少个header,
//后续根据个数去依次每次读取8个字节转换String去表示key，读取8个字节转换String去表示value，然后组装成一个header
//最后把header都装进list

return new CacheHeader(//最后根据读取出来的所有自己数据转换成一个内存中的缓存头
key, etag, serverDate, lastModified, ttl, softTtl, allResponseHeaders);
}

static int readInt(InputStream is) throws IOException {
int n = 0;
n |= (read(is) << 0);//读取下一个字节
n |= (read(is) << 8);///读取下一个字节
n |= (read(is) << 16);//读取下一个字节
n |= (read(is) << 24);//读取下一个字节
return n;//读取出来的是4个字节，正好是int类型的4字节数字
}

/**
* Simple wrapper around {@link InputStream#read()} that throws EOFException
* instead of returning -1.
*/
private static int read(InputStream is) throws IOException {
int b = is.read();
if (b == -1) {
throw new EOFException();
}
return b;
}

这段代码的本质就是按照指定的规律一个字节一个字节的读取出来。组成缓存头

得到CacheHeader以后，接下来回到DiskBaseCache.java initialize方法中：

...
for (File file : files) {//遍历每个文件
try {
long entrySize = file.length();//获取文件的大小
CountingInputStream cis = new CountingInputStream(
new BufferedInputStream(createInputStream(file)), entrySize);//构造输入流，将文件信息读取到内存
try {
CacheHeader entry = CacheHeader.readHeader(cis);//读取缓存头的信息
// NOTE: When this entry was put, its size was recorded as data.length, but
// when the entry is initialized below, its size is recorded as file.length()
entry.size = entrySize;//把文件的大小录入进去
putEntry(entry.key, entry);
} finally {
// Any IOException thrown here is handled by the below catch block by design.
//noinspection ThrowFromFinallyBlock
cis.close();
}
} catch (IOException e) {
//noinspection ResultOfMethodCallIgnored
file.delete();
}
}
...

然后利用CacheHeader对象从文件读取出来的key作为索引，CacheHeader对象本身最为value存进内存的Map中

/** Map of the Key, CacheHeader pairs */
private final Map<String, CacheHeader> mEntries =
new LinkedHashMap<String, CacheHeader>(16, .75f, true);

/**
* Puts the entry with the specified key into the cache.
* @param key The key to identify the entry by.
* @param entry The entry to cache.
*/
private void putEntry(String key, CacheHeader entry) {
if (!mEntries.containsKey(key)) {
mTotalSize += entry.size;
} else {
CacheHeader oldEntry = mEntries.get(key);
mTotalSize += (entry.size - oldEntry.size);
}
mEntries.put(key, entry);
}

总结：至此硬盘中的所有缓存文件的重要值都读取出来了，并且在内存中以Map的形式存在。

分析processRequest();过程

CacheDispatcher.java

@Override
public void run() {
if (DEBUG) VolleyLog.v("start new dispatcher");
Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);

// Make a blocking call to initialize the cache.
mCache.initialize();

while (true) {
try {
processRequest();
} catch (InterruptedException e) {
// We may have been interrupted because it was time to quit.
if (mQuit) {
return;
}
}
}
}

注意到这个线程也是一个死循环。除非发生线程中断异常

进去processRequest方法啊：

private void processRequest() throws InterruptedException {
// Get a request from the cache triage queue, blocking until
// at least one is available.
final Request<?> request = mCacheQueue.take();//这是一个BlockingQueue当元素为空时会阻塞
request.addMarker("cache-queue-take");

// If the request has been canceled, don't bother dispatching it.
if (request.isCanceled()) {
request.finish("cache-discard-canceled");
return;
}

// Attempt to retrieve this item from cache. 利用url从Map中拿entry对象
Cache.Entry entry = mCache.get(request.getCacheKey());//request.getCacheKey()拿到key,这个key默认是请求的url
if (entry == null) {//如果null,表示没有缓存过
request.addMarker("cache-miss");
// Cache miss; send off to the network dispatcher.
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {//
mNetworkQueue.put(request);
}
return;
}

// If it is completely expired, just send it to the network.
if (entry.isExpired()) {//缓存过期，需要网络获取资源
request.addMarker("cache-hit-expired");
request.setCacheEntry(entry);
if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
mNetworkQueue.put(request);
}
return;
}

// We have a cache hit; parse its data for delivery back to the request.
request.addMarker("cache-hit");//命中了缓存资源
Response<?> response = request.parseNetworkResponse(
new NetworkResponse(entry.data, entry.responseHeaders));//利用缓存中的entry.data数据构造
//NetworkResponse对象.这里是资源解析
request.addMarker("cache-hit-parsed");

if (!entry.refreshNeeded()) {//资源新鲜则直接返回
// Completely unexpired cache hit. Just deliver the response.
mDelivery.postResponse(request, response);
} else {//不新鲜则需要做新鲜度验证
// Soft-expired cache hit. We can deliver the cached response,
// but we need to also send the request to the network for
// refreshing.
request.addMarker("cache-hit-refresh-needed");
request.setCacheEntry(entry);
// Mark the response as intermediate.
response.intermediate = true;

if (!mWaitingRequestManager.maybeAddToWaitingRequests(request)) {
// Post the intermediate response back to the user and have
// the delivery then forward the request along to the network.
mDelivery.postResponse(request, response, new Runnable() {
@Override
public void run() {
try {
mNetworkQueue.put(request);
} catch (InterruptedException e) {
// Restore the interrupted status
Thread.currentThread().interrupt();
}
}
});
} else {
// request has been added to list of waiting requests
// to receive the network response from the first request once it returns.
mDelivery.postResponse(request, response);
}
}
}

分析mWaitingRequestManager.maybeAddToWaitingRequests(request)方法

这个WaitiRequestManager是CahcemManager的静态内部类

private static class WaitingRequestManager implements Request.NetworkRequestCompleteListener {
...省略内容

/**
* Staging area for requests that already have a duplicate request in flight.
*
* <ul>
*     <li>containsKey(cacheKey) indicates that there is a request in flight for the given cache
*          key.</li>
*     <li>get(cacheKey) returns waiting requests for the given cache key. The in flight request
*          is <em>not</em> contained in that list. Is null if no requests are staged.</li>
* </ul>
*/
private final Map<String, List<Request<?>>> mWaitingRequests = new HashMap<>();

/**
* For cacheable requests, if a request for the same cache key is already in flight,
* add it to a queue to wait for that in-flight request to finish.
* @return whether the request was queued. If false, we should continue issuing the request
* over the network. If true, we should put the request on hold to be processed when
* the in-flight request finishes.
*/
private synchronized boolean maybeAddToWaitingRequests(Request<?> request) {
String cacheKey = request.getCacheKey();
// Insert request into stage if there's already a request with the same cache key
// in flight.
if (mWaitingRequests.containsKey(cacheKey)) {
// There is already a request in flight. Queue up.
List<Request<?>> stagedRequests = mWaitingRequests.get(cacheKey);
if (stagedRequests == null) {
stagedRequests = new ArrayList<Request<?>>();
}
request.addMarker("waiting-for-response");
stagedRequests.add(request);
mWaitingRequests.put(cacheKey, stagedRequests);
if (VolleyLog.DEBUG) {
VolleyLog.d("Request for cacheKey=%s is in flight, putting on hold.", cacheKey);
}
return true;
} else {
// Insert 'null' queue for this cacheKey, indicating there is now a request in
// flight.
mWaitingRequests.put(cacheKey, null);
request.setNetworkRequestCompleteListener(this);
if (VolleyLog.DEBUG) {
VolleyLog.d("new request, sending to network %s", cacheKey);
}
return false;
}
}
...
}

mWaitingRequests是一个Map类型。他存在的目的就是为了确保下面这样的场景可以正常运行：

有两个相同的请求AB，那么他们的request.getCacheKey();值一定是一样的，现在假如A正在请求中，B开始了请求。A会走else的流程，然后交给网络线程去处理。而B再进入这个方法的时候mWaitingRequests.containsKey(cacheKey)就是true了 。这个时候表示已经有一个同样的请求发出去了，所以没有必要再次的发送。

假如stagedRequests是空的，那说明A请求还在请求中，然后把这个B请求就需要添加到mWaitingRequests中。

这里就是获取缓存中的资源的全过程，那么缓存资源是如何被存起来的呢？这个当然实在网络请求的流程中出现的了！

现在进入网络请求流程：

NetworkDispatcher.java

private void processRequest() throws InterruptedException {
// Take a request from the queue.
Request<?> request = mQueue.take();

long startTimeMs = SystemClock.elapsedRealtime();
try {
request.addMarker("network-queue-take");

// If the request was cancelled already, do not perform the
// network request.
if (request.isCanceled()) {
request.finish("network-discard-cancelled");
request.notifyListenerResponseNotUsable();
return;
}

addTrafficStatsTag(request);

// Perform the network request.
NetworkResponse networkResponse = mNetwork.performRequest(request);
request.addMarker("network-http-complete");

// If the server returned 304 AND we delivered a response already,
// we're done -- don't deliver a second identical response.
if (networkResponse.notModified && request.hasHadResponseDelivered()) {
request.finish("not-modified");
request.notifyListenerResponseNotUsable();
return;
}

// Parse the response here on the worker thread.
Response<?> response = request.parseNetworkResponse(networkResponse);
request.addMarker("network-parse-complete");

// Write to cache if applicable.//这里就是最核心的一环，这里根据url为key把返回结果的entry存入Map
// TODO: Only update cache metadata instead of entire record for 304s.
if (request.shouldCache() && response.cacheEntry != null) {
mCache.put(request.getCacheKey(), response.cacheEntry);
request.addMarker("network-cache-written");
}

// Post the response back.
request.markDelivered();
mDelivery.postResponse(request, response);
request.notifyListenerResponseReceived(response);
} catch (VolleyError volleyError) {
volleyError.setNetworkTimeMs(SystemClock.elapsedRealtime() - startTimeMs);
parseAndDeliverNetworkError(request, volleyError);
request.notifyListenerResponseNotUsable();
} catch (Exception e) {
VolleyLog.e(e, "Unhandled exception %s", e.toString());
VolleyError volleyError = new VolleyError(e);
volleyError.setNetworkTimeMs(SystemClock.elapsedRealtime() - startTimeMs);
mDelivery.postError(request, volleyError);
request.notifyListenerResponseNotUsable();
}
}

DiskBasedCache.java

/**
* Puts the entry with the specified key into the cache.
*/
@Override
public synchronized void put(String key, Entry entry) {
pruneIfNeeded(entry.data.length);
File file = getFileForKey(key);
try {
BufferedOutputStream fos = new BufferedOutputStream(createOutputStream(file));
CacheHeader e = new CacheHeader(key, entry);//根据返回数据构建返回头
boolean success = e.writeHeader(fos);//写入文件
if (!success) {
fos.close();
VolleyLog.d("Failed to write header for %s", file.getAbsolutePath());
throw new IOException();
}
fos.write(entry.data);//把网络请求输入写入文件
fos.close();
putEntry(key, e);//把头保存在内存中
return;
} catch (IOException e) {
}
boolean deleted = file.delete();
if (!deleted) {
VolleyLog.d("Could not clean up file %s", file.getAbsolutePath());
}
}

总结：到这里，网络请求会把请求结果保存到本地。然后把请求头的若干信息保存在内存中。当下次请求触发，用到本地缓存的内容時，会依据url这个key去内存的Map去找。如果代表请求头存在，那么就去本地硬盘找相应的缓存资源。然后把文件中读取的data网络请求返回的结果和内存中的请求头资源结合起来一并返回。

后面从子线程返回UI线程的操作和网络请求是一致的。