Android MediaScanner MediaProvider流程
Android MediaScanner MediaProvider流程
- 源码解析
- 链接: [github链接,欢迎交流](https://github.com/Tecinno).
- 时序图
- MediaSacannerReeiver.java
- MediaScannerService.java
- MediaProvider.java
- MediaScanner.java
- MediaScanner.cpp
- StagefrightMediaScanner.cpp
源码解析
链接: github链接,欢迎交流.
时序图
时序图是根据我自己修剪过的框架画的,有些地方跟源码不一样,但是大体是差不多的。
链接:
MediaScanner时序图.
MediaSacannerReeiver.java
接收android.intent.action.MEDIA_MOUNTED广播启动MediaScannerService
// An highlighted block private void scan(Context context, String volume) { Bundle args = new Bundle(); args.putString("volume", volume); context.startService( new Intent(context, MediaScannerService.class).putExtras(args)); }
MediaScannerService.java
第一次被启动走onCreate,将自己的线程启动
// A code block @Override public void onCreate() { PowerManager pm = (PowerManager)getSystemService(Context.POWER_SERVICE); mWakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK, TAG); StorageManager storageManager = (StorageManager)getSystemService(Context.STORAGE_SERVICE); mExternalStoragePaths = storageManager.getVolumePaths(); // Start up the thread running the service. Note that we create a // separate thread because the service normally runs in the process's // main thread, which we don't want to block. Thread thr = new Thread(null, this, "MediaScannerService"); thr.start(); }
第二次启动走onStartCommand,从广播里面获取信息发送给mServiceHandler
// An highlighted block @Override public int onStartCommand(Intent intent, int flags, int startId) { while (mServiceHandler == null) { synchronized (this) { try { wait(100); } catch (InterruptedException e) { } } } if (intent == null) { Log.e(TAG, "Intent is null in onStartCommand: ", new NullPointerException()); return Service.START_NOT_STICKY; } Message msg = mServiceHandler.obtainMessage(); msg.arg1 = startId; msg.obj = intent.getExtras(); mServiceHandler.sendMessage(msg); // Try again later if we are killed before we can finish scanning. return Service.START_REDELIVER_INTENT; }
mServiceHandler 解析路径和volume信息然后开始扫描:
// An highlighted block private final class ServiceHandler extends Handler { @Override public void handleMessage(Message msg) { ... scan(directories, volume); ... } };
scan(String[] directories, String volumeName)方法中先 openDatabase(volumeName);发消息给MedaiProvider,让数据库先准备好,然后MediaScanner scanner = new MediaScanner(this, volumeName),scanner.scanDirectories(directories);MediaScanner.java开始扫描。
MediaProvider.java
MediaScanner.java
具体流程可以在上面提供的时序图查看,这里主要讲解几个重要的方法:
1、prescan
prescan主要是做老数据删除,先从数据库将数据读取出来,然后判断文件存不存在,不存在就删除。
// An highlighted block private void prescan(String filePath, boolean prescanFiles) throws RemoteException { Cursor c = null; String where = null; String[] selectionArgs = null; mPlayLists.clear();//清除列表,这个列表后面用来保存每个媒体问的信息:id,修改时间等 if (filePath != null) {//获取单个数据 // query for only one file where = MediaStore.Files.FileColumns._ID + ">?" + " AND " + Files.FileColumns.DATA + "=?"; selectionArgs = new String[] { "", filePath }; } else {//从数据库files表获取所有数据 where = MediaStore.Files.FileColumns._ID + ">?"; selectionArgs = new String[] { "" }; } mDefaultRingtoneSet = wasRingtoneAlreadySet(Settings.System.RINGTONE); mDefaultNotificationSet = wasRingtoneAlreadySet(Settings.System.NOTIFICATION_SOUND); mDefaultAlarmSet = wasRingtoneAlreadySet(Settings.System.ALARM_ALERT); // Tell the provider to not delete the file. // If the file is truly gone the delete is unnecessary, and we want to avoid // accidentally deleting files that are really there (this may happen if the // filesystem is mounted and unmounted while the scanner is running). Uri.Builder builder = mFilesUri.buildUpon(); builder.appendQueryParameter(MediaStore.PARAM_DELETE_DATA, "false"); MediaBulkDeleter deleter = new MediaBulkDeleter(mMediaProvider, builder.build()); // Build the list of files from the content provider try { if (prescanFiles) { // First read existing files from the files table. // Because we'll be deleting entries for missing files as we go, // we need to query the database in small batches, to avoid problems // with CursorWindow positioning. long lastId = Long.MIN_VALUE; //每次操作限制读取1000个数据 Uri limitUri = mFilesUri.buildUpon().appendQueryParameter("limit", "1000").build(); while (true) { selectionArgs[0] = "" + lastId; if (c != null) { c.close(); c = null; } c = mMediaProvider.query(limitUri, FILES_PRESCAN_PROJECTION, where, selectionArgs, MediaStore.Files.FileColumns._ID, null); if (c == null) { break; } int num = c.getCount(); //获取到的数据个数判断是否为0,空的话就不用处理了 if (num == 0) { break; } //对1000个数据进行处理 while (c.moveToNext()) { long rowId = c.getLong(FILES_PRESCAN_ID_COLUMN_INDEX); String path = c.getString(FILES_PRESCAN_PATH_COLUMN_INDEX); int format = c.getInt(FILES_PRESCAN_FORMAT_COLUMN_INDEX); //数据库里获取文件最后修改时间 long lastModified = c.getLong(FILES_PRESCAN_DATE_MODIFIED_COLUMN_INDEX); lastId = rowId; // Only consider entries with absolute path names. // This allows storing URIs in the database without the // media scanner removing them. if (path != null && path.startsWith("/")) { boolean exists = false; try { //查询文件在系统里是否存在 exists = Os.access(path, android.system.OsConstants.F_OK); } catch (ErrnoException e1) { } if (!exists && !MtpConstants.isAbstractObject(format)) { // do not delete missing playlists, since they may have been // modified by the user. // The user can delete them in the media player instead. // instead, clear the path and lastModified fields in the row MediaFile.MediaFileType mediaFileType = MediaFile.getFileType(path); int fileType = (mediaFileType == null ? 0 : mediaFileType.fileType); //添加要删除的id deleter.delete(rowId); //如果.nomedia文件被删除了,那么就需要重新扫描这个文件夹,因为之前没有扫描。 if (path.toLowerCase(Locale.US).endsWith("/.nomedia")) { //开始删除老数据 deleter.flush(); String parent = new File(path).getParent(); mMediaProvider.call(MediaStore.UNHIDE_CALL, parent, null); } } } } } } } } finally { if (c != null) { c.close(); } //开始删除老数据 deleter.flush(); } // compute original size of images mOriginalCount = 0; c = mMediaProvider.query(mImagesUri, ID_PROJECTION, null, null, null, null); if (c != null) { mOriginalCount = c.getCount(); c.close(); } }
2、beginFile
3、endFile
每个文件处理都会调用一次endfile,主要是来判别文件的类型,插入到对应的表,不过并不是每次都插入,MediaIsert.java文件会对插入的数据计数,超过250条数据就一起插入数据库,调用bulkInsert。
// An highlighted block private Uri endFile(FileEntry entry, boolean ringtones, boolean notifications, boolean alarms, boolean music, boolean podcasts) throws RemoteException { // update database // use album artist if artist is missing if (mArtist == null || mArtist.length() == 0) { mArtist = mAlbumArtist; } //toValues保存专辑、艺术家、标题等信息,这些信息是从navie函数handleStringTag获取的, //这个函数是通过JNI:android_media_MediaScanner.cpp调用到MediaScannerClient的 //成员函数,这个函数被MediaScannerClient::addStringTag包装,其实最终还是被 //StagefrightMediaScanner.cpp调用,StagefrightMediaScanner中主要是从歌曲或者视频 //文件读取专辑信息,然后通过addStringTag传递给Java侧MediaScanner.java ContentValues values = toValues(); String title = values.getAsString(MediaStore.MediaColumns.TITLE); if (title == null || TextUtils.isEmpty(title.trim())) { title = MediaFile.getFileTitle(values.getAsString(MediaStore.MediaColumns.DATA)); values.put(MediaStore.MediaColumns.TITLE, title); } String album = values.getAsString(Audio.Media.ALBUM); if (MediaStore.UNKNOWN_STRING.equals(album)) { album = values.getAsString(MediaStore.MediaColumns.DATA); // extract last path segment before file name int lastSlash = album.lastIndexOf('/'); if (lastSlash >= 0) { int previousSlash = 0; while (true) { int idx = album.indexOf('/', previousSlash + 1); if (idx < 0 || idx >= lastSlash) { break; } previousSlash = idx; } if (previousSlash != 0) { album = album.substring(previousSlash + 1, lastSlash); values.put(Audio.Media.ALBUM, album); } } } //entry是在beginFile中就创建好,主要保存从数据库获取的该文件的信息,如果是新数据, //rowId就是0 long rowId = entry.mRowId; if (MediaFile.isAudioFileType(mFileType) && (rowId == 0 || mMtpObjectHandle != 0)) { // Only set these for new entries. For existing entries, they // may have been modified later, and we want to keep the current // values so that custom ringtones still show up in the ringtone // picker. values.put(Audio.Media.IS_RINGTONE, ringtones); values.put(Audio.Media.IS_NOTIFICATION, notifications); values.put(Audio.Media.IS_ALARM, alarms); values.put(Audio.Media.IS_MUSIC, music); values.put(Audio.Media.IS_PODCAST, podcasts); } else if ((mFileType == MediaFile.FILE_TYPE_JPEG || MediaFile.isRawImageFileType(mFileType)) && !mNoMedia) { ExifInterface exif = null; try { exif = new ExifInterface(entry.mPath); } catch (IOException ex) { // exif is null } if (exif != null) { float[] latlng = new float[2]; if (exif.getLatLong(latlng)) { values.put(Images.Media.LATITUDE, latlng[0]); values.put(Images.Media.LONGITUDE, latlng[1]); } long time = exif.getGpsDateTime(); if (time != -1) { values.put(Images.Media.DATE_TAKEN, time); } else { // If no time zone information is available, we should consider using // EXIF local time as taken time if the difference between file time // and EXIF local time is not less than 1 Day, otherwise MediaProvider // will use file time as taken time. time = exif.getDateTime(); if (time != -1 && Math.abs(mLastModified * 1000 - time) >= 86400000) { values.put(Images.Media.DATE_TAKEN, time); } } int orientation = exif.getAttributeInt( ExifInterface.TAG_ORIENTATION, -1); if (orientation != -1) { // We only recognize a subset of orientation tag values. int degree; switch(orientation) { case ExifInterface.ORIENTATION_ROTATE_90: degree = 90; break; case ExifInterface.ORIENTATION_ROTATE_180: degree = 180; break; case ExifInterface.ORIENTATION_ROTATE_270: degree = 270; break; default: degree = 0; break; } values.put(Images.Media.ORIENTATION, degree); } } } Uri tableUri = mFilesUri; MediaInserter inserter = mMediaInserter; if (!mNoMedia) { //判断是什么类型的文件,同时创建对应的URI if (MediaFile.isVideoFileType(mFileType)) { tableUri = mVideoUri; } else if (MediaFile.isImageFileType(mFileType)) { tableUri = mImagesUri; } else if (MediaFile.isAudioFileType(mFileType)) { tableUri = mAudioUri; } } Uri result = null; boolean needToSetSettings = false; // Setting a flag in order not to use bulk insert for the file related with // notifications, ringtones, and alarms, because the rowId of the inserted file is // needed. if (notifications && !mDefaultNotificationSet) { if (TextUtils.isEmpty(mDefaultNotificationFilename) || doesPathHaveFilename(entry.mPath, mDefaultNotificationFilename)) { needToSetSettings = true; } } else if (ringtones && !mDefaultRingtoneSet) { if (TextUtils.isEmpty(mDefaultRingtoneFilename) || doesPathHaveFilename(entry.mPath, mDefaultRingtoneFilename)) { needToSetSettings = true; } } else if (alarms && !mDefaultAlarmSet) { if (TextUtils.isEmpty(mDefaultAlarmAlertFilename) || doesPathHaveFilename(entry.mPath, mDefaultAlarmAlertFilename)) { needToSetSettings = true; } } if (rowId == 0) { if (mMtpObjectHandle != 0) { values.put(MediaStore.MediaColumns.MEDIA_SCANNER_NEW_OBJECT_ID, mMtpObjectHandle); } if (tableUri == mFilesUri) { int format = entry.mFormat; if (format == 0) { format = MediaFile.getFormatCode(entry.mPath, mMimeType); } values.put(Files.FileColumns.FORMAT, format); } // New file, insert it. // Directories need to be inserted before the files they contain, so they // get priority when bulk inserting. // If the rowId of the inserted file is needed, it gets inserted immediately, // bypassing the bulk inserter. if (inserter == null || needToSetSettings) { if (inserter != null) { //把数据插入数据库 inserter.flushAll(); } result = mMediaProvider.insert(tableUri, values); } else if (entry.mFormat == MtpConstants.FORMAT_ASSOCIATION) { inserter.insertwithPriority(tableUri, values); } else { inserter.insert(tableUri, values); } if (result != null) { rowId = ContentUris.parseId(result); entry.mRowId = rowId; } } else { // updated file result = ContentUris.withAppendedId(tableUri, rowId); // path should never change, and we want to avoid replacing mixed cased paths // with squashed lower case paths values.remove(MediaStore.MediaColumns.DATA); int mediaType = 0; if (!MediaScanner.isNoMediaPath(entry.mPath)) { int fileType = MediaFile.getFileTypeForMimeType(mMimeType); if (MediaFile.isAudioFileType(fileType)) { mediaType = FileColumns.MEDIA_TYPE_AUDIO; } else if (MediaFile.isVideoFileType(fileType)) { mediaType = FileColumns.MEDIA_TYPE_VIDEO; } else if (MediaFile.isImageFileType(fileType)) { mediaType = FileColumns.MEDIA_TYPE_IMAGE; } else if (MediaFile.isPlayListFileType(fileType)) { mediaType = FileColumns.MEDIA_TYPE_PLAYLIST; } values.put(FileColumns.MEDIA_TYPE, mediaType); } mMediaProvider.update(result, values, null, null); } if(needToSetSettings) { if (notifications) { setRingtoneIfNotSet(Settings.System.NOTIFICATION_SOUND, tableUri, rowId); mDefaultNotificationSet = true; } else if (ringtones) { setRingtoneIfNotSet(Settings.System.RINGTONE, tableUri, rowId); mDefaultRingtoneSet = true; } else if (alarms) { setRingtoneIfNotSet(Settings.System.ALARM_ALERT, tableUri, rowId); mDefaultAlarmSet = true; } } return result; }
MediaScanner.cpp
StagefrightMediaScanner.cpp
配置修改
修改数据库路径
修改数据库WAL模式
- 设置db.enableWriteAheadLogging();可以在MediaProvider写数据的时候,UI读数据不会被阻塞。读写不会阻塞,但是只允许同时只有一用户写。设置WAL模式可以提高数据库写速度,降低磁盘IO,但是读数据就会变慢。具体原理可以参考
链接: SQLite分析之WAL机制.
存在的问题
- Android MediaProvider框架对于手机扫描来说是很贴切的,但是对于汽车车载系统来说就不是很友好,因为车机可能需要用USB来存储媒体数据,新USB扫描速度很慢,手机一般不需要插USB。这节和下一节是针对车载系统的一些关于USB扫描的探讨和优化。
问题:
1、对于不同USB,MediaProvider会保留不同的数据库,占用多余磁盘空间;
2、扫描时会读取音视频文件title等信息,读取文件磁盘IO会导致扫描速度变慢,原本需要5分钟,可能就变成20分钟;
3、prescan预扫描时候可以读取数据,并且prescan后也没有明确的广播通知,如果数据被大量删除,UI会读取到已经删除的数据;
4、扫描是顺序扫描,如果一直在扫描歌曲还没扫描到视频,那么视频要等好久才能检索到;
性能优化
一、扫描方案优化
-
对于IO读写慢的问题是无法回避的,为什么要读取文件信息,因为需要添加歌曲视频的专辑标题等信息,在UI侧才能做成专辑等列表。但是从用户的角度分析,如果我U盘插入车机,要听音乐,我大体上用打开歌曲列表,或者文件夹,就可以快速找到自己想要播放的歌曲,或者是收藏列表。可能专辑风格艺术家等列表被打开的概率只有20%,但是这20%的概率却占用了扫描80%的时间,我觉得是不合理的,但是又不能不做,所以我觉得,可以先用很短的时间做成一个快表,这个快表能够提供歌曲视频列表,文件夹信息和收藏列表,之后再做专辑列表。我自己也尝试去做这个快表,如果整体扫描时间是20分钟的话,做快表的时间在5-15秒就可以完成,理论上可以达到5秒。等这个快表做成,就开始走正常的扫描流程,或者说两者一起并行运行也是可以的。
-
这个方案的缺陷就是CPU占比在一瞬间会比较高,而且原来扫描流程会慢几秒(我感觉可以省略)。
-
快表的做成我大部分是用c++/c语言写的,生成一个so库,Java调用so库来扫描和做成数据库,同时提供一个Provider给客户端调用。快速扫描功能包括预扫描删除没用的老数据、扫描数据时会判别是否存在、是否更改过,每个文件都保存上级文件夹的id,有一个歌曲表一个视频表和一个文件夹表。
-
快表的做成我已经完成80%了,由于工作问题暂时停了,如果需要可以看我GitHub: 链接:
MediaScanner快表做成.
UI部分我做的很随意,不要吐槽,主要还是想快速实现,所以代码也很乱,之后会整理好。
二、数据库优化
-
wal模式可以提高数据的写速度;
-
同时也可以通过事务,来减少磁盘IO;
-
在插入数据时会判断数据是否存在是否更新过,这个时候需要通过路径去数据库query,所以建议做个绝对路径的索引,索引就做这个就够用了;
-
文件夹如果没有文件就不需要插入到数据库中了,所以在插入歌曲视频的时候才插入上级文件夹就可以了;
待续。。。
- Android 多媒体扫描过程(Android Media Scanner Process)
- check android Media Scanner is running-检查Android Media Scanner 是否在运行
- Android 7.0后SettingProvider ContactsProvider TelephonyProvider MediaProvider数据库位置
- Android MediaProvider数据库分析
- Android 多媒体扫描过程(Android Media Scanner Process)
- 多媒体数据库:android.provider.MediaStore
- android中媒体扫描服务mediaScannerService
- Android 多媒体扫描过程(Android Media Scanner Process)
- Android MediaScannerJNI源码详解
- java.lang.SecurityException: Permission Denial: opening provider com.android.providers.media.MediaDo
- Android MediaScannerService源码分析
- Android 多媒体扫描过程(Android Media Scanner Process)
- Android Media Scanner Mechanism Analyze
- Android4.2.2 关于unmount SDcard流程中updateExternalMediaStatus部分
- 关于CTS android.provider.cts.MediaStore_Images_MediaTest 重跑一轮fail的问题
- android Media Route Provider
- Android之Intent.ACTION_MEDIA_SCANNER_SCAN_FILE:扫描指定文件
- android MediaScanner 原生bug 关于系统铃音
- Android Media Scanner Mechanism Analyze
- Android MediaScanner:(一)MediaScanner总体架构