Import SQL into MySQL with a progress meter

There is nice tool called

pv

# On Ubuntu/Debian system
$ sudo apt-get install pv

# On Redhat/CentOS
$ sudo yum install pv

then e.g. you can use it like this

pv sqlfile.sql | mysql -u xxx -p xxxx dbname

$ zcat dbpackfile.sql.gz | pv -cN zcat | mysql -uuser -ppass dbname

ps: check this blog http://blog.larsstrand.org/2011/12/tip-pipe-viewer.html

UPDATE: seems like above link is broken but I found same article here http://blog.larsstrand.no/2011/12/tip-pipe-viewer.html

UPDATE 2: Even better solution with FULL progress bar. To do it you need to use 2 build in

pv

options. One is

--progress

to indicate progress bar and second is

--size

to tell

pv

how large the overall file is.

pv --progress --size UNPACKED-FILE-SIZE-IN-BYTES

..the problem is with

.gz

original file size. You need somehow get unpacked original file size information without unpacking it self, otherwise you will lost our precious time to unpack this file twice (first time

pv

and second time

zcat

). But fortunately you have

gzip -l

option that contain uncompressed information about our gziped file. Unfortunattly you have it in table format so you need to extract before it can be use it. All together can be seen below:

gzip -l /path/to/our/database.sql.gz | sed -n 2p | awk '{print $2}'

Uff.. so the last thing you need to do is just combine all together.

zcat /path/to/our/database.sql.gz | pv --progress --size `gzip -l %s | sed -n 2p | awk '{print $2}'` | mysql -uuser -ppass dbname

To make it even nicer you can add progres NAME like this

zcat /path/to/our/database.sql.gz | pv --progress --size `gzip -l %s | sed -n 2p | awk '{print $2}'` --name '  Importing.. ' | mysql -uuser -ppass dbname

Final result:

Importing.. : [===========================================>] 100%

=================================================================
#sudo yum install pv
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
* base: mirror.neu.edu.cn
* extras: mirror.neu.edu.cn
* updates: mirror.neu.edu.cn
Setting up Install Process
No package pv available.
Error: Nothing to do
------------------------------------------------------------------ http://wiki.centos.org/AdditionalResources/Repositories/RPMForge#head-f0c3ecee3dbb407e4eed79a56ec0ae92d1398e01 wget http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt rpm -K rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
pm -i rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm
yum install pv

=================================================================

ref:
http://www.ivarch.com/programs/pv.shtml http://unix.stackexchange.com/questions/36769/get-a-progress-indicator-when-importing-mysql-databases https://major.io/2010/11/24/monitor-mysql-restore-progress-with-pv/ http://dba.stackexchange.com/questions/17367/how-can-i-monitor-the-progress-of-an-import-of-a-large-sql-file http://www.commandlinefu.com/commands/view/5884/import-sql-into-mysql-with-a-progress-meter