pandas的read_sql报错UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd3 in position 0: invalid con
2017-10-18 23:08
676 查看
1.这个错误一旦发生,很难找出来,下面是错误代码
问题主要针对数据库中字段含中文,编码方式gbk等其他的。
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('oracle://scott:tiger@localhost:1521/orcl1?charset=utf8')
df3=pd.read_sql_table('ys_table1',engine)
从问题看好像编码错误,但是按网上的怎么修改都还是错误。
第2种情况
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('oracle://scott:tiger@localhost:1521/orcl1?charset=utf8')
df3=pd.read_sql_table('ys_table1',engine)
报错后加了
import os
os.environ['NLS_LANG'] = 'SIMPLIFIED CHINESE_CHINA.UTF8'
df3=pd.read_sql_table('ys_table1',engine)
依然报错
第3种情况
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
import os
os.environ['NLS_LANG'] = 'SIMPLIFIED CHINESE_CHINA.UTF8'
engine = create_engine('oracle://scott:tiger@localhost:1521/orcl1?charset=utf8')
df3=pd.read_sql_table('ys_table1',engine)
成功执行df3。问题解决了,有没有大牛知道为什么要先加os.environ才能执行成功,后加就不能成功,问题浪费太多时间,求解答。
2.另外针对cx_oracle的安装成功运行sql报错问题,主要涉及软件版本兼容问题,如oracle32位,python64位出现编码错误,
其实是版本兼容问题,要安装一致才行,这个问题不详解。
python数据库的读写
import pandas as pd
import numpy as np
from datetime import date,datetime
import os
import cx_Oracle
connection=cx_Oracle.connect('scott','tiger','localhost:1521/orcl1')
os.environ['NLS_LANG']='SIMPLIFIED CHINESE _CHINA.UTF8'
df=pd.read_sql_query("select * from emp_test",engine)
备注:从oracle数据库读取到python里默认设置了空值
写回数据库
connection1=cx_Oracle.connect('scott','tiger','localhost:1521/orcl1')
cursor=connection1.cursor()
try:
cursor.execute("create table emp_test1 as select * from emp_test where 0=1")
except:
cursor.execute("truncate table emp_test1")
sql="""INSERT INTO emp_test1 values(:1,:2,:3,:4,:5,:6,:7,:8)"""
cursor.executemany(sql,df.values.tolist())
connection1.commit()
cursor.close()
connection1.close()
备注:默认将空值改为0及其他。
问题主要针对数据库中字段含中文,编码方式gbk等其他的。
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('oracle://scott:tiger@localhost:1521/orcl1?charset=utf8')
df3=pd.read_sql_table('ys_table1',engine)
从问题看好像编码错误,但是按网上的怎么修改都还是错误。
第2种情况
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
engine = create_engine('oracle://scott:tiger@localhost:1521/orcl1?charset=utf8')
df3=pd.read_sql_table('ys_table1',engine)
报错后加了
import os
os.environ['NLS_LANG'] = 'SIMPLIFIED CHINESE_CHINA.UTF8'
df3=pd.read_sql_table('ys_table1',engine)
依然报错
第3种情况
import numpy as np
import pandas as pd
from sqlalchemy import create_engine
import os
os.environ['NLS_LANG'] = 'SIMPLIFIED CHINESE_CHINA.UTF8'
engine = create_engine('oracle://scott:tiger@localhost:1521/orcl1?charset=utf8')
df3=pd.read_sql_table('ys_table1',engine)
成功执行df3。问题解决了,有没有大牛知道为什么要先加os.environ才能执行成功,后加就不能成功,问题浪费太多时间,求解答。
2.另外针对cx_oracle的安装成功运行sql报错问题,主要涉及软件版本兼容问题,如oracle32位,python64位出现编码错误,
其实是版本兼容问题,要安装一致才行,这个问题不详解。
python数据库的读写
import pandas as pd
import numpy as np
from datetime import date,datetime
import os
import cx_Oracle
connection=cx_Oracle.connect('scott','tiger','localhost:1521/orcl1')
os.environ['NLS_LANG']='SIMPLIFIED CHINESE _CHINA.UTF8'
df=pd.read_sql_query("select * from emp_test",engine)
备注:从oracle数据库读取到python里默认设置了空值
写回数据库
connection1=cx_Oracle.connect('scott','tiger','localhost:1521/orcl1')
cursor=connection1.cursor()
try:
cursor.execute("create table emp_test1 as select * from emp_test where 0=1")
except:
cursor.execute("truncate table emp_test1")
sql="""INSERT INTO emp_test1 values(:1,:2,:3,:4,:5,:6,:7,:8)"""
cursor.executemany(sql,df.values.tolist())
connection1.commit()
cursor.close()
connection1.close()
备注:默认将空值改为0及其他。
相关文章推荐
- python报错:SyntaxError: (unicode error) 'utf-8' codec can't decode byte 0xb4 in position 2:invalid sta
- python3 读入csv文件报错('utf-8' codec can't decode byte 0x8b in position 2: invalid start byte)的解决
- Pickle:UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 884: invalid start byte
- read_csv UnicodeDecodeError: 'utf8' codec can't decode byte 0xb1 in position 0: invalid start byte
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 2892: invalid continuation byte
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd5 in position 0: invalid continuation byte
- Python爬虫报UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa1 in position 225: invalid start byt
- 【转】UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc7 in position 1: invalid continuation 汉字编码
- python3 读入csv文件报错('utf-8' codec can't decode byte 0x8b in position 2: invalid start byte)的解决
- pandas读取出错:UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd6 in position 0
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0x89 in position 0: invalid start byte
- 解决 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
- 解决 python爬虫'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte‘错误
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb8 in position 0: invalid start byte
- 解决 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
- UnicodeDecodeError: 'utf8' codec can't decode byte 0xd3 in position 88: invalid continuation byte
- 报错信息:UnicodeDecodeError: 'utf-8' codec can't decode byte 0x88 in position 6382: invalid start byte
- [python]pandas的read_csv读入csv文件报错 'utf-8' codec can't decode byte 0xbe in position 0
- UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc8 in position 12: invalid continuation byte