hive-19994 bug java.sql.BatchUpdateException
事件背景:
大数据应用跑业务时候,涉及对hive表删除操作。删除过程中,会引起hive metastore报错:java.sql.BatchUpdateException: Cannot delete or update a parent row: a foreign key constraint fails ("hivemetastore_emtig3vtq7qp1tiooo07sb70ud"."COLUMNS_V2", CONSTRAINT "COLUMNS_V2_FK1" FOREIGN KEY ("CD_ID") REFERENCES "CDS" ("CD_ID"))。
问题分析:
此为hive 的一个bug,详细信息如
https://issues.apache.org/jira/browse/HIVE-19994
hive的表元数据存储在mysql中,mysql中两表之间有外键约束(关于mysql外键约束详见下文),导致无法删除。
mysql 外键约束介绍:
l RESTRICT: 拒绝删除或者更新父表。指定RESTRICT(或者NO ACTION)和忽略ON DELETE或者ON UPDATE选项的效果是一样的。(mysql创建表时,外键约束默认值)
l CASCADE: 从父表中删除或更新对应的行,同时自动的删除或更新自表中匹配的行。ON DELETE CANSCADE和ON UPDATE CANSCADE都被InnoDB所支持。
解决方案:
l 官方推荐解决方案:
更改hive-metastore.jar包中的package.jdo,添加如下内容:
此行下边添加如下内容:
</datastore-identity> 此行下边添加如下内容: <field name="cols" table="COLUMNS_V2" > <collection element-type="MFieldSchema"/> <foreign-key name="COLUMNS_V2_FK1" delete-action="cascade"/> <join> <primary-key name="COLUMNS_PK"> <column name="COLUMN_NAME"/> </field>
l 其他可行方案:(公司采取)
在mysql中删除表COLUMNS_V2的外键,重新创建外键,并制定外键约束为ON DELETE CANSCADE和ON UPDATE CANSCADE
ALTER TABLE COLUMNS_V2 DROP FOREIGN KEY COLUMNS_V2_FK1
ALTER TABLE COLUMNS_V2 ADD CONSTRAINT COLUMNS_V2_FK1 FOREIGN KEY (CD_ID) REFERENCES CDS(CD_ID)ON DELETE CANSCADE ON UPDATE CANSCADE;
- Caused by: java.sql.BatchUpdateException: You have an error in your SQL syntax; check the manual tha
- java.sql.BatchUpdateException: Field 'id' doesn't have a default value
- java.sql.BatchUpdateException: ORA-02290: check...
- java.sql.BatchUpdateException: internal error
- Caused by:java.sql.BatchUpdateException:ORA-02291
- java.sql.BatchUpdateException
- java.sql.BatchUpdateException: ORA-02291: integrity constraint
- java.sql.BatchUpdateException
- Caused by: java.sql.BatchUpdateException: ORA-00001: 违反唯一约束条件
- Caused by: java.sql.BatchUpdateException: Can not issue executeUpdate() for SELECTs
- Error updating database. Cause: java.sql.BatchUpdateException: Field 'id' doesn't have a default value
- Caused by:java.sql.BatchUpdateException:ORA-02291
- Nutch抓取错误——java.sql.BatchUpdateException: Incorrect string value: '\xF2\xA3\xAC\xB7\xEF\xBF.
- [SSH] java.sql.BatchUpdateException: Cannot delete or update a parent row:
- java.sql.BatchUpdateException(转载)
- java.sql.BatchUpdateException: Incorrect string value: '\xF0\x9D\x94\xB9+)...' for column 'NAME_' at
- UTF8编码的原理及白名单过滤utf8mb4(Caused by: java.sql.BatchUpdateException: Incorrect string value)
- java.sql.BatchUpdateException:
- java.sql.BatchUpdateException: ORA-00001: unique constraint (IEVSP.PK_C_CSERVICES) violated
- oracle Caused by: java.sql.BatchUpdateException: ORA-01461: 仅能绑定要插入 LONG 列的 LONG 值