cancel
Showing results for 
Search instead for 
Did you mean: 

Error migrating DB to v5.11 schema

brianshowers
Champ in-the-making
Champ in-the-making
I just tried to upgrade to the new v5.11 version, but unfortunately, the schema upgrade SQL is not valid.  If you attempt to run it on MySQL instances that don't allow invalid Dates to be created, you'll get the following error:


2012-12-05 16:43:23 ERROR org.activiti.engine.impl.db.DbSqlSession problem during schema upgrade, statement 'create table ACT_RE_MODEL (
ID_ varchar(64) not null,
REV_ integer,
NAME_ varchar(255),
KEY_ varchar(255),
CATEGORY_ varchar(255),
CREATE_TIME_ timestamp,
LAST_UPDATE_TIME_ timestamp,
VERSION_ integer,
META_INFO_ varchar(4000),
DEPLOYMENT_ID_ varchar(64),
EDITOR_SOURCE_VALUE_ID_ varchar(64),
EDITOR_SOURCE_EXTRA_VALUE_ID_ varchar(64),
primary key (ID_)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_bin
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Invalid default value for 'LAST_UPDATE_TIME_'
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_04]
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_04]
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_04]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:525) ~[na:1.7.0_04]
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.Util.getInstance(Util.java:386) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1053) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4074) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4006) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2468) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2629) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2713) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2663) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:888) ~[mysql-connector-java-5.1.21.jar:na]
        at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:730) ~[mysql-connector-java-5.1.21.jar:na]
        at org.activiti.engine.impl.db.DbSqlSession.executeSchemaResource(DbSqlSession.java:1015) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.db.DbSqlSession.executeSchemaResource(DbSqlSession.java:966) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.db.DbSqlSession.dbSchemaUpgrade(DbSqlSession.java:945) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.db.DbSqlSession.dbSchemaUpdate(DbSqlSession.java:826) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.db.DbSqlSession.performSchemaOperationsProcessEngineBuild(DbSqlSession.java:1100) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.SchemaOperationsProcessEngineBuild.execute(SchemaOperationsProcessEngineBuild.java:25) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.interceptor.CommandExecutorImpl.execute(CommandExecutorImpl.java:24) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:60) [activiti-engine-5.11.jar:5.11]
        at org.activiti.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:42) [activiti-spring-5.11.jar:na]
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130) [spring-tx-3.1.2.RELEASE.jar:3.1.2.RELEASE]
        at org.activiti.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:40) [activiti-spring-5.11.jar:na]
        at org.activiti.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:32) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.ProcessEngineImpl.<init>(ProcessEngineImpl.java:75) [activiti-engine-5.11.jar:5.11]
        at org.activiti.engine.impl.cfg.ProcessEngineConfigurationImpl.buildProcessEngine(ProcessEngineConfigurationImpl.java:311) [activiti-engine-5.11.jar:5.11]
        at org.activiti.spring.SpringProcessEngineConfiguration.buildProcessEngine(SpringProcessEngineConfiguration.java:60) [activiti-spring-5.11.jar:na]

The problem is that the upgrade attempts to add TIMESTAMP columns without specifying a valid default value.  When this happens, MySQL will implicitly set the default to the string "0000-00-00 00:00:00".  See the following description of the schema after a migration to v5.11:


mysql> describe ACT_RE_MODEL;
+——————————-+—————+——+—–+———————+—————————–+
| Field                         | Type          | Null | Key | Default             | Extra                       |
+——————————-+—————+——+—–+———————+—————————–+
| ID_                           | varchar(64)   | NO   | PRI | NULL                |                             |
| REV_                          | int(11)       | YES  |     | NULL                |                             |
| NAME_                         | varchar(255)  | YES  |     | NULL                |                             |
| KEY_                          | varchar(255)  | YES  |     | NULL                |                             |
| CATEGORY_                     | varchar(255)  | YES  |     | NULL                |                             |
| CREATE_TIME_                  | timestamp     | NO   |     | CURRENT_TIMESTAMP   | on update CURRENT_TIMESTAMP |
| LAST_UPDATE_TIME_             | timestamp     | NO   |     | 0000-00-00 00:00:00 |                             |
| VERSION_                      | int(11)       | YES  |     | NULL                |                             |
| META_INFO_                    | varchar(4000) | YES  |     | NULL                |                             |
| DEPLOYMENT_ID_                | varchar(64)   | YES  | MUL | NULL                |                             |
| EDITOR_SOURCE_VALUE_ID_       | varchar(64)   | YES  | MUL | NULL                |                             |
| EDITOR_SOURCE_EXTRA_VALUE_ID_ | varchar(64)   | YES  | MUL | NULL                |                             |
+——————————-+—————+——+—–+———————+—————————–+
12 rows in set (0.02 sec)

However, "0000-00-00 00:00:00" is not a value that can be marshaled into either a java.sql.Date or a java.util.Date.  If you ever attempt to do so, Java will throw a parse exception.  As a result, MySQL has the ability to configure it's sql-mode to disallow these invalid date strings.  This can be done by settings one/all of the following sql-mode values: TRADITIONAL, NO_ZERO_IN_DATE, NO_ZERO_DATE.  These settings make MySQL behave more like a traditional database by disallowing many of the non-standard MySQL-isms.  I've verified that if you relax the sql-mode, the Activiti upgrade can proceed.  However, that's not a viable solution since relaxing the sql-mode is a global change that applies to all databases on that server (not just the Activiti DB).

As a result, the only viable fix is to change the upgrade SQL to provide a valid default value (NULL or some valid date).
17 REPLIES 17

brianshowers
Champ in-the-making
Champ in-the-making
@jbarrez and @trademak

Out of the box, MySQL uses a very lenient set of sql-mode settings that allow you to do all sorts of non-compliant SQL.  In fact, the out of the box settings are pretty poor for running a high availability system since they attempt to "fix" bad SQL for you with what the server "thinks" you meant.  This is pretty awful of MySQL because it can hide bugs in code and corrupt data in ways that you don't expect.

If you change your QA environment to set sql-mode=TRADITIONAL or even just sql-mode=NO_ZERO_IN_DATE, NO_ZERO_DATE, you should be able to reproduce the problem pretty easily.  That sql-mode setting is much more indicative of what a production system would use since it enforces much more error checking by MySQL without the DB trying to guess at what the invalid SQL was trying to do.

jbarrez
Star Contributor
Star Contributor
I definitely see your point of why it is useful in production.
But I'm not a DBA, and so far this is the first time I hear about it (having done 5+ years of jbpm and Activiti on MySQL).

Ie. I'm having a hard time finding how widespread this setting is in the wild. What is the impact on the SQL, on performance, etc?
What would we need to do to make the current scripts compatible?

brianshowers
Champ in-the-making
Champ in-the-making
I'm certain that the reason you've never run into it is that you almost certainly always write ANSI compatible SQL without thinking about it Smiley Very Happy .  It's just this one time that you've run into the case where MySQL's default settings will create non ANSI compatible SQL.

As far as the impact goes, there will be none.  There is no performance impact since putting a server into TRADITIONAL mode simply causes the query parser to reject non-standard SQL syntax.  In the default setting, the parser will "correct" invalid syntax for you.  There will also be no impact on backwards compatibility since any SQL query that is valid in TRADITIONAL mode is also valid in the default MySQL mode.  If you want to learn more about MySQL's TRADITIONAL mode, this is a pretty good (and quick) read: http://answers.oreilly.com/topic/171-how-to-use-the-sql-mode-to-control-bad-input-data-handling-in-m...

As far as the fix goes, I think it's extremely simple.  In the activiti.mysql.upgradestep.510.to.511.engine.sql, when creating the ACT_RE_MODEL table, you simply need to provide a default value for the CREATE_TIME_ and LAST_UPDATE_TIME_ columns.  Presumably, 'null' would be fine.  When you don't specify a default for a TIMESTAMP type, MySQL will use the value "0000-00-00 00:00:00" which is not a valid ANSI SQL timestamp.  So the fix should simply be:


create table ACT_RE_MODEL (
    ID_ varchar(64) not null,
    REV_ integer,
    NAME_ varchar(255),
    KEY_ varchar(255),
    CATEGORY_ varchar(255),
    CREATE_TIME_ timestamp null,
    LAST_UPDATE_TIME_ timestamp null,
    VERSION_ integer,
    META_INFO_ varchar(4000),
    DEPLOYMENT_ID_ varchar(64),
    EDITOR_SOURCE_VALUE_ID_ varchar(64),
    EDITOR_SOURCE_EXTRA_VALUE_ID_ varchar(64),
    primary key (ID_)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE utf8_bin;

Of course, the same change will also have to be made in activiti.mysql.create.engine.sql

trademak
Star Contributor
Star Contributor
Hi,

We've changed the MySQL scripts according to your suggestion. Could you check if it's ok now?
So the changes are made on the current master on Github.

Best regards,

brianshowers
Champ in-the-making
Champ in-the-making
That did it!  All of the activiti-engine unit tests continue to pass for me, and I am now able to both upgrade an existing 5.10 schema and create a new schema from scratch.

Is there a plan to cut a 5.11.x release that will contain this fix, or will we be forced to wait for the 5.12 release?  I could fork the code and release my own 5.11.x release, but I would be nervous that the eventual upgrade path from our custom 5.11.x to the official 5.12 would not handle the schema migration correctly.

Also, for the final fix, you may also want a 5.12 (or 5.11.1) patch that will add the default values for anyone who upgraded without the fix.  Otherwise, there would be two different schema's in the wild that both say they are v5.11 but have different table definitions.

trademak
Star Contributor
Star Contributor
Hi,

Thanks for the feedback. We'll publish 5.12-SNAPSHOTS that contain the fix.
If there are more people that run into this issue we might consider updating the 5.11 release.
For now people can use the updated script in the repository.

Best regards,

brianshowers
Champ in-the-making
Champ in-the-making
How would that work using a 5.12-SNAPSHOT version? Based on what I've seen, when I run the upgrade using a v5.12-SNAPSHOT copy of the code, the DB schema will then be marked as v5.12-SNAPSHOT.  Once that happens, it doesn't seem like any other 5.12-SNAPSHOT versions could ever be applied since the schema upgrade will have been partially applied.  Furthermore, would migration from 5.12-SNAPSHOT to the eventual 5.12 work? 

It seems to me like there would need to be a new official 5.11 release since that's which version the fix is labeled as.  If a 5.12-SNAPSHOT version was ever applied to a DB, that DB will be left in a state that can't be auto-migrated from.

trademak
Star Contributor
Star Contributor
I would like to know how many people are using this MySQL strict option, because right now I've only heard this issue from you.
If there are more people with this issue then doing a fix 5.11 release again would make sense.
Also the automatic schema update option for Activiti is something I myself would not use on a non-development machine db.
I would always use the database scripts directly. And then the current fix already helps out.

Best regards,